Sign in
Transforms

Create transform

Create a new transform. Provide sql_template with {{weldTag}} references. Set publish=true to immediately materialize in the data warehouse. With publish=true and materialization "table", cron_expression puts the transform on its own materialization schedule.

post/transforms

Request body

namestring required

The name of the transform.

folder_pathstring

The folder path for the transform. Intermediate folders will be auto-created if they do not exist.

sql_templatestring required

The SQL template with {{weldTag}} parameter references.

materialization'view' | 'table'

The materialization type. Defaults to "view".

documentationstring

Documentation for the transform.

publishboolean

If true, the transform will be published (materialized to DWH) immediately after creation.

cron_expressionstring

Cron expression (UTC) defining the transform's own materialization schedule, applied once it has been published. Requires publish=true and materialization "table". Omit for a transform that should only be materialized on demand.

wait_for_completionboolean

If true, the API will wait for table materialization to complete before returning, up to a configured timeout (default ~45 seconds; may differ per environment). Only relevant when publishing with materialization set to "table". If the materialization does not complete within the timeout, the response will still succeed with materialization_status "running".

Example request

{
  "name": "marketing_attribution",
  "folder_path": "analytics/marketing",
  "sql_template": "SELECT * FROM {{raw.stripe.payments}} WHERE created_at > CURRENT_DATE - 30",
  "materialization": "view",
  "documentation": "Calculates marketing attribution from ad spend and conversions.",
  "cron_expression": "0 6 * * *"
}

Response

The transform has been successfully created.

idstring required

The ID of the transform.

namestring required

The name of the transform.

folder_pathstring required

The folder path for the transform.

status'draft' | 'published' required

The status of the transform.

materialization'view' | 'table' required

The materialization type.

sql_templatestring required

The SQL template with {{weldTag}} parameter references. For a published transform this is the currently published SQL — an unpublished draft edit does NOT change it. Check has_unpublished_changes and read draft_sql_template to see such an edit.

has_unpublished_changesboolean required

True when the transform is published and a newer, still-unpublished SQL version exists — for example after updating it without publish=true. Always false for a transform that has never been published, where status "draft" already says so and sql_template is the draft itself.

draft_sql_templatestring

The SQL template of the newest unpublished version, present only when has_unpublished_changes is true. Publish it with the publish endpoint (or publish_transform) to make it live.

documentationstring

Documentation for the transform.

materialization_status'running' | 'completed' | 'failed' nullable

The status of the table materialization in the data warehouse. Set to "running", "completed", or "failed" for published table transforms, or null for drafts and view-type transforms.

materialization_errorstring

Error message from the data warehouse if table materialization failed.

orchestration_workflow_idstring

The ID of the Orchestration Workflow that this transform is attached to, if any. Only reported while the attachment is actually in effect, so it can be relied on as the answer to "does this transform build as part of a workflow?".

rebuild_trigger'none' | 'not_applicable' | 'not_scheduled' | 'orchestration' | 'schedule' required

What will cause this transform to be rebuilt from now on. orchestration — it runs as part of an Orchestration Workflow. schedule — it runs on its own cron_expression. not_scheduled — it is deliberately set to build once and never update. not_applicable — it is a view, which has no independent build; a view is always current because it is a query, not a stored table. none — nothing will rebuild it and that is not intentional: either it is still a draft, or it is a published table that lost its schedule, which can happen after detaching it from a workflow. none is the only value that indicates a problem, so to find transforms that have silently stopped building look for status "published" with rebuild_trigger "none".

cron_expressionstring nullable

Cron expression (UTC) configured as the transform's own materialization schedule, or null if none has been set. It is only acted on while the transform is published, materialized as a "table", and not attached to an Orchestration Workflow. It is still reported in the other cases — a draft, a view, or a transform running as part of a workflow (see orchestration_workflow_id) — so you can see the schedule the transform would fall back to once it is published as a table again or detached from the workflow. Read rebuild_trigger to tell whether it is currently in effect: it reports "schedule" exactly when this cron is what rebuilds the transform.

created_atstring date-time required

The datetime when the transform was created.

updated_atstring date-time required

The datetime when the transform was last updated.

Example response

{
  "id": "OoORhGXUVjsqFG",
  "name": "marketing_attribution",
  "folder_path": "analytics/marketing",
  "status": "published",
  "materialization": "view",
  "sql_template": "SELECT * FROM {{raw.stripe.payments}} WHERE created_at > CURRENT_DATE - 30",
  "draft_sql_template": "SELECT * FROM {{raw.stripe.payments}} WHERE created_at > CURRENT_DATE - 7",
  "parameters": [
    {
      "weld_tag": "raw.stripe.payments",
      "type": "raw_view",
      "dw_item_id": "stripe_schema.payments"
    }
  ],
  "documentation": "Calculates marketing attribution.",
  "materialization_status": "completed",
  "materialization_error": "Column \"foo\" not found in table \"bar\".",
  "orchestration_workflow_id": "3LIw2FdxoByya9",
  "rebuild_trigger": "schedule",
  "cron_expression": "0 6 * * *",
  "created_at": "2020-01-01T00:00:00.000Z",
  "updated_at": "2020-01-01T00:00:00.000Z"
}

Changes