KEkeboola
Tables

Asynchronous Export

Exports data into File Storage. The Id of the created file is returned. This task is asynchronous; Id of the created file is stored in the results of the created job when the job is finished.

Rows filtering

Exported rows can be filtered by whereFilters. Examples of filtering by userId column:

  • Single value filter: {"whereFilters": [{"column": "UserID", "values": ["123"]}]} equals to UserId = 123.
  • Multiple values: {"whereFilters": [{"column": "UserID", "values": ["123", "234"]}]} equals to UserId IN (123,234).
  • Non equals operator: {"whereFilters": [{"column": "UserID", "values": ["123"], "operator": "ne"}]} equals to UserId != 123.
  • Greater than with cast: {"whereFilters": [{"column": "Price", "values": ["40.3"], "operator": "gt", "dataType": "DOUBLE"}]} equals to try_to_double(Price) > 40.3.
  • Date/time values: {"whereFilters": [{"column": "created_at", "values": ["2023-01-01 00:00:00"], "operator": "ge", "type": "TIME_STRING"}]}; values can also be unix timestamps ("1672531200") or relative ("-2 days").
  • Multiple filter objects are joined by AND.

The comparison operator essentially behaves like the SQL IN (and NOT IN) operator: eq matches if any value matches, ne matches if none of the values match.

Rows sorting

Exported rows can be sorted by multiple columns via orderBy:

  • {"orderBy": [{"column": "UserId"}]} equals to ORDER BY UserId ASC.
  • {"orderBy": [{"column": "UserId", "order": "DESC"}]} equals to ORDER BY UserId DESC.
  • {"orderBy": [{"column": "Price", "order": "DESC", "dataType": "DOUBLE"}]} equals to ORDER BY try_to_double(Price) DESC.

Caching

Asynchronous exports support internal caching. When the export file for certain table export parameters was created and there were no changes in the table data since, the previous cached file will be returned.

Job results

Once the job is finished, the job result shows information about the created file and caching: {"file": {"id": 234}, "cacheHit": false}.

Initial response returns job details JobResponse. The result of the asynchronous job can be fetched via /v2/storage/jobs/{id}. When the job completes, final result is TableExportAsyncJobResponse.

Parameters of job will be TableExportJobParams.

post/v2/storage/tables/{tableId}/export-async

Request

Path parameters

tableIdstring required

Table Id.

Headers

contextstring nullable

Request body

changedSincestring nullable

Export only rows imported since this time. Timestamp of import is stored within each row. Accepts a unix timestamp or a strtotime() compatible string (e.g. -2 days).

changedUntilstring nullable

Export only rows imported until this time. Accepts a unix timestamp or a strtotime() compatible string.

daysinteger nullable

Deprecated (use changedSince instead). Accepted for backward compatibility and ignored.

whereColumnstring nullable

Deprecated (use whereFilters instead). Column for exporting only matching rows.

whereValuesstring[] nullable

Deprecated (use whereFilters instead). Values for exporting only matching rows.

whereOperator'eq' | 'ne' nullable

Deprecated (use whereFilters instead). Operator for value comparison.

limitinteger nullable

Limit the number of returned rows.

columnsstring nullable

Comma separated list of columns to export; by default all columns are exported.

format'rfc' | 'json' nullable

File format: rfc (CSV according to RFC 4180) or json.

sourceBranchIdinteger nullable

Branch id to load the source from. If not provided, the default branch is used. Honoured only for JSON requests.

gzipboolean

The exported file will be gzipped if set to true.

includeInternalTimestampboolean

Include the internal _timestamp column (timestamp of the last row change) in the exported data, even when not in the columns list. Snowflake only.

fileType'csv' | 'parquet'

Type of the file created in File Storage. Parquet is supported on Snowflake only. Parquet writes TIMESTAMP_LTZ and TIMESTAMP_TZ columns as UTC wall-clock TIMESTAMP_NTZ; the original offset is not kept.

timezonestring nullable

IANA timezone name (e.g. America/Los_Angeles, Europe/Prague). When set, the Snowflake session timezone is configured before export, affecting TIMESTAMP_LTZ column display. Not supported for Parquet exports. Snowflake only.

legacyTimestampFilterKindstring nullable

Example request

{
  "changedSince": "-2 days",
  "changedUntil": "-1 day",
  "limit": 1000,
  "columns": "id,name",
  "whereFilters": [
    {
      "column": "UserID",
      "operator": "eq",
      "values": [
        "123",
        "234"
      ]
    }
  ],
  "orderBy": [
    {
      "column": "Price",
      "order": "DESC",
      "dataType": "DOUBLE"
    }
  ],
  "sourceBranchId": 1234,
  "timezone": "Europe/Prague"
}

Response

Job accepted; returns job details.

idstring required

Job ID.

statusstring required

Current status of the job.

urlstring required

API detail URL for the job.

tableIdstring nullable required

ID of the table affected by the job.

operationNamestring required

Type of operation performed.

operationParamsobject required

Parameters for the operation.

createdTimestring date-time required

Job creation timestamp.

startTimestring date-time nullable required

Job start timestamp.

endTimestring date-time nullable required

Job end timestamp.

runIdstring nullable required

Run ID for the job.

Example response

{
  "id": "22077337",
  "status": "waiting",
  "url": "https://connection.keboola.com/v2/storage/jobs/22077337",
  "tableId": null,
  "operationName": "workspaceLoad",
  "operationParams": {
    "workspaceId": "78423",
    "preserve": false,
    "input": [
      {
        "source": "in.c-application-testing.cashier-data",
        "destination": "my-table"
      }
    ],
    "queue": "main_fast"
  },
  "createdTime": "2017-02-13T16:41:18+0100",
  "startTime": null,
  "endTime": null,
  "runId": null,
  "results": null,
  "creatorToken": {
    "id": "27978",
    "description": "john.doe@keboola.com"
  },
  "actor": {
    "type": "token",
    "id": "27978",
    "name": "john.doe@keboola.com",
    "adminId": 1234
  },
  "metrics": {
    "inCompressed": false,
    "inBytes": 0,
    "inBytesUncompressed": 0,
    "outCompressed": false,
    "outBytes": 0,
    "outBytesUncompressed": 0
  }
}

Changes

No recorded changes to this operation across all 1 revision of this API.