FAfastino-ai
datasets

Get Upload Url

Changed on

Creates a reserved dataset row and presigned upload URL.

post/v1/datasets/upload/url

Request

Request body

column_mappingobject nullable

Column mapping from source to standard names (e.g., {"sentence": "text", "category": "label"}). Valid standard targets: text, label, labels, entities.

content_lengthinteger nullable

Exact size in bytes of the file you will PUT, at most 5 GiB (one S3 PUT). When set, presigned_url is signed for this Content-Length, and storage refuses a body of any other size.

dataset_namestring required

Name for the dataset, 1 to 128 characters. It must not be blank, contain '/', '', '<', '>' or control characters, or be '.' or '..': such a name cannot be fetched or deleted by name afterwards.

dataset_type'classification' | 'ner' | 'custom' | 'decoder'

Type of dataset

experiment_idstring nullable

Optional Experiment ID (UUID) the upload was started from. The dataset is linked to the experiment when the caller has access.

filenamestring nullable

Original filename (used for format detection if format not provided)

format'jsonl' | 'csv' | 'tsv' | 'parquet' | 'json' nullable

File format (auto-detected from filename if not provided)

generation_type'upload' nullable

Public uploads are always recorded as upload. Other generation methods are assigned by server-owned workflows.

project_idstring nullable

Optional project ID (UUID) to assign this dataset to

schemastring nullable

Expected schema as JSON object (e.g., {"age": "Int64", "name": "Utf8"}). Enforces column types during parsing. Valid types: Int8, Int16, Int32, Int64, UInt8-64, Float32, Float64, Utf8, Boolean, Date, Datetime, Time, Duration, Categorical, Binary

sha256string nullable

Hex sha256 of the file you will PUT. When set, presigned_url is signed for its x-amz-checksum-sha256, and storage refuses bytes with another digest.

split_ratioobject nullable

Split ratio when type is 'split', e.g. {'training': 0.8, 'evaluation': 0.2}

type'training' | 'evaluation' | 'benchmark' nullable

Dataset purpose: 'training' (trainable), 'evaluation' (not trainable), 'benchmark' (system-managed, evaluation-only; cannot be trained on or directly accessed).

visibility'private' | 'public' nullable

Dataset visibility: 'private' (owner only) or 'public' (anyone can see)

Response

Successful Response

dataset_idstring required

Dataset ID for subsequent processing

dataset_namestring required

Dataset name (for polling status via GET /{name}/{version})

expires_ininteger required

URL expiration time in seconds

presigned_urlstring required

S3 presigned URL for PUT upload

upload_headersobject required

Headers the PUT to presigned_url must send, with exactly these values. They are part of the URL's signature, so storage answers 403 when one is missing or different. Always holds Content-Type: application/octet-stream; also Content-Length and x-amz-checksum-sha256 when the request declared content_length and sha256. Most HTTP clients set Content-Length from the body.

upload_instructionsstring

Instructions for completing the upload

version_numberstring required

Version number for this dataset

Changes

  • 81465e18c1d2Revision changes
    • breaking

      request property / was restricted to a list of enum values

    • warning

      the request property's maxLength was set to 128

    • info

      added the new private enum value to the request property /

    • info

      added the new public enum value to the request property /

  • ef285860f952Revision changes
    • info

      added the new optional request property

    • info

      added the new optional request property

    • info

      the response's property default value changed from Upload file via HTTP PUT to presigned_url then call /datasets/upload/process with dataset_id format and schema (if applicable) to PUT the raw file bytes to presigned_url and send every header in upload_headers with exactly its value: always Content-Type: application/octet-stream plus Content-Length and x-amz-checksum-sha256 when listed. A missing or different header is refused with 403. Then call POST /datasets/upload/process with dataset_id. for the status

    • info

      added the required property to the response with the status