PIpioneer
datasets

Get Upload Url

Changed on

Creates a reserved dataset row and presigned upload URL.

post/v1/datasets/upload/url

Request

Request body

dataset_namestring required

Name for the dataset

dataset_type'classification' | 'ner' | 'custom' | 'decoder'

Type of dataset

format'jsonl' | 'csv' | 'tsv' | 'parquet' | 'json' nullable

File format (auto-detected from filename if not provided)

filenamestring nullable

Original filename (used for format detection if format not provided)

schemastring nullable

Expected schema as JSON object (e.g., {"age": "Int64", "name": "Utf8"}). Enforces column types during parsing. Valid types: Int8, Int16, Int32, Int64, UInt8-64, Float32, Float64, Utf8, Boolean, Date, Datetime, Time, Duration, Categorical, Binary

project_idstring nullable

Optional project ID (UUID) to assign this dataset to

experiment_idstring nullable

Optional Experiment ID (UUID) the upload was started from. The dataset is linked to it so it appears in that Experiment's Datasets tab (ENG-7109). Ignored when the caller may not access the Experiment.

type'training' | 'evaluation' | 'benchmark' nullable

Dataset purpose: 'training' (trainable), 'evaluation' (not trainable), 'benchmark' (system-managed, evaluation-only; cannot be trained on or directly accessed).

visibilitystring nullable

Dataset visibility: 'private' (owner only) or 'public' (anyone can see)

generation_type'upload' nullable

Public uploads are always recorded as upload. Other generation methods are assigned by server-owned workflows.

split_ratioobject nullable

Split ratio when type is 'split', e.g. {'training': 0.8, 'evaluation': 0.2}

column_mappingobject nullable

Column mapping from source to standard names (e.g., {"sentence": "text", "category": "label"}). Valid standard targets: text, label, labels, entities.

Response

Successful Response

presigned_urlstring required

S3 presigned URL for PUT upload

dataset_idstring required

Dataset ID for subsequent processing

dataset_namestring required

Dataset name (for polling status via GET /{name}/{version})

version_numberstring required

Version number for this dataset

expires_ininteger required

URL expiration time in seconds

upload_instructionsstring

Instructions for completing the upload

Changes