ADadaptionlabs
Dataset

Create a dataset from file upload, HuggingFace, Kaggle, Google Sheets, or a public S3 or Cloud Storage object

Changed on

Unified ingest endpoint. Pass source.url to start an async import — the provider is inferred from the URL host (HuggingFace, Kaggle, Google Sheets, or one public S3 or Cloud Storage object ending in .csv, .json, .jsonl, or .parquet). Omit it to get upload instructions for a presigned S3 PUT. Adapt the result with POST /datasets/{dataset_id}/adapt, or send it straight to training with POST /autoscientist when it already holds prompt and completion columns.

post/api/v1/datasets

Request

Request body

Example request

{
  "source": {
    "url": "https://huggingface.co/datasets/owner/name",
    "files": [
      "train.csv"
    ],
    "sheet_names": [
      "Sheet1",
      "Sales"
    ],
    "sheet_name": "Sheet1",
    "name": "sales-data.csv",
    "file_format": "csv",
    "processing_mode": "raw",
    "column_mapping": {
      "prompt": "question",
      "completion": "answer",
      "chosen": "chosen",
      "rejected": "rejected",
      "context": [
        "article_body"
      ]
    }
  }
}

Response

dataset_idstring required

ID of the newly created dataset

statusstring required

Current dataset status

Example response

{
  "upload_instructions": {
    "url": "https://s3.amazonaws.com/bucket/key?X-Amz-Signature=...",
    "method": "PUT"
  }
}

Changes

  • 7398ff1eeebaRevision changes
    • warning

      removed the request property

  • 8fb9be26396bRevision changes
    • info

      added the new optional request property //////////

    • info

      added the new optional request property //////////

    • info

      the request property ////////// became optional