build dataset

Create a dataset: from rows you have, or from a prompt Riveter finds rows for. What a dataset is: Datasets.

Start a dataset from your data

This is the preferred way to enrich data you already hold. The request is the rows plus the columns to add: the enrichment_id of an enrichment you already have — from a dataset in the app (the dataset page's API button shows it), from GET /datasets/{id}, or from GET /enrichments; or save a new one with POST /enrichments — or, with no enrichment yet, attributes, the output column names: Riveter creates the dataset's enrichment from them (its columns typed from their names; set a column's prompt, format, or tool with PATCH /datasets/{id}). Rows are objects keyed by input column name (an enrichment's input columns, GET /enrichments/{id} lists them; with attributes, input_columns or the keys of the rows), up to 100 per request; add more with POST /datasets/{id}/rows or a CSV on POST /datasets/{id}/rows/import.

curl -X POST "https://api.riveterhq.com/v1/datasets" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Q4 targets",
    "enrichment_id": "enr_YOUR_ENRICHMENT_ID",
    "rows": [
      { "Company": "Apple", "Website": "apple.com" },
      { "Company": "Shopify", "Website": "shopify.com" }
    ]
  }'

The response is the dataset: id (ds_...), input_columns, enrichment_id, output_columns, row_count. Nothing has run yet. Fill the cells:

curl -X POST "https://api.riveterhq.com/v1/enrich" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "dataset_id": "ds_YOUR_DATASET_ID" }'

This fills every empty cell and returns a run (id: run_...). Add "estimate_cost": true to see the price without running anything, or "max_credits": 50 to refuse the run above a ceiling. Then read the results:

curl "https://api.riveterhq.com/v1/runs/RUN_ID/result?wait=50" \
  -H "Authorization: Bearer YOUR_API_KEY"

wait holds the request for up to 50 seconds. output is null until the run finishes, so call again until it is not; or pass webhook_url on the fill and skip polling. output.rows is the rows the fill ran, each with its input, its cells by output column name (value, state, sources), and the credits it cost. GET /datasets/{id}/rows is the whole grid at any time, filled cells included.

Large datasets

output.rows holds up to 5,000 rows. Above that, output is null and the run's output_files are the result: output_files.jsonl and output_files.csv are the whole grid as files on object storage, one row per line. Each has a state (pending until the export lands, shortly after the run finishes; then ready), a signed url, and expires_at (one hour; fetch the result again for a fresh link).

curl -o rows.jsonl "$(curl -s "https://api.riveterhq.com/v1/runs/RUN_ID/result" \
  -H "Authorization: Bearer YOUR_API_KEY" | jq -r '.output_files.jsonl.url')"

Two more ways to read a large grid: page GET /datasets/{id}/rows with limit (up to 1,000) and cursor, or updated_since for only the rows changed since a time; or download the grid at any time with GET /datasets/{id}/export (output_format jsonl, csv, or xlsx).

Build a list from a prompt

No rows yet? Describe them. Riveter's list generator finds count rows for the prompt; input_columns names the identity columns to generate. The output columns come from attributes (an enrichment is created for the dataset from the prompt and the column names) or from an enrichment_id. auto_fill: true fills the cells as soon as the rows land. Generation and the fill are both paid runs; estimate_cost: true prices them first.

Or send prompt and count alone and the whole dataset is set up from the prompt: the identity columns, and one output column per fact the prompt asks to find about each row, each with its own prompt and type ("Top 50 US fintech startups with their CEO, total funding, and founding year" → rows under Company, columns CEO, Total funding, Founding year). A prompt about rows alone gets no output columns. Name input_columns yourself and the output columns come from attributes or enrichment_id only, as above.

curl -X POST "https://api.riveterhq.com/v1/datasets" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "SF coffee roasters",
    "prompt": "Specialty coffee roasters in San Francisco",
    "count": 50,
    "input_columns": ["Roaster", "Website"],
    "attributes": ["Founded year", "Number of cafes"],
    "auto_fill": true
  }'

The response is the dataset plus run, the generation run; poll it like any run. The rows land in the dataset when generation finishes, and with auto_fill the fill run starts then: it appears under the dataset's runs on GET /datasets/{id}, and fill_webhook_url is POSTed when it finishes. Read the grid as under Large datasets above. For more rows later, POST /datasets/{id}/rows/build.