Datasets

Validate a workflow or a dataset without running it

Two things can be validated, told apart by whether the body carries dataset_id.

Workflow (no dataset_id) — checks the operation graph against the operation manifest: unknown operations, missing required parameters, broken $ref wiring, structural rules.

Dataset (dataset_id present) — checks a dataset's data against its optional semantic-type schema. Pass source to check a file you have uploaded but not yet committed; omit it to report on the dataset as it stands. Reports unresolvable symbols (against the securities reference), unparseable dates, columns that differ from the stored data only by case, duplicate keys, out-of-range numbers and which dates are new — i.e. what will fire a dataset_trigger.

Advisory only. This never blocks a commit and changes nothing; ok: false is information. Both shapes answer the same issues array (severity, code, message, scope). Column meanings come from the semantic type registry — GET /schema returns it as semantic_manifest.

post/validate

Request

  • Base URL: https://api.indexone.io
  • URL: https://api.indexone.io/validate
  • Auth: API key in header x-api-key and API key in header Authorization

Request body

dataset_idstring

Validate this dataset. Omit to validate a workflow instead.

sourcestring

s3 uri/key of an uploaded file to check BEFORE committing it. Omit to check the dataset's current data.

mutation'append' | 'replace'

How source would be committed. append compares against the stored rows, so new dates and duplicates are visible; replace judges the file on its own.

schemaobject

Column meanings to check against; defaults to the dataset's own. Omit on a dataset with no schema to get a suggested_schema back.

resolveboolean

Confirm identifiers against the securities reference. false checks shape only and is much faster.

preview_rowsnumber

Rows of the incoming data to return with the report.

index_parametersobject

Workflow to validate, when no dataset_id is given.

Example request

{
  "dataset_id": "dst_R4mVnQ8xL27p",
  "source": "s3://prod-core-i1-datasets/dst_R4mVnQ8xL27p/fil_9c2/holdings.csv"
}

Response

Validation report.

okboolean

No errors were found. Never means "you may not commit".

typestring

dataset for a dataset report.

row_countnumber
stateobject

Aggregate flags: error / warning / info.

issuesobject[]

severity, code, message, scope, and for a dataset the column and example values.

columnsobject[]

Per column: dtype, declared semantic type, invalid counts and reference-resolution results.

datesobject

Parsed/unparsed counts, min, max, and new_dates — the ones that will fire a dataset trigger.

suggested_schemaobject

Returned when no schema was supplied. A suggestion to pre-fill a form, never applied on its own.

Changes

No recorded changes to this operation across all 1 revision of this API.