Sign in
public-api

Public Create Eval Run

Record a terminal eval run (with optional per-case results) for a task.

post/v1/evals/{slug}/runs

Path parameters

slugstring required

Headers

authorizationstring nullable

Request body

modelstring required

Model name snapshot, e.g. wilson-v3-step200

dataset_trainstring

Training dataset snapshot

dataset_evalstring

Eval dataset snapshot, e.g. apex-curated-eval-subset@2026-07-10

variantstring

Mutation/question variant within the task, e.g. v2-polar-positive

reference_labelstring

Mark this run as a reference instead of a scored datapoint, e.g. "random", "SOTA", "gpt-5 baseline". Reference runs are excluded from every score rollup and the group chart draws them as dotted horizontal lines captioned with this label. Empty (default) = an ordinary scored run.

input_templatestring

The {vars} question template that defines the variant

gradingstring

How the run was graded / answer logic

metricstring

Name of the headline metric overall_score normalizes

output_locationstring

Where the pipeline stored the raw outputs

overall_scorenumber required
status'completed' | 'failed' required
metricsobject
started_atstring date-time nullable
finished_atstring date-time nullable
created_atstring date-time nullable

Optional original record time, preserved by export→replay syncs. The timeline falls back to it when finished_at is unset; omitted = now.

sourcestring

Which pipeline pushed this run

errorstring

Response

Successful Response

idstring uuid required
model_slugstring required
dataset_trainstring required
dataset_evalstring required
variantstring required
reference_labelstring required

Non-empty for reference runs, which are excluded from the score rollups

overall_scorenumber required
status'completed' | 'failed' required
errorstring required
sourcestring required
created_atstring required
started_atstring nullable required
finished_atstring nullable required

Changes