TRtrainloop
public-api

Public Create Eval Run

Changed on

Record a terminal eval run (with optional per-case results) for a task.

post/v1/evals/{slug}/runs

Request

  • Base URL
    https://api.trainloop.ai (the document declares no server; this is the origin it was published from)
  • Auth
    none declared

Path parameters

slugstring required

Headers

authorizationstring nullable

Request body

modelstring required

Model name snapshot, e.g. wilson-v3-step200

dataset_trainstring

Training dataset snapshot

dataset_evalstring

Eval dataset snapshot, e.g. apex-curated-eval-subset@2026-07-10

variantstring

Mutation/question variant within the task, e.g. v2-polar-positive

reference_labelstring

Mark this run as a reference instead of a scored datapoint, e.g. "random", "SOTA", "gpt-5 baseline". Reference runs are excluded from every score rollup and the group chart draws them as dotted horizontal lines captioned with this label. Empty (default) = an ordinary scored run.

input_templatestring

The {vars} question template that defines the variant

gradingstring

How the run was graded / answer logic

metricstring

Name of the headline metric overall_score normalizes

output_locationstring

Where the pipeline stored the raw outputs

overall_scorenumber required

Headline score in the group's own unit. It must be finite and at least 0, and it has no upper bound.

status'completed' | 'failed' required
metricsobject
started_atstring date-time nullable
finished_atstring date-time nullable
created_atstring date-time nullable

Optional original record time, preserved by export→replay syncs. The timeline falls back to it when finished_at is unset; omitted = now.

sourcestring

Which pipeline pushed this run

idempotency_keystring nullable

Optional stable id for this result. Re-posting the same key for the same task returns the run already recorded instead of adding a second one, so a retry after a lost response cannot duplicate a score. Omit it and every POST records a new run.

errorstring

Response

Successful Response

idstring uuid required
model_slugstring required
dataset_trainstring required
dataset_evalstring required
variantstring required
reference_labelstring required

Non-empty for reference runs, which are excluded from the score rollups

overall_scorenumber required
status'completed' | 'failed' required
errorstring required
sourcestring required

Which pipeline pushed the run. "rerun-task" is a staff rerun of the task (TRA-1375): listed, tagged, and excluded from every score like a reference run.

created_atstring required
started_atstring nullable required
finished_atstring nullable required

Changes