Compare a run against a baseline
Compare this run against a baseline run: per-case status (regressed, fixed, new_case, removed_case, changed), per-scorer pass-rate and mean deltas from the evaluation contract, and whether the evaluation config changed. Omit baseRunId to compare against the nearest earlier completed run in the same suite. Returns 404 with details.reason = BASELINE_NOT_FOUND when there is no comparable predecessor — that is an incomplete comparison, not a failing one. A scorer whose definitionChanged is true was graded by a different definition on each side, so its delta is not a regression.
Path parameters
ID of the hosted project that contains the server.
Eval run ID, as returned by POST /eval-runs.
Query parameters
Run ID to compare against. Omit to use the nearest earlier completed run in the same suite. Mutually exclusive with baseCommitSha.
Source commit SHA to compare against, resolved to the completed run in this suite recorded against it. Mutually exclusive with baseRunId; sending both is a 400. A SHA matching no completed run is the ordinary 404 with details.reason = BASELINE_NOT_FOUND - an incomplete comparison, not a regression.
Response
The comparison.