Sign in
Evals

Returns side-by-side stats for 2-10 eval runs over the same scenarios. Scenarios linked to every run are grouped and labeled with the dimensions that vary (e.g. model). Scenarios linked to only some runs are listed as uncomparable. Responds 400 when fewer than 2 or more than 10 distinct run IDs are given.

get/experimental/eval-run-comparisons

Query parameters

string uuid required
OR
string[] required

Eval-run UUIDs as a repeated query parameter (runs=a&runs=b). Must resolve to 2-10 distinct IDs.

Response

Default Response

Changes

Changed in 1 of the 29 revisions of this API.1