Sign in
Function Accuracy

Create a model comparison

Compare several function versions on one dataset.

Scores a saved Golden Data Set against each entry's function version (each as an eval-score run) and reports per-entry accuracy, latency, and cost, plus lift of every entry against the baseline (the first entry). Entries may span different functions and different versions.

Runs asynchronously. The response carries a comparisonID; poll GET /v3/model-comparisons/{comparisonID} until status is complete.

post/v3/model-comparisons

Request body

namestring

Optional human-facing name.

datasetIDstring required

Saved Golden Data Set (gds_…) to score against. It must have an input-role column (the input files) and a corrected-role column (the expected JSON).

Response

The request has been accepted for processing, but processing has not yet completed.

comparisonIDstring required
status'pending' | 'running' | 'complete' | 'partial' | 'cancelled' | 'error' required

Rolled-up status of a comparison, derived from its entries' scoring runs.

Changes