Sign in
Function Accuracy

Score Function Against (input, expected) Pairs

Score a function against a list of (input, expected) pairs.

Submits a batch of (input, expected) pairs, runs the named function over each input, and returns per-pair + aggregate accuracy metrics comparing the function's actual output to the provided expected JSON.

Scoring runs asynchronously. The response carries a scoreRunID; poll GET /v3/eval/score/{scoreRunID} until status is one of completed, error, or cancelled.

This request says only what to extract. How the output is compared against the expected value happens on the GET, recomputed from stored JSON each time.

post/v3/eval/score

Request body

functionNamestring required

Name of the function to score. Must be of type extract, transform, or analyze.

functionVersionNuminteger

Optional version number to score against. P0: only the function's current version is accepted; passing a different version returns 422.

datasetIDstring

A saved Golden Data Set (gds_…) to score against. Mutually exclusive with pairs; provide exactly one. Its input / corrected / schema columns are resolved by column role. When it carries a schema-role column, scoring types each row against that ground-truth schema instead of the function's own schema — so results hold up as functions/schemas evolve.

Response

The request has been accepted for processing, but processing has not yet completed.

scoreRunIDstring required

Run identifier. Use with GET /v3/eval/score/{scoreRunID}.

status'pending' | 'initializing' | 'running' | 'completed' | 'error' | 'cancelled' required

Status values for an eval-score run.

Changes