evals

Grade answers against the engine

Each case's inputs are run through the engine and the answer is compared key by key at the Concordance tolerances. Same key boundary as running the model.

post/api/v1/grade/{model}

Request

  • Base URL: https://worthune.com
  • URL: https://worthune.com/api/v1/grade/{model}
  • Auth: one of:
    • none
    • HTTP bearer
    • API key in header X-Api-Key

Request body

Response

Per-case verdicts and a summary.

okunknown required
modelstring required
specVersionstring required
gradedinteger required
passedinteger required
failedinteger required
invalidInputsinteger required
passRatenumber nullable required

passed / (passed + failed); null when nothing was comparable.

provenancestring required
disclaimerstring required

Changes

No recorded changes to this operation across all 1 revision of this API.