---
title: "Grade answers against the engine"
method: POST
path: "/api/v1/grade/{model}"
tags: ["evals"]
---

# Grade answers against the engine

`POST /api/v1/grade/{model}`

## Request

- Base URL: `https://worthune.com`
- URL: `https://worthune.com/api/v1/grade/{model}`
- Auth: one of:
  - none
  - HTTP bearer
  - API key in header `X-Api-Key`

Each case's inputs are run through the engine and the answer is compared key by key at the Concordance tolerances. Same key boundary as running the model.

## Request body

- object
  - `cases` object[], required
    - `inputs` object, required
    - `answer` object, required
  - `tolerance` GradeTolerance: Comparison tolerance; defaults to the Concordance harness tolerance (rel 1e-9, abs 0.000001). Never looser than 1e-2 relative.
    - `rel` number: Clamped to [1e-9, 1e-2].
    - `abs` number: Clamped to [0.000001, 1].

## Response `200`

Per-case verdicts and a summary.

- object
  - `ok` unknown, required
  - `model` string, required
  - `specVersion` string, required
  - `tolerance` object, required
    - `rel` number, required
    - `abs` number, required
  - `graded` integer, required
  - `passed` integer, required
  - `failed` integer, required
  - `invalidInputs` integer, required
  - `passRate` number, nullable, required: passed / (passed + failed); null when nothing was comparable.
  - `cases` GradedCase[], required
    - `index` integer, required
    - `verdict` 'pass' | 'fail' | 'invalid-inputs', required
    - `errors` FieldError[]: Present on invalid-inputs.
      - `field` string: Input name (or `name[i].field` for array inputs); absent for envelope-level errors.
      - `message` string, required
    - `diffs` object[]: Present on fail: the first diffs found.
      - `key` string, required
      - `expected` unknown, required
      - `got` unknown, required
      - `note` 'value-mismatch' | 'missing' | 'type-mismatch', required
    - `compared` integer: How many expected output keys were compared.
  - `provenance` string, required
  - `disclaimer` string, required

## Other responses

- `400`: Malformed JSON or a case of the wrong shape.
- `402`: A paid model (or its spec / dataset / grading) was called without a key, or with an unrecognized or revoked key. Free without a key: emergency-fund, relocation, side-business.
- `404`: Unknown model.
- `413`: More than 500 cases.
- `429`: Hourly per-org stateful quota, a per-route durable limit, or the 120 requests/minute per-IP burst backstop.

---

[API](https://skmtc.dev/worthune/apis/worthune-api.md) · [All operations](https://skmtc.dev/worthune/apis/worthune-api/llms.txt) · [OpenAPI document](https://skmtc.dev/worthune/apis/worthune-api/revisions/9f2241072555?raw)