Grade an eval run with LLM as Judge

Spends. Runs the goal-completion judge over the finished run, scoring each case's final answer against its expected output.

202: scheduled, not done. Read the grades from the run detail's judges.goalCompletion rather than re-requesting — a second POST only spends again.

A run's grading config is pinned when the run is created, so turning the judge on for the suite does not reach an already-recorded run: enable: true is what grades one, and it changes nothing beyond that run. Omitting model and threshold clears any override a previous request left on the run.

post/projects/{projectId}/eval-runs/{runId}/judge

Request body

forceboolean

Re-grade a run that already has a result. SPENDS AGAIN. Must be a real boolean — "false" is rejected rather than read as consent.

enableboolean

Grade this run even though the judge was off when it ran. A per-RUN answer, not a suite edit: grading reads the config pinned when the run was created, so enabling the judge on the suite does not reach an already-recorded run.

modelstring

Judge model for this run only.

thresholdnumber

Pass threshold for this run only.

Response

Scheduled.

runIdstring required

projectIdstring required

status'pending' required

Changes

Changed in 3 of the 122 revisions of this API.3

Sep 14, 202656b9d1dda7ea1See the full diff
- ○
  added the new optional header request parameter x-mcpjam-eval-vocabulary to all path's operations
  new-optional-request-default-parameter-to-existing-path
Sep 12, 2026de57cfcdd8241See the full diff
- ○
  added the non-success response with the status
  response-non-success-status-added
Aug 19, 2026fac4c14468c71See the full diff
- ○
  endpoint added
  endpoint-added

Full history