Eval runs

Close an open ingestion run

Marks the run complete and computes its rollups. Until this is called the run stays open in the dashboard.

How eval runs executed OUTSIDE the platform (local dev, CI) reach the Evals dashboard. Authenticate like any other /api/v1 route (typically an sk_ key); the gateway swaps in a delegated org-scoped token so the backend's fail-closed org scoping applies.

The {projectId} segment declares where results land and always wins over any projectId in the body. The literal default resolves to the key org's Default project — the zero-config CI case.

STATUS AND BODY PASS THROUGH VERBATIM: success shapes are the legacy { ok: true, ... } envelopes the SDK reporter parses, not the v1 resource envelope.

post/projects/{projectId}/eval-ingest/runs/finalize

Path parameters

projectIdstring required

Project id, or the literal default for the key org's Default project.

Headers

x-mcpjam-eval-vocabulary'1' | '2'

Which vocabulary this request and its response speak. Absent means 1, which is byte-for-byte today's contract: the same request fields, the same refusals, the same response projection. 2 is the canonical vocabulary. Any other value is a 400 with code: "VALIDATION_ERROR".

Today it decides one thing: the spelling of an evaluator's policy role. Vocabulary 1 accepts and returns gating; vocabulary 2 accepts both spellings and returns the canonical required. Sending required without the header is a 400, deliberately — vocabulary 1 is not widened to meet vocabulary 2 half way, because a boundary that accepts a spelling it does not announce is one two implementations can disagree about.

A response that varies by vocabulary sends Vary: x-mcpjam-eval-vocabulary.

Request body

EvalIngestRequest required

Forwarded VERBATIM to the backend's eval-ingestion surface, which owns the schema. The projectId path segment always wins: it overwrites any projectId in the body, and the literal default omits it so the key's org Default project is used. Bodies are capped at ~6 MiB here and 5 MiB by the backend.

Response

The backend's response, passed through.

EvalIngestResponse required

The backend's response, passed through verbatim along with its status code. Success shapes are the legacy { ok: true, ... } envelopes the SDK reporter parses — deliberately NOT the v1 resource envelope. Errors are canonical v1 { code, message }.

Changes