--- title: "Get one eval case" method: GET path: "/projects/{projectId}/eval-suites/{suiteId}/cases/{caseId}" tags: ["Eval runs"] --- # Get one eval case `GET /projects/{projectId}/eval-suites/{suiteId}/cases/{caseId}` A case that belongs to this suite and project. A case in another suite is a 404, not a 403. ## Path parameters - `projectId` string, required - `suiteId` string, required - `caseId` string, required ## Response `200` The case. - EvalCase — A persisted eval case, in the public steps-first shape. Note this is NOT `EvalTestCase`, which is the INLINE authoring shape accepted by suite creation. - `id` string, required - `declaredId` string — The case's effective declared id. Absent on cases authored before declared identity existed. - `title` string, required - `steps` EvalTestStep[], required — Ordered test steps. A `prompt` step is a model turn; a single model-free `toolCall` step is a render-check; `assert` steps hold the expectations. - `id` string, required - `kind` 'prompt' | 'toolCall' | 'interact' | 'assert', required - `prompt` string — User message (`kind: prompt`). - `serverName` string — Server that owns the tool (`kind: toolCall`). - `toolName` string — Tool name (`kind: toolCall` / `interact`). - `arguments` object — Tool-call arguments (`kind: toolCall`). - `action` object — Widget action (`kind: interact`). - `assertion` object — The rule an `assert` step checks (`kind: assert`). NOT a narrower spelling of a case's `checks`: this is `WidgetAssertion | Predicate`, and that union is why the field is called an assertion rather than a check. A `Predicate` is evaluated against the PERSISTED transcript, so a stored run can be re-graded against it months later; a `WidgetAssertion` is evaluated against a LIVE DOM and can never be replayed. Calling this a check would promise the replayability only one half of it has. - `expectedOutput` string - `iterations` integer, required - `repetitions` integer — Trials this case runs under verdict policy 2, overriding the suite default. Absent means the case inherits it. NOT a second spelling of `iterations`: that one is the legacy count, which the legacy resolver reads as a FLOOR (`max(iterations, suite.minimumIterations)`) and which a policy-2 case still reports for compatibility. This one is exact. - `passThreshold` number — Fraction of this case's trials that must pass, overriding the suite default. Absent means the case inherits it. Never derived from the suite's `minimumAccuracy`, which is a PERCENT under a different resolver. - `isNegative` boolean, required — When true, the case passes if NO tools are called. - `scenario` string - `intent` string — Optional authored analytics grouping label. Must be already trimmed; absent means unlabelled. - `kind` 'capability' | 'regression' — Authored case kind for the simple editor. Absent means the editor derives it from matchOptions. - `models` object[], required - `model` string, required - `provider` string - `matchOptions` object — Absent when the case sets none — omitted from the response rather than sent as `null`. - `checks` object — Absent when the case sets none — omitted from the response rather than sent as `null`. - `mode` 'inherit' | 'replace' | 'extend' - `list` EvalAssertion[] - union — An SDK assertion reported through UVC. Discovery assertions require complete raw tool declarations; missing evidence is an evaluator error. Observation kinds, including noDeprecatedToolExposed, require role advisory. Structural schema; runtime also validates policy and unique required annotation keys. - object - `type` 'toolDescriptionsPresent', required - `minLength` integer - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'toolAnnotationsPresent', required - `require` string[] - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'toolNamesUnique', required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'noDeprecatedToolExposed', required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'toolInputSchemasWellFormed', required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'toolOutputSchemasPresent', required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'toolCalledWith', required - `toolName` string, required - `args` object, required - `args` object, required - `argumentMatching` 'exact' | 'partial' | 'ignore' - `minCount` integer - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'toolCalledAtLeastOnce', required - `toolName` string, required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'toolNeverCalled', required - `toolName` string, required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'onlyToolsCalled', required - `toolNames` string[], required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'firstToolWas', required - `toolName` string, required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'responseContains', required - `needle` string, required - `caseSensitive` boolean - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'responseMatches', required - `pattern` string, required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'noToolErrors', required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'finalAssistantMessageNonEmpty', required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'tokenBudgetUnder', required - `tokens` integer, required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'widgetRendered', required - `toolName` string - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'widgetRenderLatencyUnder', required - `ms` integer, required - `toolName` string - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'widgetNoConsoleErrors', required - `toolName` string - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'turnCountUnder', required - `turns` integer, required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'noEndingQuestion', required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'toolLatencyUnder', required - `ms` integer, required - `toolName` string - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'toolResultContains', required - `needle` string, required - `caseSensitive` boolean - `toolName` string - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'toolResultMatchesSchema', required - `schema` unknown, required - `toolName` string - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'toolResultSizeUnder', required - `maxBytes` integer, required - `toolName` string - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'argumentsMatchToolSchema', required - `toolName` string - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'noRepeatedIdenticalCall', required - `toolName` string - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'toolCallCountUnder', required - `count` integer, required - `toolName` string - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'toolCalledBefore', required - `toolName` string, required - `beforeToolName` string, required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'noDeprecatedToolCalled', required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'noDestructiveToolCalled', required - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'toolErrorNamesInput', required - `toolName` string - `role` 'gating' | 'advisory' - `severity` 'warn' - object - `type` 'fullPageHasContinuation', required - `toolName` string - `role` 'gating' | 'advisory' - `severity` 'warn' - `import` EvalCaseImportClaim — What a converter CLAIMED about one imported case. `exact` is CONVERTER-CLAIMED exact — the converter says it applied a structural mapping rule, cited in `note`. MCPJam has NOT verified semantic equivalence, so user-facing copy must say "claimed exact", never "verified" or "accepted". Claim-only: who approved an approximation, when, and why is a PER-RUN decision frozen on the run (`ImportEligibility.approvedApproximationReceipts`), never stored on the case — an approval that lived on a case would outlive the run it was granted for and the edit that invalidated it. Approval and internal keys are rejected with 400, never stripped. - `status` 'exact' | 'approximated' | 'unsupported' | 'unresolved', required — `exact`: the converter claims a 1:1 structural mapping, and must cite it in `note`. `approximated`: behaviour was intentionally approximated; a human must approve it for EVERY run. `unsupported`: the source behaviour cannot currently be represented. `unresolved`: a deterministic reference does not resolve against the live target. A selected `unsupported` or `unresolved` case cannot run. - `sourceCaseKey` string — The case's identity in the source system, when it had one. - `note` string — Why the status is what it is — the mapping rule cited, or what was lost. REQUIRED when `status` is `exact`. - `source` EvalCaseSource — Where an AI-assisted Markdown case was authored from. Provenance only: the file, its hash, and the line range the extractor read, so a reader can trace a case back to its source document. It is not an import claim (see `EvalCaseImportClaim`) and carries no approval semantics. - `format` 'markdown', required - `method` 'ai', required — How the case was derived from the document. `ai`: an extractor model proposed it and a person reviewed it before it was saved. - `fileName` string, required - `fileHash` string, required — SHA-256 of the source document, lowercase hex. - `excerpt` string, required — The passage the case was extracted from. - `startLine` integer, required - `endLine` integer, required — Inclusive; never less than `startLine`. - `extractorVersion` string, required - `createdAt` number, nullable - `updatedAt` number, nullable - `suppressedSuiteStandardCheckIds` string[] — Stable standard assertion family IDs suppressed from suite defaults. Applies before inherit/extend/replace resolution; explicit case and step assertions are preserved. Omitted updates preserve; [] clears. At most 64 IDs, normalized to unique sorted values. ## Other responses - `401` — Missing, invalid, revoked, or orphaned key (`UNAUTHORIZED`) — or the **target MCP server** needs an OAuth grant (`OAUTH_REQUIRED`), which is a property of the server, not your key. - `403` — Key is valid but not allowed to do this. - `404` — Unknown project, server, or resource. - `429` — Per-key rate limit exceeded (60 requests/minute sustained, bursts up to 10). Honor `Retry-After` and back off with jitter. - `500` — Something failed on MCPJam's side. - `502` — Could not connect to the target MCP server. ## Changes - **2026-09-13** `16cc71fe1f33` — 2 breaking, 1 info - added `subschema #1, subschema #2, subschema #3, subschema #4, subschema #5, subschema #6, subschema #7, subschema #8, subschema #9, subschema #10, subschema #11, subschema #12, subschema #13, subschema #14, subschema #15, subschema #16, subschema #17, subschema #18, subschema #19, subschema #20, subschema #21, subschema #22, subschema #23, subschema #24, subschema #25, subschema #26, subschema #27, subschema #28, subschema #29, subschema #30, subschema #31, subschema #32, subschema #33` to the `checks/list/items/` response property `oneOf` list for the response status `200` - the `checks/list/items/` response's property type changed from `object` to no type for status `200` - added the optional property `suppressedSuiteStandardCheckIds` to the response with the `200` status - **2026-09-10** `cec7381898e2` — 1 info - added the optional property `source` to the response with the `200` status - **2026-09-04** `7584cc19e5ff` — 1 info - added the optional property `kind` to the response with the `200` status - **2026-08-27** `d9a00becd476` — 1 info - added the optional property `intent` to the response with the `200` status - **2026-08-26** `e81e8ba8baae` — 1 info - added the optional property `import` to the response with the `200` status [Full history](https://skmtc.dev/mcpjam/apis/mcpjam-api/changes/projects/:projectId/eval-suites/:suiteId/cases/:caseId/get.md) --- [API](https://skmtc.dev/mcpjam/apis/mcpjam-api.md) · [All operations](https://skmtc.dev/mcpjam/apis/mcpjam-api/llms.txt) · [OpenAPI document](https://skmtc.dev/mcpjam/apis/mcpjam-api/revisions/2349c6b8a3b8?raw)