Sign in
Eval runs

Author eval cases from a document

The sibling of cases/generate: generation invents cases from the suite's tools, import reads them out of something a person already wrote — a test plan, a spreadsheet of scenarios, a JSON export. MCPJam's own model turns the document into full cases (ordered steps, tool calls, assertions), so the caller does not have to guess the case shape. No format is declared: the model reads the document's shape itself, so Markdown, JSON, CSV and plain notes all work.

Spends the organization's credits, unlike generation, which is on MCPJam. Pass an idempotency key so a retry replays the same job instead of authoring — and billing — the document twice.

Like cases/generate, this discovers the suite's tools over a live MCP connection first, so the authored cases are grounded in tools the suite can actually call.

Answers 200 when the job finishes inside the wait window and 202 with a jobId when it does not — poll the job, then commit it. A case the commit could not write comes back under skipped, with a reviewUrl that opens the app on exactly those drafts.

post/projects/{projectId}/eval-suites/{suiteId}/cases/import

Path parameters

projectIdstring required

ID of the hosted project that contains the server.

suiteIdstring required

Eval suite ID, as returned by POST /eval-runs.

Headers

x-mcpjam-idempotency-keystring

Makes a retry replay the recorded job instead of re-authoring — and re-billing — the document.

Request body

contentstring required

The document, as UTF-8 text. Markdown, JSON, CSV or plain notes — no format is declared, because the authoring model reads the document's shape itself. At most 100 KiB.

fileNamestring

Recorded on each case's provenance so a reviewer can find the source. Optional: a pasted document has no file behind it.

serversstring[]

Server ids or names to discover tools from. Mutually exclusive with environmentId.

environmentIdstring

Author against this environment's closed server set. Mutually exclusive with servers.

duplicatePolicy'block' | 'warn' | 'create_anyway'

Applied when the cases are written, not when the job starts. Anything but block requires overrideReason.

overrideReasonstring
idempotencyKeystring

Makes a retry replay the recorded job instead of authoring — and billing — the document a second time.

Response

The cases the job authored and committed, or the job's own status when it finished without committing (failed, cancelled, or still pending at the end of the wait).

OR

Changes