Eval runs

Create several eval cases

Adds up to 100 cases to the suite in one call — the bulk form of the single create, and the way to import a suite or convert a repo's test files without a round trip per case. Every entry needs a title and a non-empty steps array. Cases are validated together and reported individually: a failed entry does not roll back its committed siblings, so the response is 201 even when some entries were refused — read failed rather than branching on the status alone. Send an Idempotency-Key header to make a retry land on the same rows.

post/projects/{projectId}/eval-suites/{suiteId}/cases/batch

Path parameters

projectIdstring required

ID of the hosted project that contains the server.

suiteIdstring required

Eval suite ID, as returned by POST /eval-runs.

Query parameters

declaredSuiteIdstring

The suite file's own suite.id, when this request is that file syncing itself. A CI-owned suite (see EvalSuiteDetail.managedBy) refuses configuration writes with 409 and details.reason: "CI_OWNED_SUITE_READ_ONLY"; naming the suite's own declared id is how the file writing itself is allowed through. Naming any other id refuses exactly as loudly as naming none, so it is not a capability — omit it for an ordinary edit.

A QUERY PARAMETER on every route that takes it, never a body field. These request bodies are strict, here and on every Inspector that predates the CI-owned lock, so a body field is a 400 against an older deployment — which would break mcpjam cloud eval run --file for anyone whose CLI is newer than their Inspector. A query parameter is read by the deployments that know it and ignored by those that do not, which is the right degradation: an Inspector with no lock has no exception to make.

Headers

Idempotency-Keystring

Makes the write retry-safe. A repeat of the same request under the same key lands on the rows the first attempt authored instead of creating a second copy; those entries come back with replayed: true.

Request body

declaredSuiteIdstring

The suite file's own suite.id, when THIS request is a suite-file sync.

A CI-owned suite (see EvalSuiteDetail.managedBy) refuses configuration writes with 409 and details.reason: "CI_OWNED_SUITE_READ_ONLY". Naming the suite's own declared id is how the file writing itself is allowed through — the platform permits the write only when the id matches, so naming any other id refuses exactly as loudly as naming none. Omit it for ordinary edits.

PREFER THE declaredSuiteId QUERY PARAMETER on this route. This body field still works here, but these request bodies are strict on every Inspector that predates the CI-owned lock, so a body field is a 400 against an older deployment while an unknown query parameter is simply ignored. The MCPJam SDK and CLI send the query parameter.

duplicatePolicy'block' | 'warn' | 'create_anyway'

What to do with a case whose definition already matches one in the suite. An unrecognized value coerces to block and reports the coercion in the response's duplicatePolicy rather than failing the call.

overrideReasonstring

Required by warn and create_anyway. Recorded on the case's revision.

Response

Per-case outcomes. Some entries may have failed; see failed.

Changes