Sign in
Eval runs

Create several eval cases

Adds up to 100 cases to the suite in one call — the bulk form of the single create, and the way to import a suite or convert a repo's test files without a round trip per case. Every entry needs a title and a non-empty steps array. Cases are validated together and reported individually: a failed entry does not roll back its committed siblings, so the response is 201 even when some entries were refused — read failed rather than branching on the status alone. Send an Idempotency-Key header to make a retry land on the same rows.

post/projects/{projectId}/eval-suites/{suiteId}/cases/batch

Path parameters

projectIdstring required

ID of the hosted project that contains the server.

suiteIdstring required

Eval suite ID, as returned by POST /eval-runs.

Headers

Idempotency-Keystring

Makes the write retry-safe. A repeat of the same request under the same key lands on the rows the first attempt authored instead of creating a second copy; those entries come back with replayed: true.

Request body

duplicatePolicy'block' | 'warn' | 'create_anyway'

What to do with a case whose definition already matches one in the suite. An unrecognized value coerces to block and reports the coercion in the response's duplicatePolicy rather than failing the call.

overrideReasonstring

Required by warn and create_anyway. Recorded on the case's revision.

Response

Per-case outcomes. Some entries may have failed; see failed.

Changes

Changed in 4 of the 80 revisions of this API.15

  • d9a00becd4761See the full diff
    • ○

      added the new optional request property cases/items/intent

      new-optional-request-property

  • e81e8ba8baae11See the full diff
    • ●

      added the new INVALID_IMPORT enum value to the failed/items/code response property for the response status 201

      response-property-enum-value-added

    • ○

      added the new optional request property cases/items/import

      new-optional-request-property

  • c479eeceb7a32See the full diff
    • ○

      added the new optional request property cases/items/passThreshold

      new-optional-request-property

    • ○

      added the new optional request property cases/items/repetitions

      new-optional-request-property

  • 879ca349c8041See the full diff
    • ○

      endpoint added

      endpoint-added