test_suites

Create a test suite

Create simulation cases with caller behavior, assertions and action mocks for an agent.

Set spec.kind to simulation and define cases. Each case needs a lowercase-slug case_id, name, caller and assertions. Add dynamic variables, HTTP action mocks and an optional trial count.

Choose a caller with either {kind: "persona", goal, brief} or {kind: "script", steps: [{says}]}. Assertion IDs are your labels; node IDs come from drafts_get or publications_get.

Deterministic assertions include node_visited, node_not_visited, final_node, path, agent_says, tool_called, variable, ended, transfer and disposition. For agent_says, use expect: "never" | "at_least_once" and match: {patterns}. A rubric assertion uses a model judge.

Mocks answer customer HTTP actions by action_id. An unmocked customer HTTP action fails with unmocked_action before sending; built-in actions have simulated defaults. Run the suite with test_runs_create.

post/api/v1/test-suites

Request

  • Base URL: https://api.cumuluslabs.io
  • URL: https://api.cumuluslabs.io/api/v1/test-suites
  • Auth: HTTP bearer

Headers

Idempotency-Keystring required

Request body

agent_idstring required
namestring required

Response

Create a test suite on an agent (201; an idempotent replay returns 200)

owner_idstring required
environmentstring required
tenant_idstring uuid required
suite_idstring uuid required
agent_idstring required
namestring required
requiredboolean required
revisioninteger required
digeststring required
created_atstring date-time required
updated_atstring date-time required
archived_atstring date-time nullable required
flaky_case_idsstring[] required

Changes

No recorded changes to this operation across all 1 revision of this API.