CEcekura
test_framework

Validate a JSON test-suite spec without running it

Changed on

Check a spec against the agent it targets: metric slugs, personalities, test profiles, channel support and every tag. Returns the plan a run would follow, with case count and estimated cost. Creates nothing and places no calls. Use this before committing a suite or opening a pull request; use scenarios_run_json to execute one.

post/test_framework/v1/scenarios/validate_scenarios_json/

Request

  • Base URL
    https://api.cekura.ai
  • URL
    https://api.cekura.ai/test_framework/v1/scenarios/validate_scenarios_json/
  • Auth
    • API key in header X-CEKURA-API-KEY
    • or HTTP bearer
    • or HTTP bearer

Request body

specunknown
filestring uri

A .json spec file. Use this instead of 'spec' for uploads.

agent_idinteger

Agent to test. Overrides 'agent_id' in the spec, letting one spec file run against several environments.

namestring

Label for this run. Defaults to the spec's suite name.

channel'voice' | 'text' | 'elevenlabs' | 'livekit_v2' | 'pipecat_v2' | 'chirp'

How to reach the agent: 'voice' for a phone call (the default), 'text' for chat, 'elevenlabs' for an ElevenLabs WebRTC session, 'livekit_v2' or 'pipecat_v2' for their respective WebRTC sessions, or 'chirp' for the agent's raw-PCM WebSocket endpoint (CHIRP protocol).

  • voice - voice
  • text - text
  • elevenlabs - elevenlabs
  • livekit_v2 - livekit_v2
  • pipecat_v2 - pipecat_v2
  • chirp - chirp
frequencyinteger

Run every test case this many times, on top of each case's own 'frequency' in the spec. Default 1.

agent_numberstring

Voice and text only. The agent's phone number for this run, in E.164 format, overriding the agent's configured contact number.

outbound_phone_numberstring

Voice and text only. For an inbound voice agent, the number Cekura dials, in E.164 format; defaults to 'agent_number', then the agent's contact number. Rejected for outbound voice agents: set 'phone_number' per test case in the spec instead. For SMS and WhatsApp agents, the contact number for the conversation.

mode'same_number' | 'different_numbers'

Voice, inbound agents only. 'same_number' (default) places every call from the same Cekura number; 'different_numbers' spreads them across the pool and cannot be combined with 'phone_number' in the spec.

  • same_number - same_number
  • different_numbers - different_numbers
concurrency_limitinteger

Cap on the number of runs executed in parallel. Overrides the spec's 'defaults.concurrency_limit'.

calls_per_secondnumber double

Most calls started per second for this run, a number from 0.1 to 4 (e.g. 0.5 starts one call every 2 seconds). Overrides the spec's 'defaults.calls_per_second'. Leave both unset for no limit.

mock_tool_namesstring[] nullable

Names of mock tools to activate for this run. Omit (or send null) to mock every tool configured on the agent; send an empty list to mock none.

livekit_dataobject

Optional LiveKit overrides. These apply only to this run and are not part of the portable test-suite spec.

pipecat_dataobject

Optional Pipecat Cloud overrides. These apply only to this run and are not part of the portable test-suite spec.

chirp_dataobject

Optional CHIRP overrides. 'websocket_url' replaces the agent's stored raw-PCM WebSocket endpoint for this run only — for example a tunnel URL that changes on every CI run. Not part of the portable test-suite spec.

Response

{dry_run, valid, plan}

Changes