Evaluators

Run evaluators against a Pipecat WebRTC deployment (manual room URLs, v1)

Changed on

PIPECAT WEBRTC RUN (manual room URLs, v1): Execute evaluators against an agent running on Pipecat / Daily.co WebRTC. Each scenario entry carries its own pipecat_room_url and optional token. For project-credential-based runs (no per-scenario URL), use scenarios_run_pipecat_v2 instead.

Run-count: one run per entry in scenarios — this endpoint does not support frequency or personality/test-profile multipliers. To run the same scenario multiple times, include it multiple times in the scenarios list.

When to use this vs others (pick the endpoint matching the agent's configured connection — check aiagents_retrieve):

  • run_scenarios — real voice / phone / SIP call
  • run_scenarios_text — text / SMS / chat mode
  • run_scenarios_with_websockets — agent exposes a plain websocket URL (non-Pipecat)

Scheduled runs: WebRTC sessions are not generally scheduled. Use run_scenarios or run_scenarios_text via CronJob for recurring evaluator runs.

post/test_framework/v2/scenarios/run_scenarios_pipecat/

Request

  • Base URL
    https://api.cekura.ai
  • URL
    https://api.cekura.ai/test_framework/v2/scenarios/run_scenarios_pipecat/
  • Auth
    • API key in header X-CEKURA-API-KEY
    • or HTTP bearer
    • or HTTP bearer

Request body

agentinteger

Agent ID to run the scenarios against. If omitted, the agent is derived from the first scenario's configured agent.

calls_per_secondnumber double nullable

Most calls started per second for this run, a number from 0.1 to 4 (e.g. 0.5 starts one call every 2 seconds). Leave unset for no limit.

mock_tool_namesstring[] nullable

Names of mock tools to activate for this run. Omit (or send null) to mock every tool configured on the agent; send an empty list to mock none so every tool keeps its original endpoint; send specific names to mock only those.

Response

idinteger

ID of the result

agentinteger

ID of the agent

status'running' | 'completed' | 'failed' | 'pending' | 'in_progress' | 'evaluating' | 'in_queue' | 'timeout' | 'cancelled' | 'scaling_up'

Status of the result. Poll until it is completed, failed, timeout or cancelled. It stays non-terminal until every run has ended, then becomes completed if any run completed — even if runs failed their checks or others errored.

run_as_textboolean

Whether the scenario ran as text or not

created_atstring date-time

Example response

{
  "runs": [
    {
      "number": "+11234567890",
      "inbound_number": "+11234567890"
    }
  ],
  "created_at": "2025-02-25T21:00:01.990052Z"
}

Changes