---
title: "Run evaluators over a voice call"
method: POST
path: "/test_framework/v1/scenarios/run_scenarios/"
tags: ["Evaluators"]
---

# Run evaluators over a voice call

`POST /test_framework/v1/scenarios/run_scenarios/`

## Request

- Base URL: `https://api.cekura.ai`
- URL: `https://api.cekura.ai/test_framework/v1/scenarios/run_scenarios/`
- Auth: one of:
  - API key in header `X-CEKURA-API-KEY`
  - HTTP bearer
  - HTTP bearer

VOICE RUN: Execute evaluators against your agent using a real voice / phone / SIP call. This is the primary 'run your evaluators' endpoint for voice-mode testing. Requires `telephony.phone_number` configured on the agent (any provider).

**Run-count math:** total calls = `len(scenarios) × frequency × max(1, len(personality_ids)) × max(1, len(test_profile_ids))`. Each call is billed at the voice-simulation rate — check `frequency` carefully.

**Concurrency:** use `concurrency_limit` to cap parallel calls. Default: unbounded (subject to provider rate limits).

**Scenario selection (pick one):**
- `scenarios` — list of integer evaluator IDs (e.g. `[101, 102]`)
- `tags` — select by tag
- `folder_path` — select by folder path (e.g. `"Sales.Inbound"`); requires `project_id`

**Phone-number overrides:**
- `outbound_phone_number` — override the number dialed
- `agent_number` — override the contact number the inbound agent receives calls on
- `mode` — `same_number` (all runs use one pair) vs `different_numbers`

**When to use this vs others** (pick the endpoint matching the agent's configured connection — check `aiagents_retrieve`):
- `run_scenarios_text` — text / SMS / chat mode; cheaper, no voice cost
- `run_scenarios_with_websockets` — each scenario carries its own websocket URL
- `run_scenarios_pipecat` — Pipecat/Daily.co WebRTC deployment

**Personality override:** `personality_ids` must be IDs that are already enabled for the agent's project. Check which are enabled via `aiagents_retrieve` → `enabled_personalities` field. To enable a new personality, call `aiagents_partial_update` with the updated `enabled_personalities` list before using it here. Note: enabling a personality applies project-wide, not just to one agent. Passing an ID that is not enabled returns `["Invalid personalities provided."]`.

**Scheduled runs:** to run on a recurring cron schedule, create a CronJob via `POST /test_framework/v1/cron_jobs/` with `execution_mode="voice"` instead of looping this endpoint.

**Polling for completion:** the response is the result itself — read its top-level `id` (`status: 'running'`). Poll `GET /test_framework/v1/results/{result_id}/` until `status` is `completed`, `failed`, `timeout` or `cancelled`. A result stays non-terminal until every run has ended, then becomes `completed` if any run completed — even if runs failed their checks or others errored. To gate, require `success_runs_count == total_runs_count` (`failed_runs_count` counts only completed runs that failed evaluation). There is no separate progress endpoint for runs. Check `failed_reasons` for infrastructure errors; check `runs[run_id].error_message` for per-run errors.

**Outbound dial window:** for outbound runs (agent `inbound=false`), your agent must dial the run's `outbound_number` before `outbound_dial_window_closes_at` (default 300s after the run reaches `waiting_for_call` / pending state). Dials received after that deadline are dropped and the run returns `status: timeout`.

## Request body

- SchemaPostRunScenarios
  - `agent_id` integer: ID of the agent to run evaluators for
  - `assistant_id` string: Alternative to agent ID - the assistant ID to use for this scenario Example: `"asst_1234567890"`
  - `scenarios` integer[]: List of evaluator IDs to run. Either scenarios, tags, or folder_path must be provided. Runs start in the order given. Example: `[11, 22, 33]` To run evaluators by tag or folder, use the `tags` or `folder_path` fields instead.
  - `tags` string[]: List of tags to filter evaluators to run. Either evaluators or tags must be provided. Example: `["tag1", "tag2", "tag3"]`
  - `folder_path` string: Dot-separated folder path to select evaluators from (e.g. `"Sales.Inbound"`). Mutually exclusive with `scenarios` and `tags`. Requires `project_id`.
  - `project_id` integer: Project ID. Required when using `folder_path` (or when passing a folder path string via `scenarios`). Example: `1376`
  - `frequency` integer: The number of times each evaluator will run. Each repeat runs the whole list in order before the next one starts (A, B, A, B), rather than grouping repeats of one evaluator. Example: `1`
  - `name` string: Label text for result
  - `outbound_phone_number` string: Override the phone number to use for outbound calls. Example: "+1234567890"
  - `personality_ids` integer[], nullable: List of personality IDs to override for this run. If not provided, uses the scenario's default personality. Example: `[123, 456, 789]`
  - `test_profile_ids` integer[], nullable: List of test profile IDs to override for this run. If not provided, uses the scenario's default test profile. Example: `[123, 456, 789]`
  - `personality_id` integer, nullable: Single personality ID override. Merged into personality_ids; prefer personality_ids for multiple.
  - `test_profile_id` integer, nullable: Single test profile ID override. Merged into test_profile_ids; prefer test_profile_ids for multiple.
  - `mode` 'same_number' | 'different_numbers': Using same or different phone numbers for each evaluation * `same_number` - same_number * `different_numbers` - different_numbers
  - `agent_number` string: Override the phone number from which the agent under test receives the call during outbound runs. This allows overriding the default agent contact number for testing purposes. Example: "+1234567890"
  - `concurrency_limit` integer: Cap on the number of evaluator runs executed in parallel. Default: unbounded (subject to your provider's rate limits). Use this to avoid hitting provider rate limits on large frequency or folder runs. Example: `5`
  - `calls_per_second` number, double, nullable: Most calls started per second for this run, a number from 0.1 to 4 (e.g. 0.5 starts one call every 2 seconds). Leave unset for no limit.
  - `mock_tool_names` string[]: Names of mock tools to activate for this run. Cekura swaps those tools' URLs to mock endpoints while the run is active and restores the originals when it finishes. Omit (or send null) to mock every tool configured on the agent; send an empty list to mock none so every tool keeps its original endpoint. Example: `["get_appointments", "cancel_booking"]`
  - `agent_version_id` string: Run against a stored version of the agent instead of the agent as it is live. Cekura makes a private copy of the agent in your provider account, puts that version's prompt on it, and runs against the copy — voice, model and tools stay as the agent is configured now, and the live agent is never changed. Accepts the version label shown in the agent's version history (`v12`, or `p1` for a version an improvement run proposed) or its numeric `history_id`. List them with `GET /test_framework/v1/aiagents/{id}/history/`; a version is usable when its `is_runnable` is true. Available for VAPI, Retell, ElevenLabs and Bland agents that are outbound and have a single prompt. Example: `"v12"`

## Response `200`

- object
  - `id` integer: ID of the result
  - `agent` integer: ID of the agent
  - `status` 'running' | 'completed' | 'failed' | 'pending' | 'in_progress' | 'evaluating' | 'in_queue' | 'timeout' | 'cancelled' | 'scaling_up': Status of the result. Poll until it is `completed`, `failed`, `timeout` or `cancelled`. It stays non-terminal until every run has ended, then becomes `completed` if any run completed — even if runs failed their checks or others errored.
  - `run_as_text` boolean: Whether the scenario ran as text or not
  - `runs` object[]
    - `id` integer: ID of the run
    - `status` 'running' | 'completed' | 'failed' | 'pending' | 'in_progress' | 'evaluating' | 'in_queue' | 'timeout' | 'cancelled' | 'scaling_up': Status of the run
    - `scenario` integer: ID of the scenario
    - `number` string: For outbound runs (agent.inbound=False). The given number that must be called from the phone number configured in the cekura agent.
    - `inbound_number` string, nullable: For inbound runs (agent.inbound=True). The agent's configured phone number will receive calls from this number.
    - `scenario_name` string: Name of the scenario
    - `test_profile_data` object, nullable: Details of the test profile associated with this run scenario
    - `outbound_dial_window_opens_at` string, date-time, nullable: When the outbound dial window opened (run transitioned to pending/waiting-for-call). Null until dispatched.
    - `outbound_dial_window_seconds` integer, nullable: Duration of the outbound dial window in seconds.
    - `outbound_dial_window_closes_at` string, date-time, nullable: Deadline by which your agent must dial the outbound number (opens_at + outbound_dial_window_seconds). Dials after this are dropped and the run returns status: timeout.
  - `created_at` string, date-time

## Other responses

- `400`

## Changes

- **2026-11-03** `a6fb6a739212`: 1 info
  - endpoint added

[Operation history](https://skmtc.dev/cekura/apis/cekura-api/changes/test_framework/v1/scenarios/run_scenarios/post.md)

---

[API](https://skmtc.dev/cekura/apis/cekura-api.md) - [All operations](https://skmtc.dev/cekura/apis/cekura-api/llms.txt) - [OpenAPI document](https://skmtc.dev/cekura/apis/cekura-api/revisions/a6fb6a739212?raw)