---
title: "Run evaluators against per-scenario websocket URLs (bulk)"
method: POST
path: "/test_framework/v2/scenarios/run_scenarios_with_websockets/"
tags: ["Evaluators"]
---

# Run evaluators against per-scenario websocket URLs (bulk)

`POST /test_framework/v2/scenarios/run_scenarios_with_websockets/`

## Request

- Base URL: `https://api.cekura.ai`
- URL: `https://api.cekura.ai/test_framework/v2/scenarios/run_scenarios_with_websockets/`
- Auth: one of:
  - API key in header `X-CEKURA-API-KEY`
  - HTTP bearer
  - HTTP bearer

WEBSOCKET RUN (per-scenario URLs, bulk): Execute evaluators where each scenario carries its own websocket URL. Useful when you have a fleet of agents or stateful websocket sessions to test in a single batch.

**Run-count:** total conversations = `len(scenarios) × frequency`. (Personality and test-profile multipliers are not supported on this endpoint.)

**`frequency` is required** on this endpoint (no default).

**When to use this vs others** (pick the endpoint matching the agent's configured connection — check `aiagents_retrieve`):
- `run_scenarios_text` — single websocket URL or SMS/chat mode (simpler)
- `run_scenarios` — real voice / phone / SIP call
- `run_scenarios_pipecat` — Pipecat/Daily.co WebRTC deployment

**Polling for completion:** the response is the result itself — read its top-level `id` (`status: 'running'`). Poll `GET /test_framework/v1/results/{result_id}/` until `status` is `completed`, `failed`, `timeout` or `cancelled`. A result stays non-terminal until every run has ended, then becomes `completed` if any run completed — even if runs failed their checks or others errored. To gate, require `success_runs_count == total_runs_count` (`failed_runs_count` counts only completed runs that failed evaluation).

**Known limitation:** `result.run_type` may show 'Text' instead of 'WebSocket' for runs triggered via this endpoint — this is a display issue only and does not affect execution.

## Request body

- RunScenariosWithWebsockets
  - `agent` integer, nullable: Agent to use. Required if none of the scenarios have an agent.
  - `scenarios` ScenarioWithWebsocket[], required: List of scenario+websocket pairs. Each element must be an object with two keys: `scenario` (integer evaluator ID) and `websocket_url` (string, ws:// or wss:// URL). Example: [{"scenario": 101, "websocket_url": "wss://agent-a.example.com/chat"}, {"scenario": 102, "websocket_url": "wss://agent-b.example.com/chat"}]
    - `scenario` integer, required: Integer ID of the evaluator (scenario) to run.
    - `websocket_url` string: WebSocket URL for this specific scenario (must start with ws:// or wss://). Overrides the agent-level websocket_url for this run only.
  - `frequency` integer, required: Frequency to run
  - `calls_per_second` number, double, nullable: Most calls started per second for this run, a number from 0.1 to 4 (e.g. 0.5 starts one call every 2 seconds). Leave unset for no limit.
  - `mock_tool_names` string[], nullable: Names of mock tools to activate for this run. Omit (or send null) to mock every tool configured on the agent; send an empty list to mock none so every tool keeps its original endpoint; send specific names to mock only those. Tools that belong to an MCP server should be selected via the MCP server's name or key, which mocks all of that server's tools as one unit. Legacy MCP sub-tool names are accepted as aliases for the whole parent MCP.

## Response `200`

- object
  - `id` integer: ID of the result
  - `agent` integer: ID of the agent
  - `status` 'running' | 'completed' | 'failed' | 'pending' | 'in_progress' | 'evaluating' | 'in_queue' | 'timeout' | 'cancelled' | 'scaling_up': Status of the result. Poll until it is `completed`, `failed`, `timeout` or `cancelled`. It stays non-terminal until every run has ended, then becomes `completed` if any run completed — even if runs failed their checks or others errored.
  - `run_as_text` boolean: Whether the scenario ran as text or not
  - `runs` object[]
    - `id` integer: ID of the run
    - `status` 'running' | 'completed' | 'failed' | 'pending' | 'in_progress' | 'evaluating' | 'in_queue' | 'timeout' | 'cancelled' | 'scaling_up': Status of the run
    - `scenario` integer: ID of the scenario
    - `number` string: For outbound runs (agent.inbound=False). The given number that must be called from the phone number configured in the cekura agent.
    - `inbound_number` string, nullable: For inbound runs (agent.inbound=True). The agent's configured phone number will receive calls from this number.
    - `scenario_name` string: Name of the scenario
    - `test_profile_data` object, nullable: Details of the test profile associated with this run scenario
    - `outbound_dial_window_opens_at` string, date-time, nullable: When the outbound dial window opened (run transitioned to pending/waiting-for-call). Null until dispatched.
    - `outbound_dial_window_seconds` integer, nullable: Duration of the outbound dial window in seconds.
    - `outbound_dial_window_closes_at` string, date-time, nullable: Deadline by which your agent must dial the outbound number (opens_at + outbound_dial_window_seconds). Dials after this are dropped and the run returns status: timeout.
  - `created_at` string, date-time

## Other responses

- `400`

## Changes

- **2026-11-03** `a6fb6a739212`: 1 info
  - endpoint added

[Operation history](https://skmtc.dev/cekura/apis/cekura-api/changes/test_framework/v2/scenarios/run_scenarios_with_websockets/post.md)

---

[API](https://skmtc.dev/cekura/apis/cekura-api.md) - [All operations](https://skmtc.dev/cekura/apis/cekura-api/llms.txt) - [OpenAPI document](https://skmtc.dev/cekura/apis/cekura-api/revisions/a6fb6a739212?raw)