---
title: "Start customer-driven (API chat) evaluator runs"
method: POST
path: "/test_framework/v1/scenarios/run_scenarios_api_chat/"
tags: ["Evaluators"]
---

# Start customer-driven (API chat) evaluator runs

`POST /test_framework/v1/scenarios/run_scenarios_api_chat/`

## Request

- Base URL: `https://api.cekura.ai`
- URL: `https://api.cekura.ai/test_framework/v1/scenarios/run_scenarios_api_chat/`
- Auth: one of:
  - API key in header `X-CEKURA-API-KEY`
  - HTTP bearer
  - HTTP bearer

API CHAT RUN (pull mode): open one or more chat conversations that YOU drive turn-by-turn over REST — Cekura plays the simulated user, you play the agent under test. No agent connection is dialed; there is no server-side conversation loop, so a conversation can span minutes or up to an hour with turns arbitrarily far apart.

**Run-count:** total conversations = `len(scenarios) × frequency`.

**Driving each run** (the response `result.runs[].id` are the run IDs):
1. If the scenario has a first message, the simulated user speaks first — read it from the run transcript. Otherwise you speak first.
2. `POST /runs/{run_id}/chat/` with your agent's message → returns a `turn_id`.
3. Poll `GET /runs/{run_id}/chat/{turn_id}/` until `status` is `ready` to read the reply.
4. Repeat until `ended` is true, or call `POST /runs/{run_id}/end-chat/`.

`timeout` is the overall run timeout: each conversation is finalized and scored automatically once it elapses (defaults to the project's duration, capped at 1 hour).

## Request body

- SchemaPostRunScenariosApiChat: Schema for the run_scenarios_api_chat endpoint (customer-driven pull mode). Same selection fields as the other run endpoints (agent_id, scenarios/tags/ folder_path, frequency, personality/test-profile overrides) plus `timeout`.
  - `agent_id` integer: ID of the agent to run evaluators for
  - `assistant_id` string: Alternative to agent ID - the assistant ID to use for this scenario Example: `"asst_1234567890"`
  - `name` string: Label text for result
  - `scenarios` integer[]: List of evaluator IDs to run. Either scenarios, tags, or folder_path must be provided. Runs start in the order given. Example: `[11, 22, 33]` To run evaluators by tag or folder, use the `tags` or `folder_path` fields instead.
  - `tags` string[]: List of tags to filter evaluators to run. Either evaluators or tags must be provided. Example: `["tag1", "tag2", "tag3"]`
  - `folder_path` string: Dot-separated folder path to select evaluators from (e.g. `"Sales.Inbound"`). Mutually exclusive with `scenarios` and `tags`. Requires `project_id`.
  - `project_id` integer: Project ID. Required when using `folder_path` (or when passing a folder path string via `scenarios`). Example: `1376`
  - `frequency` integer: The number of times each evaluator will run. Each repeat runs the whole list in order before the next one starts (A, B, A, B), rather than grouping repeats of one evaluator. Example: `1`
  - `personality_ids` integer[], nullable: Personality IDs to override for this run. Defaults to each scenario's personality.
  - `test_profile_ids` integer[], nullable: Test-profile IDs to override for this run. Defaults to each scenario's test profile.
  - `timeout` integer, nullable: Overall run timeout in seconds — the whole conversation's max lifetime (NOT a per-message timeout). When it elapses the run is finalized and scored automatically. Defaults to the project's configured duration. Capped at 3600 (1 hour).
  - `agent_version_id` string: Run against a stored version of the agent instead of the agent as it is live. Cekura makes a private copy of the agent in your provider account, puts that version's prompt on it, and runs against the copy — voice, model and tools stay as the agent is configured now, and the live agent is never changed. Accepts the version label shown in the agent's version history (`v12`, or `p1` for a version an improvement run proposed) or its numeric `history_id`. List them with `GET /test_framework/v1/aiagents/{id}/history/`; a version is usable when its `is_runnable` is true. Available for VAPI, Retell, ElevenLabs and Bland agents that are outbound and have a single prompt. Example: `"v12"`

## Response `200`

- RunScenarioResult
  - `id` integer: Unique identifier for the Result Example: `123`
  - `name` string, required: Name of the Result Example: `"Test Result 1"`
  - `agent` integer: ID of the AI agent that was tested Example: `123`
  - `agent_inbound` boolean: Whether the agent is configured for inbound calls (true) or outbound calls (false) Example: `true` or `false`
  - `status` 'running' | 'completed' | 'failed' | 'pending' | 'in_progress' | 'evaluating' | 'in_queue' | 'timeout' | 'cancelled' | 'scaling_up': Current status of the result * `running` - Running * `completed` - Completed * `failed` - Failed * `pending` - Pending * `in_progress` - In Progress * `evaluating` - Evaluating * `in_queue` - In Queue * `timeout` - Timeout * `cancelled` - Cancelled * `scaling_up` - Scaling Up
  - `success_rate` number, double: Percentage of runs that passed, 0 to 100
  - `run_as_text` boolean: Whether this test was run in text mode instead of voice mode Example: `true` or `false`
  - `is_cronjob` string: Whether this result was created by a scheduled cronjob Example: `true` or `false`
  - `runs` object: Run summaries for this result. For full run data including transcripts and metric scores, fetch the result detail endpoint — there `runs` is a dict keyed by run ID.
  - `created_at` string, date-time: Timestamp when this test result was created Example: `2021-01-01 00:00:00`
  - `updated_at` string, date-time: Timestamp when this test result was last updated

## Other responses

- `400`

## Changes

- **2026-11-03** `a6fb6a739212`: 1 info
  - endpoint added

[Operation history](https://skmtc.dev/cekura/apis/cekura-api/changes/test_framework/v1/scenarios/run_scenarios_api_chat/post.md)

---

[API](https://skmtc.dev/cekura/apis/cekura-api.md) - [All operations](https://skmtc.dev/cekura/apis/cekura-api/llms.txt) - [OpenAPI document](https://skmtc.dev/cekura/apis/cekura-api/revisions/a6fb6a739212?raw)