---
title: "Generate evaluator"
method: POST
path: "/test_framework/v3/scenarios/generate/"
tags: ["test_framework"]
---

# Generate evaluator

`POST /test_framework/v3/scenarios/generate/`

## Request

- Base URL: `https://api.cekura.ai`
- URL: `https://api.cekura.ai/test_framework/v3/scenarios/generate/`
- Auth: one of:
  - API key in header `X-CEKURA-API-KEY`
  - HTTP bearer
  - HTTP bearer

Generate evaluators for an agent and return them once generation finishes. Generation never pauses for clarification questions. Usually takes 1–4 minutes.

## Request body

- SchemaGenerateScenarios
  - `agent_id` integer: ID of the agent to generate evaluator for
  - `assistant_id` string: Alternative to agent_id - the assistant ID to generate evaluators for
  - `personalities` integer[]: List of personality IDs to use for generated evaluators
  - `num_scenarios` integer, required: Number of evaluators to generate (1-100)
  - `test_profile` integer: ID of the test profile to assign to generated evaluators
  - `scenario_type` 'workflow' | 'red_teaming_voice' | 'red_teaming_text': Type of scenarios to generate: workflow (standard scenarios), red_teaming_voice (red teaming with voice-suitable prompts only), or red_teaming_text (red teaming with all prompts) * `workflow` - workflow * `red_teaming_voice` - red_teaming_voice * `red_teaming_text` - red_teaming_text
  - `attack_type` '' | 'system_prompt_leak' | 'data_leak' | 'harmful_content' | 'biased_output' | 'unauthorized_actions' | 'off_task' | 'null', nullable: Attack category for red teaming scenarios. Required when scenario_type is red_teaming_voice or red_teaming_text. Categories: system_prompt_leak (attempts to extract the system prompt), data_leak (attempts to extract sensitive data), harmful_content (attempts to generate harmful output), biased_output (attempts to generate biased responses), unauthorized_actions (attempts to trigger unauthorized tool calls or actions), off_task (attempts to derail the conversation away from the agent's purpose) * `` - * `system_prompt_leak` - system_prompt_leak * `data_leak` - data_leak * `harmful_content` - harmful_content * `biased_output` - biased_output * `unauthorized_actions` - unauthorized_actions * `off_task` - off_task
  - `expected_outcome` string, nullable: Custom expected outcome for the scenario. If not provided, a default is generated based on the attack type.
  - `extra_instructions` string: Additional instructions to guide evaluator generation
  - `generation_files` string[]: Files used only as context for this generation run. Supports PDF, TXT, JSON, CSV, XML, and Markdown; the combined upload size cannot exceed 50 MB. Upload as multipart/form-data. From a Cekura chat conversation (MCP), pass the name of a file already attached to that conversation instead of the file body.
  - `simulation_type` 'instruction' | 'conditional_actions': Output format for generated workflow evaluators. Use instruction for prompt-based evaluators or conditional_actions for structured condition/action evaluators. * `instruction` - instruction * `conditional_actions` - conditional_actions
  - `inbound_phone_number` integer, nullable: Phone number ID to use for inbound evaluator
  - `first_message` string: Initial message to start the conversation
  - `generate_expected_outcomes` boolean: If true, expected outcomes will be generated for each evaluator
  - `tags` string[]: List of tags to apply to all generated scenarios
  - `tool_ids` string[]: List of tool IDs to use for generated evaluators
  - `folder_path` string, nullable: Folder path to assign generated scenarios to (e.g., 'folder_name' or 'parent.child')

## Response `200`

- object
  - `status` 'completed' | 'failed': `failed` when the job could not produce its result; see `error` / `errors`.
  - `agent_id` integer
  - `total_scenarios` integer
  - `completed_scenarios` integer
  - `failed_scenarios` integer
  - `scenarios_list` object[]: The generated evaluators.
  - `errors` unknown[]: Why individual evaluators failed.
    - unknown
  - `warnings` unknown[]
    - unknown
  - `session_id` integer, nullable
  - `current_step` string

## Other responses

- `400`
- `403`: No access to the agent, project, metric or result the job runs on.
- `504`: The job did not finish within 15 minutes. It may still complete in the background.

## Changes

- **2026-11-03** `a6fb6a739212`: 1 info
  - endpoint added

[Operation history](https://skmtc.dev/cekura/apis/cekura-api/changes/test_framework/v3/scenarios/generate/post.md)

---

[API](https://skmtc.dev/cekura/apis/cekura-api.md) - [All operations](https://skmtc.dev/cekura/apis/cekura-api/llms.txt) - [OpenAPI document](https://skmtc.dev/cekura/apis/cekura-api/revisions/a6fb6a739212?raw)