CEcekura
test_framework

Generate evaluator

Changed on

Generate evaluators for an agent and return them once generation finishes. Generation never pauses for clarification questions. Usually takes 1–4 minutes.

post/test_framework/v3/scenarios/generate/

Request

  • Base URL
    https://api.cekura.ai
  • URL
    https://api.cekura.ai/test_framework/v3/scenarios/generate/
  • Auth
    • API key in header X-CEKURA-API-KEY
    • or HTTP bearer
    • or HTTP bearer

Request body

agent_idinteger

ID of the agent to generate evaluator for

assistant_idstring

Alternative to agent_id - the assistant ID to generate evaluators for

personalitiesinteger[]

List of personality IDs to use for generated evaluators

num_scenariosinteger required

Number of evaluators to generate (1-100)

test_profileinteger

ID of the test profile to assign to generated evaluators

scenario_type'workflow' | 'red_teaming_voice' | 'red_teaming_text'

Type of scenarios to generate: workflow (standard scenarios), red_teaming_voice (red teaming with voice-suitable prompts only), or red_teaming_text (red teaming with all prompts)

  • workflow - workflow
  • red_teaming_voice - red_teaming_voice
  • red_teaming_text - red_teaming_text
attack_type'' | 'system_prompt_leak' | 'data_leak' | 'harmful_content' | 'biased_output' | 'unauthorized_actions' | 'off_task' | 'null' nullable

Attack category for red teaming scenarios. Required when scenario_type is red_teaming_voice or red_teaming_text. Categories: system_prompt_leak (attempts to extract the system prompt), data_leak (attempts to extract sensitive data), harmful_content (attempts to generate harmful output), biased_output (attempts to generate biased responses), unauthorized_actions (attempts to trigger unauthorized tool calls or actions), off_task (attempts to derail the conversation away from the agent's purpose)

  • `` -
  • system_prompt_leak - system_prompt_leak
  • data_leak - data_leak
  • harmful_content - harmful_content
  • biased_output - biased_output
  • unauthorized_actions - unauthorized_actions
  • off_task - off_task
expected_outcomestring nullable

Custom expected outcome for the scenario. If not provided, a default is generated based on the attack type.

extra_instructionsstring

Additional instructions to guide evaluator generation

generation_filesstring[]

Files used only as context for this generation run. Supports PDF, TXT, JSON, CSV, XML, and Markdown; the combined upload size cannot exceed 50 MB. Upload as multipart/form-data. From a Cekura chat conversation (MCP), pass the name of a file already attached to that conversation instead of the file body.

simulation_type'instruction' | 'conditional_actions'

Output format for generated workflow evaluators. Use instruction for prompt-based evaluators or conditional_actions for structured condition/action evaluators.

  • instruction - instruction
  • conditional_actions - conditional_actions
inbound_phone_numberinteger nullable

Phone number ID to use for inbound evaluator

first_messagestring

Initial message to start the conversation

generate_expected_outcomesboolean

If true, expected outcomes will be generated for each evaluator

tagsstring[]

List of tags to apply to all generated scenarios

tool_idsstring[]

List of tool IDs to use for generated evaluators

folder_pathstring nullable

Folder path to assign generated scenarios to (e.g., 'folder_name' or 'parent.child')

Example request

{
  "tool_ids": [
    "TOOL_DTMF",
    "TOOL_END_CALL",
    "TOOL_END_CALL_ONLY_ON_TRANSFER"
  ]
}

Response

status'completed' | 'failed'

failed when the job could not produce its result; see error / errors.

agent_idinteger
total_scenariosinteger
completed_scenariosinteger
failed_scenariosinteger
scenarios_listobject[]

The generated evaluators.

session_idinteger nullable
current_stepstring

Changes