CEcekura
test_framework

Clarify what evaluator scenarios should cover

Changed on

Clarify what a set of evaluator scenarios should cover before generating them.

Send the agent and a natural-language brief; the AI comes back with the questions it needs answered (or confirms it has enough) so scenarios_generate_bg produces scenarios that match what you actually meant.

Response: Returns progress_id for polling status at /scenario-agent-progress/

Improving existing evaluators is not done here. Read the evaluator with scenarios_retrieve (or the list endpoint), decide the edit yourself, and write it back with scenarios_partial_update or scenarios_bulk_update. This tool changes evaluator scenarios only; it never changes an AI agent's prompt, configuration, tools, or provider settings — for improve agent <id>, use the cekura:cekura-self-improving-agent skill.

post/test_framework/v2/scenarios/scenario-agent/

Request

  • Base URL
    https://api.cekura.ai
  • URL
    https://api.cekura.ai/test_framework/v2/scenarios/scenario-agent/
  • Auth
    • API key in header X-CEKURA-API-KEY
    • or HTTP bearer
    • or HTTP bearer

Request body

user_messagestring

What the generated evaluators should cover (e.g., 'cover refund disputes and angry callers'). Do not use for agent prompt/config improvements.

agent_idinteger nullable

Agent the generated evaluators are for. Required on the first turn; on a follow-up it may be omitted and is read off session_id.

project_idinteger nullable

Project ID - optional, automatically inferred from evaluators if not provided

session_idinteger nullable

Session ID for tracking - automatically created if not provided

num_scenariosinteger

Number of scenarios the clarification should be sized for

knowledge_baseinteger[]

Knowledge Base file IDs for additional context

tool_idsstring[]

Tools that will be enabled on the generated evaluators (validated up front)

inbound_phone_numbersinteger[]

Inbound numbers that will be attached to the generated evaluators (validated up front)

Response

progress_idstring required

Use this to poll for results at /scenario-agent-progress/?progress_id={id}

session_idinteger required

Session tracking ID

Changes