CEcekura
test_framework

Create multiple metrics in a single request

Changed on

Create multiple metrics in a single request

post/test_framework/v2/metrics/bulk_create/

Request

  • Base URL
    https://api.cekura.ai
  • URL
    https://api.cekura.ai/test_framework/v2/metrics/bulk_create/
  • Auth
    • API key in header X-CEKURA-API-KEY
    • or HTTP bearer
    • or HTTP bearer
    • or HTTP bearer

Request body

idinteger
namestring required

Name of the metric

descriptionstring required

Description of what this metric evaluates

audio_enabledboolean required

Whether this metric evaluates audio content

promptstring required

The evaluation prompt used for this metric

projectinteger required

ID of the project this metric belongs to.

agentsinteger[]

List of agent IDs to enable this project-level metric for. Only applicable when project is set.

assistant_idstring required

External identifier for the assistant

type'basic' | 'custom_prompt' | 'custom_code' | 'llm_judge' required

Type of metric (llm_judge recommended; basic and custom_prompt are deprecated)

  • basic - Basic (Deprecated in favor of LLM Judge)
  • custom_prompt - Custom Prompt ( Deprecated in favor of LLM Judge)
  • custom_code - Custom Code
  • llm_judge - LLM Judge
eval_type'binary' | 'continuous_qualitative' | 'numeric' | 'enum' required

Output shape of the evaluation score. Accepts canonical (binary, continuous_qualitative, numeric, enum); legacy binary_workflow_adherence and binary_qualitative are accepted and mapped to binary.

  • binary - Binary
  • continuous_qualitative - Continuous Qualitative
  • numeric - Numeric
  • enum - Enum
enum_valuesstring[]

Possible values for enum-type metrics (list of strings, e.g. ["resolved", "escalated", "abandoned"])

display_orderinteger required

Order in which to display this metric in the UI

configurationobject required

Per-metric configuration.

Some predefined metrics carry per-agent configuration, keyed by agent id: configuration[<key>] = {"<agent_id>": <value>}. On read, every agent attached to the metric is returned; on write, only the agent ids you include are updated.

Call-level keys (not per-agent):

  • Detect Silence — silence_duration (int, seconds, default 10): minimum mutual silence before flagging a failure.
  • Infrastructure Issues — infra_issues_timeout (int, seconds, default 10): max seconds before the Main Agent must respond after the Testing Agent finishes speaking.

Per-agent keys (each value is {"<agent_id>": <value>}):

  • Dropoff Node — nodes: {"<agent_id>": [{name, description}]} — conversation stages used to classify where a call dropped off.
  • Topic of Call — nodes: {"<agent_id>": [{name, description}]} — topic categories used to classify each call.
  • Pronunciation Check — pronunciation_words: {"<agent_id>": [[word, phonemes], ...]} — how specific words should be pronounced (e.g. {"42": [["Cekura", "suh-KYUR-uh"]]}).
  • Letterwise Pronunciation — spelling_word_types: {"<agent_id>": ["name", "email", ...]} — word categories the Main Agent must spell out letter-by-letter.
  • Hallucination — hallucination_kb_files: {"<agent_id>": [file_id, ...]} — KnowledgeBaseFile IDs used as the source of truth for fact-checking.

Any metric (call-level key):

  • metric_dependencies: list of metric IDs (names or slugs also accepted) whose results this metric reads. Those metrics are evaluated first; an LLM-judge prompt can reference them as {{metrics.<slug>.score}} / {{metrics.<slug>.explanation}} and custom code as data["<Metric name>"] or data["metrics"]["<slug>"]. Omit the key to infer inputs from those references; pass [] to declare none. Cycles, more than 10 inputs, or chains deeper than 5 levels are rejected.
add_to_new_agentsboolean nullable

When enabled, this metric is automatically assigned to new agents created in the project.

simulation_enabledboolean

Enable this metric for simulations. Example: true or false

observability_enabledboolean

Enable this metric for observability. Example: true or false

sampling_enabledboolean

Enable sampling for this metric using its sample_rate, or the project sample rate when unset

sample_ratenumber double nullable

Sampling rate for this metric (0.0 to 1.0). Overrides the project sample rate when set.

evaluation_trigger'always' | 'automatic' | 'custom'

When to run this metric.

  • always — evaluate every call (default)

  • automatic — system decides based on call content

  • custom — only evaluate when evaluation_trigger_prompt condition is met

  • always - Always

  • automatic - Automatic

  • custom - Custom

evaluation_trigger_promptstring

LLM prompt that decides whether to evaluate this call. Only used when evaluation_trigger=custom and trigger_type=llm_judge. Example: "Did the agent offer a refund?"

trigger_type'llm_judge' | 'custom_code'

How to evaluate the trigger condition. Only relevant when evaluation_trigger=custom.

  • llm_judge — use evaluation_trigger_prompt (default)

  • custom_code — use evaluation_trigger_custom_code

  • llm_judge - LLM Judge

  • custom_code - Custom Code

evaluation_trigger_custom_codestring

Python code to evaluate the trigger condition. Only used when evaluation_trigger=custom and trigger_type=custom_code.

custom_codestring

Python code that implements the metric evaluation. Required when type=custom_code. Must define a function evaluate(transcript, ...) -> bool | float | str.

Response

idinteger
namestring required

Name of the metric

descriptionstring required

Description of what this metric evaluates

audio_enabledboolean required

Whether this metric evaluates audio content

promptstring required

The evaluation prompt used for this metric

projectinteger required

ID of the project this metric belongs to.

agentsinteger[]

List of agent IDs to enable this project-level metric for. Only applicable when project is set.

assistant_idstring required

External identifier for the assistant

type'basic' | 'custom_prompt' | 'custom_code' | 'llm_judge' required

Type of metric (llm_judge recommended; basic and custom_prompt are deprecated)

  • basic - Basic (Deprecated in favor of LLM Judge)
  • custom_prompt - Custom Prompt ( Deprecated in favor of LLM Judge)
  • custom_code - Custom Code
  • llm_judge - LLM Judge
eval_type'binary' | 'continuous_qualitative' | 'numeric' | 'enum' required

Output shape of the evaluation score. Accepts canonical (binary, continuous_qualitative, numeric, enum); legacy binary_workflow_adherence and binary_qualitative are accepted and mapped to binary.

  • binary - Binary
  • continuous_qualitative - Continuous Qualitative
  • numeric - Numeric
  • enum - Enum
enum_valuesstring[]

Possible values for enum-type metrics (list of strings, e.g. ["resolved", "escalated", "abandoned"])

display_orderinteger required

Order in which to display this metric in the UI

configurationobject required

Per-metric configuration.

Some predefined metrics carry per-agent configuration, keyed by agent id: configuration[<key>] = {"<agent_id>": <value>}. On read, every agent attached to the metric is returned; on write, only the agent ids you include are updated.

Call-level keys (not per-agent):

  • Detect Silence — silence_duration (int, seconds, default 10): minimum mutual silence before flagging a failure.
  • Infrastructure Issues — infra_issues_timeout (int, seconds, default 10): max seconds before the Main Agent must respond after the Testing Agent finishes speaking.

Per-agent keys (each value is {"<agent_id>": <value>}):

  • Dropoff Node — nodes: {"<agent_id>": [{name, description}]} — conversation stages used to classify where a call dropped off.
  • Topic of Call — nodes: {"<agent_id>": [{name, description}]} — topic categories used to classify each call.
  • Pronunciation Check — pronunciation_words: {"<agent_id>": [[word, phonemes], ...]} — how specific words should be pronounced (e.g. {"42": [["Cekura", "suh-KYUR-uh"]]}).
  • Letterwise Pronunciation — spelling_word_types: {"<agent_id>": ["name", "email", ...]} — word categories the Main Agent must spell out letter-by-letter.
  • Hallucination — hallucination_kb_files: {"<agent_id>": [file_id, ...]} — KnowledgeBaseFile IDs used as the source of truth for fact-checking.

Any metric (call-level key):

  • metric_dependencies: list of metric IDs (names or slugs also accepted) whose results this metric reads. Those metrics are evaluated first; an LLM-judge prompt can reference them as {{metrics.<slug>.score}} / {{metrics.<slug>.explanation}} and custom code as data["<Metric name>"] or data["metrics"]["<slug>"]. Omit the key to infer inputs from those references; pass [] to declare none. Cycles, more than 10 inputs, or chains deeper than 5 levels are rejected.
add_to_new_agentsboolean nullable

When enabled, this metric is automatically assigned to new agents created in the project.

simulation_enabledboolean

Enable this metric for simulations. Example: true or false

observability_enabledboolean

Enable this metric for observability. Example: true or false

sampling_enabledboolean

Enable sampling for this metric using its sample_rate, or the project sample rate when unset

sample_ratenumber double nullable

Sampling rate for this metric (0.0 to 1.0). Overrides the project sample rate when set.

evaluation_trigger'always' | 'automatic' | 'custom'

When to run this metric.

  • always — evaluate every call (default)

  • automatic — system decides based on call content

  • custom — only evaluate when evaluation_trigger_prompt condition is met

  • always - Always

  • automatic - Automatic

  • custom - Custom

evaluation_trigger_promptstring

LLM prompt that decides whether to evaluate this call. Only used when evaluation_trigger=custom and trigger_type=llm_judge. Example: "Did the agent offer a refund?"

trigger_type'llm_judge' | 'custom_code'

How to evaluate the trigger condition. Only relevant when evaluation_trigger=custom.

  • llm_judge — use evaluation_trigger_prompt (default)

  • custom_code — use evaluation_trigger_custom_code

  • llm_judge - LLM Judge

  • custom_code - Custom Code

evaluation_trigger_custom_codestring

Python code to evaluate the trigger condition. Only used when evaluation_trigger=custom and trigger_type=custom_code.

custom_codestring

Python code that implements the metric evaluation. Required when type=custom_code. Must define a function evaluate(transcript, ...) -> bool | float | str.

Changes