--- title: "POST /test_framework/v1/metrics/generate_evaluation_trigger/" method: POST path: "/test_framework/v1/metrics/generate_evaluation_trigger/" tags: ["test_framework"] --- # POST /test_framework/v1/metrics/generate_evaluation_trigger/ `POST /test_framework/v1/metrics/generate_evaluation_trigger/` ## Request - Base URL: `https://api.cekura.ai` - URL: `https://api.cekura.ai/test_framework/v1/metrics/generate_evaluation_trigger/` - Auth: one of: - API key in header `X-CEKURA-API-KEY` - HTTP bearer - HTTP bearer - HTTP bearer Metrics generate evaluation trigger ## Request body - MetricDetailV2 - `id` integer - `slug` string, nullable: URL-friendly unique identifier in snake_case format. Example: "customer_satisfaction_1562" - `project` integer, nullable - `agents` integer[] - `name` string, required: Name of the metric. Example: `"Customer Satisfaction"` or `"Appointment Booking"` - `description` string: Description of what the metric measures. Example: `"Measures how satisfied customers are with the service provided"` - `type` 'basic' | 'custom_prompt' | 'custom_code' | 'llm_judge': Type of metric * `basic` - Basic (Deprecated in favor of LLM Judge) * `custom_prompt` - Custom Prompt ( Deprecated in favor of LLM Judge) * `custom_code` - Custom Code * `llm_judge` - LLM Judge - `eval_type` 'binary' | 'continuous_qualitative' | 'numeric' | 'enum', required: * `binary` - Binary * `continuous_qualitative` - Continuous Qualitative * `numeric` - Numeric * `enum` - Enum - `enum_values` unknown - `audio_enabled` boolean: Whether this metric requires audio analysis. Example: `true` or `false` - `prompt` string: Evaluation prompt for the metric. Example: `"Evaluate customer satisfaction based on conversation"` - `evaluation_trigger` 'always' | 'automatic' | 'custom': * `always` - Always * `automatic` - Automatic * `custom` - Custom - `trigger_type` 'llm_judge' | 'custom_code': Type of trigger evaluation: LLM judge or custom code. Only used when evaluation_trigger is CUSTOM. Example: `"llm_judge"` or `"custom_code"` * `llm_judge` - LLM Judge * `custom_code` - Custom Code - `evaluation_trigger_prompt` string: Evaluation trigger prompt for the metric. Example: `"Evaluate metric only if call ended reason is main-agent-ended-call"` - `evaluation_trigger_custom_code` string: Python custom code to determine metric relevance. Code should set _result (bool) and _explanation (str). Example: ```python _result = True _explanation = "Metric is relevant" if "call_end_reason" in data and data["call_end_reason"] == "customer-hung-up": _result = False _explanation = "Customer hung up, metric not applicable" ``` - `priority_assignment_prompt` string: Priority assignment prompt for the metric. - `configuration` unknown - `overall_score` string: The overall score for this metric across all test sets - `total_score` string: The total score for this metric - `knowledge_base_files` string: Knowledge base files associated with this metric - `observability_enabled` boolean: Enable this metric for observability. Example: `true` or `false` - `simulation_enabled` boolean: Enable this metric for simulations. Example: `true` or `false` - `sampling_enabled` boolean: Enable sampling for this metric using its sample_rate, or the project sample rate when unset - `sample_rate` number, double, nullable: Sampling rate for this metric (0.0 to 1.0). Overrides the project sample rate when set. - `alert_enabled` boolean: Enable alerts for this metric when it fails (value=false). Only applicable to binary metrics (eval_type=binary). For other metric types, use significant_change_alert_status instead. Example: `true` or `false` - `alert_type` 'disabled' | 'normal' | 'significant_change': * `disabled` - Alerts Disabled * `normal` - Normal Alerts * `significant_change` - Significant Change Alerts - `alert_filters` unknown - `slack_workspace` integer, nullable: Slack workspace to send alerts to - `slack_channel_id` string, nullable: Override channel ID for this metric's alerts - `significant_change_alert_status` 'enabled' | 'disabled': Alert status: enabled or disabled. * `enabled` - Enabled * `disabled` - Disabled - `significant_change_alert_direction` '' | 'increase' | 'decrease': Alert direction: increase only, decrease only, or both (empty = both). Example: `"increase"`, `"decrease"`, or `"both"` * `` - Both (Increase and Decrease) * `increase` - Increase Only * `decrease` - Decrease Only - `function_name` string, nullable: Predefined function name Example: `"get_latency"` or `"check_critical_deviations"` - `custom_code` string: Python custom code for the metric. Example: ```python _resul = True _explanation = None if "call_end_reason" in data and data["call_end_reason"] == "customer_satisfaction": _result = True _explanation = "Customer expressed satisfaction with service" ``` - `vocera_defined_metric_code` string: Vocera defined metric code for the metric. Example: `"7fd534f5"` - `scenarios` unknown - `reviews` MetricReviewDetail[]: Reviews associated with the metric - `id` integer - `test_set` TestSetInline - `id` integer - `agent` integer, required - `name` string: Name of the test set Example: `"Test Set 1"` - `transcript` string: Full text transcript of the call. Example: ```text [00:01] Testing Agent: Hello. [00:02] Main Agent: Hello, how can I help you today? [00:03] Testing Agent: Well, I mean, sure. What time exactly are we talking about here [00:04] Main Agent: 6 PM. [00:05] Testing Agent: Great. I'll book that for you. Just a sec. [00:06] Main Agent: Okay. ``` - `voice_recording_url` string - `call_end_reason` string: Reason why the call ended. Example: - `"customer-ended-call"` - `"agent-ended-call"` - `duration` string - `source_model` 'CallLog' | 'Run': * `CallLog` - Call Log * `Run` - Run - `source_id` string - `created_at` string, date-time: When this record was created. Example: `2024-03-15T10:30:45Z` - `updated_at` string, date-time: When this record was last updated. Example: `2024-03-15T10:35:11Z` - `metric` integer, required - `expected_value` unknown - `actual_value` unknown - `explanation` unknown - `feedback` string: Metric feedback Example: - `"Great Call"` - `created_at` string, date-time: Timestamp when the metric review was created Example: `"2021-01-01T00:00:00Z"` - `updated_at` string, date-time: Timestamp when the metric review was last updated Example: `"2021-01-01T00:00:00Z"` - `window_size` integer: Window size for rolling statistics calculation. Example: `50` - `std_multiplier` number, double: Standard deviation multiplier for threshold calculation. Example: `2.0` - `ewma_alpha` number, double: Alpha value for exponentially weighted moving average (EWMA) calculation. Example: `0.1` - `add_to_new_agents` boolean, nullable: When enabled, this metric is automatically assigned to new agents created in the project. - `pending_optimisation_proposal` unknown ## Response `200` - MetricDetailV2 - `id` integer - `slug` string, nullable: URL-friendly unique identifier in snake_case format. Example: "customer_satisfaction_1562" - `project` integer, nullable - `agents` integer[] - `name` string, required: Name of the metric. Example: `"Customer Satisfaction"` or `"Appointment Booking"` - `description` string: Description of what the metric measures. Example: `"Measures how satisfied customers are with the service provided"` - `type` 'basic' | 'custom_prompt' | 'custom_code' | 'llm_judge': Type of metric * `basic` - Basic (Deprecated in favor of LLM Judge) * `custom_prompt` - Custom Prompt ( Deprecated in favor of LLM Judge) * `custom_code` - Custom Code * `llm_judge` - LLM Judge - `eval_type` 'binary' | 'continuous_qualitative' | 'numeric' | 'enum', required: * `binary` - Binary * `continuous_qualitative` - Continuous Qualitative * `numeric` - Numeric * `enum` - Enum - `enum_values` unknown - `audio_enabled` boolean: Whether this metric requires audio analysis. Example: `true` or `false` - `prompt` string: Evaluation prompt for the metric. Example: `"Evaluate customer satisfaction based on conversation"` - `evaluation_trigger` 'always' | 'automatic' | 'custom': * `always` - Always * `automatic` - Automatic * `custom` - Custom - `trigger_type` 'llm_judge' | 'custom_code': Type of trigger evaluation: LLM judge or custom code. Only used when evaluation_trigger is CUSTOM. Example: `"llm_judge"` or `"custom_code"` * `llm_judge` - LLM Judge * `custom_code` - Custom Code - `evaluation_trigger_prompt` string: Evaluation trigger prompt for the metric. Example: `"Evaluate metric only if call ended reason is main-agent-ended-call"` - `evaluation_trigger_custom_code` string: Python custom code to determine metric relevance. Code should set _result (bool) and _explanation (str). Example: ```python _result = True _explanation = "Metric is relevant" if "call_end_reason" in data and data["call_end_reason"] == "customer-hung-up": _result = False _explanation = "Customer hung up, metric not applicable" ``` - `priority_assignment_prompt` string: Priority assignment prompt for the metric. - `configuration` unknown - `overall_score` string: The overall score for this metric across all test sets - `total_score` string: The total score for this metric - `knowledge_base_files` string: Knowledge base files associated with this metric - `observability_enabled` boolean: Enable this metric for observability. Example: `true` or `false` - `simulation_enabled` boolean: Enable this metric for simulations. Example: `true` or `false` - `sampling_enabled` boolean: Enable sampling for this metric using its sample_rate, or the project sample rate when unset - `sample_rate` number, double, nullable: Sampling rate for this metric (0.0 to 1.0). Overrides the project sample rate when set. - `alert_enabled` boolean: Enable alerts for this metric when it fails (value=false). Only applicable to binary metrics (eval_type=binary). For other metric types, use significant_change_alert_status instead. Example: `true` or `false` - `alert_type` 'disabled' | 'normal' | 'significant_change': * `disabled` - Alerts Disabled * `normal` - Normal Alerts * `significant_change` - Significant Change Alerts - `alert_filters` unknown - `slack_workspace` integer, nullable: Slack workspace to send alerts to - `slack_channel_id` string, nullable: Override channel ID for this metric's alerts - `significant_change_alert_status` 'enabled' | 'disabled': Alert status: enabled or disabled. * `enabled` - Enabled * `disabled` - Disabled - `significant_change_alert_direction` '' | 'increase' | 'decrease': Alert direction: increase only, decrease only, or both (empty = both). Example: `"increase"`, `"decrease"`, or `"both"` * `` - Both (Increase and Decrease) * `increase` - Increase Only * `decrease` - Decrease Only - `function_name` string, nullable: Predefined function name Example: `"get_latency"` or `"check_critical_deviations"` - `custom_code` string: Python custom code for the metric. Example: ```python _resul = True _explanation = None if "call_end_reason" in data and data["call_end_reason"] == "customer_satisfaction": _result = True _explanation = "Customer expressed satisfaction with service" ``` - `vocera_defined_metric_code` string: Vocera defined metric code for the metric. Example: `"7fd534f5"` - `scenarios` unknown - `reviews` MetricReviewDetail[]: Reviews associated with the metric - `id` integer - `test_set` TestSetInline - `id` integer - `agent` integer, required - `name` string: Name of the test set Example: `"Test Set 1"` - `transcript` string: Full text transcript of the call. Example: ```text [00:01] Testing Agent: Hello. [00:02] Main Agent: Hello, how can I help you today? [00:03] Testing Agent: Well, I mean, sure. What time exactly are we talking about here [00:04] Main Agent: 6 PM. [00:05] Testing Agent: Great. I'll book that for you. Just a sec. [00:06] Main Agent: Okay. ``` - `voice_recording_url` string - `call_end_reason` string: Reason why the call ended. Example: - `"customer-ended-call"` - `"agent-ended-call"` - `duration` string - `source_model` 'CallLog' | 'Run': * `CallLog` - Call Log * `Run` - Run - `source_id` string - `created_at` string, date-time: When this record was created. Example: `2024-03-15T10:30:45Z` - `updated_at` string, date-time: When this record was last updated. Example: `2024-03-15T10:35:11Z` - `metric` integer, required - `expected_value` unknown - `actual_value` unknown - `explanation` unknown - `feedback` string: Metric feedback Example: - `"Great Call"` - `created_at` string, date-time: Timestamp when the metric review was created Example: `"2021-01-01T00:00:00Z"` - `updated_at` string, date-time: Timestamp when the metric review was last updated Example: `"2021-01-01T00:00:00Z"` - `window_size` integer: Window size for rolling statistics calculation. Example: `50` - `std_multiplier` number, double: Standard deviation multiplier for threshold calculation. Example: `2.0` - `ewma_alpha` number, double: Alpha value for exponentially weighted moving average (EWMA) calculation. Example: `0.1` - `add_to_new_agents` boolean, nullable: When enabled, this metric is automatically assigned to new agents created in the project. - `pending_optimisation_proposal` unknown ## Changes - **2026-11-03** `a6fb6a739212`: 1 info - endpoint added [Operation history](https://skmtc.dev/cekura/apis/cekura-api/changes/test_framework/v1/metrics/generate_evaluation_trigger/post.md) --- [API](https://skmtc.dev/cekura/apis/cekura-api.md) - [All operations](https://skmtc.dev/cekura/apis/cekura-api/llms.txt) - [OpenAPI document](https://skmtc.dev/cekura/apis/cekura-api/revisions/a6fb6a739212?raw)