CEcekura
test_framework

Run metric reviews

Changed on

Re-evaluate every review of the metric, optionally with overridden settings, and return each verdict next to its expected value. With skip_save the reviews are left unchanged. Each review is billed as a metric evaluation. Usually takes under a minute.

post/test_framework/v3/metrics/{id}/run_reviews/

Request

  • Base URL
    https://api.cekura.ai
  • URL
    https://api.cekura.ai/test_framework/v3/metrics/{id}/run_reviews/
  • Auth
    • API key in header X-CEKURA-API-KEY
    • or HTTP bearer
    • or HTTP bearer
    • or HTTP bearer

Path parameters

idstring required

Request body

llm_provider'openai' | 'gemini' | 'dspy-gepa-gemini-flash' | 'deepseek-v4-flash'

Override the judge provider while running unsaved Labs changes

  • openai - OpenAI
  • gemini - Gemini
  • dspy-gepa-gemini-flash - Dspy Gepa Gemini Flash
  • deepseek-v4-flash - DeepSeek V4 Flash
evaluation_triggerstring

Override evaluation trigger setting

evaluation_trigger_promptstring

Override evaluation trigger prompt

trigger_type'llm_judge' | 'custom_code'

Override evaluation trigger implementation

  • llm_judge - LLM Judge
  • custom_code - Custom Code
evaluation_trigger_custom_codestring

Override custom-code evaluation trigger

descriptionstring

Override metric description

type'basic' | 'custom_prompt' | 'custom_code' | 'llm_judge'

Override metric type

  • basic - Basic (Deprecated in favor of LLM Judge)
  • custom_prompt - Custom Prompt ( Deprecated in favor of LLM Judge)
  • custom_code - Custom Code
  • llm_judge - LLM Judge
custom_codestring

Override custom code

promptstring

Override evaluation prompt

metric_description_programunknown
metric_description_variablesunknown
evaluation_trigger_programunknown
evaluation_trigger_variablesunknown
skip_saveboolean

If true, don't save evaluation results to database (used for internal previews)

Response

status'completed' | 'failed'

failed when the job could not produce its result; see error / errors.

metric_idinteger
agent_idinteger nullable
project_idinteger nullable
total_reviewsinteger
completed_reviewsinteger
failed_reviewsinteger
reviews_listinteger[]

Ids of the reviews evaluated.

failed_reviews_listinteger[]

Ids of the reviews that could not be evaluated.

metricobject

The metric, as the metrics API returns it.

computed_resultsobject[]

Only with skip_save: the same items as output.

Changes