CEcekura
test_framework

Improve a metric from feedback

Changed on

Improve a metric using feedback from test sets. Returns improved metric configuration. Use the Update Metric endpoint to save changes.

post/test_framework/metric-reviews/process_feedbacks/

Request

  • Base URL
    https://api.cekura.ai
  • URL
    https://api.cekura.ai/test_framework/metric-reviews/process_feedbacks/
  • Auth
    • API key in header X-CEKURA-API-KEY
    • or HTTP bearer
    • or HTTP bearer

Request body

metric_idinteger required

ID of the metric to process feedbacks for

metric_type'basic' | 'custom_prompt' | 'custom_code' | 'llm_judge'

Optional metric type

  • basic - Basic (Deprecated in favor of LLM Judge)
  • custom_prompt - Custom Prompt ( Deprecated in favor of LLM Judge)
  • custom_code - Custom Code
  • llm_judge - LLM Judge
test_set_idsinteger[]

Test set IDs whose feedback drives the optimization. If omitted, defaults to every test set already added to this metric's Lab.

simplified_promptstring

Simplified prompt to use as initial prompt for optimization. If provided, the optimizer will start from this prompt instead of the metric's original description.

skip_evaluationboolean

If true, skip running LLM evaluations on test sets after optimization. This speeds up the process but won't show evaluation results.

Response

Returns progress_id for tracking progress

Changes