judges

Update judge

Partially updates a judge; omitted fields stay unchanged. Returns 409 while a running or paused A/B test uses the judge.

Required permission: judges.update.

patch/v1/projects/{project_id}/judges/{judge_id}

Request

  • Base URL: https://api.connic.co
  • URL: https://api.connic.co/v1/projects/{project_id}/judges/{judge_id}
  • Auth: HTTP bearer

Path parameters

project_idstring uuid required

ID of the project the API key belongs to.

judge_idstring uuid required

Judge ID from GET /judges.

Request body

namestring nullable

Display name, at most 200 characters.

agent_namestring nullable

Agent whose runs the judge evaluates.

modelstring nullable

Model that scores runs, for example a connic/* model. At most 512 characters.

system_promptstring nullable

Extra instructions for the scoring model. At most 8,000 characters.

include_agent_system_promptboolean nullable

Include the evaluated agent's system prompt in the scoring context.

filter_expressionstring nullable

Expression over the run's input, output and context that must be true for automatic evaluation. Empty matches every run.

trigger_mode'automatic' | 'manual' nullable

automatic evaluates runs when they complete; manual evaluates only when triggered.

sample_ratenumber nullable

Share of matching runs evaluated automatically, from 0 to 1.

notification_thresholdnumber nullable

Sends a judge_score_low notification when the average score, as a share of the maximum, drops below this value (0 to 1). null disables it.

notification_windowinteger nullable

Number of most recent evaluations averaged for notification_threshold (1 to 10,000). null averages all.

enabledboolean nullable

When false, runs are not evaluated automatically; on-demand triggers still work.

Response

Successful Response

idstring uuid required
project_idstring uuid required

Project the judge belongs to.

environment_idstring uuid required

Environment whose runs the judge evaluates.

namestring required
agent_namestring required

Agent whose runs the judge evaluates.

modelstring required

Model that scores runs.

system_promptstring required

Extra instructions for the scoring model.

include_agent_system_promptboolean required

Whether the evaluated agent's system prompt is included in the scoring context.

filter_expressionstring required

Expression a run must match to be evaluated automatically. Empty matches every run.

trigger_modestring required

automatic evaluates runs when they complete; manual evaluates only when triggered.

sample_ratenumber required

Share of matching runs evaluated automatically, from 0 to 1.

notification_thresholdnumber nullable

Average score share (0 to 1) below which a judge_score_low notification is sent. null when disabled.

notification_windowinteger nullable

Number of most recent evaluations averaged for notification_threshold. null averages all.

enabledboolean required

When false, runs are not evaluated automatically.

created_atstring date-time required
updated_atstring date-time required
avg_scorenumber nullable

Average overall score of completed evaluations. null before the first one.

avg_max_scorenumber nullable

Average maximum possible score of completed evaluations. null before the first one.

total_evaluatedinteger

Number of completed evaluations.

Changes

No recorded changes to this operation across all 1 revision of this API.