data

prompts

rows

Update

patch/public/v2/scorers/{scorer_id}

Path parameters

scorer_idstring uuid4 required

Request body

namestring nullable

descriptionstring nullable

tagsstring[] nullable

model_type'slm' | 'llm' | 'code'

ground_truthboolean nullable

default_version_idstring uuid4 nullable

user_promptstring nullable

scoreable_node_typesstring[] nullable

Enumeration of output types.

Enumeration of input types.

multimodal_capabilitiesMultimodalCapability[] nullable

roll_up_method'average' | 'sum' | 'max' | 'min' | 'category_count' | 'percentage_true' | 'percentage_false'

Display options for roll up methods when showing rolled up metrics in the UI.

Separates display intent from computation methods. The computation methods (NumericRollUpMethod, CategoricalRollUpMethod) control what aggregations are available. This enum controls how the UI displays the selected roll-up value for a scorer.

Example request

{
  "defaults": {
    "filters": [
      {
        "value": "ENUM_VALUE"
      }
    ]
  }
}

Response

Successful Response

idstring uuid4 required

namestring required

labelstring nullable

scorer_type'llm' | 'code' | 'luna' | 'preset' required

model_type'slm' | 'llm' | 'code'

ground_truthboolean nullable

default_version_idstring uuid4 nullable

user_promptstring nullable

scoreable_node_typesstring[] nullable

Enumeration of output types.

Enumeration of input types.

multimodal_capabilitiesMultimodalCapability[] nullable

required_scorersstring[] nullable

required_metric_idsstring[] nullable

deprecatedboolean nullable

roll_up_method'average' | 'sum' | 'max' | 'min' | 'category_count' | 'percentage_true' | 'percentage_false'

Display options for roll up methods when showing rolled up metrics in the UI.

tagsstring[] required

included_fieldsstring[]

Fields that can be used in the scorer to configure it. i.e. model, num_judges, etc. This enables the ui to know which fields a user can configure when they're setting a scorer

descriptionstring nullable

created_bystring uuid4 nullable

created_atstring date-time nullable

updated_atstring date-time nullable

metric_namestring nullable

is_globalboolean

Example response

{
  "defaults": {
    "filters": [
      {
        "value": "ENUM_VALUE"
      }
    ]
  }
}

Changes

Changed in 7 of the 15 revisions of this API.6217

Sep 2, 20264bc0315072465See the full diff
- ●
  added the new change_role_to_capability_based enum value to the /// response property for the response status
  response-property-enum-value-added
- ●
  added the new create_global enum value to the /// response property for the response status
  response-property-enum-value-added
- ●
  added the new create_shared enum value to the /// response property for the response status
  response-property-enum-value-added
- ●
  added the new unsafe_output enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new unsafe_output enum value to the ///// response property for the response status
  response-property-enum-value-added
This revision also has 4 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog
Aug 24, 2026a0fa505ce96711See the full diff
- ●
  added the new read_unredacted_data enum value to the /// response property for the response status
  response-property-enum-value-added
- ○
  added the optional property to the response with the status
  response-optional-property-added
Aug 22, 202683e6317250381See the full diff
- ●
  added the new read_ai_usage_organization enum value to the /// response property for the response status
  response-property-enum-value-added
This revision also has 15 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog
Aug 13, 2026344acdc36dbb4See the full diff
- ○
  added to the //// response property anyOf list for the response status
  response-property-any-of-added
- ○
  added to the //// response property anyOf list for the response status
  response-property-any-of-added
- ○
  removed from the //// response property anyOf list for the response status
  response-property-any-of-removed
- ○
  removed from the //// response property anyOf list for the response status
  response-property-any-of-removed
This revision also has 44 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog
Aug 6, 2026933e8c8e636628See the full diff
- ●
  added the new agent_flow_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new agent_flow_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new agent_flow_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new agent_flow_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new conversation_quality_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new conversation_quality_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new conversation_quality_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new conversation_quality_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new input_sexist_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new input_sexist_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new input_sexist_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new input_sexist_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new instruction_adherence_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new instruction_adherence_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new instruction_adherence_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new instruction_adherence_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new output_sexist_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new output_sexist_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new output_sexist_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new output_sexist_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new prompt_injection_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new prompt_injection_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new prompt_injection_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new prompt_injection_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new tool_selection_quality_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new tool_selection_quality_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new tool_selection_quality_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new tool_selection_quality_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
This revision also has 8 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog
Aug 5, 20261f7a6a4a81ad63See the full diff
- ●
  removed the optional property // from the response with the status
  response-optional-property-removed
- ●
  removed the optional property // from the response with the status
  response-optional-property-removed
- ●
  removed the optional property // from the response with the status
  response-optional-property-removed
- ●
  removed the optional property // from the response with the status
  response-optional-property-removed
- ●
  removed the optional property // from the response with the status
  response-optional-property-removed
- ●
  removed the optional property // from the response with the status
  response-optional-property-removed
- ○
  added the new optional request property //
  new-optional-request-property
- ○
  added the optional property // to the response with the status
  response-optional-property-added
- ○
  added the optional property to the response with the status
  response-optional-property-added
This revision also has 2 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog
Jul 26, 202605003e7835a2219See the full diff
- ●
  added the new action_completion_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new action_completion_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new action_completion_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new action_completion_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new correctness_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new correctness_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new correctness_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new correctness_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new ground_truth_adherence_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new ground_truth_adherence_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new ground_truth_adherence_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new ground_truth_adherence_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new read_ai_usage enum value to the /// response property for the response status
  response-property-enum-value-added
- ●
  added the new reasoning_coherence_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new reasoning_coherence_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new reasoning_coherence_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new reasoning_coherence_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new user_intent_change_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new user_intent_change_audio enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new user_intent_change_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ●
  added the new user_intent_change_vision enum value to the ///// response property for the response status
  response-property-enum-value-added
- ○
  the endpoint scheme security ClassicAPIKeyHeader was added to the API
  api-security-added
- ○
  removed the bleu enum value from the ///// response property for the response status
  response-property-enum-value-removed
- ○
  removed the bleu enum value from the ///// response property for the response status
  response-property-enum-value-removed
- ○
  removed the prompt_perplexity enum value from the ///// response property for the response status
  response-property-enum-value-removed
- ○
  removed the prompt_perplexity enum value from the ///// response property for the response status
  response-property-enum-value-removed
- ○
  removed the rouge enum value from the ///// response property for the response status
  response-property-enum-value-removed
- ○
  removed the rouge enum value from the ///// response property for the response status
  response-property-enum-value-removed
- ○
  removed the uncertainty enum value from the ///// response property for the response status
  response-property-enum-value-removed
- ○
  removed the uncertainty enum value from the ///// response property for the response status
  response-property-enum-value-removed
This revision also has 15 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

Full history