GAgalileo
data
prompts
rows

Create

post/v2/scorers

Request

  • Base URL
    https://api.galileo.ai
  • URL
    https://api.galileo.ai/v2/scorers
  • Auth
    • API key in header Galileo-API-Key
    • or API key in header Splunk-AO-API-Key
    • or OAuth 2
      Password
      Token URL: https://api.galileo.ai/login
    • or HTTP basic

Request body

namestring required
idstring uuid4 nullable
labelstring nullable
descriptionstring nullable
tagsstring[]
scorer_type'llm' | 'code' | 'luna' | 'preset' required
deprecatedboolean nullable
model_type'slm' | 'llm' | 'code'
ground_truthboolean nullable
default_version_idstring uuid4 nullable
user_promptstring nullable
scoreable_node_typesstring[] nullable
output_type'boolean' | 'categorical' | 'count' | 'discrete' | 'freeform' | 'percentage' | 'multilabel' | 'retrieved_chunk_list_boolean' | 'boolean_multilabel'

Enumeration of output types.

input_type'basic' | 'llm_spans' | 'retriever_spans' | 'sessions_normalized' | 'sessions_trace_io_only' | 'tool_spans' | 'trace_input_only' | 'trace_io_only' | 'trace_normalized' | 'trace_output_only' | 'agent_spans' | 'workflow_spans'

Enumeration of input types.

multimodal_capabilitiesMultimodalCapability[] nullable
required_scorersstring[] nullable
required_metric_idsstring[] nullable
roll_up_method'average' | 'sum' | 'max' | 'min' | 'category_count' | 'percentage_true' | 'percentage_false'

Display options for roll up methods when showing rolled up metrics in the UI.

Separates display intent from computation methods. The computation methods (NumericRollUpMethod, CategoricalRollUpMethod) control what aggregations are available. This enum controls how the UI displays the selected roll-up value for a scorer.

is_globalboolean nullable
project_idsstring[]

Example request

{
  "defaults": {
    "filters": [
      {
        "value": "ENUM_VALUE"
      }
    ]
  }
}

Response

Successful Response

idstring uuid4 required
namestring required
scorer_type'llm' | 'code' | 'luna' | 'preset' required
model_type'slm' | 'llm' | 'code'
ground_truthboolean nullable
default_version_idstring uuid4 nullable
user_promptstring nullable
scoreable_node_typesstring[] nullable
output_type'boolean' | 'categorical' | 'count' | 'discrete' | 'freeform' | 'percentage' | 'multilabel' | 'retrieved_chunk_list_boolean' | 'boolean_multilabel'

Enumeration of output types.

input_type'basic' | 'llm_spans' | 'retriever_spans' | 'sessions_normalized' | 'sessions_trace_io_only' | 'tool_spans' | 'trace_input_only' | 'trace_io_only' | 'trace_normalized' | 'trace_output_only' | 'agent_spans' | 'workflow_spans'

Enumeration of input types.

multimodal_capabilitiesMultimodalCapability[] nullable
required_scorersstring[] nullable
required_metric_idsstring[] nullable
deprecatedboolean nullable
roll_up_method'average' | 'sum' | 'max' | 'min' | 'category_count' | 'percentage_true' | 'percentage_false'

Display options for roll up methods when showing rolled up metrics in the UI.

Separates display intent from computation methods. The computation methods (NumericRollUpMethod, CategoricalRollUpMethod) control what aggregations are available. This enum controls how the UI displays the selected roll-up value for a scorer.

labelstring nullable
tagsstring[] required
included_fieldsstring[]

Fields that can be used in the scorer to configure it. i.e. model, num_judges, etc. This enables the ui to know which fields a user can configure when they're setting a scorer

descriptionstring nullable
created_bystring uuid4 nullable
created_atstring date-time nullable
updated_atstring date-time nullable
metric_namestring nullable
is_globalboolean

Example response

{
  "defaults": {
    "filters": [
      {
        "value": "ENUM_VALUE"
      }
    ]
  }
}

Changes

No recorded changes to this operation across all 1 revision of this API.