GAgalileo
data
prompts
rows

Create Llm Scorer Version

Changed on
post/scorers/{scorer_id}/version/llm

Request

Path parameters

scorer_idstring uuid4 required

Query parameters

organization_idstring uuid4 nullable

Request body

model_namestring nullable
num_judgesinteger nullable
scoreable_node_typesstring[] nullable
cot_enabledboolean nullable
output_type'boolean' | 'categorical' | 'count' | 'discrete' | 'freeform' | 'percentage' | 'multilabel' | 'retrieved_chunk_list_boolean' | 'boolean_multilabel' | 'boolean_probability'

Enumeration of output types.

input_type'basic' | 'llm_spans' | 'retriever_spans' | 'sessions_normalized' | 'sessions_trace_io_only' | 'tool_spans' | 'trace_input_only' | 'trace_io_only' | 'trace_normalized' | 'trace_output_only' | 'agent_spans' | 'workflow_spans'

Enumeration of input types.

instructionsstring nullable
user_promptstring nullable

Response

Successful Response

idstring uuid4 required
versioninteger required
scorer_idstring uuid4 required
model_namestring nullable
num_judgesinteger nullable
cot_enabledboolean nullable
runtimestring nullable

The runtime a code metric version runs on, such as python3.14. Null for a version saved before runtimes were recorded, which runs on the legacy Python 3.12 runtime.

code_sha256string nullable

SHA-256 of the code bytes uploaded for a code metric version. Null for a version saved before digests were recorded.

created_atstring date-time required
updated_atstring date-time required
scoreable_node_typesstring[] nullable
output_type'boolean' | 'categorical' | 'count' | 'discrete' | 'freeform' | 'percentage' | 'multilabel' | 'retrieved_chunk_list_boolean' | 'boolean_multilabel' | 'boolean_probability'

Enumeration of output types.

allowed_modelboolean nullable
created_bystring uuid4 nullable
validation_task_idstring uuid4 nullable
request_idstring nullable

Changes

  • 4cf49e96b632Revision changes
    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

  • 0b20eade6391Revision changes
    • info

      added subschema #2 to the // response property anyOf list for the response status