Sign in
Evaluators

Get evaluator

Returns an evaluator and a resolved version. By default, the latest version is included. Use the version_id query parameter to resolve a specific version.

<Note>This endpoint is in beta, read more here.</Note>

get/v2/evaluators/{evaluator_id}

Path parameters

evaluator_idstring required

A universally unique identifier (base64-encoded opaque string).

Example:RW50aXR5OjEyMzQ1

The unique evaluator identifier (base64)

Query parameters

version_idstring

A universally unique identifier (base64-encoded opaque string).

Example:RW50aXR5OjEyMzQ1

Return the evaluator with this specific version (base64 identifier (base64)). If omitted, returns the latest version.

Response

Returns an evaluator with a resolved version

idstring required

The unique identifier for the evaluator

namestring required

The name of the evaluator

descriptionstring nullable

The description of the evaluator

type'TEMPLATE' | 'CODE' | 'HARNESS' | 'REMOTE' required

The evaluator type:

  • TEMPLATE — LLM-based evaluator.
  • CODE — managed built-in evaluators or custom Python code (both are subtypes of CODE, discriminated by the nested CodeConfig.type = MANAGED | CUSTOM).
  • HARNESS — test harness evaluator.
  • REMOTE — remote evaluator.

Applies to both the parent Evaluator.type field and every version's type discriminator — a version's type must always match its parent evaluator's type.

space_idstring required

The unique identifier for the space the evaluator belongs to

created_atstring date-time required

When the evaluator was created

updated_atstring date-time required

When the evaluator was last updated

created_by_user_idstring nullable required

The unique identifier for the user who created the evaluator

Changes

Changed in 2 of the 16 revisions of this API.12

  • 9263b98aaefb1See the full diff
    • ○

      added the optional property allOf[subschema #2]/version/oneOf[EvaluatorVersionTemplate]/allOf[subschema #2]/template_config/llm_config/invocation_parameters/service_tier to the response with the 200 status

      response-optional-property-added

  • c26a00e9b31811See the full diff
    • ▲

      removed the required property allOf[subschema #2]/version/oneOf[EvaluatorVersionTemplate]/allOf[subschema #2]/template_config/use_function_calling_if_available from the response with the 200 status

      response-required-property-removed

    • ○

      added the optional property allOf[subschema #2]/version/oneOf[EvaluatorVersionTemplate]/allOf[subschema #2]/template_config/use_function_calling to the response with the 200 status

      response-optional-property-added

Of the 16 revisions, 1 has no diff computed.