Sign in
Evaluators

Create evaluator version

Endpoint: POST /v2/evaluators/{evaluator_id}/versions

Create a new version of an existing evaluator. The new version becomes the latest version immediately (versioning is append-only).

Payload Requirements

  • commit_message describes the changes in this version.
  • Provide either template_config or code_config to match the evaluator's type. code_config.type is a separate inner discriminator (MANAGED or CUSTOM) and is unrelated to the top-level type. Schema and constraints match Create Evaluator.
  • For a template version, template_config.llm_config.ai_integration_id must reference an AI integration that exists and is accessible to the evaluator's space; otherwise the request fails with 404.

Responses

  • 201 — version created; returns the new EvaluatorVersion.
  • 400 — malformed request: evaluator_id fails ID-format validation, the request body fails schema validation (e.g. malformed JSON), or type/config mismatch a documented invalid shape.
  • 401 — missing or invalid credentials.
  • 403 — the evaluator is readable but the caller lacks permission to create a version on it.
  • 404 — evaluator_id does not exist or is not readable by the caller (Evaluator not found), or template_config.llm_config.ai_integration_id does not exist or is not accessible to this space (LLM integration not found or not accessible to this space).
  • 422 — the body is well-formed JSON but fails business validation (e.g. missing commit_message, invalid template column name).
  • 429 — rate limit exceeded.

Valid example (template version)

{
  "commit_message": "Improve prompt template for better accuracy",
  "template_config": {
    "name": "hallucination",
    "template": "Given the input: {input}\nand output: {output}\nIs the output a hallucination? Explain your reasoning.",
    "include_explanations": true,
    "use_function_calling": true,
    "classification_choices": {"hallucinated": 0, "factual": 1},
    "llm_config": {
      "ai_integration_id": "TGxtSW50ZWdyYXRpb246MTI6YUJjRA==",
      "model_name": "gpt-4o",
      "invocation_parameters": {"temperature": 0},
      "provider_parameters": {}
    }
  }
}

Invalid example (missing required commit_message)

{
  "template_config": {
    "name": "hallucination",
    "template": "Is this a hallucination?",
    "include_explanations": false,
    "use_function_calling": false,
    "llm_config": {
      "ai_integration_id": "TGxtSW50ZWdyYXRpb246MTI6YUJjRA==",
      "model_name": "gpt-4o",
      "invocation_parameters": {},
      "provider_parameters": {}
    }
  }
}

Response 422:

{
  "status": 422,
  "title": "Unprocessable Entity",
  "type": "https://arize.com/docs/ax/rest-reference/errors#validation-error",
  "detail": "Invalid input"
}

Invalid example (ai_integration_id does not exist or is not accessible to this space)

{
  "commit_message": "Try a nonexistent integration",
  "template_config": {
    "name": "hallucination",
    "template": "Given {input} and {output}, is it a hallucination?",
    "include_explanations": true,
    "use_function_calling_if_available": true,
    "classification_choices": {"hallucinated": 0, "factual": 1},
    "llm_config": {
      "ai_integration_id": "TGxtSW50ZWdyYXRpb246OTk5OTk6ZmFrZQ==",
      "model_name": "gpt-4o",
      "invocation_parameters": {},
      "provider_parameters": {}
    }
  }
}

Response 404:

{
  "status": 404,
  "title": "Not Found",
  "type": "https://arize.com/docs/ax/rest-reference/errors#resource-not-found",
  "detail": "LLM integration not found or not accessible to this space"
}

<Note>This endpoint is in beta, read more here.</Note>

post/v2/evaluators/{evaluator_id}/versions

Path parameters

evaluator_idstring required

A universally unique identifier (base64-encoded opaque string).

Example:RW50aXR5OjEyMzQ1

The unique evaluator identifier (base64)

Request body

OR

Response

Returns an evaluator version

OR
OR
OR

Changes

Changed in 4 of the 16 revisions of this API.867

  • 9263b98aaefb1See the full diff
    • ○

      added the optional property oneOf[EvaluatorVersionTemplate]/allOf[subschema #2]/template_config/llm_config/invocation_parameters/service_tier to the response with the 201 status

      response-optional-property-added

  • c26a00e9b318211See the full diff
    • ▲

      added the new required request property oneOf[CreateTemplateEvaluatorVersionRequest]/template_config/use_function_calling

      new-required-request-property

    • ▲

      removed the required property oneOf[EvaluatorVersionTemplate]/allOf[subschema #2]/template_config/use_function_calling_if_available from the response with the 201 status

      response-required-property-removed

    • ●

      removed the request property oneOf[CreateTemplateEvaluatorVersionRequest]/template_config/use_function_calling_if_available

      request-property-removed

    • ○

      added the optional property oneOf[EvaluatorVersionTemplate]/allOf[subschema #2]/template_config/use_function_calling to the response with the 201 status

      response-optional-property-added

  • 4d95080d8ffb11See the full diff
    • ▲

      added ToolConfigRequest to the oneOf[CreateTemplateEvaluatorVersionRequest]/template_config/llm_config/invocation_parameters/tool_config request property allOf list

      request-property-all-of-added

    • ●

      removed ToolConfig from the oneOf[CreateTemplateEvaluatorVersionRequest]/template_config/llm_config/invocation_parameters/tool_config request property allOf list

      request-property-all-of-removed

  • 1e87d8a4cf69545See the full diff
    • ▲

      added AnthropicHeadersRequest to the oneOf[CreateTemplateEvaluatorVersionRequest]/template_config/llm_config/provider_parameters/anthropic_headers request property allOf list

      request-property-all-of-added

    • ▲

      added AzureParamsRequest to the oneOf[CreateTemplateEvaluatorVersionRequest]/template_config/llm_config/provider_parameters/azure_params request property allOf list

      request-property-all-of-added

    • ▲

      added BedrockOptionsRequest to the oneOf[CreateTemplateEvaluatorVersionRequest]/template_config/llm_config/provider_parameters/bedrock_options request property allOf list

      request-property-all-of-added

    • ▲

      added ResponseFormatRequest to the oneOf[CreateTemplateEvaluatorVersionRequest]/template_config/llm_config/invocation_parameters/response_format request property allOf list

      request-property-all-of-added

    • ▲

      removed ManagedCodeConfig CustomCodeConfig from the oneOf[CreateCodeEvaluatorVersionRequest]/code_config request property oneOf list

      request-property-one-of-removed

    • ●

      removed AnthropicHeaders from the oneOf[CreateTemplateEvaluatorVersionRequest]/template_config/llm_config/provider_parameters/anthropic_headers request property allOf list

      request-property-all-of-removed

    • ●

      removed AzureParams from the oneOf[CreateTemplateEvaluatorVersionRequest]/template_config/llm_config/provider_parameters/azure_params request property allOf list

      request-property-all-of-removed

    • ●

      removed BedrockOptions from the oneOf[CreateTemplateEvaluatorVersionRequest]/template_config/llm_config/provider_parameters/bedrock_options request property allOf list

      request-property-all-of-removed

    • ●

      removed ResponseFormat from the oneOf[CreateTemplateEvaluatorVersionRequest]/template_config/llm_config/invocation_parameters/response_format request property allOf list

      request-property-all-of-removed

    • ○

      the request property oneOf[CreateTemplateEvaluatorVersionRequest]/template_config/classification_choices became nullable

      request-property-became-nullable

    • ○

      the request property oneOf[CreateTemplateEvaluatorVersionRequest]/template_config/classification_choices became optional

      request-property-became-optional

    • ○

      mapped value for discriminator key CUSTOM changed from CustomCodeConfig to CustomCodeConfigRequest for oneOf[CreateCodeEvaluatorVersionRequest]/code_config request property

      request-property-discriminator-mapping-changed

    • ○

      mapped value for discriminator key MANAGED changed from ManagedCodeConfig to ManagedCodeConfigRequest for oneOf[CreateCodeEvaluatorVersionRequest]/code_config request property

      request-property-discriminator-mapping-changed

    • ○

      added ManagedCodeConfigRequest CustomCodeConfigRequest to the oneOf[CreateCodeEvaluatorVersionRequest]/code_config request property oneOf list

      request-property-one-of-added

Of the 16 revisions, 1 has no diff computed.