---
title: "Run an evaluator"
method: POST
path: "/api/evaluators/{evaluator_id}/run/"
tags: ["evaluators"]
---

# Run an evaluator

`POST /api/evaluators/{evaluator_id}/run/`

Run an evaluator against raw unified inputs. The evaluator ID may include a version suffix such as `evl_abc123:2` to run a specific version.

## Path parameters

- `evaluator_id` string, required

## Headers

- `Authorization` string, required

## Request body

- object
  - `inputs` ApiEvaluatorsEvaluatorIdRunPostRequestBodyContentApplicationJsonSchemaInputs, required — Unified evaluator inputs.
    - `input` unknown
    - `output` unknown
    - `expected_output` unknown
    - `metrics` ApiEvaluatorsEvaluatorIdRunPostRequestBodyContentApplicationJsonSchemaInputsMetrics — Runtime metrics such as cost, latency, and token counts.
    - `metadata` ApiEvaluatorsEvaluatorIdRunPostRequestBodyContentApplicationJsonSchemaInputsMetadata — Request metadata such as model, environment, trace IDs, or customer identifiers.
    - `llm_input` string — Legacy alias for `input`.
    - `llm_output` string — Legacy alias for `output`.
  - `generation_method` 'auto' | 'llm' | 'code' — Optional method override for evaluators that support multiple execution modes.
  - `evaluation_id` string — Legacy evaluator ID field. Prefer the path parameter or `evaluator_id`.
  - `evaluator_id` string — Optional evaluator ID override. Supports version suffixes such as `evl_abc123:2`.

## Response `200`

Evaluation result.

- EvaluatorsRunEvaluatorResponse200
  - `id` string — Score/result ID.
  - `created_at` string, date-time
  - `type` string
  - `environment` string
  - `numerical_value` number, double, nullable
  - `string_value` string, nullable
  - `boolean_value` boolean, nullable
  - `categorical_value` string[], nullable
  - `json_value` string, nullable
  - `is_passed` boolean, nullable
  - `cost` number, double, nullable
  - `evaluator_id` string
  - `evaluator_slug` string
  - `scorer` string, nullable
  - `log_id` string, nullable
  - `prompt_id` string, nullable
  - `prompt_version_number` integer, nullable
  - `dataset_id` string, nullable
  - `automation_id` string, nullable
  - `status` string, nullable
  - `error_message` string, nullable
  - `inputs` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaInputs — Unified evaluator inputs.
    - `input` unknown
    - `output` unknown
    - `expected_output` unknown
    - `metrics` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaInputsMetrics — Runtime metrics such as cost, latency, and token counts.
    - `metadata` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaInputsMetadata — Request metadata such as model, environment, trace IDs, or customer identifiers.
    - `llm_input` string — Legacy alias for `input`.
    - `llm_output` string — Legacy alias for `output`.
  - `evaluator` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaEvaluator
    - `id` string, required — Stable evaluator ID shared by all versions.
    - `version_id` string — Unique ID for this evaluator version.
    - `version` integer — Version number. The first draft is version 0.
    - `is_read_only` boolean — Whether this version is committed and immutable.
    - `version_description` string — Commit message for this version.
    - `name` string, required
    - `evaluator_slug` string — Organization-scoped evaluator slug.
    - `type` 'llm' | 'human' | 'code', required
    - `score_value_type` 'numerical' | 'boolean' | 'percentage' | 'single_select' | 'multi_select' | 'text' | 'json' | 'comment' | 'categorical', required
    - `eval_class` 'keywordsai_custom_llm' | 'custom_code' — Optional pre-built evaluator template.
    - `description` string, nullable
    - `score_config` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaEvaluatorScoreConfig — Score configuration. For numerical/percentage scores, use `min_score` and `max_score`. For select scores, use `choices`.
      - `min_score` number, double
      - `max_score` number, double
      - `choices` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaEvaluatorScoreConfigChoicesItems[]
        - `name` string, required
        - `value` union, required
          - string
          - number, double
          - boolean
    - `passing_conditions` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaEvaluatorPassingConditions — Passing conditions in the standard Respan filter format.
    - `llm_config` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaEvaluatorLlmConfig — LLM grader configuration. The backend validates this against the selected evaluator form.
      - `model` string
      - `evaluator_definition` string — Prompt/template used by the LLM grader.
      - `scoring_rubric` string
      - `temperature` number, double
      - `max_tokens` integer
      - `top_p` number, double
      - `frequency_penalty` number, double
      - `presence_penalty` number, double
      - `stop` string[]
      - `response_format` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaEvaluatorLlmConfigResponseFormat
      - `tools` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaEvaluatorLlmConfigToolsItems[]
      - `tool_choice` unknown
      - `verbosity` string
    - `code_config` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaEvaluatorCodeConfig — Code grader configuration.
      - `eval_code_snippet` string — Python code defining `main(eval_inputs)`.
    - `configurations` object — Legacy user-facing configuration object. New clients should prefer `llm_config`, `code_config`, `score_config`, and `passing_conditions`.
    - `categorical_choices` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaEvaluatorCategoricalChoicesItems[], nullable
      - `name` string, required
      - `value` union, required
        - string
        - number, double
        - boolean
    - `starred` boolean
    - `created_at` string, date-time
    - `updated_at` string, date-time
    - `created_by` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaEvaluatorCreatedBy
      - `id` integer
      - `first_name` string
      - `last_name` string
      - `email` string, email
    - `updated_by` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaEvaluatorUpdatedBy
      - `id` integer
      - `first_name` string
      - `last_name` string
      - `email` string, email
    - `editor` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaEvaluatorEditor
      - `id` integer
      - `first_name` string
      - `last_name` string
      - `email` string, email
    - `tags` ApiEvaluatorsEvaluatorIdRunPostResponsesContentApplicationJsonSchemaEvaluatorTagsItems[]
      - `id` integer
      - `name` string
      - `color` string

## Other responses

- `400` — Bad Request
- `401` — Unauthorized - Missing or invalid authentication
- `404` — Not Found

## Changes

- **2026-08-13** `4e064cf81dae` — 27 info
  - added optional request body
  - added the non-success response with the status `400`
  - added the non-success response with the status `401`
  - added the non-success response with the status `404`
  - …23 more
- **2026-07-31** `c26d550029f8` — 1 breaking, 23 warning, 3 info
  - removed the request body
  - removed the optional property `automation_id` from the response with the `200` status
  - removed the optional property `boolean_value` from the response with the `200` status
  - removed the optional property `categorical_value` from the response with the `200` status
  - …23 more

[Change history](https://skmtc.dev/keywordsai/apis/api-reference/changes/api/evaluators/:evaluator_id/run/post.md)

---

[API](https://skmtc.dev/keywordsai/apis/api-reference.md) · [All operations](https://skmtc.dev/keywordsai/apis/api-reference/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/keywordsai/api-reference/revisions/c61efaf24756/schema)