---
title: "Evaluate"
method: POST
path: "/v1/evaluate"
tags: ["AI"]
---

# Evaluate

`POST /v1/evaluate`

Evaluate unstructured `state` against a map of typed System One questions
(noul / choice / score). Returns structured answers with calibrated
probabilities and confidence. Chat completions are not supported for these
models — use this endpoint instead of `/v1/chat/completions`.

## Request body

- EvaluateRequest — Request body for `POST /v1/evaluate` and provider System One calls.
  - `model` string, required — Spicepod model name (runtime) or provider model id (provider forward).
  - `questions` object, required
  - `state` union, required — Evaluation `state`: string, object, or array (not bool/number/null).
    - string
    - unknown[]
      - unknown
    - object

## Response `200`

Evaluation succeeded

- EvaluateResponse — Response body for evaluation: typed answers plus provider metadata.
  - `answers` object, required
  - `model` string, required — Versioned model id that answered (e.g. `jev-1.13.0`), when the provider reports it.
  - `usage` Usage — Token usage reported by the provider.
    - `input_tokens` integer — Defaulted: the provider may report one count without the other, and a partial `usage` block must not fail an otherwise successful evaluation.
    - `output_tokens` integer

## Other responses

- `400` — Invalid request
- `401` — Upstream authentication failed
- `403` — Upstream permission denied
- `404` — Model not found
- `422` — Malformed JSON request body (Axum Json extractor)
- `429` — Rate limited
- `500` — Evaluation failed
- `503` — Upstream provider unavailable

## Changes

- **2026-09-22** `899fb3203a1b` — 1 info
  - endpoint added

[Change history](https://skmtc.dev/spiceai/apis/runtime/changes/v1/evaluate/post.md)

---

[API](https://skmtc.dev/spiceai/apis/runtime.md) · [All operations](https://skmtc.dev/spiceai/apis/runtime/llms.txt) · [OpenAPI document](https://skmtc.dev/spiceai/apis/runtime/revisions/899fb3203a1b?raw)