---
title: "List eval runs"
method: GET
path: "/v1/projects/{project_id}/evals/{eval_id}/runs"
tags: ["Evaluations"]
---

# List eval runs

`GET /v1/projects/{project_id}/evals/{eval_id}/runs`

## Request

- Base URL: `https://api.naturali.ai`
- URL: `https://api.naturali.ai/v1/projects/{project_id}/evals/{eval_id}/runs`
- Auth: one of:
  - HTTP bearer
  - OAuth 2 (authorization code, authorization URL `https://api.naturali.ai/authorize`, token URL `https://api.naturali.ai/token`, scopes: `mcp:access`)

Returns an eval's runs, newest first

## Path parameters

- `eval_id` string, required

## Query parameters

- `limit` integer
- `offset` integer

## Response `200`

List of eval runs

- object
  - `data` EvalRun[], required
    - `id` string
    - `eval_id` string
    - `agent_version` integer: The one agent version every item in this run executed against
    - `decider_versions` object, nullable: The decider version each `decider` scorer grades under, keyed by scorer name and resolved when the run started. Null when the eval has no decider scorer.
    - `status` 'queued' | 'running' | 'completed' | 'failed' | 'cancelled'
    - `baseline_run_id` string, nullable
    - `trigger_id` string, nullable: The trigger that started this run — set when a schedule (or a manual trigger fire) started it, null for a run started through this API. Kept even if the trigger is later deleted.
    - `aggregate_scores` AggregateScores, nullable: Per-scorer rollup plus the run-level pass rate; null until the run is terminal
      - `scorers` object
      - `pass_rate` number, nullable: Passed items over non-errored items; null when nothing was scorable
      - `pass_rate_interval` PassRateInterval, nullable: 95% Wilson score interval around `pass_rate`: the range the pass rate of this agent over items like these plausibly spans, given how many items were scored. Null exactly when `pass_rate` is.
        - `low` number
        - `high` number
        - `level` number: The confidence level, always 0.95
      - `scored_item_count` integer: Items that produced a score — errored items are excluded
      - `baseline` BaselineComparison, nullable: Comparison against the run named by `baseline_run_id`; absent when the run named none. Every number here is computed over the **item intersection** — the dataset items present and scorable in both runs — because a delta only means something when both sides answered the same question. The compared/added/removed counts make any dataset drift visible instead of letting it read as agent regression. Positive deltas mean this run scored higher than the baseline.
        - `run_id` string
        - `compared_item_count` integer: Items scorable in both runs — the basis of every delta
        - `added_item_count` integer: Scorable here but not in the baseline (added, or errored there)
        - `removed_item_count` integer: Scorable in the baseline but not here (removed, or errored here)
        - `pass_rate_delta` number, nullable: Run-level pass-rate delta over the intersection; null when the two runs share no comparable item
        - `flipped` FlippedItems: Compared items that changed verdict. `pass_rate_delta` is `(improved - regressed) / compared_item_count`.
          - `improved` integer: Passed in this run, failed in the baseline
          - `regressed` integer: Failed in this run, passed in the baseline
        - `p_value` number, nullable: Exact McNemar p-value of the `flipped` split: the probability of a split at least this uneven were an item equally likely to flip either way, which is what an unchanged agent produces. `1` when no item flipped; null when the runs share no comparable item.
        - `scorers` object: Per-scorer deltas, keyed by scorer type. A scorer only one of the two runs ran is omitted rather than compared against nothing.
        - `grouping` BaselineGrouping: The baseline comparison per group; absent when the eval declares no `group_by`. Both runs' items are grouped by the same labels, so an item relabelled between the runs is compared within one group.
          - `group_by` string
          - `groups` object
          - `ungrouped_item_count` integer: Items scorable in both runs that name no group
      - `grouping` RunGrouping: Scores rolled up per value of the eval's `group_by` key; absent when the eval declares none. An item belongs to the group its `metadata` names under that key, read when the run settles; only a string names a group. Errored items count in no group.
        - `group_by` string: The metadata key the groups were read from
        - `groups` object: Group value → that group's figures
        - `ungrouped_item_count` integer: Scored items whose `metadata` holds no string under `group_by`, or whose dataset item was deleted
    - `passed` boolean, nullable: Null when the eval declares no pass_threshold, and until the run is terminal
    - `item_count` integer
    - `completed_count` integer
    - `errored_count` integer
    - `metadata` MetadataBag: Caller-owned annotations on a resource, stored as the object they were written as: the types a value was written with are the types a read returns, so a filter can ask an ordering question about a number. Unlike other body fields, keys are stored and returned verbatim in the casing supplied — they are not converted between snake_case and camelCase. No key is reserved, and that is the point: every piece of state the platform owns lives in its own typed column, so nothing written here reaches platform state. The platform never reads the bag — it is not an IAM context, not a policy input and not part of a prompt — which is what separates it from a tag bag.
    - `started_at` string, date-time, nullable
    - `finished_at` string, date-time, nullable
    - `created_at` string, date-time
  - `total` integer, required
  - `limit` integer, required
  - `offset` integer, required

## Other responses

- `401`: Unauthorized
- `403`: Forbidden
- `404`: Eval not found

---

[API](https://skmtc.dev/naturali/apis/naturali-ai-api.md) · [All operations](https://skmtc.dev/naturali/apis/naturali-ai-api/llms.txt) · [OpenAPI document](https://skmtc.dev/naturali/apis/naturali-ai-api/revisions/e4905b78f422?raw)