---
title: "run every case of a dataset as one immutable experiment"
method: POST
path: "/w/{workspace}/ai_evals/experiments/run"
tags: ["ai_evals"]
---

# run every case of a dataset as one immutable experiment

`POST /w/{workspace}/ai_evals/experiments/run`

## Path parameters

- `workspace` string, required

## Request body

- object
  - `dataset` string, required
  - `subject` EvalSubject, required — What an eval run is executed against.
    - `kind` 'agent' | 'agent_draft' | 'agent_version', required — `agent` runs the ai_agent resource as it is deployed when the run opens, `agent_draft` the caller's unsaved edits of it as the editor holds them (carried in `draft`), and `agent_version` one past version named by `version`. The first and last are read server-side; all three are inlined into the run, so every case of a run executes one configuration: a deploy part-way through changes what the next run measures, never this one.
    - `path` string, required — Path of the ai_agent resource.
    - `version` integer, nullable — The agent's per-path version number when the run opened: how many times the resource had been saved, not a resource_version row id. For `agent` and `agent_draft` it names the configuration the run read and every case executed. For `agent_version` it is the request's own, says which version to inline, and is required.
    - `draft` AgentDraft — The brain and tools of an agent, as the flow editor holds them. Carried by the request and present exactly when the subject kind is `agent_draft` — the edits exist only in the editor — where it is the whole definition of what ran: the run goes through the same unlinked branch of the agent executor the editor's own test uses.
      - `input_transforms` object — The agent's input transforms: provider, system prompt, output type and the rest. The message and attachments come from the case and override anything named here.
      - `tools` object[]
    - `draft_hash` string — Hash of the configuration a draft run executed, stamped server-side. A draft moves without the version moving, so this is what dates a run of one. It is also what recognises a draft run whose configuration was later deployed: when it matches the agent as deployed, the run's kind and version are rewritten to that version, once, and the hash is kept as what the resolution rests on.

## Response `200`

id of the created experiment

## Changes

- **2026-08-24** `5afc9f1a93f5` — 1 info
  - endpoint added

[Change history](https://skmtc.dev/windmill/apis/windmill-api/changes/w/:workspace/ai_evals/experiments/run/post.md)

---

[API](https://skmtc.dev/windmill/apis/windmill-api.md) · [All operations](https://skmtc.dev/windmill/apis/windmill-api/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/windmill/windmill-api/revisions/7b281701c340/schema)