---
title: "create an eval dataset"
method: POST
path: "/w/{workspace}/ai_evals/datasets/create"
tags: ["ai_evals"]
---

# create an eval dataset

`POST /w/{workspace}/ai_evals/datasets/create`

## Path parameters

- `workspace` string, required

## Request body

- object
  - `path` string, required
  - `summary` string
  - `scorers` Scorer[]
    - `id` string
    - `name` string — Column header. Defaults to the last segment of the path.
    - `pass_if` number — A score at or above this counts as a pass, and the column reports a pass rate beside its mean. Applied when results are read rather than when they are produced, so moving the line re-reads every score already recorded instead of invalidating them.
    - `kind` 'script' | 'agent', required
    - `path` string, required — The script, or the ai_agent resource used as a judge.
  - `cases` NewEvalCase[] — The cases to create the dataset holding, so one can be assembled in a single act rather than created empty and filled in afterwards.
    - `input` EvalCaseInput — The inputs a standalone run feeds the agent.
      - `user_message` string
      - `user_attachments` object[]
    - `expected` unknown

## Response `200`

eval dataset created

## Changes

- **2026-08-24** `5afc9f1a93f5` — 1 info
  - endpoint added

[Change history](https://skmtc.dev/windmill/apis/windmill-api/changes/w/:workspace/ai_evals/datasets/create/post.md)

---

[API](https://skmtc.dev/windmill/apis/windmill-api.md) · [All operations](https://skmtc.dev/windmill/apis/windmill-api/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/windmill/windmill-api/revisions/7b281701c340/schema)