---
title: "Create Evaluation Suite"
method: POST
path: "/projects/{project_id}/evaluation-suites"
tags: ["projects"]
---

# Create Evaluation Suite

`POST /projects/{project_id}/evaluation-suites`

Materialize and pin a project Evaluation Suite from one supported source.

Args:
    project_id: Owning project.
    request: Source, canonicalization, and scorer configuration.
    auth: Authenticated requester.

Returns:
    Project-owned immutable Evaluation Suite.

Raises:
    HTTPException: If source cases are invalid or project access is denied.

## Path parameters

- `project_id` string, required

## Request body

- EvaluationSuiteCreate — Create a project-owned suite from an existing dataset or Hugging Face.
  - `name` string, required
  - `source_kind` 'agent_generated' | 'huggingface' | 'platform_catalog' | 'upload', required
  - `dataset_id` string, nullable
  - `agent_run_id` string, nullable
  - `catalog_key` string, nullable
  - `huggingface` HuggingFaceSuiteSource — Pinned Hugging Face dataset source used to materialize suite cases.
    - `repo_id` string, required
    - `revision` string, nullable
    - `config` string, nullable
    - `split` string
    - `token` string, password, nullable — Read token used only for import; it is never persisted.
  - `task_type` string, required
  - `field_mapping` object
  - `scorer_kind` 'harness' | 'llmaj' | 'native'
  - `scorer_key` string — Reviewed scorer key. Native suites accept native.v1, plus exact_match for decoder tasks; unsupported native task/key pairs are rejected. LLMAJ and harness scorers retain their own reviewed string namespaces, so this field cannot be a global enum.
  - `harness` HarnessScorerProgram — Content-addressed benchmark program executed in the agent sandbox.
    - `scorer_id` string, required
    - `version` string, required
    - `source_digest` string, required
    - `dependency_digest` string, required
    - `entrypoint` string, required
    - `runtime_configuration` JsonObjectInput
    - `metric_names` string[], required
    - `declared_egress` string[]
  - `max_cases` integer, nullable

## Response `200`

Successful Response

- EvaluationSuiteResponse — A project-owned pinned Evaluation Suite.
  - `id` string, required
  - `project_id` string, required
  - `dataset_id` string, required
  - `name` string, required
  - `task_type` string, required
  - `source_kind` 'agent_generated' | 'huggingface' | 'platform_catalog' | 'upload', required
  - `source_provenance` JsonObjectOutput, required
  - `field_mapping` JsonObjectOutput, required
  - `scorer_kind` 'harness' | 'llmaj' | 'native', required
  - `scorer_config` JsonObjectOutput, required
  - `case_count` integer, required
  - `created_at` string, date-time, required
  - `updated_at` string, date-time, required

## Other responses

- `404` — Project not found.
- `422` — Validation Error

## Changes

- **2026-09-24** `1cffaad2a921` — 1 info
  - endpoint added

[Change history](https://skmtc.dev/pioneer/apis/brain-api/changes/projects/:project_id/evaluation-suites/post.md)

---

[API](https://skmtc.dev/pioneer/apis/brain-api.md) · [All operations](https://skmtc.dev/pioneer/apis/brain-api/llms.txt) · [OpenAPI document](https://skmtc.dev/pioneer/apis/brain-api/revisions/1cffaad2a921?raw)