---
title: "POST /projects/{projectId}/agent-evaluators/drafts"
method: POST
path: "/projects/{projectId}/agent-evaluators/drafts"
---

# POST /projects/{projectId}/agent-evaluators/drafts

`POST /projects/{projectId}/agent-evaluators/drafts`

## Request

- Base URL: `https://api.caveman.so/api/v1`
- URL: `https://api.caveman.so/api/v1/projects/{projectId}/agent-evaluators/drafts`
- Auth: one of:
  - OAuth 2
    - Authorization code: `/api/v1/oauth/authorize`
    - Token URL: `/api/v1/oauth/token` scopes: `eval:write`
  - HTTP bearer
  - API key in cookie `cave_access`
  - API key in cookie `__Secure-cave_identity.session_token`

POST /api/v1/projects/{projectId}/agent-evaluators/drafts on the Caveman Cloud control plane. Requires the eval:write scope. Exposed to autonomous agents as the draft_evaluator action of the caveman_authoring_write tool (write access). Draft an evaluator or a scenario, list the qualification sets that can qualify a draft, qualify it and request its activation, each bound to the Ask or author session whose case token calls it. A draft is unqualified and not live; a person's connection has no session run and is refused.

## Request body

- object
  - `name` string, required
  - `description` string, required
  - `config` object, required
    - `schema_version` unknown, required
    - `grader_type` 'exact_match' | 'contains' | 'not_contains' | 'regex' | 'not_regex' | 'blocklist' | 'json_schema' | 'json_path_assertion' | 'tool_called' | 'tool_not_called' | 'tool_sequence' | 'tool_argument_assertion' | 'http_status' | 'latency_threshold' | 'cost_threshold' | 'token_threshold' | 'bleu_score' | 'rouge_score' | 'context_f1' | 'no_pii' | 'localization_f1' | 'llm_judge' | 'llm_score' | 'llm_category' | 'llm_pairwise' | 'llm_answer_match', required: One of the platform's grader families; any other name is refused before any effect. Free-text contracts: contains (required facts) and not_contains (forbidden literals). Tool use: tool_called, tool_not_called, tool_sequence, tool_argument_assertion. Spend: cost_threshold, token_threshold, latency_threshold.
    - `options` object, required: The grader family's own options. The draft records them; they are checked when the evaluator is qualified, not here.
    - `input_mapping` object, required: Only candidate, reference and baseline are accepted; any other key is refused.
      - `candidate` object, required: what is graded: {"source":"target","key":"$target"} for the target's own output
        - `source` 'dataset' | 'target', required
        - `key` union, required: a dataset column key, or $target (source target only) for the graded output itself
          - string
          - '$target'
      - `reference` object: optional expected value, usually a dataset column
        - `source` 'dataset' | 'target', required
        - `key` union, required: a dataset column key, or $target (source target only) for the graded output itself
          - string
          - '$target'
      - `baseline` object: optional comparison value for pairwise graders
        - `source` 'dataset' | 'target', required
        - `key` union, required: a dataset column key, or $target (source target only) for the graded output itself
          - string
          - '$target'
  - `evidence_queries` string[], required

## Response `2XX`

Successful response. Exact status depends on synchronous or durable delivery outcome.

## Other responses

- `4XX`: Structured failure. `error.code` is stable; `error.request_id` is echoed in the x-cave-request-id header.
- `5XX`: Structured failure. `error.code` is stable; `error.request_id` is echoed in the x-cave-request-id header.

---

[API](https://skmtc.dev/caveman/apis/caveman-cloud-api.md) - [All operations](https://skmtc.dev/caveman/apis/caveman-cloud-api/llms.txt) - [OpenAPI document](https://skmtc.dev/caveman/apis/caveman-cloud-api/revisions/508312d01cfc?raw)