---
title: "Start a custom-rubric evaluation for an adapted dataset"
method: POST
path: "/api/v1/datasets/{dataset_id}/custom-evals"
tags: ["datasets"]
---

# Start a custom-rubric evaluation for an adapted dataset

`POST /api/v1/datasets/{dataset_id}/custom-evals`

## Request

- The document declares no server URL.
- Auth: HTTP bearer
- TypeScript: <https://skmtc.dev/adaptionlabs/apis/adaption-api/docs/api/v1/datasets/:dataset_id/custom-evals/post.ts> (a function with typed inputs; its response is checked against the schema)

## Path parameters

- `dataset_id` string, required

## Request body

- CreateCustomEvalDto
  - `judge_prompt` string, required: The judge rubric. Scored per sample against the original and adapted pairs.
  - `name` string: Optional label so a history of rubrics stays legible.

## Response `202`

- CustomEvalResponseDto
  - `custom_eval_id` string, required
  - `dataset_id` string, required
  - `status` string, required: pending | running | succeeded | failed
  - `judge_prompt` string, required
  - `name` string, nullable, required
  - `score_before` number, nullable, required: Mean rubric score on the original pairs. Null until the eval succeeds.
  - `score_after` number, nullable, required: Mean rubric score on the adapted pairs. Null until the eval succeeds.
  - `improvement_percent` number, nullable, required: Percentage change from score_before to score_after. Null until the eval succeeds.
  - `scored_rows` integer, nullable, required: Rows the judge scored. A rubric score is an average over this sample, not the whole dataset.
  - `error_message` string, nullable, required
  - `created_at` string, date-time, required
  - `completed_at` string, date-time, nullable, required

## Other responses

- `400`: The judge prompt failed rubric validation (`code: invalid_judge_prompt`, with per-issue feedback), or the request body is invalid or the dataset is not ready (generic error).

## Changes

- **2026-10-29** `b38d4dc031db`: 1 info
  - endpoint added
- **2026-09-15** `f4d1edd8f3d1`: 1 breaking
  - api path removed without deprecation

[Operation history](https://skmtc.dev/adaptionlabs/apis/adaption-api/changes/api/v1/datasets/:dataset_id/custom-evals/post.md)

---

[API](https://skmtc.dev/adaptionlabs/apis/adaption-api.md) - [All operations](https://skmtc.dev/adaptionlabs/apis/adaption-api/llms.txt) - [OpenAPI document](https://skmtc.dev/adaptionlabs/apis/adaption-api/revisions/d4cf908c19a0?raw)