---
title: "Changes to Create eval"
method: POST
path: "/evals"
---

# Changes to Create eval

`POST /evals`

> Every recorded change to this operation, newest first.

## Timeline

Changed in 14 of 397 revisions.

- **2026-10-26** `6f5c52c2ff04`: 8 info
- **2026-10-06** `e2f8d8cbc41f`: 2 info
- **2026-09-25** `d86203de41ae`: 1 info
- **2026-09-08** `bb9b3b410112`: 1 info
- **2026-09-03** `9430b00d75b4`: 1 info
- **2026-08-14** `6ac2618eefd7`: 1 info
- **2026-08-05** `ab0c5306e390`: 1 warning, 10 info
- **2026-07-22** `0e6756eca8e0`: 1 info
- **2026-07-15** `4065aec90c5d`: 1 warning, 9 info
- **2026-07-09** `e0508a503b84`: 1 warning, 9 info
- **2026-05-13** `74cbcf73838f`: 7 breaking, 6 info
- **2026-04-29** `5b98524f8dee`: 4 breaking
- **2026-04-10** `7c540cce6eb3`: 4 breaking, 11 info
- **2025-04-29** `6a6c681b1820`: 1 info

## Changes

- **2026-10-26** `6f5c52c2ff04`: 8 info
  - added the new optional request property `testing_criteria/items/oneOf[subschema #1: LabelModelGrader]/input/items/oneOf[subschema #2: Eval message object]/content/oneOf[subschema #1: Eval content item]/oneOf[subschema #2: Input text]/prompt_cache_breakpoint/ttl`
  - added the new optional request property `testing_criteria/items/oneOf[subschema #1: LabelModelGrader]/input/items/oneOf[subschema #2: Eval message object]/content/oneOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/oneOf[subschema #2: Input text]/prompt_cache_breakpoint/ttl`
  - added the new optional request property `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/input/items/content/oneOf[subschema #1: Eval content item]/oneOf[subschema #2: Input text]/prompt_cache_breakpoint/ttl`
  - added the new optional request property `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/input/items/content/oneOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/oneOf[subschema #2: Input text]/prompt_cache_breakpoint/ttl`
  - added the optional property `testing_criteria/items/oneOf[subschema #1: LabelModelGrader]/allOf[subschema #1: LabelModelGrader]/input/items/content/oneOf[subschema #1: Eval content item]/oneOf[subschema #2: Input text]/prompt_cache_breakpoint/ttl` to the response with the `201` status
  - added the optional property `testing_criteria/items/oneOf[subschema #1: LabelModelGrader]/allOf[subschema #1: LabelModelGrader]/input/items/content/oneOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/oneOf[subschema #2: Input text]/prompt_cache_breakpoint/ttl` to the response with the `201` status
  - added the optional property `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/input/items/content/oneOf[subschema #1: Eval content item]/oneOf[subschema #2: Input text]/prompt_cache_breakpoint/ttl` to the response with the `201` status
  - added the optional property `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/input/items/content/oneOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/oneOf[subschema #2: Input text]/prompt_cache_breakpoint/ttl` to the response with the `201` status
- **2026-10-06** `e2f8d8cbc41f`: 2 info
  - added the optional property `error/misalignment/review_target` to the response with the `400` status
  - added the optional property `error/misalignment/review_target` to the response with the `429` status
- **2026-09-25** `d86203de41ae`: 1 info
  - added the non-success response with the status `400`
- **2026-09-08** `bb9b3b410112`: 1 info
  - added the non-success response with the status `429`
- **2026-09-03** `9430b00d75b4`: 1 info
  - added the optional property `error/misalignment` to the response with the `429` status
- **2026-08-14** `6ac2618eefd7`: 1 info
  - added the non-success response with the status `429`
- **2026-08-05** `ab0c5306e390`: 1 warning, 10 info
  - added the new `max` enum value to the `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/sampling_params/reasoning_effort/anyOf[subschema #1]/` response property for the response status `201`
  - added the new optional request property `testing_criteria/items/oneOf[subschema #1: LabelModelGrader]/input/items/oneOf[subschema #2: Eval message object]/content/oneOf[subschema #1: Eval content item]/oneOf[subschema #2: Input text]/prompt_cache_breakpoint`
  - added the new optional request property `testing_criteria/items/oneOf[subschema #1: LabelModelGrader]/input/items/oneOf[subschema #2: Eval message object]/content/oneOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/oneOf[subschema #2: Input text]/prompt_cache_breakpoint`
  - added the new optional request property `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/input/items/content/oneOf[subschema #1: Eval content item]/oneOf[subschema #2: Input text]/prompt_cache_breakpoint`
  - added the new optional request property `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/input/items/content/oneOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/oneOf[subschema #2: Input text]/prompt_cache_breakpoint`
  - added the new `max` enum value to the request property `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/sampling_params/reasoning_effort/anyOf[subschema #1]/`
  - added the optional property `testing_criteria/items/oneOf[subschema #1: LabelModelGrader]/allOf[subschema #1: LabelModelGrader]/input/items/content/oneOf[subschema #1: Eval content item]/oneOf[subschema #2: Input text]/prompt_cache_breakpoint` to the response with the `201` status
  - added the optional property `testing_criteria/items/oneOf[subschema #1: LabelModelGrader]/allOf[subschema #1: LabelModelGrader]/input/items/content/oneOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/oneOf[subschema #2: Input text]/prompt_cache_breakpoint` to the response with the `201` status
  - added the optional property `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/input/items/content/oneOf[subschema #1: Eval content item]/oneOf[subschema #2: Input text]/prompt_cache_breakpoint` to the response with the `201` status
  - added the optional property `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/input/items/content/oneOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/oneOf[subschema #2: Input text]/prompt_cache_breakpoint` to the response with the `201` status
  - the `testing_criteria` response's property default value `eval` was removed for the status `201`
- **2026-07-22** `0e6756eca8e0`: 1 info
  - the `testing_criteria` response's property default value `eval` was removed for the status `201`
- **2026-07-15** `4065aec90c5d`: 1 warning, 9 info
  - added the new `max` enum value to the `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/sampling_params/reasoning_effort/anyOf[subschema #1]/` response property for the response status `201`
  - added the new optional request property `testing_criteria/items/oneOf[subschema #1: LabelModelGrader]/input/items/oneOf[subschema #2: Eval message object]/content/oneOf[subschema #1: Eval content item]/oneOf[subschema #2: Input text]/prompt_cache_breakpoint`
  - added the new optional request property `testing_criteria/items/oneOf[subschema #1: LabelModelGrader]/input/items/oneOf[subschema #2: Eval message object]/content/oneOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/oneOf[subschema #2: Input text]/prompt_cache_breakpoint`
  - added the new optional request property `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/input/items/content/oneOf[subschema #1: Eval content item]/oneOf[subschema #2: Input text]/prompt_cache_breakpoint`
  - added the new optional request property `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/input/items/content/oneOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/oneOf[subschema #2: Input text]/prompt_cache_breakpoint`
  - added the new `max` enum value to the request property `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/sampling_params/reasoning_effort/anyOf[subschema #1]/`
  - added the optional property `testing_criteria/items/oneOf[subschema #1: LabelModelGrader]/allOf[subschema #1: LabelModelGrader]/input/items/content/oneOf[subschema #1: Eval content item]/oneOf[subschema #2: Input text]/prompt_cache_breakpoint` to the response with the `201` status
  - added the optional property `testing_criteria/items/oneOf[subschema #1: LabelModelGrader]/allOf[subschema #1: LabelModelGrader]/input/items/content/oneOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/oneOf[subschema #2: Input text]/prompt_cache_breakpoint` to the response with the `201` status
  - added the optional property `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/input/items/content/oneOf[subschema #1: Eval content item]/oneOf[subschema #2: Input text]/prompt_cache_breakpoint` to the response with the `201` status
  - added the optional property `testing_criteria/items/oneOf[subschema #5: ScoreModelGrader]/allOf[subschema #1: ScoreModelGrader]/input/items/content/oneOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/oneOf[subschema #2: Input text]/prompt_cache_breakpoint` to the response with the `201` status
- **2026-07-09** `e0508a503b84`: 1 warning, 9 info
  - added the new `max` enum value to the `testing_criteria/items/anyOf[subschema #5: EvalGraderScoreModel]/allOf[subschema #1: ScoreModelGrader]/sampling_params/reasoning_effort/anyOf[subschema #1]/` response property for the response status `201`
  - added the new optional request property `testing_criteria/items/anyOf[subschema #1: LabelModelGrader]/input/items/anyOf[subschema #2: EvalItem]/content/anyOf[subschema #1: Eval content item]/anyOf[subschema #2: Input text]/prompt_cache_breakpoint`
  - added the new optional request property `testing_criteria/items/anyOf[subschema #1: LabelModelGrader]/input/items/anyOf[subschema #2: EvalItem]/content/anyOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/anyOf[subschema #2: Input text]/prompt_cache_breakpoint`
  - added the new optional request property `testing_criteria/items/anyOf[subschema #5: EvalGraderScoreModel]/allOf[subschema #1: ScoreModelGrader]/input/items/content/anyOf[subschema #1: Eval content item]/anyOf[subschema #2: Input text]/prompt_cache_breakpoint`
  - added the new optional request property `testing_criteria/items/anyOf[subschema #5: EvalGraderScoreModel]/allOf[subschema #1: ScoreModelGrader]/input/items/content/anyOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/anyOf[subschema #2: Input text]/prompt_cache_breakpoint`
  - added the new `max` enum value to the request property `testing_criteria/items/anyOf[subschema #5: EvalGraderScoreModel]/allOf[subschema #1: ScoreModelGrader]/sampling_params/reasoning_effort/anyOf[subschema #1]/`
  - added the optional property `testing_criteria/items/anyOf[subschema #1: LabelModelGrader]/allOf[subschema #1: LabelModelGrader]/input/items/content/anyOf[subschema #1: Eval content item]/anyOf[subschema #2: Input text]/prompt_cache_breakpoint` to the response with the `201` status
  - added the optional property `testing_criteria/items/anyOf[subschema #1: LabelModelGrader]/allOf[subschema #1: LabelModelGrader]/input/items/content/anyOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/anyOf[subschema #2: Input text]/prompt_cache_breakpoint` to the response with the `201` status
  - added the optional property `testing_criteria/items/anyOf[subschema #5: EvalGraderScoreModel]/allOf[subschema #1: ScoreModelGrader]/input/items/content/anyOf[subschema #1: Eval content item]/anyOf[subschema #2: Input text]/prompt_cache_breakpoint` to the response with the `201` status
  - added the optional property `testing_criteria/items/anyOf[subschema #5: EvalGraderScoreModel]/allOf[subschema #1: ScoreModelGrader]/input/items/content/anyOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/anyOf[subschema #2: Input text]/prompt_cache_breakpoint` to the response with the `201` status
- **2026-05-13** `74cbcf73838f`: 7 breaking, 6 info
  - the request property `metadata` became not nullable
  - removed `#/components/schemas/EvalStringCheckGrader, #/components/schemas/EvalTextSimilarityGrader, #/components/schemas/EvalPythonGrader, #/components/schemas/EvalScoreModelGrader` from the `testing_criteria/items/` request property `oneOf` list
  - removed `#/components/schemas/InputTextContent, subschema #1: Text input, subschema #3: Output text` from the `testing_criteria/items/oneOf[subschema #1: LabelModelGrader]/input/items/oneOf[subschema #2: Eval message object]/content` request property `oneOf` list
  - response property `data_source_config/oneOf[subschema #2: StoredCompletionsDataSourceConfig -> subschema #3: StoredCompletionsDataSourceConfig]/metadata` list-of-types was widened by adding types `null` to media type `application/json` of response `201`
  - response property `metadata` list-of-types was widened by adding types `null` to media type `application/json` of response `201`
  - added `#/components/schemas/EvalGraderLabelModel, #/components/schemas/EvalGraderStringCheck, #/components/schemas/EvalGraderTextSimilarity, #/components/schemas/EvalGraderPython, #/components/schemas/EvalGraderScoreModel` to the `testing_criteria/items/` response property `oneOf` list for the response status `201`
  - added `#/components/schemas/EvalLogsDataSourceConfig` to the `data_source_config` response property `oneOf` list for the response status `201`
  - request property `metadata` list-of-types was widened by adding types `null` to media type `application/json`
  - added `#/components/schemas/CreateEvalStoredCompletionsDataSourceConfig` to the `data_source_config` request property `oneOf` list
  - added `#/components/schemas/EvalGraderStringCheck, #/components/schemas/EvalGraderTextSimilarity, #/components/schemas/EvalGraderPython, #/components/schemas/EvalGraderScoreModel` to the `testing_criteria/items/` request property `oneOf` list
  - added `#/components/schemas/EvalItemContentItem, #/components/schemas/EvalItemContentArray` to the `testing_criteria/items/oneOf[subschema #1: LabelModelGrader]/input/items/oneOf[subschema #2: Eval message object]/content` request property `oneOf` list
  - response property `data_source_config/oneOf[subschema #2: StoredCompletionsDataSourceConfig -> subschema #3: StoredCompletionsDataSourceConfig]/` deprecated
  - removed `#/components/schemas/EvalLabelModelGrader, #/components/schemas/EvalStringCheckGrader, #/components/schemas/EvalTextSimilarityGrader, #/components/schemas/EvalPythonGrader, #/components/schemas/EvalScoreModelGrader` from the `testing_criteria/items/` response property `oneOf` list for the response status `201`
- **2026-04-29** `5b98524f8dee`: 4 breaking
  - the `testing_criteria/items/anyOf[subschema #1: LabelModelGrader]/input/items/anyOf[subschema #2: EvalItem]/content/anyOf[subschema #1: Eval content item]/anyOf[subschema #4: Input image]/image_url` request property format changed from no format to `uri`
  - the `testing_criteria/items/anyOf[subschema #1: LabelModelGrader]/input/items/anyOf[subschema #2: EvalItem]/content/anyOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/anyOf[subschema #4: Input image]/image_url` request property format changed from no format to `uri`
  - the `testing_criteria/items/anyOf[subschema #5: EvalGraderScoreModel]/allOf[subschema #1: ScoreModelGrader]/input/items/content/anyOf[subschema #1: Eval content item]/anyOf[subschema #4: Input image]/image_url` request property format changed from no format to `uri`
  - the `testing_criteria/items/anyOf[subschema #5: EvalGraderScoreModel]/allOf[subschema #1: ScoreModelGrader]/input/items/content/anyOf[subschema #2: An array of Input text, Output text, Input image, and Input audio]/items/anyOf[subschema #4: Input image]/image_url` request property format changed from no format to `uri`
- **2026-04-10** `7c540cce6eb3`: 4 breaking, 11 info
  - the request property `metadata` became not nullable
  - removed `#/components/schemas/CreateEvalCustomDataSourceConfig, #/components/schemas/CreateEvalLogsDataSourceConfig` from the `data_source_config` request property `oneOf` list
  - removed `#/components/schemas/CreateEvalLabelModelGrader, #/components/schemas/EvalStringCheckGrader, #/components/schemas/EvalTextSimilarityGrader, #/components/schemas/EvalPythonGrader, #/components/schemas/EvalScoreModelGrader` from the `testing_criteria/items/` request property `oneOf` list
  - response property `metadata` list-of-types was widened by adding types `null` to media type `application/json` of response `201`
  - added `#/components/schemas/CreateEvalCustomDataSourceConfig, #/components/schemas/CreateEvalLogsDataSourceConfig, #/components/schemas/CreateEvalStoredCompletionsDataSourceConfig` to the `data_source_config` request property `anyOf` list
  - added `#/components/schemas/CreateEvalLabelModelGrader, #/components/schemas/EvalGraderStringCheck, #/components/schemas/EvalGraderTextSimilarity, #/components/schemas/EvalGraderPython, #/components/schemas/EvalGraderScoreModel` to the `testing_criteria/items/` request property `anyOf` list
  - added discriminator to `data_source_config` request property
  - added discriminator to `testing_criteria/items/` request property
  - request property `metadata` list-of-types was widened by adding types `null` to media type `application/json`
  - added `#/components/schemas/EvalCustomDataSourceConfig, #/components/schemas/EvalLogsDataSourceConfig, #/components/schemas/EvalStoredCompletionsDataSourceConfig` to the `data_source_config` response property `anyOf` list for the response status `201`
  - added `#/components/schemas/EvalGraderLabelModel, #/components/schemas/EvalGraderStringCheck, #/components/schemas/EvalGraderTextSimilarity, #/components/schemas/EvalGraderPython, #/components/schemas/EvalGraderScoreModel` to the `testing_criteria/items/` response property `anyOf` list for the response status `201`
  - the `testing_criteria` response's property default value `eval` was removed for the status `201`
  - added discriminator to `data_source_config` response property for the response status `201`
  - removed `#/components/schemas/EvalCustomDataSourceConfig, #/components/schemas/EvalStoredCompletionsDataSourceConfig` from the `data_source_config` response property `oneOf` list for the response status `201`
  - removed `#/components/schemas/EvalLabelModelGrader, #/components/schemas/EvalStringCheckGrader, #/components/schemas/EvalTextSimilarityGrader, #/components/schemas/EvalPythonGrader, #/components/schemas/EvalScoreModelGrader` from the `testing_criteria/items/` response property `oneOf` list for the response status `201`
- **2025-04-29** `6a6c681b1820`: 1 info
  - endpoint added

---

[Operation](https://skmtc.dev/openai/apis/openapi/docs/evals/post.md) - [API](https://skmtc.dev/openai/apis/openapi.md) - [Page](https://skmtc.dev/openai/apis/openapi/changes/evals/post)