Benchmark

Replaces a benchmark's prompt structure with the submitted one.

Prompts that would no longer satisfy the structure are only removed when deleteInvalidPrompts is set; otherwise the request is rejected with a message naming how many would have to go. Deleting them also removes the samples and generation items they produced. Returns 409 when a leaderboard's rapid context, or the benchmark's HuggingFace export configuration, still references a segment the new structure removes or retypes. Content stored under a removed segment is discarded.

put/benchmark/{benchmarkId}/prompt-structure

Path parameters

benchmarkIdstring required

The id of the benchmark to change.

Request body

deleteInvalidPromptsboolean

Whether prompts the new structure invalidates may be deleted. Defaults to false, which refuses the change and reports how many prompts would have to go.

Response

OK

deletedPromptCountinteger required

How many prompts were deleted to satisfy the new structure.

Changes

Changed in 1 of the 14 revisions of this API.1