WAwarp
FactoriesBenchmarks

Create a benchmark suite task

Changed on

This adds a single hand-authored task to a suite, without needing to resend the whole task list. A title, prompt, and success criteria are required, and if a source_run_id is included, the caller must have permission to read that run. Adding the exact same task twice simply returns the existing one (so retrying is safe), while reusing a title with different content is rejected as a conflict — and on a code.storage-managed Factory, the task is saved as a file and returned from that file.

post/factory/{factory_uid}/benchmarks/{suite_uid}/tasks

Request

  • Base URL
    https://app.warp.dev/api/v1
  • Auth
    HTTP bearer

Path parameters

factory_uidstring required

UID of the factory that owns the benchmark suite.

suite_uidstring required

UID of the benchmark suite within this factory.

Request body

titlestring required
promptstring required
success_criteriastring required

Plain-text criteria the built-in Correctness scorer grades each trial against, as the authoritative requirements. The task prompt provides context only.

source_run_idstring nullable

Optional provenance. The caller must be able to read the run.

tagsstring[]

Response

Task created

uidstring uuid required

Immutable public task identifier.

titlestring required
promptstring required
success_criteriastring required

Plain-text criteria the built-in Correctness scorer grades each trial against, as the authoritative requirements. The task prompt provides context only.

source_run_idstring nullable
labelsobject
tagsstring[]

Changes