---
title: "Launch a benchmark run from the current suite version"
method: POST
path: "/factory/{factory_uid}/benchmarks/{suite_uid}/runs"
tags: ["Factories", "Benchmarks"]
---

# Launch a benchmark run from the current suite version

`POST /factory/{factory_uid}/benchmarks/{suite_uid}/runs`

## Request

- Base URL: `https://app.warp.dev/api/v1`
- URL: `https://app.warp.dev/api/v1/factory/{factory_uid}/benchmarks/{suite_uid}/runs`
- Auth: HTTP bearer

Each successful request creates a new run. Launch is not idempotent:
disable automatic retries for this request and reconcile an uncertain
outcome by listing runs before launching again.
Retries are a client transport option, not a request-body field.
In the TypeScript SDK, pass { maxRetries: 0 } as the request options.
In the Python SDK, call create through client.with_options(max_retries=0).
These options do not change execution retries within the benchmark run.

## Path parameters

- `factory_uid` string, required
- `suite_uid` string, required

## Request body

- LaunchBenchmarkRunRequest
  - `display_name` string: Optional display name. Surrounding whitespace is trimmed; empty clears the name.
  - `configurations` BenchmarkConfigurationInput[]: Optional launch-time configurations. When omitted, the suite's persistent configurations are used. An explicit empty list, or a suite with neither launch-time nor persistent configurations, is rejected.
    - `display_name` string: Optional display name. Surrounding whitespace is trimmed; empty clears the name.
    - `harness` string
    - `model` string
    - `runner_id` string, nullable: Optional runner UID, constrained to the Factory's existing runner roster. Omitted or empty uses the agent's or environment's default.
    - `harness_auth_secret_name` string, nullable: Optional managed-secret name this configuration's third-party harness authenticates with. Empty or omitted is valid for Oz. For a third-party harness that requires auth, empty or omitted means from the worker environment on a self-hosted host, or a launch violation on Warp-hosted.
    - `agent_configs` BenchmarkAgentConfigInput[]: Optional per-named-Agent overrides keyed by stable Agent UID.
      - `agent_uid` string, uuid, required
      - `agent_name` string
      - `harness` string, required
      - `model` string, required
      - `harness_auth_secret_name` string, nullable
  - `repetition_count` integer, required: Number of repetitions per (task, configuration) pair, frozen onto the run. Like configurations and scorer_selection, this is chosen fresh at each launch rather than persisted on the suite.
  - `scorer_selection` union[]: Allowlist of live scorer UUIDs (or temporary positive numeric IDs) for this factory, applied to every trial in the run. Empty or absent means all applicable scorers. IDs must belong to the suite's factory and team (any status except deleted).
    - union
      - string, uuid: Stable scorer UUID; new scorers use UUIDv4. Nil UUIDs are invalid.
      - integer
  - `tasks` string[]: Optional allowlist of immutable UIDs for current tasks in the addressed suite. Omitted runs all current suite tasks. Explicit empty, invalid, unknown, stale, cross-suite, and duplicate selections are rejected.

## Response `201`

Run launched

- BenchmarkRunLaunchResponse
  - `uid` string, required
  - `state` 'pending' | 'running' | 'scoring' | 'completed' | 'failed' | 'cancelled', required: Lifecycle state of a benchmark run.

## Other responses

- `400`: Suite not runnable; body contains violations list
- `401`: Authentication required
- `403`: Factory access is not enabled for the authenticated principal
- `404`: Suite not found

## Changes

- **2026-11-04** `4b5d145669be`: 1 warning, 1 info
  - the `configurations/items/display_name` request property's maxLength was set to `200`
  - added the new optional request property `display_name`
- **2026-10-21** `7ee3ba55debc`: 3 info
  - api tag `factory-benchmarks` added
  - api tag `benchmark` removed
  - api tag `factory` removed
- **2026-10-19** `2c45dcc0a13e`: 3 breaking
  - the request property `configurations/items/agent_configs/items/harness_auth_secret_name` became not nullable
  - the request property `configurations/items/harness_auth_secret_name` became not nullable
  - the request property `configurations/items/runner_id` became not nullable
- **2026-10-15** `f4630aae17ba`: 1 info
  - endpoint added

[Operation history](https://skmtc.dev/warp/apis/oz-agent-api/changes/factory/:factory_uid/benchmarks/:suite_uid/runs/post.md)

---

[API](https://skmtc.dev/warp/apis/oz-agent-api.md) - [All operations](https://skmtc.dev/warp/apis/oz-agent-api/llms.txt) - [OpenAPI document](https://skmtc.dev/warp/apis/oz-agent-api/revisions/4b5d145669be?raw)