---
title: "[Beta] List BenchmarkJobs."
method: GET
path: "/v1/benchmark_jobs"
tags: ["Benchmark"]
---

# [Beta] List BenchmarkJobs.

`GET /v1/benchmark_jobs`

[Beta] List all BenchmarkJobs matching filter.

## Query parameters

- `name` string
- `search` string
- `limit` integer
- `starting_after` string
- `include_total_count` boolean

## Response `200`

OK

- BenchmarkJobListView
  - `jobs` BenchmarkJobView[], required — List of BenchmarkJobs matching filter.
    - `id` string, required — The ID of the BenchmarkJob.
    - `name` string, required — The unique name of the BenchmarkJob.
    - `state` 'initializing' | 'queued' | 'running' | 'completed' | 'failed' | 'cancelled' | 'timeout', required
    - `job_source` union
      - HarborJobSource — Harbor job source with inline YAML configuration
        - `inline_yaml` string, required — The Harbor job configuration as inline YAML content
        - `type` 'harbor', required
      - BenchmarkDefJobSource — Benchmark definition job source
        - `benchmark_id` string, required — The ID of the benchmark definition
        - `benchmark_name` string, nullable — Optional user-provided name for the benchmark definition
        - `type` 'benchmark', required
      - ScenariosJobSource — Scenarios job source with a list of scenario definition IDs
        - `scenario_ids` string[], required — List of scenario definition IDs to execute
        - `type` 'scenarios', required
    - `job_spec` JobSpec — Job specification describing scenarios and execution configuration
      - `scenario_ids` string[], required — List of scenario IDs to execute
      - `orchestrator_config` JobOrchestratorConfig — Orchestrator configuration for benchmark job execution
        - `n_concurrent_trials` integer, nullable — Number of concurrent trials to run (default: 1). Controls parallelism for scenario execution. Default is 1.
        - `n_attempts` integer, nullable — Number of retry attempts on failure (default: 0). This is the retry policy for failed scenarios. Default is 0.
        - `timeout_multiplier` number, float, nullable — Timeout multiplier for retries (default: 1.0). Each retry will multiply the timeout by this factor.
        - `quiet` boolean, nullable — Suppress verbose output (default: false)
      - `agent_configs` JobAgentConfig[], required — Agent configurations for this job
        - `agent_id` string, nullable — ID of the agent to use (optional if agent exists by name)
        - `name` string, required — Name of the agent
        - `model_name` string, nullable — Model name override for this agent
        - `timeout_seconds` number, float, nullable — Timeout in seconds for this agent
        - `kwargs` object, nullable — Additional kwargs for agent configuration
        - `agent_environment` JobAgentEnvironment — Environment configuration for an agent in a benchmark job
          - `environment_variables` object, nullable — Environment variables to set when launching the agent.
          - `secrets` object, nullable — Secrets to inject as environment variables when launching the agent. Map of environment variable names to secret IDs.
        - `type` 'job_agent', required
    - `failure_reason` string, nullable — Failure reason if job failed.
    - `benchmark_outcomes` BenchmarkOutcomeView[], nullable — Detailed outcome data for each benchmark run created by this job. Includes per-agent results and scenario-level details.
      - `benchmark_run_id` string, required — The ID of the benchmark run.
      - `agent_name` string, required — The name of the agent configuration used.
      - `model_name` string, nullable — The model name used by the agent.
      - `n_completed` integer, required — Number of scenarios that completed successfully.
      - `n_failed` integer, required — Number of scenarios that failed.
      - `n_timeout` integer, required — Number of scenarios that timed out.
      - `average_score` number, float, nullable — Average score across all completed scenarios (0.0 to 1.0).
      - `duration_ms` integer, nullable — Total duration of the benchmark run in milliseconds.
      - `scenario_outcomes` ScenarioOutcomeView[], required — Detailed outcomes for each scenario in this benchmark run.
        - `scenario_run_id` string, nullable — The ID of the scenario run. May be absent if the scenario failed during setup before a run was created.
        - `scenario_definition_id` string, required — The ID of the scenario definition that was executed.
        - `scenario_name` string, required — The name of the scenario.
        - `state` 'COMPLETED' | 'FAILED' | 'TIMEOUT' | 'CANCELED', required
        - `score` number, float, nullable — The score achieved for this scenario (0.0 to 1.0). Only present if state is COMPLETED.
        - `duration_ms` integer, nullable — Duration of the scenario execution in milliseconds.
        - `failure_reason` FailureReason — Information about why a scenario execution failed
          - `exception_type` string, required — The exception class name (e.g., 'TimeoutException', 'AgentTimeoutError')
          - `exception_message` string, required — The exception message providing context
    - `in_progress_runs` InProgressRunView[], nullable — Benchmark runs currently in progress for this job. Shows runs that have not yet completed.
      - `benchmark_run_id` string, required — The ID of the benchmark run.
      - `agent_config` union
        - ExternalApiAgentConfig — Configuration for externally-driven benchmark runs via API
          - `info` string, nullable — Placeholder for future external agent metadata
          - `type` 'external_api', required
        - JobAgentConfig — Configuration for an agent in a benchmark job
          - `agent_id` string, nullable — ID of the agent to use (optional if agent exists by name)
          - `name` string, required — Name of the agent
          - `model_name` string, nullable — Model name override for this agent
          - `timeout_seconds` number, float, nullable — Timeout in seconds for this agent
          - `kwargs` object, nullable — Additional kwargs for agent configuration
          - `agent_environment` JobAgentEnvironment — Environment configuration for an agent in a benchmark job
            - `environment_variables` object, nullable — Environment variables to set when launching the agent.
            - `secrets` object, nullable — Secrets to inject as environment variables when launching the agent. Map of environment variable names to secret IDs.
          - `type` 'job_agent', required
      - `state` 'running' | 'canceled' | 'completed' | 'failed', required
      - `start_time_ms` integer, required — Start time (Unix milliseconds).
      - `duration_ms` integer, nullable — Duration so far in milliseconds.
    - `create_time_ms` integer, required — Timestamp when job was created (Unix milliseconds).
  - `has_more` boolean, required
  - `total_count` integer, nullable

---

[API](https://skmtc.dev/runloopai/apis/runloop-api.md) · [All operations](https://skmtc.dev/runloopai/apis/runloop-api/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/runloopai/runloop-api/revisions/e9dc776053e4/schema)