--- title: "List Benchmarks" method: GET path: "/api/v0/agent/evals/benchmarks" tags: ["Agent Evals"] --- # List Benchmarks `GET /api/v0/agent/evals/benchmarks` ## Request - Base URL: `https://api.outerport.com` (the document declares no server; this is the origin it was published from) - URL: `https://api.outerport.com/api/v0/agent/evals/benchmarks` - Auth: HTTP bearer List the user's benchmarks with each one's newest run, and the models a run uses by default. ## Response `200` Successful Response - AgentEvalBenchmarkListResponse: The user's benchmarks and the models a run would use by default. - `benchmarks` AgentEvalBenchmarkSummary[], required: The user's benchmarks, most recently changed first. - `id` string, uuid, required: The ID of the benchmark. - `name` string, required: The benchmark's name. - `persona_id` string, uuid, nullable: The persona a run uses unless it names another. - `case_count` integer, required: Cases in the benchmark. - `latest_run` AgentEvalRunResponse: One run of a benchmark. - `id` string, uuid, required: The ID of the run. - `status` 'running' | 'completed' | 'cancelled', required: running while any case is queued or running; otherwise how the run ended. - `persona_id` string, uuid, nullable: The persona the run used. Null for the default persona or once it is deleted. - `persona_name` string, required: The persona's name when the run started. - `agent_model` string, required: The model the run started the agent on. - `judge_model` string, required: The model the run started the judge on. - `agent_reasoning_level` string, nullable: The reasoning depth the agent ran at. Null: the model's provider default. - `judge_reasoning_level` string, nullable: The reasoning depth the judge graded at. Null: the model's provider default. - `rubric` string, required: The judge instructions the run graded with. - `passed_count` integer, required: Cases the judge passed. - `finished_count` integer, required: Cases that ended, in any status. - `case_count` integer, required: Cases in the run. - `results` AgentEvalResultResponse[], required: One result per case, in order. - `id` string, uuid, required: The ID of the result. - `case_id` string, uuid, nullable: The case this result ran. Null once the case is deleted. - `position` integer, required: The case's place in the run, from 1. - `status` 'queued' | 'running' | 'completed' | 'error' | 'cancelled', required: Where the case is: queued, running, or how it ended. - `passed` boolean, nullable: The judge's verdict. Set only when status is completed. - `input` string, nullable: The prompt the agent was given. - `expected_output` string, nullable: The answer the judge compared against. - `output` string, nullable: The agent's answer. - `judge_reason` string, nullable: What decided the judge's verdict. - `error_message` string, nullable: Why the case ended in error. - `agent_session_id` string, uuid, nullable: The agent session the case ran in. - `started_at` string, date-time, nullable: When the case started. - `completed_at` string, date-time, nullable: When the case ended. - `input_tokens` integer, nullable: Input tokens the agent used on the case, sub-agents included and the judge excluded. Null before the agent recorded any usage. - `output_tokens` integer, nullable: Output tokens the agent used on the case, reasoning included. Null before the agent recorded any usage. - `cost_usd` number, nullable: Estimated cost of the agent's calls on the case in USD, from the pricing table; 0 for a model the table does not price. Null before the agent recorded any usage. - `agent_model` string, nullable: The model that answered the case, which differs from the run's agent_model when the agent reran on another model. Null while the case is queued or running, and when it ended before the agent started. - `judge_model` string, nullable: The model that graded the case, which differs from the run's judge_model when the call reran on another model. Null until the judge answers. - `created_at` string, date-time, required: When the run started. - `cancelled_at` string, date-time, nullable: When the run was stopped. - `answering_agent_models` string[]: The models that answered the run's cases (the results' agent_model), each once, in the order first named. A case that is queued or running, or whose agent never started, adds none. - `answering_judge_models` string[]: The models that graded the run's cases (the results' judge_model), each once, in the order first named. A case the judge has not answered adds none. - `updated_at` string, date-time, required: When the benchmark or its cases last changed. - `agent_model` string, required: The model a run uses for the agent unless it names another. - `judge_model` string, required: The model a run grades with, from Settings. - `agent_models` string[], required: Models a run may name for the agent. Empty when the tier has no model selection. ## Changes - **2026-11-09** `ecd5b922e78a`: 4 info - added the optional property `benchmarks/items/latest_run/anyOf[subschema #1: AgentEvalRunResponse]/answering_agent_models` to the response with the `200` status - added the optional property `benchmarks/items/latest_run/anyOf[subschema #1: AgentEvalRunResponse]/answering_judge_models` to the response with the `200` status - added the optional property `benchmarks/items/latest_run/anyOf[subschema #1: AgentEvalRunResponse]/results/items/agent_model` to the response with the `200` status - added the optional property `benchmarks/items/latest_run/anyOf[subschema #1: AgentEvalRunResponse]/results/items/judge_model` to the response with the `200` status - **2026-10-22** `fd2664afd5a3`: 1 info - endpoint added [Operation history](https://skmtc.dev/outerport/apis/outerport-api/changes/api/v0/agent/evals/benchmarks/get.md) --- [API](https://skmtc.dev/outerport/apis/outerport-api.md) - [All operations](https://skmtc.dev/outerport/apis/outerport-api/llms.txt) - [OpenAPI document](https://skmtc.dev/outerport/apis/outerport-api/revisions/08b5bf6f7f18?raw)