---
title: "Run Tests"
method: POST
path: "/v1/agent/agents/{agent_id}/tests/run"
tags: ["Agent Tests"]
---

# Run Tests

`POST /v1/agent/agents/{agent_id}/tests/run`

## Request

- Base URL: `https://api.fish.audio`
- URL: `https://api.fish.audio/v1/agent/agents/{agent_id}/tests/run`
- Auth: HTTP bearer

Start a batch that runs the agent's attached tests against its current
draft, one run per test and repeat. A simulation test repeats
`simulation.repeat_count` times, and a `repeat_count` in the body repeats
every selected test that many times instead. A batch starts at most 1000
runs. Runs are queued and finish in the background, simulations take
minutes. Poll `GET /v1/agent/agents/{agent_id}/test-batches/{batch_id}`
until `completed` is true, then gate on `pass_rate`, which leaves out
`error` runs.

Each simulation run that ends `passed` or `failed` is billed: a platform
fee per agent reply plus the LLM tokens of the agent, the simulated user
and the judge. `error` runs are not billed. A batch with a simulation
test returns 402 when the account has no trial balance or API credit
left, and 403 when the team's agreement does not allow agent usage.

## Path parameters

- `agent_id` string, required

## Request body

- object
  - `test_ids` string[]: Attached tests to run. Omit to run every test attached to the agent.
  - `repeat_count` integer: Run every selected test this many times in this batch, in place of each test's own repeat count.

## Response `202`

Request accepted, processing continues off-line

- object
  - `batch_id` string, required
  - `agent_id` string, required
  - `status` 'running' | 'completed', required: running until every run has finished.
  - `completed` boolean, required
  - `passed` integer, required
  - `failed` integer, required
  - `errors` integer, required
  - `pass_rate` number, nullable: passed / (passed + failed), null until a run passed or failed. Error runs never judged the agent and are left out.
  - `runs` PublicAgentTestRun[], required
    - `run_id` string, required
    - `test_id` string, required
    - `agent_id` string, required
    - `batch_id` string, nullable
    - `test_type` 'next_reply' | 'tool' | 'simulation', required
    - `repeat_index` integer: Which repeat of the test in its batch this run is.
    - `status` 'queued' | 'running' | 'passed' | 'failed' | 'error', required
    - `needs_review` boolean: The judge could not decide, or a tool call had no mock answer.
    - `error` string
    - `usage` PublicAgentTestRunUsage
      - `agent_replies` integer
      - `cost_usd` string, nullable: What the run was charged in USD. `0` on a finished run that is not charged: runs that end in `error` and tests other than simulations. Null until the run's charge is settled.
      - `platform_fee` PublicAgentTestRunPlatformFee
        - `rate_usd` string, required: USD per agent reply.
        - `amount_usd` string, required: `agent_replies` times `rate_usd`.
      - `llm` PublicAgentTestRunLLMCharge[]: LLM tokens, one entry per participant and model, by token category. With `platform_fee` they add up to `cost_usd`. Empty unless charged.
        - `participant` 'agent' | 'simulated_user' | 'judge', required: Whose tokens: the agent, the simulated user, or the judge.
        - `model` string, required
        - `amount_usd` string, required: What these tokens charged, in USD.
        - `input` PublicAgentTestRunTokenCharge
          - `tokens` integer, required
          - `amount_usd` string, required
        - `cache_read` PublicAgentTestRunTokenCharge
          - `tokens` integer, required
          - `amount_usd` string, required
        - `cache_write_5m` PublicAgentTestRunTokenCharge
          - `tokens` integer, required
          - `amount_usd` string, required
        - `cache_write_1h` PublicAgentTestRunTokenCharge
          - `tokens` integer, required
          - `amount_usd` string, required
        - `output` PublicAgentTestRunTokenCharge
          - `tokens` integer, required
          - `amount_usd` string, required
    - `created_at` string, date-time, required
    - `started_at` string, date-time, nullable
    - `finished_at` string, date-time, nullable
    - `agent_reply` string: The reply a scripted test judged.
    - `judge_reasoning` string
    - `config_hash` string: The agent draft's config hash the run tested.
    - `latency_ms` integer
    - `simulation_result` AgentTestSimulationResult: The simulation run's result: what was said, what the agent called, the deterministic assertions and the judge's per-condition verdicts.
      - `transcript` AgentTestTranscriptMessage[]
        - `role` 'user' | 'agent', required
        - `text` string
        - `turn_id` integer, nullable
        - `node_id` string, nullable
        - `created_at` string, nullable
        - `seq` integer, nullable
      - `tool_calls` AgentTestToolCallRecord[]
        - `id` string
        - `tool_name` string
        - `tool_source` string
        - `input` object
        - `status` string
        - `output` string, nullable
        - `error` string, nullable
        - `mocked` boolean
        - `mock_source` 'test' | 'tool' | 'missing' | 'unmatched' | 'real', nullable
        - `node_id` string
        - `turn_id` integer, nullable
        - `created_at` string, nullable
        - `seq` integer, nullable
      - `assertions` AgentTestAssertionResult[]
        - `kind` 'tool_call' | 'forbidden_tool' | 'ended_by', required
        - `target` string, required
        - `tool_id` string, nullable
        - `tool_type` 'webhook' | 'client' | 'integration', nullable
        - `passed` boolean, required
        - `detail` string
      - `conditions` AgentTestConditionResult[]
        - `name` string, required
        - `result` 'success' | 'failure' | 'unknown', required
        - `rationale` string
      - `summary` string
      - `passed` boolean
      - `llm_endpoint_errors` AgentSessionLlmEndpointError[]
        - `occurred_at` string, nullable
        - `turn_id` integer, nullable
        - `type` 'http_error' | 'timeout' | 'connection_error' | 'invalid_response', required
        - `reason` 'dns_failed' | 'connection_refused' | 'connection_reset' | 'tls_failed' | 'address_not_allowed' | 'not_event_stream' | 'stream_incomplete' | 'empty_reply' | 'unknown', nullable
        - `status_code` integer, nullable
        - `response_body` string, nullable
        - `duration_ms` integer, nullable
        - `consecutive_failures` integer, required
    - `channel` 'web_voice' | 'phone_inbound' | 'phone_outbound', nullable
    - `turns_used` integer
    - `ended_by` 'agent' | 'user' | 'transfer' | 'max_turns' | 'timeout' | 'error', nullable
    - `mock_gaps` AgentTestMockGap[]
      - `tool_name` string, required
      - `tool_id` string, nullable
      - `tool_type` 'webhook' | 'client' | 'integration', nullable
      - `source` 'missing' | 'unmatched', required
      - `calls` integer

## Other responses

- `400`: Bad request syntax or unsupported method
- `401`: No permission -- see authorization schemes
- `402`: No payment -- see charging schemes
- `403`: Request forbidden -- authorization will not help
- `404`: Nothing matches the given URI
- `422`
- `503`: The server cannot process the request due to a high load

## Changes

- **2026-11-03** `42ac421f7a12`: 1 info
  - endpoint added

[Operation history](https://skmtc.dev/fish/apis/fishaudio-openapi/changes/v1/agent/agents/:agent_id/tests/run/post.md)

---

[API](https://skmtc.dev/fish/apis/fishaudio-openapi.md) - [All operations](https://skmtc.dev/fish/apis/fishaudio-openapi/llms.txt) - [OpenAPI document](https://skmtc.dev/fish/apis/fishaudio-openapi/revisions/42ac421f7a12?raw)