Agent Tests

Start a batch that runs the agent's attached tests against its current draft, one run per test and repeat. A simulation test repeats simulation.repeat_count times, and a repeat_count in the body repeats every selected test that many times instead. A batch starts at most 1000 runs. Runs are queued and finish in the background, simulations take minutes. Poll GET /v1/agent/agents/{agent_id}/test-batches/{batch_id} until completed is true, then gate on pass_rate, which leaves out error runs.

Each simulation run that ends passed or failed is billed: a platform fee per agent reply plus the LLM tokens of the agent, the simulated user and the judge. error runs are not billed. A batch with a simulation test returns 402 when the account has no trial balance or API credit left, and 403 when the team's agreement does not allow agent usage.

post/v1/agent/agents/{agent_id}/tests/run

Request

  • Base URL
    https://api.fish.audio
  • URL
    https://api.fish.audio/v1/agent/agents/{agent_id}/tests/run
  • Auth
    HTTP bearer

Path parameters

agent_idstring required

Request body

test_idsstring[]

Attached tests to run. Omit to run every test attached to the agent.

repeat_countinteger

Run every selected test this many times in this batch, in place of each test's own repeat count.

Response

Request accepted, processing continues off-line

batch_idstring required
agent_idstring required
status'running' | 'completed' required

running until every run has finished.

completedboolean required
passedinteger required
failedinteger required
errorsinteger required
pass_ratenumber nullable

passed / (passed + failed), null until a run passed or failed. Error runs never judged the agent and are left out.

Changes