Run Tests - Eval Configs

Run new evaluations on test executions

Runs new evaluations on completed test executions. Either test_execution_ids or select_all=true must be provided.

post/simulate/run-tests/{run_test_id}/run-new-evals/

Request

  • Base URL: http://localhost:8000
  • URL: http://localhost:8000/simulate/run-tests/{run_test_id}/run-new-evals/
  • Auth: one of:
    • API key in header X-Api-Key
    • API key in header X-Secret-Key

Request body

test_execution_idsstring[]

List of specific test execution IDs to run evaluations on

select_allboolean

Whether to run evaluations on all test executions in the run test

eval_config_idsstring[] required

List of SimulateEvalConfig IDs to run on the test executions

enable_tool_evaluationboolean

Whether to enable tool evaluation for this run (if not provided, uses the run test's current setting)

Response

OK

messagestring required
run_test_idstring uuid required
call_execution_countinteger required
dispatchedboolean required

Changes

No recorded changes to this operation across all 1 revision of this API.