simulate

Grade this finished run's calls again with chosen evals of the environment, without rerunning the calls.

post/simulate/api/harness-environments/{id}/runs/{execution_id}/evaluations/run/

Request

  • Base URL: http://localhost:8000
  • URL: http://localhost:8000/simulate/api/harness-environments/{id}/runs/{execution_id}/evaluations/run/
  • Auth: one of:
    • API key in header X-Api-Key
    • API key in header X-Secret-Key

Request body

eval_config_idsstring[] required
enable_tool_evaluationboolean nullable

Saved on the environment before grading starts; left as it is when absent.

Response

Accepted

call_execution_countinteger required

How many of the run's calls were queued for grading.

Changes

No recorded changes to this operation across all 1 revision of this API.