Sign in
evals
evals

(dev only) Fabricate failing eval runs and fire the real eval.failed emitter

Fabricate failing eval runs for an agent and run the REAL ENG-604 emitter.

Exercises the full classification/escalation/digest/dedupe-key path that plain test-send bypasses: builds the eval.failed payload exactly as the eval orchestrator would, then enqueues dispatch (the notify Celery task) — so a background worker is required for the message to actually deliver.

post/v1/evals/synthesize-failure

Request body

agent_idstring uuid required
n_failuresinteger
failure_kind'quality' | 'error'
scorenumber
thresholdnumber

Response

Successful Response

session_idstring uuid required
eval_run_idsstring[] required
payloadobject required
severitystring nullable required
sustained_countinteger required
dispatchedboolean required
task_idstring nullable required

Changes

Changed in 1 of the 3 revisions of this API.4

  • 5a5597ebb2d64See the full diff
    • ○

      added the optional property retry_after to the response with the 400 status

      response-optional-property-added

    • ○

      added the optional property retry_after to the response with the 401 status

      response-optional-property-added

    • ○

      added the optional property retryable to the response with the 400 status

      response-optional-property-added

    • ○

      added the optional property retryable to the response with the 401 status

      response-optional-property-added

    This revision also has 40 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog