ONoneloop-hq
evals
evals

(dev only) Fabricate failing eval runs and fire the real eval.failed emitter

Changed on

Fabricate failing eval runs for an agent and run the REAL ENG-604 emitter.

Exercises the full classification/escalation/digest/dedupe-key path that plain test-send bypasses: builds the eval.failed payload exactly as the eval orchestrator would, then enqueues dispatch (the notify Celery task) — so a background worker is required for the message to actually deliver.

post/v1/evals/synthesize-failure

Request

Request body

agent_idstring uuid required
n_failuresinteger
failure_kind'quality' | 'error'
scorenumber
thresholdnumber

Response

Successful Response

session_idstring uuid required
eval_run_idsstring[] required
payloadobject required
severitystring nullable required
sustained_countinteger required
dispatchedboolean required
task_idstring nullable required

Changes

  • 5a5597ebb2d6Revision changes
    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status