observability

Run Eval

Grade content against your workspace data and get a run id to poll. Use when scoring a prompt or a drafted message: pass inputs (prompt and/or message, optional audience, account, mode=rewrite|advisory|gate) and context.facts, what you know first-hand about the recipient. A blinded judge scores relevant positioning, grounding, verified specifics, differentiation and CTA clarity against real customer quotes. Returns eval-run://<id> plus a console_url to give the user.

post/api/platform/v1/evals/run

Request

  • Base URL: https://docs.amdahl.ai
  • URL: https://docs.amdahl.ai/api/platform/v1/evals/run
  • Auth: one of:
    • API key in header X-API-Key
    • HTTP bearer

Request body

evalstring

Eval slug or id. Default 'prompt-and-message-eval'. Discover options + their input fields via eval://list.

inputsobject

Run inputs for the eval's declared fields. prompt-and-message-eval requires at least one of prompt / message, and also accepts audience (a seniority or title to scope the cohort), account (the company it is going to — retrieves what THAT account said), and mode: rewrite (the default, a full improved prompt and message), advisory (anchored suggestions against what you wrote), or gate (grade ONLY what you sent and stop — read the verdict from eval-run://<id>/gate). Read the exact schema from eval://<slug>.

reuse'cached' | 'force'

cached (default) reuses an active run for the same target (fingerprint); force always starts a fresh run.

evidence_from_runstring

Run id whose frozen customer quotes this run should be graded against, instead of retrieving fresh ones. Use to A/B two drafts: without it each run retrieves its own evidence, so the score difference mixes your edit with a different set of quotes. The source run must be in this workspace and have reached retrieval; the report states the pin and how old that evidence is.

include_externalboolean

Add public web/market evidence to the grade (default false). One web fan-out runs and its snippets join the pool tagged EXTERNAL — the tier that backs market claims ("analysts report"), never customer voice. Needs the external_search:execute scope (refused without it, never silently dropped) and cannot combine with a pinned run — a pin holds the evidence frozen, and its pool already carries any external quotes it was graded on.

external_capinteger

How many external quotes to retrieve when include_external is set. Clamped to 1..25; default 15. The ack echoes the clamped value as external.cap.

generate_modelstring

STAGING ONLY, ignored in production: the Anthropic model id the writer (generate stage) uses on this run, for a writer bake-off without a redeploy. The judge never moves. Not part of the content address, so pass reuse=force or a cached run of another arm returns.

generate_thinking'off' | 'low' | 'medium' | 'high' | 'default'

STAGING ONLY, ignored in production: how deeply the writer thinks on this run. Omitted uses the shipped depth (medium); default sends no thinking field. Same reuse=force caveat as generate_model.

candidate_from_runstring

Run id whose improved version this run should GRADE, instead of writing a new one. Use to re-run the judge over fixed text — the run makes no generate call, so a score difference between two such runs is the judge, not the writer. Pass reuse=force when sampling: two identical pinned requests share one content address, so on the default reuse=cached the second is served the first run and an N-sample spread reads as zero. Also pins that run's quotes (the citations in the frozen text address them), so pass the same id to evidence_from_run or omit it. Refused if this run submits different copy or a different mode than the pinned candidate was written for.

recipient_notesstring

Beta, per workspace: refused unless the workspace has recipient evidence on. Free text about the recipient, such as your own research or a pasted CRM note. Up to 10000 characters.

message_typestring

Beta, per workspace: ignored unless the workspace has recipient evidence on. The kind of message: cold, re-engage, follow-up, deal. follow-up is a reply in a thread that is in progress.

Response

OK

Changes

No recorded changes to this operation across all 1 revision of this API.