observability

Create Eval

Create a custom eval — a configured grading pipeline (declared input fields + cases + graders) stored for your workspace. Use when the built-in graders don't fit and you want your own scored report card. Pass slug, name, input_schema, and cases; the whole definition is validated and you get back the new eval's slug + id, or a structured validation_failed with the exact field errors. Built-in slugs are reserved.

post/api/platform/v1/evals

Request

  • Base URL: https://docs.amdahl.ai
  • URL: https://docs.amdahl.ai/api/platform/v1/evals
  • Auth: one of:
    • API key in header X-API-Key
    • HTTP bearer

Request body

slugstring required

kebab-case identifier, unique in your workspace.

namestring required
descriptionstring
versionstring

semver, default 1.0.0.

input_schemaobject

Declared run-input fields ({ fields: [...] }).

casesobject[] required

Cases: each a subject + its graders.

reliabilityobject

Required for any grader that reports a MAGNITUDE (judge / evidence_judge / improvement_loop): the measured flip rate and the floor a reported lift must clear. Validation refuses without it, so declare it rather than discovering the refusal.

Response

Created

Changes

No recorded changes to this operation across all 1 revision of this API.