--- title: "Submit Tier 0 answers for automatic scoring" method: POST path: "/api/eval/submit" tags: ["certification"] --- # Submit Tier 0 answers for automatic scoring `POST /api/eval/submit` ## Request - Base URL: `https://aioq.org` - URL: `https://aioq.org/api/eval/submit` - Auth: none declared Scores a submission and, for every completed dual-gate attempt, issues a signed SCORE REPORT into the public registry. **There is no pass threshold.** A score report is issued whatever the scores are, and it carries no `passed` field: it reports the Gate A score, the Gate B score, the per-provision breakdown under the real article names, the conditions the measurement ran under, and a descriptive `referenceBand`. The 0.7 gate figure and the 0.5 per-provision figure are still reported, under `diagnostics`, so a reader can see where a score sits; they issue and withhold nothing. Tier 0 no longer issues certificates — the registry keeps the earlier ones (id AIO-C0-…) untouched. With an `attemptId` (from POST /api/eval/attempt) both gates are scored, each on the items that attempt served: Gate A on the public set, Gate B on the private variants drawn for the attempt. Gate B answers are keyed by the opaque per-attempt handles the attempt served and are resolved to their bank items server-side; `gateB.perItem` comes back under those same handles. Answers for ids the attempt did not serve are ignored and reported in `ignoredItemIds`. The attempt is consumed on the first submission, and a used or expired attempt returns 410. Without an `attemptId` only Gate A is scored, the response reports methodology `v0-draft`, and **nothing is issued** — the response explains the dual-gate flow. Scoring is deterministic and identical for both gates: per-item conformance 0–1 (exact hierarchy match 1.0, a code adjacent in the vocabulary 0.5, otherwise 0), averaged across the layers an item declares, then a weighted mean across items. Unanswered items score 0 and remain in the denominator. On the report, `score` and `gateAScore` are the same Gate A number (the first name is kept because legacy signatures cover it); the Gate B score is `gateBScore`. Every report also carries a signed `margin`: the Gate A model-nondeterminism floor, the Gate B draw-noise standard error predicted for that pack (null when the pack has no reference data), and the empirical upper bound observed across the reference campaign. Two scores that differ by less than the margin are not meaningfully different, and the figures are provisional pending large-N repeats. Where the pack's reference entry shows adaptation clearing the noise floor on Gate A but not on Gate B, the report additionally carries a `gateNote` describing which gate currently separates models on that pack — an observation about the reference panel, not a verdict on the measurement. Optional `conditions: { maxTokens?, temperature? }` are self-declared runner conditions. They are echoed into the signed payload under `conditions.runner` with `selfDeclared: true`, because AIO cannot observe how the model was called: the signature attests that the operator stated them. A value that is not sent is recorded as null — undeclared, never defaulted. Tier 0 is free, but the model name, model version, and operator are mandatory (taken from the attempt when one is supplied) and are published in the registry. `registrationId` is recorded but not verified. An `apiKey` is verified when supplied and raises the reported trust level to `registered`; it is optional, and neither its absence nor its invalidity rejects a submission. Rate-limited to 10 requests per 10 minutes per client. A score report records only the judgment distribution observed on AIO formalized items; it is not certification, not a legal conformity assessment, and it does not assess the organizational or management-system obligations of the reference norm. ## Request body - object - `attemptId` string: The attempt id from POST /api/eval/attempt. Required for a score report: with it both gates are scored on the items that attempt served, and the identity recorded on the attempt is authoritative (identity fields sent here are only cross-checked). Without it only Gate A is scored and nothing is issued. - `modelName` string: Required when no attemptId is supplied. - `modelVersion` string: The score report is pinned to this version. - `operator` object - `name` string, required: Published in the registry. - `email` string, email, required: Recorded only; never published. - `url` string, uri - `contact` string, email: Defaults to operator.email. - `packId` string: Required when no attemptId is supplied. - `registrationId` string: Recorded, not verified. - `conditions` object: Self-declared runner conditions. Both keys are optional and neither is verifiable by AIO, so they are echoed into the report's signed `conditions.runner` block marked `selfDeclared: true`: the signature attests that the operator stated these values, not that the model was called that way. A key that is not sent is recorded as null — undeclared, never defaulted. Only accepted meaningfully on a dual-gate submission; without an attemptId nothing is issued, so nothing is recorded. - `maxTokens` integer: The max_tokens the runner used. - `temperature` number: The sampling temperature the runner used. - `apiKey` string: An agent key (aio_k_…) with the 'eval:submit' scope. Verified when supplied, and the response reports the resulting trust level. Optional: Tier 0 is open, so a missing or invalid key never rejects the submission. - `answers` object[], required: With an attemptId, the answers to both gates go in this one array — the server splits them by the ids the attempt served, and ids it did not serve are ignored and reported in `ignoredItemIds`. Gate A answers are keyed by the published item id; Gate B answers are keyed by the opaque per-attempt handle the attempt served (a real Gate B bank id was not served, so it lands in `ignoredItemIds`). - `itemId` string, required: Exactly the `id` as served: the published bank id for a Gate A item, the opaque per-attempt handle (`h_<16 hex>`) for a Gate B item. - `response` unknown, required ## Response `200` Scored on Gate A alone, because no attemptId was supplied. Nothing is issued. - object - `scored` 'gateA' - `score` number: Gate A score. - `reportedThreshold` number: Reported diagnostic only. It gates nothing on any path. - `atOrAboveReportedThreshold` boolean: Reported diagnostic only. - `perItem` object[]: Gate A detail. - `scoreReport` unknown - `notes` string[] ## Other responses - `201`: The attempt was completed and a signed score report was issued. This is the only outcome of a dual-gate submission — there is nothing to fail. - `400`: Validation failed; `details` lists each problem. Also returned when a declared model, version, or pack does not match the attempt. - `404`: No public item bank for that pack, or no attempt with that id. - `410`: The attempt was already submitted or has expired. Start a new one at /api/eval/attempt. - `429`: Rate limit exceeded. - `503`: The attempt store or the private Gate B bank is unavailable on this deployment. The attempt is not consumed. --- [API](https://skmtc.dev/aioq/apis/aio-public-api.md) · [All operations](https://skmtc.dev/aioq/apis/aio-public-api/llms.txt) · [OpenAPI document](https://skmtc.dev/aioq/apis/aio-public-api/revisions/377b76120162?raw)