Submit Tier 0 answers for automatic scoring
Scores a submission and, for every completed dual-gate attempt, issues a signed SCORE REPORT into the public registry. There is no pass threshold. A score report is issued whatever the scores are, and it carries no passed field: it reports the Gate A score, the Gate B score, the per-provision breakdown under the real article names, the conditions the measurement ran under, and a descriptive referenceBand. The 0.7 gate figure and the 0.5 per-provision figure are still reported, under diagnostics, so a reader can see where a score sits; they issue and withhold nothing. Tier 0 no longer issues certificates — the registry keeps the earlier ones (id AIO-C0-…) untouched. With an attemptId (from POST /api/eval/attempt) both gates are scored, each on the items that attempt served: Gate A on the public set, Gate B on the private variants drawn for the attempt. Gate B answers are keyed by the opaque per-attempt handles the attempt served and are resolved to their bank items server-side; gateB.perItem comes back under those same handles. Answers for ids the attempt did not serve are ignored and reported in ignoredItemIds. The attempt is consumed on the first submission, and a used or expired attempt returns 410. Without an attemptId only Gate A is scored, the response reports methodology v0-draft, and nothing is issued — the response explains the dual-gate flow. Scoring is deterministic and identical for both gates: per-item conformance 0–1 (exact hierarchy match 1.0, a code adjacent in the vocabulary 0.5, otherwise 0), averaged across the layers an item declares, then a weighted mean across items. Unanswered items score 0 and remain in the denominator. On the report, score and gateAScore are the same Gate A number (the first name is kept because legacy signatures cover it); the Gate B score is gateBScore. Every report also carries a signed margin: the Gate A model-nondeterminism floor, the Gate B draw-noise standard error predicted for that pack (null when the pack has no reference data), and the empirical upper bound observed across the reference campaign. Two scores that differ by less than the margin are not meaningfully different, and the figures are provisional pending large-N repeats. Where the pack's reference entry shows adaptation clearing the noise floor on Gate A but not on Gate B, the report additionally carries a gateNote describing which gate currently separates models on that pack — an observation about the reference panel, not a verdict on the measurement. Optional conditions: { maxTokens?, temperature? } are self-declared runner conditions. They are echoed into the signed payload under conditions.runner with selfDeclared: true, because AIO cannot observe how the model was called: the signature attests that the operator stated them. A value that is not sent is recorded as null — undeclared, never defaulted. Tier 0 is free, but the model name, model version, and operator are mandatory (taken from the attempt when one is supplied) and are published in the registry. registrationId is recorded but not verified. An apiKey is verified when supplied and raises the reported trust level to registered; it is optional, and neither its absence nor its invalidity rejects a submission. Rate-limited to 10 requests per 10 minutes per client. A score report records only the judgment distribution observed on AIO formalized items; it is not certification, not a legal conformity assessment, and it does not assess the organizational or management-system obligations of the reference norm.
Request
- Base URL: https://aioq.org
- URL: https://aioq.org/api/eval/submit
- Auth: none declared
Request body
Example request
{
"packId": "eu-ai-act",
"answers": [
{
"itemId": "eu-ai-act-001"
}
]
}Response
Scored on Gate A alone, because no attemptId was supplied. Nothing is issued.
Changes
No recorded changes to this operation across all 1 revision of this API.