--- title: "Debug Requirement" method: GET path: "/internal/evals/rollout-consistency/sweeps/{sweep_id}/requirements/{requirement_id}" tags: ["internal"] --- # Debug Requirement `GET /internal/evals/rollout-consistency/sweeps/{sweep_id}/requirements/{requirement_id}` ## Request - Base URL: `https://api.kobaltlabs.com` (the document declares no server; this is the origin it was published from) - URL: `https://api.kobaltlabs.com/internal/evals/rollout-consistency/sweeps/{sweep_id}/requirements/{requirement_id}` - Auth: none declared One requirement, as the thing to be judged rather than a row in a ranking. Three questions in one response, because answering them separately is what makes a reader guess: what does this rule say now, where in the standard did it come from, and what has it actually done to real collateral. ## Path parameters - `sweep_id` string, uuid, required - `requirement_id` string, uuid, required ## Response `200` Successful Response - RequirementDebug: Everything needed to judge one requirement: what it says, where it came from, what it did. - `requirement_id` string, uuid, required - `revision_id` string, uuid, required - `revision_number` integer, required - `state` 'active' | 'disabled' | 'archived' | 'superseded' | 'proposed', required: Whether a requirement is in the working library, and if not, why not. All of them are recorded as revisions, so every one is reversible and readable. ARCHIVED and SUPERSEDED are deliberately distinct: ARCHIVED is a person deciding this obligation does not apply to them, SUPERSEDED is a newer decomposition of the same guideline taking over. A regeneration must never overwrite the first with the second — that would erase a human decision. - `requirement_text` string, required - `judging_guidance` string, nullable, required - `trigger_terms` string[], required - `collateral_types` string[], nullable, required - `marketing_team_ids` string[], nullable, required - `provenance` RequirementProvenance, required: Where in the standards document this obligation came from. The point of the whole panel: a reviewer deciding whether a rule is worth keeping has to read what it was extracted *from*, not just what the model wrote. `source_text` is the paragraph the grounding pass located; it is a lexical best guess and never a quotation, which `match_kind` qualifies and the surface says out loud. `score` is deliberately **not** here. It reaches 1.0 exactly when the paraphrase dropped the discriminating word, so it is highest when it is least trustworthy — surfacing it, or ordering anything by it, reads as confidence while measuring the opposite. - `guideline_file_id` string, uuid, nullable, required - `guideline_filename` string, nullable, required - `guideline_upload_id` string, uuid, nullable, required - `section_label` string, nullable, required - `title` string, nullable, required - `source_page` integer, nullable, required - `source_text` string, nullable, required - `matched_terms` string[], required - `match_kind` string, nullable, required - `boxes` object[], required - `flagged_on` RequirementFlaggedOn[], required - `check_id` string, uuid, required - `run_id` string, uuid, required - `file_id` string, uuid, nullable, required - `upload_id` string, uuid, nullable, required - `filename` string, nullable, required - `outcome` 'passed' | 'failed' | 'not_determined', required: What happened when one requirement met one piece of collateral. Stored rather than derived. "We checked 34 requirements and 3 failed" cannot be reconstructed from the risks alone: a requirement that produced no finding is indistinguishable in `document_risks` from one that was never evaluated, and that difference is the whole feature. **`NOT_DETERMINED` is why the model is asked to answer for every rule** rather than only to report failures. Silence has two meanings — "the collateral satisfies this" and "I cannot tell from what the collateral shows" — and a panel that reads the first from the absence of a finding claims a pass the model never gave. The judging contract therefore makes each rule land in exactly one of these three, and a rule the model omits altogether lands here too. - `risk_id` string, uuid, nullable, required - `risk_summary` string, nullable, required - `risk_severity` string, nullable, required - `link_source` string, nullable, required - `from_this_sweep` boolean, required - `run_time` string, date-time, nullable, required - `flagged_on_truncated` boolean - `drift` DriftRow[], required - `requirement_id` string, uuid, required - `guideline_file_id` string, uuid, nullable, required - `guideline_filename` string, nullable, required - `kind` 'presence' | 'outcome' | 'revision', required: Why two runs of the same collateral disagreed about one requirement. Three kinds, kept apart because they have three different fixes and one "disagreement" count would point at none of them. - `outcome_a` 'passed' | 'failed' | 'not_determined', required: What happened when one requirement met one piece of collateral. Stored rather than derived. "We checked 34 requirements and 3 failed" cannot be reconstructed from the risks alone: a requirement that produced no finding is indistinguishable in `document_risks` from one that was never evaluated, and that difference is the whole feature. **`NOT_DETERMINED` is why the model is asked to answer for every rule** rather than only to report failures. Silence has two meanings — "the collateral satisfies this" and "I cannot tell from what the collateral shows" — and a panel that reads the first from the absence of a finding claims a pass the model never gave. The judging contract therefore makes each rule land in exactly one of these three, and a rule the model omits altogether lands here too. - `outcome_b` 'passed' | 'failed' | 'not_determined', required: What happened when one requirement met one piece of collateral. Stored rather than derived. "We checked 34 requirements and 3 failed" cannot be reconstructed from the risks alone: a requirement that produced no finding is indistinguishable in `document_risks` from one that was never evaluated, and that difference is the whole feature. **`NOT_DETERMINED` is why the model is asked to answer for every rule** rather than only to report failures. Silence has two meanings — "the collateral satisfies this" and "I cannot tell from what the collateral shows" — and a panel that reads the first from the absence of a finding claims a pass the model never gave. The judging contract therefore makes each rule land in exactly one of these three, and a rule the model omits altogether lands here too. - `visible_flip` 'unmet_in_a_only' | 'unmet_in_b_only', required: Which way one (collateral, requirement) pair flipped *in the reviewer's view*. The strict agreement figure counts a pair as agreeing only when both runs judged the same revision of the rule and returned the identical verdict, which is the right denominator for tuning — `DriftKind` splits the disagreements three ways because they have three different fixes. It is the wrong denominator for the question a rollout decision actually turns on: *would the customer have seen a different report?* Met against not-conclusive reads as drift and changes nothing on screen; a rule that only reached the judge in one run reads as presence drift and changes everything if that run found it unmet. So this projects both sides through `RISK_RAISING_OUTCOMES` — unmet, or not — and names the two directions that survive. Never summed with `DriftKind` counts: this is the same set of disagreements read a second way, not more of them. Named `A_ONLY` / `B_ONLY` rather than lost/gained on purpose. The two trial runs are the same path run twice in sequence, so neither is the "after"; and the regression half of this dashboard already means something specific by a *lost* finding — a risk the customer's reviewer flagged that the new path stopped producing. A skew between these two is worth a look on its own (one run reaching more of the library than the other), but neither direction is a verdict. - `matched_trigger_terms` string[], required - `requirement_text` string, nullable, required - `filename` string, nullable, required - `drift_truncated` boolean ## Other responses - `422`: Validation Error ## Changes - **2026-09-02** `425d5b8a3c17`: 1 warning - added the new `proposed` enum value to the `state` response property for the response status `200` [Operation history](https://skmtc.dev/kobaltlabs/apis/fastapi/changes/internal/evals/rollout-consistency/sweeps/:sweep_id/requirements/:requirement_id/get.md) --- [API](https://skmtc.dev/kobaltlabs/apis/fastapi.md) - [All operations](https://skmtc.dev/kobaltlabs/apis/fastapi/llms.txt) - [OpenAPI document](https://skmtc.dev/kobaltlabs/apis/fastapi/revisions/8587332d5e43?raw)