---
title: "Post-maintenance node re-entry sweep checklist (Slurm NHC)"
method: GET
path: "/api/maintenance/node-sweep-checklist"
tags: ["Maintenance"]
---

# Post-maintenance node re-entry sweep checklist (Slurm NHC)

`GET /api/maintenance/node-sweep-checklist`

## Request

- Base URL: `https://api.fiber.ac` (the document declares no server; this is the origin it was published from)
- URL: `https://api.fiber.ac/api/maintenance/node-sweep-checklist`
- Auth: none declared

Read-only checklist for returning a compute node to service after maintenance (MLSys 2026 Guard, arXiv 2605.17879: sweep before re-entry). Resolves `hostname` (FQDN or short) to its leaf uplink port via gpu_nodes (fallback: last 24 h of gpu_telemetry) and evaluates: `uplink` — risk-engine risk_score / severity and telemetry silence of the uplink optic; `los` — OpenConfig transceiver v1.0.0 RX LOS/LOL reset-safe increase over the last 1 h (only when the columns exist); `gpu` — latest DCGM health per GPU (detachment, XID, ECC DBE, DCGM health, thermal, roster); `nccl` — recent NCCL Inspector / RAS events (a non-failing `ncclras -D` diagnostic — OK/PASS or SKIP, which share `ras_severity='info'` — is counted as `nccl.diag_info_checks` ("OK or skipped", not a pass count), never as a RAS event or a low-bandwidth collective; a diagnostic FAIL still counts as RAS critical). Each check is pass | warn | fail | unknown; `recommended` is `hold` when any check fails (or warns with `strict=true`), else `sweep_pass`. `sections=uplink` is the cheap mode used by scripts/nhc/check_optical_uplink.nhc (one PostgreSQL lookup + two ClickHouse point reads): the NHC check dies when `uplink.risk_score >= risk_hold` or `uplink.silent` is true. Sections degrade to `unknown` (never 500) when a backend or column is missing. 404 only when the hostname is genuinely neither registered in gpu_nodes nor seen in gpu_telemetry; a lookup that could not be answered because a store is down returns 503 (`lookup_backend_error`) so the client's fail-mode — not its unknown-node policy — decides. Tenant-scoped keys (non-`default`) only resolve nodes whose switch belongs to that tenant, and every optical read is filtered by `tenant_id`.

## Query parameters

- `hostname` string, required: Compute node hostname (FQDN or short; matched against gpu_nodes.hostname)
- `sections` string: `all` or comma list of uplink,los,gpu,nccl — `uplink` alone is the cheap NHC mode
- `risk_hold` number: uplink risk_score at/above which the node is held (default 60 = warning+critical)
- `silence_sec` integer: uplink telemetry older than this is `silent` (default 180 s = collector watchdog floor)
- `nccl_minutes` integer: NCCL / RAS look-back window in minutes
- `strict` boolean: Escalate warn checks (thermal, partial roster, RAS, LOS warn, risk warning) to hold

## Response `200`

Per-check verdicts, uplink/los/gpu/nccl detail blocks and recommended sweep_pass | hold

- unknown

## Other responses

- `422`: Validation Error

---

[API](https://skmtc.dev/fiber-ac/apis/fiber-api.md) · [All operations](https://skmtc.dev/fiber-ac/apis/fiber-api/llms.txt) · [OpenAPI document](https://skmtc.dev/fiber-ac/apis/fiber-api/revisions/58267250d793?raw)