Health Drain

Graceful-drain probe for Kubernetes preStop hooks.

Disabled by default and returns 404 unless general_settings sets enable_drain_endpoint: true. Calling it flips a process-wide shutting-down flag, so a successful call permanently takes the worker out of rotation until the pod restarts.

Because the kubelet calls preStop hooks without proxy credentials, the endpoint does not require user_api_key_auth. To prevent any pod-reachable caller from triggering shutdown, set general_settings.drain_endpoint_token (or the DRAIN_ENDPOINT_TOKEN env var) and supply the same value on the X-Drain-Token header from the preStop hook. Calls without the header (or with a wrong value) get a 401 and have no side effect.

When enabled, it marks the worker as shutting down (so /health/readiness and /health/liveliness immediately start returning 503, removing the pod from service) and blocks until the in-flight request counter drains to zero or GRACEFUL_SHUTDOWN_TIMEOUT elapses. Unlike a fixed sleep, this returns as soon as real in-flight work is done.

Wire it up as:

lifecycle:
  preStop:
    httpGet:
      path: /health/drain
      port: 4000
      httpHeaders:
        - name: X-Drain-Token
          value: <same value as drain_endpoint_token>