Sign in
Internal

Start or abort an in-place rolling rollout (platform admin)

start bumps the deployment's Version IN PLACE and replaces its replicas one at a time; drain, bounded in-flight window, terminate, admit into the freed units, wait ready; so a saturated pool can revise without capacity headroom (which is exactly what blue/green needs and rolling does not). Requires the deployment's rolloutStrategy to be "rolling". The lane lock is held for the campaign's lifetime: create/revise answer 409 while it runs. pause/resume reuse the existing rollout verb. abort stops the campaign and releases the lane; drained victims re-activate on the next reconcile pass (membership heals level-triggered), so abort never leaves a replica out of rotation permanently.

post/v1/admin/inference/deployments/{id}/rolling-rollout

Path parameters

idstring required

Request body

action'start' | 'abort' required
causestring

Free-text reason, recorded in the version history

versionstring

Required for start: the target version tag replicas are replaced to

Response

The refreshed deployment

autoscalePausedboolean

True while an operator has frozen the load-driven autoscaler. The min/max band is kept as bounds and reconcile still converges the fleet to replicasDesired; the autoscaler just stops moving replicasDesired on its own. An operator can still hand-set replicasDesired within the band through /scale. Absent when unset.

backendCatalogWeightinteger

Live catalog routing weight of this deployment's backend. Only present when backendRegistered is true. May differ from backendWeight (the row mirror) while a rollout ramp is in flight.

backendHealthyboolean

Live catalog health of this deployment's backend (the kill switch). Only present when backendRegistered is true.

backendRegisteredboolean

Whether this deployment's pool:// backend is currently present on the catalog model row. Absent when the live catalog state could not be read; false until the first replica passes readiness.

backendVersionstring

The rollout tag this deployment's backend reports to the gateway as the version metric label on ig_model_requests_total and ig_stream_ttft_seconds. Use it to separate the two sides of a rollout in a dashboard query. Only present when backendRegistered is true.

backendWeightinteger required

Routing weight of the catalog backend. 0 = standby (blue/green starts here)

changeCausestring

Operator-supplied reason for the revision that created this deployment

createdAtstring required
dtypestring
enginestring required

Inference engine, e.g. vllm or sglang

engineImagestring

Engine container image override; absent = the engine's default

envobject
extraArgsstring[]
generationinteger

1 for a hand-created deployment, parent+1 for each revision. History ordering only: the controller never reads it, because rollout adoption is the explicit rolloutRole opt-in and nothing else.

gpuCountPerReplicainteger required
gpuModelIdstring required
idstring required
isolatedboolean required

True when replicas and their private endpoint pool are for direct operator testing only. Isolated deployments never register a catalog backend, never participate in blue/green rollout, and cannot receive gateway traffic.

maxConcurrencyPerReplicainteger
maxModelLeninteger
maxPromptBytesinteger

Exclusive text-request byte ceiling for routing eligibility; 0 or absent = unbounded. Routing hint only, not a context limit.

maxUnavailableinteger

Rolling only: how many replicas may be simultaneously not-ready (absolute count; surfaces show the percentage of desired).

minPromptBytesinteger

Inclusive text-request byte floor for routing eligibility; 0 or absent = unbounded. Routing hint only, not a context limit.

nativeFamiliesstring[]
nodePoolstring

Stable logical capacity lane within the public model. Blue/green only replaces versions in the same node pool; empty preserves legacy model-wide grouping.

organizationIdstring required

Org that owns the managed replicas (and their metering)

parentDeploymentIdstring

The deployment this one was revised from. Set only by /revise, so an absent value means this deployment was created by hand.

poolIdstring

Endpoint pool the deployment's replicas register in

publicModelIdstring required

Catalog model this deployment serves capacity for

quantizationstring
readyReplicasinteger required
replicaRuntime'' | 'vm' | 'pod'

How each replica is placed. Absent or "" or "vm" = a QEMU/VFIO VM on the baremetal fleet (the legacy shape; every pre-existing deployment reads as this). "pod" = a hardened GPU pod on a provider-onboarded runc node. One deployment is one runtime; a public model mixes runtimes through lanes.

replicaVmIdsstring[]
replicasDesiredinteger required
replicasMaxinteger required
replicasMininteger required
rollingActiveboolean

True while an in-place rolling rollout is in flight on this deployment.

rolloutPausedboolean

True while an operator has paused the ramp. Traffic is untouched: the deployment keeps the weight it holds and keeps serving, and the controller takes no rollout action until resume. The pause is also on rollout.paused; this raw column is what a surface falls back to when the lane could not be read at all.

rolloutRole'' | 'green'

Opt-in blue/green role. Empty (default) = an INDEPENDENT weighted capacity member of its lane, never blue or green. "green" = the lane's successor generation: the controller ramps it against the older members under a capacity guard and cuts them over once it reaches capacity parity. Rollouts are never implicit.

rolloutStrategy'blue_green' | 'rolling'

How a version bump reaches the fleet. blue_green (default) mints a ramped sibling; rolling replaces this deployment's replicas in place one at a time.

servedModelIdstring required

Model id the engine serves; must equal the catalog backend ref

status'pending' | 'deploying' | 'ready' | 'degraded' | 'scaling' | 'draining' | 'stopped' | 'failed' required
statusReasonstring

Human-readable cause for degraded/failed states

updatedAtstring required
versionstring

Generation marker for blue/green within one public model and node pool; the target a rolling rollout replaces replicas to

weightsRefstring

Staged-checkpoint reference; absent = the engine loads from its own source

Changes