BAbaseten

Updates a deployment's autoscaling settings

Updates a deployment's autoscaling settings and returns the update status.

patch/v1/models/{model_id}/deployments/{deployment_id}/autoscaling_settings

Request

Request body

min_replicainteger nullable

Minimum number of replicas

max_replicainteger nullable

Maximum number of replicas

autoscaling_windowinteger nullable

Timeframe of traffic considered for autoscaling decisions

scale_down_delayinteger nullable

Waiting period before scaling down any active replica

concurrency_targetinteger nullable

Number of requests per replica before scaling up

target_utilization_percentageinteger nullable

Target utilization percentage for scaling up/down.

target_in_flight_tokensinteger nullable

Target number of in-flight tokens for autoscaling decisions. Early access only.

max_scale_down_rateinteger nullable

Maximum percentage of replicas that can be removed per autoscaling window (1–50). E.g. 20 means at most 20% of replicas are removed per window.

Example request

{
  "max_replica": 7,
  "autoscaling_window": 600,
  "scale_down_delay": 120,
  "concurrency_target": 2,
  "target_utilization_percentage": 70,
  "target_in_flight_tokens": 40000,
  "max_scale_down_rate": 20
}

Response

status'ACCEPTED' | 'QUEUED' | 'UNCHANGED' required

The status of a request to update autoscaling settings.

messagestring required

A message describing the status of the request to update autoscaling settings

Changes

No recorded changes to this operation across all 1 revision of this API.