DeploymentService

Update a deployment

Changed on

Updates mutable deployment fields such as its model, configuration, autoscaling bounds, or LoRA support. Changes that affect serving may trigger asynchronous reprovisioning.

patch/v2/projects/{projectId}/endpoints/{endpointId}/deployments/{id}

Request

  • Base URL
    https://api.together.ai
  • URL
    https://api.together.ai/v2/projects/{projectId}/endpoints/{endpointId}/deployments/{id}
  • Auth
    HTTP bearer

Path parameters

projectIdstring required

Project identifier.

endpointIdstring required

Endpoint identifier.

idstring required

Deployment identifier.

Query parameters

updateMaskstring

Fields to update. If not set, the fields populated on deployment are updated.

Request body

namestring

Updated endpoint string.

inactiveTimeoutinteger

Updated inactive timeout in minutes. Use 0 to disable automatic stopping; otherwise accepted values are 30 through 1440.

maxConcurrentRequestsPerReplicastring

Updated maximum number of inference requests that may be in flight to a single replica. Values above the deployment config's per-replica concurrency limit minus one are reduced on update; 0 means unlimited when the config limit is 1 or less. Changes take effect without restarting replicas.

etagstring

Current deployment version. The update is rejected if this value no longer matches.

Example request

{
  "autoscaling": {
    "scaleUp": {
      "policies": [
        {
          "type": "SCALING_POLICY_TYPE_PERCENT",
          "value": 25,
          "periodSeconds": 60
        },
        {
          "type": "SCALING_POLICY_TYPE_PODS",
          "value": 10,
          "periodSeconds": 60
        }
      ],
      "selectPolicy": "SCALING_POLICY_SELECT_MIN"
    },
    "scaleDown": {
      "policies": [
        {
          "type": "SCALING_POLICY_TYPE_PERCENT",
          "value": 25,
          "periodSeconds": 60
        },
        {
          "type": "SCALING_POLICY_TYPE_PODS",
          "value": 10,
          "periodSeconds": 60
        }
      ],
      "selectPolicy": "SCALING_POLICY_SELECT_MIN"
    }
  }
}

Response

OK

idstring required

Unique deployment identifier.

projectIdstring required

ID of the project that owns the deployment.

endpointIdstring required

ID of the endpoint that contains the deployment.

namestring required

Project- and endpoint-qualified deployment name in the form <project_slug>/<endpoint_name>/<deployment_name>. Pass it as model in an inference request to target this deployment directly instead of using the endpoint's traffic split.

createdAtstring date-time required

Timestamp when the deployment was created.

updatedAtstring date-time required

Timestamp when the deployment was last updated.

modelIdstring required

Deprecated. Use model. Model identifier being served, populated during migration.

modelRevisionIdstring required

Deprecated. Use model with a /revisions/{revisionId} segment. Pin to a specific model revision.

modelstring required

Pinned model resource in the form projects/{projectId}/models/{modelId}/revisions/{revisionId}.

configIdstring required

Deprecated. Use config. Config revision identifier, populated during migration.

configstring required

Immutable config revision in the form projects/{projectId}/configs/{configRevisionId}.

speculatorIdstring

Deprecated. Use speculator. Speculative decoding model identifier derived from the deployment config.

speculatorRevisionIdstring

Deprecated. Use speculator. ID of the speculative decoding draft-model revision pinned at creation time.

speculatorstring

Pinned draft-model resource used for speculative decoding, in the same form as model. Omitted when speculative decoding is disabled.

estimatedEffectiveTrafficSharenumber double

Estimated fraction in [0, 1] of endpoint traffic that reaches this deployment under the current routing configuration. Absent or unrouted deployments are 0.

maxConcurrentRequestsPerReplicastring

Maximum number of inference requests that may be in flight to a single replica. If omitted, the platform uses one less than the config's per-replica concurrency limit to reserve a health-check slot. Values above that maximum are reduced on create and update; 0 means unlimited when the config limit is 1 or less. Changes take effect without restarting replicas.

inactiveTimeoutinteger

Minutes without an inference request before the deployment stops automatically. Omitted or 0 means automatic stopping is disabled.

etagstring required

Opaque version tag for optimistic concurrency control. Supply on update/delete to ensure consistent read-modify-write. If not set, the write overwrites based on current state.

hardwarestring required

Hardware selected by the deployment config, including GPU type and count.

trafficMode'TRAFFIC_MODE_LIVE' | 'TRAFFIC_MODE_SHADOW' required

Whether the deployment serves client-visible responses or only mirrored shadow traffic.

desiredReplicasinteger

Number of replicas the autoscaler currently wants across all regions. Not settable on any request; steer it through autoscaling.minReplicas and autoscaling.maxReplicas.

Example response

{
  "autoscaling": {
    "scaleUp": {
      "policies": [
        {
          "type": "SCALING_POLICY_TYPE_PERCENT",
          "value": 25,
          "periodSeconds": 60
        },
        {
          "type": "SCALING_POLICY_TYPE_PODS",
          "value": 10,
          "periodSeconds": 60
        }
      ],
      "selectPolicy": "SCALING_POLICY_SELECT_MIN"
    },
    "scaleDown": {
      "policies": [
        {
          "type": "SCALING_POLICY_TYPE_PERCENT",
          "value": 25,
          "periodSeconds": 60
        },
        {
          "type": "SCALING_POLICY_TYPE_PODS",
          "value": 10,
          "periodSeconds": 60
        }
      ],
      "selectPolicy": "SCALING_POLICY_SELECT_MIN"
    }
  }
}

Changes