DeploymentService

Create a deployment

Changed on

Creates a model deployment under an endpoint. The deployment provisions asynchronously; monitor its status before routing live traffic to it.

post/v2/projects/{projectId}/endpoints/{endpointId}/deployments

Request

  • Base URL
    https://api.together.ai
  • URL
    https://api.together.ai/v2/projects/{projectId}/endpoints/{endpointId}/deployments
  • Auth
    HTTP bearer

Path parameters

projectIdstring required

ID of the project that owns the endpoint.

endpointIdstring required

ID of the endpoint that will contain the deployment.

Query parameters

validateOnlyboolean

When true, validates the request without creating or provisioning a deployment.

Request body

namestring required

Name for the deployment within its endpoint. Returned as a fully-qualified endpoint string.

modelIdstring

Deprecated. Use model. Model identifier to serve, accepted when model is unset.

modelRevisionIdstring

Deprecated. Use model with a /revisions/{revisionId} segment. If omitted, the latest revision is resolved at creation.

configIdstring

Deprecated. Use config. Config revision identifier to deploy, accepted when config is unset.

modelstring

Model resource name in the form projects/{projectId}/models/{modelId}[/revisions/{revisionId}]. Omit the revision segment to pin the latest revision at creation time.

configstring

Immutable config revision in the form projects/{projectId}/configs/{configRevisionId}. The config must be compatible with the model.

inactiveTimeoutinteger

Inactive timeout in minutes. Use 0 or omit to disable automatic stopping; otherwise accepted values are 30 through 1440.

maxConcurrentRequestsPerReplicastring

Maximum number of inference requests that may be in flight to a single replica. If omitted, the platform uses one less than the config's per-replica concurrency limit to reserve a health-check slot. Values above that maximum are reduced on create; 0 means unlimited when the config limit is 1 or less.

Example request

{
  "autoscaling": {
    "scaleUp": {
      "policies": [
        {
          "type": "SCALING_POLICY_TYPE_PERCENT",
          "value": 25,
          "periodSeconds": 60
        },
        {
          "type": "SCALING_POLICY_TYPE_PODS",
          "value": 10,
          "periodSeconds": 60
        }
      ],
      "selectPolicy": "SCALING_POLICY_SELECT_MIN"
    },
    "scaleDown": {
      "policies": [
        {
          "type": "SCALING_POLICY_TYPE_PERCENT",
          "value": 25,
          "periodSeconds": 60
        },
        {
          "type": "SCALING_POLICY_TYPE_PODS",
          "value": 10,
          "periodSeconds": 60
        }
      ],
      "selectPolicy": "SCALING_POLICY_SELECT_MIN"
    }
  }
}

Response

OK

idstring required

Unique deployment identifier.

projectIdstring required

ID of the project that owns the deployment.

endpointIdstring required

ID of the endpoint that contains the deployment.

namestring required

Project- and endpoint-qualified deployment name in the form <project_slug>/<endpoint_name>/<deployment_name>. Pass it as model in an inference request to target this deployment directly instead of using the endpoint's traffic split.

createdAtstring date-time required

Timestamp when the deployment was created.

updatedAtstring date-time required

Timestamp when the deployment was last updated.

modelIdstring required

Deprecated. Use model. Model identifier being served, populated during migration.

modelRevisionIdstring required

Deprecated. Use model with a /revisions/{revisionId} segment. Pin to a specific model revision.

modelstring required

Pinned model resource in the form projects/{projectId}/models/{modelId}/revisions/{revisionId}.

configIdstring required

Deprecated. Use config. Config revision identifier, populated during migration.

configstring required

Immutable config revision in the form projects/{projectId}/configs/{configRevisionId}.

speculatorIdstring

Deprecated. Use speculator. Speculative decoding model identifier derived from the deployment config.

speculatorRevisionIdstring

Deprecated. Use speculator. ID of the speculative decoding draft-model revision pinned at creation time.

speculatorstring

Pinned draft-model resource used for speculative decoding, in the same form as model. Omitted when speculative decoding is disabled.

estimatedEffectiveTrafficSharenumber double

Estimated fraction in [0, 1] of endpoint traffic that reaches this deployment under the current routing configuration. Absent or unrouted deployments are 0.

maxConcurrentRequestsPerReplicastring

Maximum number of inference requests that may be in flight to a single replica. If omitted, the platform uses one less than the config's per-replica concurrency limit to reserve a health-check slot. Values above that maximum are reduced on create and update; 0 means unlimited when the config limit is 1 or less. Changes take effect without restarting replicas.

inactiveTimeoutinteger

Minutes without an inference request before the deployment stops automatically. Omitted or 0 means automatic stopping is disabled.

etagstring required

Opaque version tag for optimistic concurrency control. Supply on update/delete to ensure consistent read-modify-write. If not set, the write overwrites based on current state.

hardwarestring required

Hardware selected by the deployment config, including GPU type and count.

trafficMode'TRAFFIC_MODE_LIVE' | 'TRAFFIC_MODE_SHADOW' required

Whether the deployment serves client-visible responses or only mirrored shadow traffic.

desiredReplicasinteger

Number of replicas the autoscaler currently wants across all regions. Not settable on any request; steer it through autoscaling.minReplicas and autoscaling.maxReplicas.

Example response

{
  "autoscaling": {
    "scaleUp": {
      "policies": [
        {
          "type": "SCALING_POLICY_TYPE_PERCENT",
          "value": 25,
          "periodSeconds": 60
        },
        {
          "type": "SCALING_POLICY_TYPE_PODS",
          "value": 10,
          "periodSeconds": 60
        }
      ],
      "selectPolicy": "SCALING_POLICY_SELECT_MIN"
    },
    "scaleDown": {
      "policies": [
        {
          "type": "SCALING_POLICY_TYPE_PERCENT",
          "value": 25,
          "periodSeconds": 60
        },
        {
          "type": "SCALING_POLICY_TYPE_PODS",
          "value": 10,
          "periodSeconds": 60
        }
      ],
      "selectPolicy": "SCALING_POLICY_SELECT_MIN"
    }
  }
}

Changes