EndpointService

Create an endpoint

Changed on

Creates a stable, inference-addressable endpoint. Add one or more deployments and configure its traffic split before sending inference requests to the endpoint name.

post/v2/projects/{projectId}/endpoints

Request

  • Base URL
    https://api.together.ai
  • URL
    https://api.together.ai/v2/projects/{projectId}/endpoints
  • Auth
    HTTP bearer

Path parameters

projectIdstring required

Project identifier.

Request body

namestring required

Inference-addressable endpoint name to create.

visibility'VISIBILITY_PRIVATE' | 'VISIBILITY_INTERNAL'

Who can discover the endpoint. VISIBILITY_PRIVATE restricts it to the project; VISIBILITY_INTERNAL shares it with the organization.

Response

OK

idstring required

Unique endpoint identifier.

projectIdstring required

ID of the project that owns the endpoint.

namestring required

Project-qualified endpoint name in the form <project_slug>/<endpoint_name>. Pass this value as model in inference requests. Create and update requests may use either a bare endpoint name or the qualified form; a supplied project slug must match the project in the request path.

createdAtstring date-time required

Timestamp when the endpoint was created.

updatedAtstring date-time required

Output only. Timestamp when the endpoint was last updated.

etagstring required

Opaque version tag for optimistic concurrency control. Supply on update/delete to ensure consistent read-modify-write. If not set, the write overwrites based on current state.

visibility'VISIBILITY_PRIVATE' | 'VISIBILITY_INTERNAL' required

Who can discover the endpoint. VISIBILITY_PRIVATE restricts it to the project; VISIBILITY_INTERNAL shares it with the organization.

endpointType'ENDPOINT_TYPE_DEDICATED' | 'ENDPOINT_TYPE_SERVERLESS' | 'ENDPOINT_TYPE_RESERVED' required

Serving class of the endpoint. Reserved endpoints use reserved capacity.

activeRolloutIdstring

ID of the currently active rollout in an in-flight state, including paused.

Example response

{
  "deployments": [
    {
      "autoscaling": {
        "scaleUp": {
          "policies": [
            {
              "type": "SCALING_POLICY_TYPE_PERCENT",
              "value": 25,
              "periodSeconds": 60
            },
            {
              "type": "SCALING_POLICY_TYPE_PODS",
              "value": 10,
              "periodSeconds": 60
            }
          ],
          "selectPolicy": "SCALING_POLICY_SELECT_MIN"
        },
        "scaleDown": {
          "policies": [
            {
              "type": "SCALING_POLICY_TYPE_PERCENT",
              "value": 25,
              "periodSeconds": 60
            },
            {
              "type": "SCALING_POLICY_TYPE_PODS",
              "value": 10,
              "periodSeconds": 60
            }
          ],
          "selectPolicy": "SCALING_POLICY_SELECT_MIN"
        }
      }
    }
  ]
}

Changes