GPUClusterService

Create a GPU cluster

Changed on

Create an Instant Cluster on Together's high-performance GPU clusters. With features like on-demand scaling, long-lived resizable high-bandwidth shared DC-local storage, Kubernetes and Slurm cluster flavors, a REST API, and Terraform support, you can run workloads flexibly without complex infrastructure management.

post/v1/compute/clusters

Request

  • Base URL
    https://api.together.ai
  • URL
    https://api.together.ai/v1/compute/clusters
  • Auth
    HTTP bearer

Request body

GPUClusterCreateRequest required(unresolved $ref)

Response

OK

cluster_idstring required
cluster_type'KUBERNETES' | 'SLURM' required

Type of cluster.

regionstring required
gpu_type'H100_SXM' | 'H200_SXM' | 'RTX_6000_PCI' | 'L40_PCIE' | 'B200_SXM' | 'H100_SXM_INF' | 'B300_SXM' | 'GB200_SXM' required

Type of GPU used by the cluster.

cluster_namestring required
duration_hoursinteger
status'WaitingForControlPlaneNodes' | 'WaitingForDataPlaneNodes' | 'WaitingForSubnet' | 'WaitingForSharedVolume' | 'InstallingDrivers' | 'RunningAcceptanceTests' | 'Paused' | 'OnDemandComputePaused' | 'Ready' | 'Degraded' | 'Deleting' required

Current status of the GPU cluster.

kube_configstring required
num_gpusinteger required
slurm_shm_size_gibinteger
capacity_pool_idstring
reservation_start_timestring date-time
reservation_end_timestring date-time
install_traefikboolean
cuda_versionstring required
nvidia_driver_versionstring required
created_atstring date-time
project_idstring required
num_cpu_workersinteger required

Number of CPU-only worker nodes in the cluster.

desired_preemptible_gpusinteger required

Customer's requested number of preemptible GPUs. Set on cluster create or update; persists until changed.

allocated_preemptible_gpusinteger required

Actual number of preemptible GPUs currently allocated to the cluster. Updated asynchronously by the fulfillment and reclamation workers; may be less than desired_preemptible_gpus when capacity is constrained.

billing_type'RESERVED' | 'ON_DEMAND' | 'SCHEDULED_CAPACITY' required

Billing type for the cluster (RESERVED, ON_DEMAND, or SCHEDULED_CAPACITY).

machine_cluster_idstring

ID of the machine cluster backing this GPU cluster.

first_ready_atstring date-time

Timestamp when the cluster first reached the Ready phase.

is_in_substrateboolean

Whether the cluster runs on the Together-managed GPU cluster platform.

control_plane_readyboolean

Whether the control plane is currently ready.

ums_project_idstring

UMS project ID associated with this cluster.

ums_org_idstring

UMS organization ID associated with this cluster.

os_imagestring

Data-volume image name for GPU worker nodes.

nvidia_driver_version_idstring

Internal NVIDIA version ID for this cluster's driver and CUDA combination.

num_capacity_pool_gpusinteger required

Number of GPUs to draw from a capacity pool. A component of the overall num_gpus, alongside num_reserved_gpus.

num_reserved_gpusinteger required

Number of prepaid reserved GPUs for this cluster. A component of the overall num_gpus, alongside num_capacity_pool_gpus.

Changes

  • 93438ddac711Revision changes
    • breaking

      the response's property type changed from no type to string for status

    • warning

      added the new GB200_SXM enum value to the response property for the response status

    • info

      added the new GB200_SXM enum value to the request property ////////

    • info

      added the new GB200_SXM enum value to the request property ////////

  • bb5257b3173aRevision changes
    • info

      endpoint added