TOtogether
RL

Estimate rollout time and price

Changed on

Estimates rollout-only time, generator GPU-hours, billed GPU-hours of the whole model resource, and price for a supported model and GPU type. Each generator replica may contain several inference servers or span multiple nodes. Assumes identical episodes, exact reuse of generated tokens between turns, turns that start together within a replica, no environment delay, and no concurrent computation of the same initial prompt. Prompt-cache reuse and capacity misses are approximate, especially for multi-turn episodes. Rates are measured in the training mode reported by lora_enabled. Cache-transfer and request-preemption delays are not included, so long multi-turn rollouts with distinct prompts can take longer than estimated.

post/v1/rl/rollouts/estimate

Request

  • Base URL
    https://api.together.ai
  • Auth
    HTTP bearer

Request body

episodesinteger required

Total number of episodes (N). Must be between 1 and 10000000.

concurrencyinteger required

Maximum concurrent episodes across all generator replicas (B). Must be positive.

prompt_tokensinteger required

Initial prompt tokens per episode (P). Must be positive.

turnsinteger required

Assistant turns per episode (T). Must be between 1 and 1024.

added_tokens_per_turninteger required

Tokens added before each later turn (K), including the prior generated tokens followed by environment tokens. Must be at least generated_tokens_per_turn.

generated_tokens_per_turninteger required

Tokens generated on each turn (O). Must be positive.

distinct_prompts_per_replicainteger

Number of distinct initial prompts among the concurrent episodes of one replica (Q). When episodes_per_prompt is given, defaults to ceil(concurrent episodes / G). Otherwise defaults to 1 when shared_prefix_tokens equals prompt_tokens, and to the concurrent episodes per replica when it does not. Must be 1 when shared_prefix_tokens equals prompt_tokens.

episodes_per_promptinteger

Maximum sibling episodes per initial prompt (G). When distinct_prompts_per_replica is given, defaults to ceil(concurrent episodes / Q). Otherwise defaults to the concurrent episodes per replica when shared_prefix_tokens equals prompt_tokens, and to 1 when it does not. When shared_prefix_tokens equals prompt_tokens, must be at least the concurrent episodes per replica.

shared_prefix_tokensinteger

Leading tokens shared by all distinct initial prompts (H). Defaults to 0; cannot exceed prompt_tokens.

base_modelstring required

Base model to estimate. The model, GPU type, and training mode must support rollout estimates.

lora_enabledboolean

Whether to estimate a resource that hosts LoRA sessions (true, default) or a single full-weight session (false), as in a model resource. The model and GPU type must support rollout estimates in this mode.

Example request

{
  "base_model": "zai-org/GLM-5.3",
  "compute_config": {
    "num_generator_replicas": 2,
    "gpu_type": "B200-SXM"
  },
  "lora_enabled": true
}

Response

Estimated rollout time and price

wall_secondsnumber double required

Estimated time until all rollout episodes finish, in seconds.

allocated_gpu_hoursnumber double required

Generator GPUs across all replicas times wall time, in GPU-hours.

effective_decode_tokens_per_secondnumber double required

generated_tokens divided by wall_seconds.

effective_distinct_prompts_per_replicainteger required

Effective distinct_prompts_per_replica (Q) for the largest concurrent batch of one replica.

effective_episodes_per_promptinteger required

Effective episodes_per_prompt (G) for the largest concurrent batch of one replica.

effective_shared_prefix_tokensinteger required

Effective shared_prefix_tokens (H).

lora_enabledboolean required

Whether the estimate is for a LoRA resource (true) or a full-weight resource (false).

billed_gpu_hoursnumber double required

GPUs of the whole model resource, trainer and generator, times wall time, in GPU-hours. A model resource is billed for all of its GPUs while the rollouts run. An approximation measured on single-wave rollouts (concurrency equal to episodes) of up to four turns on one generator replica: actual usage can be up to about 45% higher when episodes share a prompt, and about 20% higher or lower otherwise. Longer multi-turn rollouts with distinct prompts can take longer than estimated.

estimated_pricenumber double required

Estimated on-demand price of the rollouts: billed_gpu_hours times the GPU type's list price per GPU-hour, the rate the model resource cost estimate uses. An approximation that inherits the uncertainty of billed_gpu_hours; it is not a quote.

currencystring required

Currency of estimated_price.

effective_prefill_tokens_per_secondnumber double required

submitted_prompt_tokens divided by wall_seconds, counting prompt tokens served from the prompt cache.

Example response

{
  "currency": "USD"
}

Changes