---
title: "Sample"
method: POST
path: "/v1/rl/training-sessions/{session_id}/operations/sample"
tags: ["RL"]
---

# Sample

`POST /v1/rl/training-sessions/{session_id}/operations/sample`

## Request

- Base URL: `https://api.together.ai`
- URL: `https://api.together.ai/v1/rl/training-sessions/{session_id}/operations/sample`
- Auth: HTTP bearer

Submits a sample operation that will asynchronously generate text completions with logprobs.

## Path parameters

- `session_id` string, required: Training session ID

## Headers

- `Idempotency-Key` string, required: Required key that makes retries return the original operation; use a new key for changed request bodies.

## Request body

- RLSampleBody
  - `model_inputs` RLModelInput[], required: Model inputs to sample from
    - `chunks` RLModelInputChunk[], required: Input chunks for the model
      - `encoded_text` RLEncodedTextChunk, required: Pre-tokenized text content for a model input chunk.
        - `tokens` union[], required: Pre-tokenized text input
          - union
            - string
            - integer
  - `sampling_params` RLSamplingParams
    - `max_tokens` integer: Maximum number of tokens to generate per completion
    - `temperature` number: Sampling temperature
    - `top_p` number: Nucleus sampling probability threshold
    - `top_k` integer: Top-k sampling limit
    - `stop` string[]: Generation stops when any of these strings is produced
    - `seed` union: Random seed for reproducible sampling for the same prompt and model state.
      - string
      - integer
  - `num_samples` integer: Number of completions to generate per prompt
  - `prompt_logprobs` boolean: When true, also compute teacher-forced log-probabilities for the model input tokens and return them in `SampleResult.prompt_logprobs`.
  - `topk_prompt_logprobs` integer: Number of most likely alternative tokens to return per model input token in `SampleResult.topk_prompt_logprobs`. 0 disables top-k prompt log-probabilities. Maximum 20.
  - `return_routed_experts` boolean: When true, enable reuse of the expert selections from sampled sequences during training. Only supported for mixture-of-experts models; ignored for other models.

## Response `200`

- RLSampleOperation: Async sample operation
  - `id` string, required: Operation ID
  - `status` 'TRAINING_OPERATION_STATUS_UNSPECIFIED' | 'TRAINING_OPERATION_STATUS_PENDING' | 'TRAINING_OPERATION_STATUS_RUNNING' | 'TRAINING_OPERATION_STATUS_COMPLETED' | 'TRAINING_OPERATION_STATUS_FAILED', required
  - `output` RLSampleBatchResult: Batched result of a sample operation
    - `results` RLSampleResult[], required: One result per model input
      - `sequences` RLSampledSequence[], required: Generated completions
        - `tokens` union[], required: Generated token IDs
          - union
            - string
            - integer
        - `logprobs` number[]: Log probabilities for each generated token
        - `stop_reason` 'STOP_REASON_LENGTH' | 'STOP_REASON_STOP', required: Reason generation stopped.
        - `prompt_cache_hit_tokens` integer, required: Number of model input tokens served from the prefix cache while generating this sequence.
        - `routed_experts_key` string: Opaque key for reusing this sequence's expert selections during training. Pass it unchanged with the corresponding training sample. Absent for non-mixture-of-experts models or when `return_routed_experts` is disabled.
      - `prompt_logprobs` number[]: Teacher-forced log-probability of each model input token. Full prompt length; entry i corresponds to prompt token i. Entry 0 is always 0 as a placeholder: the first prompt token has no conditioning context, so it has no log-probability. Present only when prompt_logprobs was set on the request.
      - `topk_prompt_logprobs` RLPromptTopLogprobs[]: The most likely alternative tokens at each model input token, up to `topk_prompt_logprobs` per position. Full prompt length; entry i corresponds to prompt token i, and entry 0 is empty. Present only when topk_prompt_logprobs was set on the request.
        - `token_ids` integer[]: Token IDs of the alternatives, ordered by descending log-probability.
        - `logprobs` number[]: Log-probability of each alternative in `token_ids`, at the same index.
      - `policy_segments` RLPolicyVersionSegment[], required: Policy versions that produced these completions
        - `version` integer, required: Model version under which this segment of tokens was generated
        - `start_token` integer, required: Index of the first token of this segment within the sampled sequence. Always 0 for the first segment.
  - `error` RLTrainingOperationError: Error details for a failed training operation
    - `code` 'TRAINING_OPERATION_ERROR_CODE_UNSPECIFIED' | 'TRAINING_OPERATION_ERROR_CODE_RESOURCE_EXHAUSTED' | 'TRAINING_OPERATION_ERROR_CODE_TIMEOUT' | 'TRAINING_OPERATION_ERROR_CODE_INTERNAL_ERROR' | 'TRAINING_OPERATION_ERROR_CODE_SESSION_NOT_ACTIVE' | 'TRAINING_OPERATION_ERROR_CODE_INVALID_INPUT' | 'TRAINING_OPERATION_ERROR_CODE_NON_FINITE_LOSS': Application error code for a failed training operation
    - `message` string: Human-readable error message

## Other responses

- `default`: An unexpected error response.

## Changes

- **2026-10-22** `bb5257b3173a`: 1 info
  - endpoint added

[Operation history](https://skmtc.dev/together/apis/together-apis/changes/v1/rl/training-sessions/:session_id/operations/sample/post.md)

---

[API](https://skmtc.dev/together/apis/together-apis.md) - [All operations](https://skmtc.dev/together/apis/together-apis/llms.txt) - [OpenAPI document](https://skmtc.dev/together/apis/together-apis/revisions/93438ddac711?raw)