---
title: "Generate tokens if `stream == false` or a stream of token if `stream == true`"
method: POST
path: "/"
tags: ["LoRAX"]
---

# Generate tokens if `stream == false` or a stream of token if `stream == true`

`POST /`

Generate tokens if `stream == false` or a stream of token if `stream == true`

## Request body

- CompatGenerateRequest
  - `inputs` string, required
  - `parameters` GenerateParameters
    - `best_of` integer, nullable
    - `decoder_input_details` boolean
    - `details` boolean
    - `do_sample` boolean
    - `max_new_tokens` integer, nullable
    - `ignore_eos_token` boolean
    - `repetition_penalty` number, float, nullable
    - `return_full_text` boolean, nullable
    - `seed` integer, nullable
    - `stop` string[]
    - `temperature` number, float, nullable
    - `top_k` integer, nullable
    - `top_p` number, float, nullable
    - `truncate` integer, nullable
    - `typical_p` number, float, nullable
    - `watermark` boolean
    - `schema` string, nullable
    - `adapter_id` string, nullable
    - `adapter_source` string, nullable
    - `merged_adapters` AdapterParameters
      - `ids` string[]
      - `weights` number[]
      - `merge_strategy` 'linear' | 'ties' | 'dare_linear' | 'dare_ties'
      - `density` number, float
      - `majority_sign_method` 'total' | 'frequency'
    - `api_token` string, nullable
    - `apply_chat_template` boolean
  - `stream` boolean

## Response `200`

Generated Text

- GenerateResponse
  - `details` Details
    - `best_of_sequences` BestOfSequence[], nullable
      - `finish_reason` 'length' | 'eos_token' | 'stop_sequence', required
      - `generated_text` string, required
      - `generated_tokens` integer, required
      - `prefill` PrefillToken[], required
        - `id` integer, required
        - `logprob` number, float, nullable, required
        - `text` string, required
      - `seed` integer, nullable
      - `tokens` Token[], required
        - `id` integer, required
        - `logprob` number, float, nullable, required
        - `special` boolean, required
        - `text` string, required
    - `finish_reason` 'length' | 'eos_token' | 'stop_sequence', required
    - `prompt_tokens` integer, required
    - `generated_tokens` integer, required
    - `prefill` PrefillToken[], required
      - `id` integer, required
      - `logprob` number, float, nullable, required
      - `text` string, required
    - `seed` integer, nullable
    - `tokens` Token[], required
      - `id` integer, required
      - `logprob` number, float, nullable, required
      - `special` boolean, required
      - `text` string, required
  - `generated_text` string, required

## Other responses

- `422` — Input validation error
- `424` — Generation Error
- `429` — Model is overloaded
- `500` — Incomplete generation

## Changes

- **2026-07-17** `d56749eaa5c4` — 4 warning, 8 info
  - the `parameters/merged_adapters/density` request property's max was set to `1.00`
  - the `parameters/repetition_penalty` request property's min was set to `0.00`
  - the `parameters/temperature` request property's min was set to `0.00`
  - the `parameters/top_k` request property's min was set to `0.00`
  - …8 more
- **2024-03-22** `6c0bacf31ca5` — 2 info
  - the request property `parameters/max_new_tokens` became nullable
  - the `max_new_tokens` request property default value changed from `20` to `null`
- **2024-03-19** `cca02625fbcd` — 1 info
  - added the new optional request property `parameters/ignore_eos_token`
- **2024-02-12** `7a5ed91a6ffa` — 1 info
  - added the new optional request property `parameters/schema`
- **2024-02-01** `bf530b15fd2b` — 1 info
  - added the new optional request property `parameters/merged_adapters`

[Full history](https://skmtc.dev/predibase/apis/lorax/changes/post.md)

---

[API](https://skmtc.dev/predibase/apis/lorax.md) · [All operations](https://skmtc.dev/predibase/apis/lorax/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/predibase/lorax/revisions/d56749eaa5c4/schema)