---
title: "Generate tokens if `stream == false` or a stream of token if `stream == true`"
method: POST
path: "/"
tags: ["LoRAX"]
---

# Generate tokens if `stream == false` or a stream of token if `stream == true`

`POST /`

Generate tokens if `stream == false` or a stream of token if `stream == true`

## Request body

- CompatGenerateRequest — unresolved $ref

## Response `200`

Generated Text

- GenerateResponse
  - `details` Details
    - `best_of_sequences` BestOfSequence[], nullable
      - `finish_reason` 'length' | 'eos_token' | 'stop_sequence', required
      - `generated_text` string, required
      - `generated_tokens` integer, required
      - `prefill` PrefillToken[], required
        - `id` integer, required
        - `logprob` number, float, nullable, required
        - `text` string, required
      - `seed` integer, nullable
      - `tokens` Token[], required
        - `id` integer, required
        - `logprob` number, float, nullable, required
        - `special` boolean, required
        - `text` string, required
    - `finish_reason` 'length' | 'eos_token' | 'stop_sequence', required
    - `prompt_tokens` integer, required
    - `generated_tokens` integer, required
    - `prefill` PrefillToken[], required
      - `id` integer, required
      - `logprob` number, float, nullable, required
      - `text` string, required
    - `seed` integer, nullable
    - `tokens` Token[], required
      - `id` integer, required
      - `logprob` number, float, nullable, required
      - `special` boolean, required
      - `text` string, required
  - `generated_text` string, required

## Other responses

- `422` — Input validation error
- `424` — Generation Error
- `429` — Model is overloaded
- `500` — Incomplete generation

## Changes

- **2024-01-10** `906cbf329039` — 1 info
  - added the new optional request property `parameters/apply_chat_template`
- **2024-01-09** `d3c70dbaab3b` — 2 info
  - added the required property `details/allOf[#/components/schemas/Details]/prompt_tokens` to the response with the `200` status (media type: application/json)
  - added the required property `details/allOf[#/components/schemas/StreamDetails]/prompt_tokens` to the response with the `200` status (media type: text/event-stream)
- **2023-12-14** `c7fca05406da` — 1 info
  - added the new optional request property `parameters/api_token`
- **2023-12-07** `a775339b4938` — 3 breaking, 6 warning, 9 info
  - the request property `parameters/max_new_tokens` became not nullable
  - removed the required property `details/allOf[#/components/schemas/Details]/prompt_tokens` from the response with the `200` status (media type: application/json)
  - removed the required property `details/allOf[#/components/schemas/StreamDetails]/prompt_tokens` from the response with the `200` status (media type: text/event-stream)
  - the `parameters/max_new_tokens` request property's exclusiveMaximum was set to `512.00`
  - …14 more

[Change history](https://skmtc.dev/predibase/apis/lorax/changes/post.md)

---

[API](https://skmtc.dev/predibase/apis/lorax.md) · [All operations](https://skmtc.dev/predibase/apis/lorax/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/predibase/lorax/revisions/906cbf329039/schema)