Text Generation Inference

Generate tokens

Generate tokens

post/generate

Request body

GenerateRequest required— unresolved $ref

Response

Generated Text

generated_textstring required

Example response

[
  {
    "details": {
      "generated_tokens": 1,
      "prefill": [
        {
          "logprob": -0.34,
          "text": "test"
        }
      ],
      "seed": 42,
      "tokens": [
        {
          "logprob": -0.34,
          "text": "test"
        }
      ]
    },
    "generated_text": "test"
  }
]

Changes