Sign in
Text Generation Inference

Generate tokens if `stream == false` or a stream of token if `stream == true`

Generate tokens if stream == false or a stream of token if stream == true

post/

Request body

CompatGenerateRequest required— unresolved $ref

Response

Generated Text

generated_textstring required

Example response

{
  "details": {
    "best_of_sequences": [
      {
        "generated_text": "test",
        "generated_tokens": 1,
        "prefill": [
          {
            "logprob": -0.34,
            "text": "test"
          }
        ],
        "seed": 42,
        "tokens": [
          {
            "logprob": -0.34,
            "text": "test"
          }
        ]
      }
    ],
    "generated_tokens": 1,
    "prefill": [
      {
        "logprob": -0.34,
        "text": "test"
      }
    ],
    "seed": 42,
    "tokens": [
      {
        "logprob": -0.34,
        "text": "test"
      }
    ]
  },
  "generated_text": "test"
}

Changes