Sign in
Text Generation Inference

Generate tokens if `stream == false` or a stream of token if `stream == true`

Generate tokens if stream == false or a stream of token if stream == true

post/

Request body

CompatGenerateRequest required— unresolved $ref

Response

Generated Text

generated_textstring required

Example response

{
  "details": {
    "best_of_sequences": [
      {
        "generated_text": "test",
        "generated_tokens": 1,
        "prefill": [
          {
            "logprob": -0.34,
            "text": "test"
          }
        ],
        "seed": 42,
        "tokens": [
          {
            "logprob": -0.34,
            "text": "test"
          }
        ]
      }
    ],
    "generated_tokens": 1,
    "prefill": [
      {
        "logprob": -0.34,
        "text": "test"
      }
    ],
    "seed": 42,
    "tokens": [
      {
        "logprob": -0.34,
        "text": "test"
      }
    ]
  },
  "generated_text": "test"
}

Changes

Changed in 4 of the 26 revisions of this API.15

  • e35edd53eed02See the full diff
    • ○

      added the new optional request property parameters/decoder_input_details

      new-optional-request-property

    • ○

      the stream request property default value false was added

      request-property-default-value-added

  • 36489c7b42f02See the full diff
    • ○

      added the media type application/json for the response with the status 200

      response-media-type-added

    • ○

      added the media type text/event-stream for the response with the status 200

      response-media-type-added

  • de34bec00da81See the full diff
    • ○

      endpoint added

      endpoint-added

  • a9059df72b921See the full diff
    • ▲

      api path removed without deprecation

      api-path-removed-without-deprecation

    This revision also has 4 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog