generate

Generate

Changed on

Generate text from a prompt using the named model.

The model path segment uses the SIE-safe id (double-underscore separator, e.g. Qwen__Qwen3-4B-Instruct). HuggingFace-style slashes are rejected with 400 to keep parity with the gateway contract.

post/v1/generate/{model}

Request

  • The document declares no server URL.
  • Auth
    none declared

Path parameters

modelstring required

Headers

X-SIE-MACHINE-PROFILEstring nullable
X-SIE-Remotestring nullable

The only accepted value is forbid. It keeps the request off remote upstreams: local capacity serves or refuses it, and a model served only by an upstream answers 400. Any other value answers 400.

Request body

promptstring required

Prompt text to generate from

max_new_tokensinteger required

Maximum number of tokens to generate

temperaturenumber nullable

Sampling temperature override

top_pnumber nullable

Nucleus-sampling probability override

optionsobject nullable

Governed generation runtime options; explicit top-level sampler fields override them

stopstring[] nullable

Stop sequences

streamboolean nullable

Return SIE-native Server-Sent Events when true

frequency_penaltynumber nullable

Frequency penalty

presence_penaltynumber nullable

Presence penalty

seedinteger nullable

Optional signed 64-bit per-request sampling seed. Reproducibility is best effort, not guaranteed, and depends on the active generation backend and deployment configuration. Non-integer or out-of-range values reject with 400 invalid_request.

logit_biasobject nullable

Token-id string to finite sampler bias in [-100, 100]

logprobsboolean nullable

Return per-token log probabilities; supported only with stream true

top_logprobsinteger nullable

Alternative tokens per position; requires logprobs and stream true

routing_keystring nullable

Optional routing hint

prompt_cache_keystring nullable

Optional prompt-cache routing hint

safety_identifierstring nullable

Sensitive client identifier; validated and dropped without logging or forwarding

Example request

{
  "max_new_tokens": 64,
  "prompt": "Write one sentence about vector search.",
  "stream": false,
  "temperature": 0.7,
  "top_p": 0.9
}

Response

Generated text, or a Server-Sent Event stream when stream is true

modelstring required

Model used for generation

textstring required

Generated text

finish_reasonstring required

Reason generation stopped

Changes

  • 2045d09d59b7Revision changes
    • info

      added the media type application/json for the response with the status

    • info

      added the media type application/json for the response with the status

    • info

      added the non-success response with the status

  • 6669031899b8Revision changes
    • info

      added the new optional header request parameter X-SIE-Remote