Sign in
Inference

Generate

Generate text based on the input prompt.

post/generate

Request body

promptobject[] required

The input prompt for the language model to generate text from.

min_output_tokensinteger

Minimum number of tokens to output.

max_output_tokensinteger

Maximum number of tokens to output.

topkinteger

Choose top k tokens.

modelstring

The name of the model or adapter to use for generation.

streamboolean nullable

Whether to stream the response back incrementally.

toolsobject[] nullable

List of tools given to model.

tool_choicestring nullable

Controls which (if any) tool is called by the model.

logprobsboolean

Whether to return log probabilities of the output tokens or not.

top_logprobsinteger

Return output tokens with top probabilities.

system_prompt_versionstring

Which system prompt to use in the backend.

tool_namestring

Which tool was selected by router LLM model.

context_for_toolstring

Context most relavant to selected tool.

guided_jsonobject nullable

A JSON schema for guided generation.

guided_regexstring nullable

A regex pattern for guided generation.

response_formatobject nullable

For JSON object output.

metadataobject nullable

Used to propagate number of prefill and decode tokens.

echoboolean

Echo back the prompt in addition to the completion.

temperaturenumber nullable

What sampling temperature to use, between 0 and 2.

ignore_eosboolean

Ignore end of sequence (stop_id) and continue generation until output limit is hit.

Response

Successful Response

{"stackTrail":"paths:/generate:post:responses:200:content:application/json:schema","oasType":"schema","type":"unknown"}

Changes

No recorded changes to this endpoint across all 1 revision of this API.