FRfriendli
Serverless.Messages

Messages

Generate responses using Anthropic Messages-style payloads.

post/serverless/v1/messages

Request

Headers

X-Friendli-Teamstring nullable

ID of team to run requests as (optional parameter).

Request body

max_tokensinteger nullable

Maximum number of tokens to generate for the assistant response. Must be greater than 0 when provided.

modelstring required

Code of the model to use. See available model list.

streamboolean nullable

Whether to stream output as server-sent events (text/event-stream). When false or omitted, returns a single JSON response.

temperaturenumber nullable

Sampling temperature. Lower values make outputs more deterministic; higher values increase diversity.

top_pnumber nullable

Nucleus sampling parameter. The model samples from the smallest token set whose cumulative probability reaches top_p.

top_kinteger nullable

Limits sampling to the k most likely tokens at each decoding step.

stop_sequencesstring[] nullable

Stop strings that terminate generation when matched in output. The matched value is returned in stop_sequence when applicable.

cache_controlobject nullable

Compatibility field accepted for request portability. Parsed but not used for generation.

containerobject nullable

Compatibility field accepted for request portability. Parsed but not used for generation.

context_managerobject nullable

Compatibility field accepted for request portability. Parsed but not used for generation.

inference_geoobject nullable

Compatibility field accepted for request portability. Parsed but not used for generation.

metadataobject nullable

Compatibility field accepted for request portability. Parsed but not used for generation.

Example request

{
  "max_tokens": 128,
  "messages": [
    {
      "content": "Hello, summarize what you can do in one sentence.",
      "role": "user"
    }
  ],
  "model": "meta-llama/Llama-3.1-8B-Instruct"
}

Response

Successfully generated a Messages-style response. For streaming (text/event-stream) event and chunk details, see Messages chunk object.

idstring required

Unique identifier for this message response.

type'message' required

Response object type (message).

role'assistant' required

Role of the output message author (assistant).

stop_reason'end_turn' | 'max_tokens' | 'tool_use' | 'stop_sequence' nullable

Why generation stopped (end_turn, max_tokens, tool_use, stop_sequence).

stop_sequencestring nullable

Matched stop string when stop_reason is stop_sequence; otherwise null.

modelstring nullable

Model used to generate the response.

Changes

No recorded changes to this operation across all 1 revision of this API.