LLllmgateway

Chat Completions

Changed on

Create a completion for the chat conversation

post/v1/chat/completions

Request

  • Base URL
    https://api.llmgateway.io
  • Auth
    HTTP bearer

Request body

modelstring
temperaturenumber nullable
max_tokensnumber nullable
top_pnumber nullable
frequency_penaltynumber nullable
presence_penaltynumber nullable
streamboolean
ninteger nullable

How many chat completion choices to generate for each input message. Only accepted when the resolved model supports it upstream (currently OpenAI Chat Completions models and Google Gemini 2.5 models via candidateCount); requests for unsupported models are rejected with 400. Streaming is supported for OpenAI models: choice deltas are demultiplexed by choices[].index on a single SSE stream. Exceptions rejected with 400: n > 1 with stream: true and function tools (the streaming tool-call aggregator can't disambiguate concurrent calls across choices; native web_search tools and the web_search: true flag are exempt), n > 1 with stream: true on Google models (Gemini rejects candidateCount on streamGenerateContent), and n > 8 on Google models (Gemini caps candidateCount at 8).

prompt_cache_keystring nullable

OpenAI prompt caching key used to improve cache routing for requests with shared prompt prefixes.

prompt_cache_retention'in_memory' | '24h' | 'null' nullable

OpenAI prompt cache retention policy. OpenAI supports in_memory and 24h for eligible models.

safety_identifierstring nullable

Opaque end-user identifier stored in log params.safety_identifier. Never forwarded upstream; credits requests use the fixed LLM Gateway API-key identifier instead.

userstring nullable

OpenAI end-user identifier. Used as a fallback sticky-routing session key when no x-session-id header or prompt_cache_key is provided.

parallel_tool_callsboolean nullable

Whether the model may call several tools in one turn. Only sent alongside tools. Forwarded to OpenAI, Azure, and OpenAI-compatible providers that list it in supported_parameters; false maps to Anthropic's disable_parallel_tool_use. Ignored elsewhere.

reasoning_effort'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max' | 'null' nullable

Controls the reasoning effort for reasoning-capable models. none is only supported by OpenAI's newer reasoning models (e.g. gpt-5.4 and later); for other providers it disables reasoning. max is the highest tier (above xhigh), supported by Anthropic models and OpenAI GPT-5.6 models. The gateway never downgrades effort tiers: providers that accept an effort parameter receive the value unchanged (an unsupported value results in a provider error), while providers that take a thinking budget instead (e.g. Anthropic, Google) translate each tier to a native budget. The exact values each provider mapping accepts are exposed as reasoning_efforts on /v1/models.

effort'low' | 'medium' | 'high' | 'null' nullable

Controls the computational effort for supported models (currently only claude-opus-4-5-20251101)

verbosity'low' | 'medium' | 'high' | 'null' nullable

Controls how detailed the model's responses are. Only supported by OpenAI GPT-5 and later models; requests to models without verbosity support return a 400 error.

service_tier'auto' | 'default' | 'flex' | 'priority'

Processing tier for the request. flex and priority are forwarded only for provider/model mappings that explicitly support the requested tier, such as supported OpenAI and Google mappings. auto/default use the standard on-demand tier. Unsupported tier requests return a 400 unsupported_service_tier error. On coding (dev) plans only auto, default and flex are allowed.

routing'auto' | 'price' | 'throughput' | 'latency'

Provider selection strategy for model-id routing, named after the factor it optimizes. auto (default) uses the full weighted smart-routing score. price, throughput, and latency each give a 90% relative weight to that factor while keeping a small uptime weight so requests still fall back to other providers when the top pick has extremely bad uptime. latency only biases streaming requests. Combining routing with a specific provider prefix (e.g. openai/gpt-4o) returns a 400. On coding (dev) plans only auto and price are allowed.

free_models_onlyboolean

When used with auto routing, only route to free models (models with zero input and output pricing)

onboardingboolean

Deprecated and ignored. This once skipped email verification for free model usage, but the flag is client-supplied, so any account could assert it. Onboarding is now recognized server-side by the API proxy; setting this grants nothing.

no_reasoningboolean

When used with auto routing, exclude reasoning models from selection

web_searchboolean

Enable native web search for models that support it. When enabled, the model can search the web for real-time information.

Example request

{
  "model": "gpt-5",
  "messages": [
    {
      "role": "user",
      "content": "Hello!",
      "tool_calls": [
        {
          "id": "call_abc123",
          "type": "function",
          "function": {
            "name": "get_current_weather",
            "arguments": "{\"location\": \"Boston, MA\"}"
          }
        }
      ]
    }
  ],
  "temperature": 0.7,
  "max_tokens": 1000,
  "top_p": 0.9,
  "response_format": {
    "type": "json_object"
  },
  "n": 1,
  "prompt": {
    "id": "support-reply",
    "variables": {
      "customer": "Ada"
    }
  },
  "prompt_cache_key": "tenant-123",
  "prompt_cache_retention": "24h",
  "prompt_cache_options": {
    "mode": "explicit"
  },
  "user": "user-123",
  "reasoning_effort": "medium",
  "reasoning": {
    "effort": "medium",
    "max_tokens": 4000,
    "context": "current_turn",
    "mode": "pro"
  },
  "effort": "medium",
  "verbosity": "low",
  "service_tier": "flex",
  "routing": "price",
  "web_search_options": {
    "search_context_size": "low"
  },
  "web_search": true,
  "plugins": [
    {
      "id": "response-healing"
    }
  ]
}

Response

User response object or streaming response.

idstring required
objectstring required
creatednumber required
modelstring required

Changes