---
title: "Chat Completions"
method: POST
path: "/v1/chat/completions"
---

# Chat Completions

`POST /v1/chat/completions`

Create a completion for the chat conversation

## Request body

- object
  - `model` string, required
  - `messages` object[], required
    - `role` 'developer' | 'system' | 'user' | 'assistant' | 'tool' | 'function', required
    - `content` union
      - string
      - union[]
        - union
          - object
            - `type` 'text', required
            - `text` string, required
            - `cache_control` object
              - …
            - `prompt_cache_breakpoint` object — OpenAI explicit prompt cache breakpoint marker (GPT-5.6 and later). Ends a cacheable prefix when the request sets prompt_cache_options.mode to 'explicit'. Stripped for providers/models without explicit prompt caching support.
              - …
          - object
            - `type` 'image_url', required
            - `image_url` object, required
              - …
            - `prompt_cache_breakpoint` object
              - …
          - object
            - `type` 'input_audio', required
            - `input_audio` object, required
              - …
            - `prompt_cache_breakpoint` object
              - …
          - object
            - `type` 'file', required
            - `file` object, required
              - …
            - `prompt_cache_breakpoint` object
              - …
      - unknown
    - `name` string
    - `tool_call_id` string
    - `tool_calls` object[] — A list of tool calls generated by the model in this message.
      - `id` string, required
      - `type` 'function', required
      - `function` object, required
        - `name` string, required
        - `arguments` string, required
    - `reasoning` string
    - `reasoning_content` string
    - `reasoning_details` object[]
      - `text` string
      - `type` string
    - `phase` 'commentary' | 'final_answer' — OpenAI Responses assistant-message phase. Replayed upstream for OpenAI Responses API models; stripped for other providers.
    - `content_before_tool_calls` boolean — Marks assistant content that preceded the message's tool calls (pre-tool commentary on OpenAI Responses API models), so replay preserves the original item order. Stripped for other providers.
    - `message_items` object[] — Separate phased assistant message items (e.g. commentary and final_answer) emitted by OpenAI Responses API models in a single turn. preceding_tool_calls records how many of the message's tool calls came before each item. Replayed upstream as individual message items in their original order; stripped for other providers.
      - `text` string, required
      - `phase` 'commentary' | 'final_answer'
      - `preceding_tool_calls` integer
    - `anthropic_native_blocks` object[] — Anthropic content blocks with no OpenAI-format equivalent. On an assistant message these are the server-side tool search blocks (`server_tool_use` + `tool_search_tool_result`), replayed ahead of the message's tool calls; on a tool message they are the `tool_result` content array, which is how a client-side tool search returns `tool_reference` blocks. Replayed on the Anthropic Messages API only and stripped for every other provider.
      - `type` string, required
    - `tool_result_cache_control` object — Prompt-cache breakpoint for the Anthropic `tool_result` block this tool message becomes. The OpenAI tool message shape has nowhere to carry it, so it rides here. Applied on the Anthropic Messages API only and stripped for every other provider.
      - `type` 'ephemeral', required
      - `ttl` '5m' | '1h'
  - `temperature` number, nullable
  - `max_tokens` number, nullable
  - `top_p` number, nullable
  - `frequency_penalty` number, nullable
  - `presence_penalty` number, nullable
  - `response_format` union
    - object
      - `type` 'text' | 'json_object', required
    - object
      - `type` 'json_schema', required
      - `json_schema` object, required
        - `name` string, required
        - `description` string
        - `schema` object, required
        - `strict` boolean
  - `stream` boolean
  - `n` integer, nullable — How many chat completion choices to generate for each input message. Only accepted when the resolved model supports it upstream (currently OpenAI Chat Completions models and Google Gemini 2.5 models via `candidateCount`); requests for unsupported models are rejected with 400. Streaming is supported for OpenAI models: choice deltas are demultiplexed by `choices[].index` on a single SSE stream. Exceptions rejected with 400: `n > 1` with `stream: true` **and** function `tools` (the streaming tool-call aggregator can't disambiguate concurrent calls across choices; native `web_search` tools and the `web_search: true` flag are exempt), `n > 1` with `stream: true` on Google models (Gemini rejects candidateCount on streamGenerateContent), and `n > 8` on Google models (Gemini caps candidateCount at 8).
  - `prompt_cache_key` string, nullable — OpenAI prompt caching key used to improve cache routing for requests with shared prompt prefixes.
  - `prompt_cache_retention` 'in_memory' | '24h', nullable — OpenAI prompt cache retention policy. OpenAI supports in_memory and 24h for eligible models.
  - `prompt_cache_options` object, nullable — OpenAI explicit prompt caching options (GPT-5.6 and later). mode 'implicit' (default) places an automatic cache breakpoint at the latest message; 'explicit' caches only content parts marked with prompt_cache_breakpoint. Only forwarded for OpenAI models that support explicit prompt caching.
    - `mode` 'implicit' | 'explicit'
    - `ttl` '30m'
  - `user` string, nullable — OpenAI end-user identifier. Used as a fallback sticky-routing session key when no x-session-id header or prompt_cache_key is provided.
  - `tools` union[]
    - union
      - object
        - `type` 'function', required
        - `function` object, required
          - `name` string, required
          - `description` string
          - `parameters` object
        - `defer_loading` boolean — Anthropic only. Keeps the tool out of the rendered tools section so it never enters the cached prompt prefix, and loads it on demand once the tool search tool discovers it. Requires a `tool_search` tool in `tools`, and Anthropic rejects a request whose tools are all deferred. Stripped for every other provider, which receives the tool eagerly instead.
        - `cache_control` object — Anthropic only. Cache breakpoint ending the tool-definitions prefix, which Anthropic renders before the system prompt and messages. Placed on the last tool it caches every tool up to and including that one, and counts toward Anthropic's limit of 4 breakpoints per request. Stripped for every other provider.
          - `type` 'ephemeral', required
          - `ttl` '5m' | '1h'
      - object
        - `type` 'tool_search', required
        - `tool_search_type` string, required — Anthropic tool search tool type, e.g. `tool_search_tool_regex_20251119` or `tool_search_tool_bm25_20251119`.
        - `name` string
      - object
        - `type` 'web_search', required
        - `user_location` object
          - `city` string
          - `region` string
          - `country` string
          - `timezone` string
        - `search_context_size` 'low' | 'medium' | 'high'
        - `max_uses` number
        - `allowed_domains` string[]
        - `blocked_domains` string[]
  - `tool_choice` union — Controls which tool the model calls. `{"type": "web_search"}` demands a search instead of offering one, and requires a `web_search` tool in `tools`. Providers whose web search is model-elected are unaffected by it — the model already decides — but it is the only way to reach providers that can search solely on demand (currently Alibaba's DashScope and its resellers), which are otherwise skipped when routing a web search request.
    - 'auto'
    - 'none'
    - 'required'
    - object
      - `type` 'function', required
      - `function` object, required
        - `name` string, required
    - object
      - `type` 'web_search', required
  - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max', nullable — Controls the reasoning effort for reasoning-capable models. `none` is only supported by OpenAI's newer reasoning models (e.g. gpt-5.4 and later); for other providers it disables reasoning. `max` is the highest tier (above `xhigh`), supported by Anthropic models and OpenAI GPT-5.6 models. The gateway never downgrades effort tiers: providers that accept an effort parameter receive the value unchanged (an unsupported value results in a provider error), while providers that take a thinking budget instead (e.g. Anthropic, Google) translate each tier to a native budget. The exact values each provider mapping accepts are exposed as `reasoning_efforts` on `/v1/models`.
  - `reasoning` object — Unified reasoning configuration object for controlling reasoning behavior
    - `effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max' — Controls the reasoning effort. Alternative to top-level reasoning_effort. Cannot be used together with reasoning_effort. `max` is the highest tier (above `xhigh`), supported by Anthropic models and OpenAI GPT-5.6 models. Tiers are never downgraded by the gateway: enum-based providers receive the value unchanged (unsupported values result in a provider error), while budget-based providers (e.g. Anthropic, Google) translate the tier to a native thinking budget. See `reasoning_efforts` on `/v1/models` for the values each mapping accepts.
    - `max_tokens` integer — Exact number of tokens to allocate for reasoning. When specified, overrides effort. Supported by Anthropic and Google thinking models.
    - `context` 'auto' | 'current_turn' | 'all_turns' — How much replayed reasoning the model considers (OpenAI Responses API models only). Omitting the field is equivalent to 'auto'. Forwarded upstream as reasoning.context; ignored by other providers.
  - `effort` 'low' | 'medium' | 'high', nullable — Controls the computational effort for supported models (currently only claude-opus-4-5-20251101)
  - `verbosity` 'low' | 'medium' | 'high', nullable — Controls how detailed the model's responses are. Only supported by OpenAI GPT-5 and later models; requests to models without verbosity support return a 400 error.
  - `service_tier` 'auto' | 'default' | 'flex' | 'priority' — Processing tier for the request. `flex` and `priority` are forwarded only for provider/model mappings that explicitly support the requested tier, such as supported OpenAI and Google mappings. `auto`/`default` use the standard on-demand tier. Unsupported tier requests return a 400 `unsupported_service_tier` error. On coding (dev) plans only `auto`, `default` and `flex` are allowed.
  - `routing` 'auto' | 'price' | 'throughput' | 'latency' — Provider selection strategy for model-id routing, named after the factor it optimizes. `auto` (default) uses the full weighted smart-routing score. `price`, `throughput`, and `latency` each give a 90% relative weight to that factor while keeping a small uptime weight so requests still fall back to other providers when the top pick has extremely bad uptime. `latency` only biases streaming requests. Combining `routing` with a specific provider prefix (e.g. `openai/gpt-4o`) returns a 400. On coding (dev) plans only `auto` and `price` are allowed.
  - `free_models_only` boolean — When used with auto routing, only route to free models (models with zero input and output pricing)
  - `onboarding` boolean — Deprecated and ignored. This once skipped email verification for free model usage, but the flag is client-supplied, so any account could assert it. Onboarding is now recognized server-side by the API proxy; setting this grants nothing.
  - `no_reasoning` boolean — When used with auto routing, exclude reasoning models from selection
  - `sensitive_word_check` object
    - `status` 'DISABLE' | 'ENABLE', required
  - `image_config` object
    - `aspect_ratio` string
    - `image_size` string
    - `image_quality` 'low' | 'medium' | 'high' | 'auto'
    - `n` number
    - `seed` number
  - `web_search` boolean — Enable native web search for models that support it. When enabled, the model can search the web for real-time information.
  - `plugins` object[] — Plugins to enable for this request. Currently supported: response-healing (automatically repairs malformed JSON responses when using response_format)
    - `id` 'response-healing', required — Plugin identifier

## Response `200`

User response object or streaming response.

- object
  - `id` string, required
  - `object` string, required
  - `created` number, required
  - `model` string, required
  - `choices` object[], required
    - `index` number, required
    - `message` object, required
      - `role` string, required
      - `content` string, nullable, required
      - `reasoning` string, nullable
      - `tool_calls` object[]
        - `id` string, required
        - `type` 'function', required
        - `function` object, required
          - `name` string, required
          - `arguments` string, required
      - `images` object[]
        - `type` 'image_url', required
        - `image_url` object, required
          - `url` string, required
    - `finish_reason` string, required
  - `usage` object, required
    - `prompt_tokens` number, required
    - `completion_tokens` number, required
    - `total_tokens` number, required
    - `reasoning_tokens` number
    - `prompt_tokens_details` object
      - `cached_tokens` number, required
      - `cache_write_tokens` number
      - `cache_creation_tokens` number
      - `cache_creation` object
        - `ephemeral_5m_input_tokens` number, required
        - `ephemeral_1h_input_tokens` number, required
      - `audio_tokens` number
      - `video_tokens` number
    - `completion_tokens_details` object
      - `reasoning_tokens` number
      - `image_tokens` number
      - `audio_tokens` number
    - `cost` number, nullable
    - `cost_details` object
      - `upstream_inference_cost` number, required
      - `upstream_inference_prompt_cost` number, required
      - `upstream_inference_completions_cost` number, required
      - `total_cost` number, nullable
      - `input_cost` number, nullable
      - `output_cost` number, nullable
      - `cached_input_cost` number, nullable
      - `cache_write_input_cost` number, nullable
      - `request_cost` number, nullable
      - `web_search_cost` number, nullable
      - `image_input_cost` number, nullable
      - `image_output_cost` number, nullable
      - `audio_input_cost` number, nullable
      - `data_storage_cost` number, nullable
    - `info` string
  - `metadata` object, required
    - `request_id` string, required
    - `requested_model` string, required
    - `requested_provider` string, nullable, required
    - `used_model` string, required
    - `used_provider` string, required
    - `used_region` string, nullable
    - `underlying_used_model` string, required
    - `log_id` string
    - `organization_id` string
    - `project_id` string
    - `discount` number, nullable
    - `cached` boolean — True when the response was replayed from the gateway response cache instead of being generated upstream. Omitted otherwise.
    - `routing` object[]
      - `provider` string, required
      - `model` string, required
      - `region` string
      - `status_code` number, required
      - `error_type` string, required
      - `succeeded` boolean, required
      - `apiKeyHash` string — Stable fingerprint of the provider credential this attempt was sent with. Use it together with credentialSource to tell attempts apart when a request rotated keys.
      - `credentialSource` 'byok' | 'platform' — Whose provider credential served this attempt. `byok` is your organization's own provider key — the provider bills you directly and no credits are deducted. `platform` is an LLM Gateway credential, billed as credits. A hybrid-mode request whose own key fails falls back to `platform`, so both values can appear in one response.
      - `providerKeyId` string — Id of your provider key that served this attempt. Set only when credentialSource is `byok`.
      - `providerKeyLabel` string — Your provider key as it is named on the provider-keys page (its name, or its masked token when unnamed), so an attempt can be tied to a key without decoding the fingerprint. Set only when credentialSource is `byok`; LLM Gateway's own credentials are never described.
      - `logId` string

## Other responses

- `400` — Invalid request body or parameters.
- `401` — Missing or invalid API key.
- `402` — Insufficient credits or plan limits reached.
- `403` — Forbidden request or upstream response.
- `404` — Unknown model or upstream not-found response.
- `410` — Archived or unavailable project.
- `429` — Rate limited (organization, endpoint, or upstream provider). Back off until Retry-After elapses.
- `500` — Internal server error.
- `502` — Failed to connect to the upstream provider.
- `503` — Service unavailable upstream response.
- `504` — Upstream provider timeout.

---

[API](https://skmtc.dev/llmgateway/apis/llmgateway-api.md) · [All operations](https://skmtc.dev/llmgateway/apis/llmgateway-api/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/llmgateway/llmgateway-api/revisions/aaf167e3d8cc/schema)