---
title: "Create a chat completion"
method: POST
path: "/api/chat/completions"
tags: ["Gateway"]
---

# Create a chat completion

`POST /api/chat/completions`

## Request

- Base URL: `https://api.respan.ai`
- URL: `https://api.respan.ai/api/chat/completions`
- Auth: HTTP bearer

Sends a chat completion request through the Respan gateway with automatic logging. Accepts [OpenAI chat completion parameters](https://platform.openai.com/docs/apis/chat) and Respan options for fallbacks, caching, and prompt management.

Pass Respan parameters in top-level body fields, under `respan_params`, or as base64-encoded JSON in the `X-Data-Respan-Params` header. If the same field is sent more than one way, `respan_params` takes precedence over the header, and both take precedence over a top-level field. The exception is `variables`: top-level `variables` are merged key by key over the `variables` from `respan_params` or the header. `respan_params`, `keywordsai_params`, and the decoded header must each be a JSON object; a top-level `respan_params` may also be a JSON string that decodes to one. Any other value is ignored, and so are the header's parameters: the request is served, but none of those parameters apply. Fields sent at the top level still apply. With the OpenAI SDK, use `extra_body`.

For legacy compatibility, `keywordsai_params` is merged into `respan_params`, and `X-Data-Keywordsai-Params` is still accepted and renamed internally.

## Headers

- `X-Data-Respan-Params` string
- `X-Respan-Route-Provider` string
- `X-Respan-Beta` string

## Request body

- object
  - `messages` object[], required: Array of messages in the conversation. Each message has `role` (`system`, `user`, `assistant`, `tool`) and `content`.
    - `role` 'system' | 'user' | 'assistant' | 'tool', required: Message role.
    - `content` union, required: Message content. Use a string for text-only requests, or an array of content parts for multimodal requests.
      - string
      - object[]
    - `name` string: Optional participant name.
    - `tool_call_id` string: Required for tool response messages.
  - `model` string, required: Model to use. See [Models](https://platform.respan.ai/platform/models) for available options.
  - `stream` boolean: Stream back partial progress token by token as server-sent events.
  - `tools` object[]: Tools the model may call. Currently only functions are supported.
  - `tool_choice` object: Controls tool selection. `"none"` = no tools, `"auto"` = model decides, or specify a tool object.
  - `frequency_penalty` number: Penalizes tokens based on frequency in text so far (-2 to 2).
  - `max_tokens` number: Maximum tokens to generate.
  - `temperature` number: Sampling temperature (0-2). Higher = more random.
  - `n` number: Number of completions to generate. Note: costs multiply with `n`.
  - `logprobs` boolean: Return log probabilities of output tokens.
  - `echo` boolean: Echo back the prompt in addition to the completion
  - `stop` string[]: Stop sequences where generation halts.
  - `presence_penalty` number: Penalizes tokens already present in text (-2 to 2).
  - `logit_bias` object: Used to modify the probability of tokens appearing in the response
  - `response_format` object: Output format. Set `{"type": "json_schema", "json_schema": {...}}` for structured output, or `{"type": "json_object"}` for JSON mode.
  - `parallel_tool_calls` boolean: Enable parallel function calling during tool use.
  - `load_balance_group` object: Load balance group selection. Use `{"group_id": "..."}` to route through a configured group.
  - `fallback_models` string[]: Backup models (ranked by priority) if the primary model fails.
  - `customer_credentials` object: Per-customer LLM provider credentials. Keys are provider names, values are API keys.
  - `credential_override` object: One-off credential overrides per provider. Overrides uploaded provider keys for this request only.
  - `cache_enabled` boolean: Enable response caching. See [Caching](/docs/documentation/features/gateway/caching).
  - `cache_ttl` number: How long a cached response is served, in seconds. A response stored without `cache_ttl` is served from the cache for at most 30 minutes.
  - `cache_options` object: Cache behavior options. Properties: `cache_by_customer`, `is_cached_by_model`, `omit_log`.
    - `cache_by_customer` boolean: Partition cache entries by customer identifier.
    - `is_cached_by_model` boolean: Partition cache entries by model name.
    - `omit_log` boolean: Suppress log creation for cache hits.
  - `prompt` object: Prompt template config. Properties: `prompt_id` (required), `variables` (template variables), `version` (number, or `"latest"` for draft), `echo` (return rendered prompt), `override` (use override_params), `override_params` (OpenAI params to override), `schema_version` (`1` = legacy, `2` = prompt config wins). See [Prompt management](/docs/documentation/features/prompt-management/advanced).
  - `retry_params` object: Has no effect on this endpoint: chat completions always uses your organization's retry settings. Per-request `retry_params` (`retry_enabled`, `num_retries`, `retry_after`) applies only to `POST /api/responses`. See [Retries and fallback](/docs/documentation/features/gateway/retries).
  - `disable_log` boolean: When `true`, omits input/output from the log. Metrics (tokens, cost, latency) are still recorded.
  - `model_name_map` object: Azure deployment name mapping. Maps your custom Azure deployment names to standard model names.
  - `models` string[]: Model list for LLM router selection.
  - `exclude_providers` string[]: Providers to exclude from routing. All models under excluded providers are skipped.
  - `exclude_models` string[]: Specific models to exclude from routing.
  - `metadata` object: Custom key-value metadata attached to the span.
  - `custom_identifier` string: Indexed custom tag for fast querying.
  - `customer_identifier` string: End user identifier for analytics and budgets.
  - `customer_params` object: Customer details. Properties: `customer_identifier` (takes precedence over the top-level `customer_identifier`), `name` and `email` (logged with the request, and saved on the customer when Respan first sees it), and `rate_limit` (requests per minute for this customer, overriding your organization's customer rate limit; requests over it get `429`). Budget fields sent here aren't saved or enforced. Set budgets with [Update a user](/docs/apis/users/update-user).
  - `request_breakdown` boolean: Return response metrics summary in the response body. For streaming, metrics appear in the final chunk.
  - `positive_feedback` boolean: User feedback. `true` = liked, `false` = disliked.
  - `load_balance_models` object[]: Inline load balancing options. Each item can include `model`, `weight`, and optional `credentials`.
  - `thread_identifier` string: Conversation thread ID. Spans with the same `thread_identifier` are grouped together.
  - `properties` object: Typed metadata preserving native types (numbers, booleans, nested objects). Unlike `metadata` which coerces to strings.
  - `retries` integer: Has no effect: chat completions always uses your organization's retry settings. See [Retries and fallback](/docs/documentation/features/gateway/retries).
  - `weight` number: Load balancing weight.
  - `span_name` string: Custom span name for tracing.
  - `respan_params` object: Namespaced container for all Respan parameters. Alternative to passing them at top level. Must be a JSON object, or a JSON string that decodes to one. A list, number or boolean is ignored and none of its params apply.

## Response `200`

Successful response for Create chat completion

- object
  - `id` string, required: Chat completion ID.
  - `object` string, required
  - `created` integer, required: Unix timestamp for when the completion was created.
  - `model` string, required: Model used for the completion.
  - `choices` object[], required
    - `index` integer
    - `message` object
      - `role` string
      - `content` string
    - `finish_reason` string
  - `usage` object
    - `prompt_tokens` integer
    - `completion_tokens` integer
    - `total_tokens` integer

## Other responses

- `400`: Invalid request or preprocessing failure.
- `401`: No usable key for the model. The code is `activation_required` when your organization has no credits and no provider key for the model.
- `403`: The API key is missing, invalid or expired.
- `404`: The model isn't available to your organization.
- `424`: The upstream model provider failed.

## Changes

- **2026-10-21** `c30bcb383ee8`: 8 breaking, 22 info
  - request body became required
  - removed the media type `application/json` for the response with the status `401`
  - the response property `error/code` became nullable for the status `400`
  - the response property `error/code` became nullable for the status `404`
  - …26 more
- **2026-10-02** `b830b79cfd52`: 2 info
  - added the non-success response with the status `403`
  - added the non-success response with the status `404`
- **2026-09-17** `6fa8242da8c2`: 1 warning, 1 info
  - deleted the `header` request parameter `Authorization`
  - the endpoint scheme security `RespanApiKey` was added to the API
- **2026-09-12** `22b868ef3f22`: 1 info
  - api operation id `create-chat-completion` removed and replaced with `gateway_createChatCompletion`

[Operation history](https://skmtc.dev/respan/apis/respan-api/changes/api/chat/completions/post.md)

---

[API](https://skmtc.dev/respan/apis/respan-api.md) - [All operations](https://skmtc.dev/respan/apis/respan-api/llms.txt) - [OpenAPI document](https://skmtc.dev/respan/apis/respan-api/revisions/900841387635?raw)