chat

Chat Completions

Changed on

OpenAI-compatible chat completions endpoint.

Supports both streaming and non-streaming responses. Handles reasoning content from otari providers.

Authentication modes:

  • Master key + user field: Use specified user (must exist)
  • API key + user field: Use specified user (must exist)
  • API key without user field: Use the shared "default" user
post/api/v1/chat/completions

Request

  • The document declares no server URL.
  • Auth: one of:
    • API key in header Otari-Key
    • API key in header x-api-key

Headers

Idempotency-Keystring nullable

A unique value, such as a UUID, that makes a non-streaming request safe to retry. A retry with the same key and body returns the original response, request ID and cost without calling the provider or billing again. A retry while the original is still running is answered 409 with Retry-After. Reusing a key for a different body is refused with 422. Ignored for streaming requests, in hybrid mode, and on a deployment without OTARI_SECRET_KEY, which encrypts the stored response.

Request body

frequency_penaltynumber nullable
logit_biasobject nullable
logprobsboolean nullable
max_completion_tokensinteger nullable

Upper bound on generated tokens. OpenAI's current name for the cap max_tokens used to carry; either field is accepted, and this one wins when a request sends both.

max_tokensinteger nullable
max_tool_iterationsinteger nullable
mcp_server_idsstring[] nullable
messagesobject[] required
metadataobject nullable

Tags for cost attribution, recorded on the request's usage rows and filterable in the usage API: up to 16 string pairs, keys up to 64 characters and values up to 512. A null value is ignored. LiteLLM's nested spend_logs_metadata object is also read, and wins over a flat key of the same name; it is never forwarded to the provider.

modelstring required
ninteger nullable
parallel_tool_callsboolean nullable
presence_penaltynumber nullable
prompt_cache_keystring nullable
reasoning_effort'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max' | 'auto' nullable
response_formatobject nullable
seedinteger nullable
service_tierstring nullable
session_labelstring nullable

Optional caller-supplied label for cost attribution (per run, experiment, or conversation). In hybrid mode it is forwarded onto the platform usage report so spend can be sliced by session without standing up OpenTelemetry. Stripped before the request is forwarded upstream to the provider. Has no effect in standalone mode, where there is no platform to report it to.

streamboolean
stream_optionsobject nullable
temperaturenumber nullable
tools_headerstring nullable

Optional override for the lead-in that the gateway prepends before the per-tool hint block in the system message. Useful for expressing global tool-selection policy (e.g. 'prefer MCP tools over code_execution'). Falls back to OTARI_TOOLS_HEADER env, then to the built-in default.

top_logprobsinteger nullable
top_pnumber nullable
userstring nullable

Response

Successful Response

unknown required

Changes