RErespan
Gateway

Create a chat completion

Changed on

Sends a chat completion request through the Respan gateway with automatic logging. Accepts OpenAI chat completion parameters and Respan options for fallbacks, caching, and prompt management.

Pass Respan parameters in top-level body fields, under respan_params, or as base64-encoded JSON in the X-Data-Respan-Params header. If the same field is sent more than one way, respan_params takes precedence over the header, and both take precedence over a top-level field. The exception is variables: top-level variables are merged key by key over the variables from respan_params or the header. respan_params, keywordsai_params, and the decoded header must each be a JSON object; a top-level respan_params may also be a JSON string that decodes to one. Any other value is ignored, and so are the header's parameters: the request is served, but none of those parameters apply. Fields sent at the top level still apply. With the OpenAI SDK, use extra_body.

For legacy compatibility, keywordsai_params is merged into respan_params, and X-Data-Keywordsai-Params is still accepted and renamed internally.

post/api/chat/completions

Request

  • Base URL
    https://api.respan.ai
  • URL
    https://api.respan.ai/api/chat/completions
  • Auth
    HTTP bearer

Headers

X-Data-Respan-Paramsstring

Base64-encoded JSON object of Respan parameters. Legacy X-Data-Keywordsai-Params is still accepted.

X-Respan-Route-Providerstring

Pin the request to a specific provider without changing the model slug. Use the provider ID shown for that model in the model catalog.

X-Respan-Betastring

Comma-separated beta feature flags. Available: token-breakdown-2026-03-26, env-scoped-integrations-2026-03-28

Request body

modelstring required

Model to use. See Models for available options.

streamboolean

Stream back partial progress token by token as server-sent events.

toolsobject[]

Tools the model may call. Currently only functions are supported.

tool_choiceobject

Controls tool selection. "none" = no tools, "auto" = model decides, or specify a tool object.

frequency_penaltynumber

Penalizes tokens based on frequency in text so far (-2 to 2).

max_tokensnumber

Maximum tokens to generate.

temperaturenumber

Sampling temperature (0-2). Higher = more random.

nnumber

Number of completions to generate. Note: costs multiply with n.

logprobsboolean

Return log probabilities of output tokens.

echoboolean

Echo back the prompt in addition to the completion

stopstring[]

Stop sequences where generation halts.

presence_penaltynumber

Penalizes tokens already present in text (-2 to 2).

logit_biasobject

Used to modify the probability of tokens appearing in the response

response_formatobject

Output format. Set {"type": "json_schema", "json_schema": {...}} for structured output, or {"type": "json_object"} for JSON mode.

parallel_tool_callsboolean

Enable parallel function calling during tool use.

load_balance_groupobject

Load balance group selection. Use {"group_id": "..."} to route through a configured group.

fallback_modelsstring[]

Backup models (ranked by priority) if the primary model fails.

customer_credentialsobject

Per-customer LLM provider credentials. Keys are provider names, values are API keys.

credential_overrideobject

One-off credential overrides per provider. Overrides uploaded provider keys for this request only.

cache_enabledboolean

Enable response caching. See Caching.

cache_ttlnumber

How long a cached response is served, in seconds. A response stored without cache_ttl is served from the cache for at most 30 minutes.

promptobject

Prompt template config. Properties: prompt_id (required), variables (template variables), version (number, or "latest" for draft), echo (return rendered prompt), override (use override_params), override_params (OpenAI params to override), schema_version (1 = legacy, 2 = prompt config wins). See Prompt management.

retry_paramsobject

Has no effect on this endpoint: chat completions always uses your organization's retry settings. Per-request retry_params (retry_enabled, num_retries, retry_after) applies only to POST /api/responses. See Retries and fallback.

disable_logboolean

When true, omits input/output from the log. Metrics (tokens, cost, latency) are still recorded.

model_name_mapobject

Azure deployment name mapping. Maps your custom Azure deployment names to standard model names.

modelsstring[]

Model list for LLM router selection.

exclude_providersstring[]

Providers to exclude from routing. All models under excluded providers are skipped.

exclude_modelsstring[]

Specific models to exclude from routing.

metadataobject

Custom key-value metadata attached to the span.

custom_identifierstring

Indexed custom tag for fast querying.

customer_identifierstring

End user identifier for analytics and budgets.

customer_paramsobject

Customer details. Properties: customer_identifier (takes precedence over the top-level customer_identifier), name and email (logged with the request, and saved on the customer when Respan first sees it), and rate_limit (requests per minute for this customer, overriding your organization's customer rate limit; requests over it get 429). Budget fields sent here aren't saved or enforced. Set budgets with Update a user.

request_breakdownboolean

Return response metrics summary in the response body. For streaming, metrics appear in the final chunk.

positive_feedbackboolean

User feedback. true = liked, false = disliked.

load_balance_modelsobject[]

Inline load balancing options. Each item can include model, weight, and optional credentials.

thread_identifierstring

Conversation thread ID. Spans with the same thread_identifier are grouped together.

propertiesobject

Typed metadata preserving native types (numbers, booleans, nested objects). Unlike metadata which coerces to strings.

retriesinteger

Has no effect: chat completions always uses your organization's retry settings. See Retries and fallback.

weightnumber

Load balancing weight.

span_namestring

Custom span name for tracing.

respan_paramsobject

Namespaced container for all Respan parameters. Alternative to passing them at top level. Must be a JSON object, or a JSON string that decodes to one. A list, number or boolean is ignored and none of its params apply.

Example request

{
  "model": "gpt-6-luna",
  "messages": [
    {
      "role": "user",
      "content": "Reply with exactly ok."
    }
  ],
  "max_tokens": 32,
  "temperature": 0,
  "reasoning_effort": "none"
}

Response

Successful response for Create chat completion

idstring required

Chat completion ID.

objectstring required
createdinteger required

Unix timestamp for when the completion was created.

modelstring required

Model used for the completion.

Example response

{
  "object": "chat.completion",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "ok"
      },
      "finish_reason": "stop"
    }
  ]
}

Changes

  • c30bcb383ee8Revision changes
    • breaking

      request body became required

    • breaking

      removed the media type application/json for the response with the status

    • breaking

      the response property / became nullable for the status

    • breaking

      the response property / became nullable for the status

    • breaking

      the response property / became nullable for the status

    • breaking

      the response property / became nullable for the status

    • breaking

      the response property / became nullable for the status

    • breaking

      the response property / became nullable for the status

    • info

      api operation id gateway_createChatCompletion removed and replaced with createChatCompletion

    • info

      api tag Gateway added

    • info

      api tag gateway removed

    • info

      the request property default value [] was added

    • info

      the request property default value [] was added

    • info

      the request property default value false was added

    • info

      the request property default value false was added

    • info

      the request property default value false was added

    • info

      the request property format was generalized from double to no format

    • info

      the request property format was generalized from double to no format

    • info

      the request property format was generalized from double to no format

    • info

      the request property format was generalized from double to no format

    • info

      the request property format was generalized from double to no format

    • info

      the request property format was generalized from double to no format

    • info

      the request property format was generalized from double to no format

    • info

      added the media type text/event-stream for the response with the status

    • info

      removed subschema #1 subschema #2 from the / response property oneOf list for the response status

    • info

      removed subschema #1 subschema #2 from the / response property oneOf list for the response status

    • info

      removed subschema #1 subschema #2 from the / response property oneOf list for the response status

    • info

      removed subschema #1 subschema #2 from the / response property oneOf list for the response status

    • info

      removed subschema #1 subschema #2 from the / response property oneOf list for the response status

    • info

      removed subschema #1 subschema #2 from the / response property oneOf list for the response status

  • b830b79cfd52Revision changes
    • info

      added the non-success response with the status

    • info

      added the non-success response with the status