PIpioneer
openai-compat

Chat Completions

Changed on

OpenAI-compatible chat completions endpoint.

Thin shell over :func:services.inference.adapters.openai_chat.run_chat_completion. The adapter handles LLM-passthrough vs Fastino-task dispatch, SSE rendering with <think>...</think> folding for reasoning models, tool-call deltas, finish-reason mapping, persistence, and error mapping. The router keeps only HTTP-shaped concerns: route declaration, auth, rate limiting.

Args: body: Validated :class:ChatCompletionRequest. request: FastAPI request (forwarded so the adapter can read API-key billing context out of request.state for streaming responses). auth: Authenticated request context.

Returns: :class:ChatCompletionResponse for non-streaming, or a :class:StreamingResponse of chat.completion.chunk SSE events terminated by data: [DONE] when body.stream is true.

post/v1/chat/completions

Request

Request body

modelstring required
temperaturenumber nullable

Sampling temperature

max_tokensinteger nullable
response_formatobject nullable
streamboolean
extra_headersobject nullable
extra_bodyobject nullable
reasoningobject nullable

Opt-in reasoning / extended-thinking controls. Accepts Fastino's normalized shape with keys enabled (bool), max_tokens (int, Anthropic-style budget), effort (one of minimal/low/medium/high/xhigh/max/none, OpenAI-style tier), and exclude (bool, hide reasoning tokens from the response). effort and max_tokens are mutually exclusive. Fastino extensions for Claude routes (Anthropic direct + Bedrock): mode (manual/adaptive — adaptive lets the model pick thinking depth per request, required on Opus 4.7+) and display (summarized/omitted — controls whether thinking text streams back; omitted preserves only the signature for multi-turn). On Chat Completions, provider reasoning text is hidden by default and is returned on message.reasoning_content / delta.reasoning_content only when the caller explicitly requests visible reasoning with exclude=false or display=summarized. Fastino canonicalizes this into InferenceRequest.reasoning at the adapter boundary and each provider renders it to its native wire field (Anthropic thinking, OpenAI reasoning_effort, Vercel AI Gateway reasoning). Fastino does not enable reasoning by default.

storeboolean
metadataobject nullable
top_pnumber nullable
ninteger nullable
presence_penaltynumber nullable
frequency_penaltynumber nullable
logit_biasobject nullable
userstring nullable
seedinteger nullable
task_typestring nullable

Deprecated. Legacy task hint (extract_entities / classify_text / extract_json / ner / schema). The unified schema disambiguates the task automatically so this field is no longer required. Submitting it emits Deprecation: true and Sunset: <RFC 7231 date> headers on the response.

include_confidenceboolean
include_spansboolean
thresholdnumber

Confidence threshold for encoder (GLiNER) predictions.

Response

Chat completion. Returns application/json (ChatCompletionResponse) by default, or text/event-stream of ChatCompletionStreamChunk events terminated by data: [DONE] when stream=true.

idstring
objectstring
createdinteger
modelstring required

Changes

  • 0e5e86e2bae5Revision changes
    • warning

      removed the request property

    • warning

      removed the request property

    • warning

      removed the optional property from the response with the status (media type: application/json)

    • warning

      removed the optional property from the response with the status (media type: text/event-stream)

    • info

      added the optional property to the response with the status (media type: application/json)

    • info

      added the optional property to the response with the status (media type: text/event-stream)

  • 1cffaad2a921Revision changes
    • warning

      the / request property's max was set to 2.00

    • warning

      the / request property's min was set to 0.00

    • warning

      removed the optional property from the response with the status

    • info

      added the new optional request property

    • info

      added the required property to the response with the status