--- title: "Chat Completions" method: POST path: "/v1/chat/completions" tags: ["openai-compat"] --- # Chat Completions `POST /v1/chat/completions` ## Request - Base URL: `https://api.fastino.ai` - URL: `https://api.fastino.ai/v1/chat/completions` - Auth: one of: - API key in header `X-API-Key` - HTTP bearer - TypeScript: (a function with typed inputs; its response is checked against the schema) Serve an OpenAI-compatible chat completion. ## Request body - ChatCompletionRequest: OpenAI-compatible chat completion request. - `extra_body` object, nullable - `extra_headers` object, nullable - `frequency_penalty` number, nullable - `include_confidence` boolean - `include_spans` boolean - `logit_bias` object, nullable - `max_tokens` integer, nullable - `messages` ChatMessage[], required - `content` union - string - object[] - `name` string, nullable - `reasoning_content` string, nullable - `role` string, required - `tool_call_id` string, nullable - `tool_calls` unknown[], nullable - unknown - `metadata` object, nullable - `model` string, required - `n` integer, nullable - `presence_penalty` number, nullable - `reasoning` object, nullable: Opt-in reasoning / extended-thinking controls. Accepts Fastino's normalized shape with keys ``enabled`` (bool), ``max_tokens`` (int, Anthropic-style budget), ``effort`` (one of minimal/low/medium/high/xhigh/max/none, OpenAI-style tier), and ``exclude`` (bool, hide reasoning tokens from the response). ``effort`` and ``max_tokens`` are mutually exclusive. Fastino extensions for Claude routes (Anthropic direct + Bedrock): ``mode`` (manual/adaptive — adaptive lets the model pick thinking depth per request, required on Opus 4.7+) and ``display`` (summarized/omitted — controls whether thinking text streams back; omitted preserves only the signature for multi-turn). On Chat Completions, provider reasoning text is hidden by default and is returned on ``message.reasoning_content`` / ``delta.reasoning_content`` only when the caller explicitly requests visible reasoning with ``exclude=false`` or ``display=summarized``. Fastino canonicalizes this into InferenceRequest.reasoning at the adapter boundary and each provider renders it to its native wire field (Anthropic ``thinking``, OpenAI ``reasoning_effort``, Vercel AI Gateway ``reasoning``). Fastino does not enable reasoning by default. - `response_format` object, nullable - `schema` union: Schema for the Fastino encoder (GLiNER models). A decoder model refuses it with a 400; send a JSON Schema in ``response_format`` for structured output. **Deprecated when supplied as a flat list** of entity labels; use the unified dict shape (``entities`` / ``classifications`` / ``structures`` / ``relations``) instead. Deprecated submissions emit ``Deprecation: true`` and ``Sunset: `` headers. Record modes (``natural``, ``latent``, ``anchorless``) are hosted when ``encoder_features`` includes ``records``, which ``fastino/gliner2.5-multi-v1`` does. ``mode``, ``anchor``, ``occurrence_policy``, and field ``cardinality`` return 400 on any other encoder. Span attributes (``entity_attributes``) are hosted when ``encoder_features`` includes ``span_attributes``, which ``fastino/gliner2.5-multi-v1`` does. Constrained classification (``version``, ``tasks``, ``constraints``) is hosted when ``encoder_features`` includes ``constrained_classification``, which ``fastino/gliner2.5-multi-v1`` does. JointIE (``constraints`` without ``tasks`` or ``version``, and relation ``head`` / ``tail`` options) is hosted when ``encoder_features`` includes ``joint_ie``, which ``fastino/gliner2.5-multi-v1`` does. Read ``encoder_features`` on ``GET /v1/base-models``. - string[] - object - `seed` integer, nullable - `stop` union - string[] - string - `store` boolean - `stream` boolean - `system` union - string - object[] - `task_type` string, nullable: **Deprecated.** Legacy task hint (``extract_entities`` / ``classify_text`` / ``extract_json`` / ``ner`` / ``schema``). The unified schema disambiguates the task automatically so this field is no longer required. Submitting it emits ``Deprecation: true`` and ``Sunset: `` headers on the response. - `temperature` number, nullable: Sampling temperature - `threshold` number: Confidence threshold for encoder (GLiNER) predictions. - `tool_choice` union - string - object - `tools` unknown[], nullable - unknown - `top_p` number, nullable - `truncate_overflow_text` boolean: Encoder (GLiNER) models only. When the text is longer than the model's max_input_tokens, cut it to fit instead of rejecting the request with input_too_long. Off by default. A response whose input was cut reports x_fastino.truncated: true; entities past the cut are not returned. Token usage still counts the full submitted text. Texts over 50,000 characters are rejected before truncation applies. Not supported for full fine-tuned encoder models, which still reject. - `user` string, nullable ## Response `200` Chat completion. Returns ``application/json`` (``ChatCompletionResponse``) by default, or ``text/event-stream`` of ``ChatCompletionStreamChunk`` events terminated by ``data: [DONE]`` when ``stream=true``. - ChatCompletionResponse: OpenAI-compatible chat completion response. - `choices` ChatCompletionChoice[], required - `finish_reason` string, nullable - `index` integer - `message` object - `created` integer - `id` string - `model` string, required - `object` string - `usage` ChatCompletionUsage, required: Token usage statistics. ``prompt_tokens`` follows the upstream wire contract — *includes* every input class (non-cached, cache read, and cache write). The breakdown is exposed under ``prompt_tokens_details`` so consumers can attribute the cached-read and cache-write subsets. - `completion_tokens` integer - `prompt_tokens` integer - `prompt_tokens_details` PromptTokensDetails: Per-input-class breakdown for OpenAI-shape usage payloads. Mirrors OpenAI's ``prompt_tokens_details`` block and the industry cache-creation extension so clients reading ``usage.prompt_tokens_details.cached_tokens`` / ``cache_write_tokens`` keep working when Fastino relays a cache-aware upstream response. Both counts are subsets of ``prompt_tokens`` on the wire — that's the upstream contract Fastino relays faithfully. Attributes: cached_tokens: Input tokens served from the upstream prompt cache (cache read). cache_write_tokens: Input tokens written into the upstream prompt cache (cache creation). ``0`` for upstreams that bill writes as plain input (OpenAI, vLLM). - `cache_write_tokens` integer - `cached_tokens` integer - `total_tokens` integer - `x_fastino` FastinoExtension: Fastino-specific extension fields appended to OpenAI-compatible responses. OpenAI's API contract reserves the unprefixed top-level keys (``id``, ``choices``, ``usage``, …); custom data must live under a clearly namespaced key. ``x_fastino`` is that key. Attributes: inference_id: The Fastino-side identifier of the persisted ``inferences`` row associated with this completion. Present when persistence is enabled (``extra_body.store == True``) and the row was successfully recorded; ``None`` for ad-hoc requests that opted out of persistence. The frontend uses this to poll ``GET /inferences/{id}`` for asynchronous judge results without coupling the inference response latency to the judge. default_schema_applied: Whether Fastino supplied the selected encoder model's default because the request did not provide a GLiNER schema. This also applies when ``response_format`` is ``text`` or ``json_object``, since neither contains a schema. truncated: Whether the encoder input was cut to fit the model's ``max_input_tokens`` because the request set ``truncate_overflow_text``. For a batch, at least one text was cut. - `default_schema_applied` boolean - `inference_id` string, nullable - `truncated` boolean ## Other responses - `422`: Validation Error - `425`: The model is temporarily warming. Wait the number of seconds in `Retry-After`, then retry the same request with a bounded retry policy. ## Changes - **2026-10-12** `43d7e06a824b`: 6 info - added the new optional request property `truncate_overflow_text` - added the non-success response with the status `425` - added the optional property `x_fastino/anyOf[subschema #1: FastinoExtension]/default_schema_applied` to the response with the `200` status (media type: application/json) - added the optional property `x_fastino/anyOf[subschema #1: FastinoExtension]/default_schema_applied` to the response with the `200` status (media type: text/event-stream) - …2 more - **2026-09-28** `54da122a2879`: 4 warning, 2 info - removed the request property `effort` - removed the request property `models` - removed the optional property `x_pioneer` from the response with the `200` status (media type: application/json) - removed the optional property `x_pioneer` from the response with the `200` status (media type: text/event-stream) - …2 more - **2026-09-23** `620970c6f5f2`: 13 breaking, 11 warning, 51 info - the request property `messages/items/role` became required - the response property `created` became optional for the status `200` (media type: application/json) - the response property `id` became optional for the status `200` (media type: application/json) - the response property `object` became optional for the status `200` (media type: application/json) - …71 more [Operation history](https://skmtc.dev/fastino-ai/apis/pioneer-inference-api/changes/v1/chat/completions/post.md) --- [API](https://skmtc.dev/fastino-ai/apis/pioneer-inference-api.md) - [All operations](https://skmtc.dev/fastino-ai/apis/pioneer-inference-api/llms.txt) - [OpenAPI document](https://skmtc.dev/fastino-ai/apis/pioneer-inference-api/revisions/81465e18c1d2?raw)