Inference

Create a Responses-style inference request

Authenticates an Infer API key, validates a canonical model selector and routing policy, reserves the request's spend ceiling, and returns a charge record reference after completion. With stream=true, Infer opens the provider stream before returning HTTP 200, forwards reasoning, text, and function-call deltas in real time, emits keepalive comments during quiet reasoning periods, and places settled usage, cache accounting, final charge, and receipt metadata in response.completed. Infer does not impose a generation wall-clock timeout while the client and upstream connection remain alive; provider-originated timeout/error events remain possible. Client disconnect cancels the upstream stream and reconciles the reservation. Exact model-family selection does not cross into another model family. Route eligibility still depends on current capabilities, privacy, health, authorization, margin, and capacity. Read /api/public/status immediately before use. A response model name, streamed output, charge, or receipt does not attest an upstream provider identity or satisfy Infer's separate strict supplier-evidence standard.

post/v1/responses

Request

  • Base URL: https://infer.flow7.org
  • URL: https://infer.flow7.org/v1/responses
  • Auth: HTTP bearer

Headers

Idempotency-Keystring

Stable retry key scoped to the organization. The header takes precedence over the body field. When neither is supplied, Infer generates a one-use key; clients that may retry should send a stable value explicitly. A failed operation keeps its key as a terminal failure record; retry that operation with a new key.

X-Session-Affinitystring

Optional stable session identifier. relay.session_id in the JSON body takes precedence.

Request body

modelstring required

Canonical selector from the model list or public catalog.

max_output_tokensinteger

Output ceiling, sent upstream and covered by the wallet reservation. When omitted, Infer sets it to the output room of the routes serving the request, at most 32768 tokens. A reply that reaches it ends with status "incomplete" and incomplete_details, and is charged for what was generated.

streamboolean
toolsobject[]

Responses-style function tools. The selected route must advertise tool support.

textobject

Responses text options, including a structured-output format. Route support is required.

response_formatobject

Accepted structured-output shorthand. Route support is required.

reasoningobject
session_idstring

Top-level session identifier accepted for compatibility. relay.session_id is preferred.

idempotency_keystring

Body fallback for Idempotency-Key. The header wins when both are present; Infer generates a one-use key when neither is supplied.

request_idstring

Optional client correlation value.

cache_controlobject

Optional provider-compatible prompt-cache control. Cache writes are metered separately. Not allowed with relay.privacy=zero-retention.

prompt_cache_keystring

Optional provider prompt-cache routing key. When relay.session_id is used on a supported native Responses route, Infer also derives a private organization-scoped sticky cache key automatically. Not allowed with zero-retention.

modalitiesstring[]

Optional native Responses modality hint. Actual media input is still admitted only when the selected route supports it.

Response

Completed response as JSON, or Responses events when stream is true. Infer currently assembles the completed upstream result into an SSE event sequence; this description does not promise token-by-token upstream delivery.

idstring required
object'response' required
created_atinteger required
status'completed' required
modelstring required

The requested Infer selector. Read relay.resolved_model_class for the class that completed.

outputobject[] required
output_textstring required

Changes

No recorded changes to this operation across all 1 revision of this API.