Sign in
completions

Generate LLM completion

Generates a completion based on the input messages and retrieval chunks. If the 'stream' parameter is set to true, the response is streamed. By default the stream is plain text (text/plain). Send the request header 'Accept: text/event-stream' to instead receive a Server-Sent Events (SSE) stream of ag-ui protocol events (delta text as TEXT_MESSAGE_CONTENT, grounding and citations as CUSTOM events, history ids and completion as RUN_STARTED / RUN_FINISHED).

Required scope: `inference:run`

post/completions

Request body

streamboolean

Whether to stream the response. If true, the response is streamed as 'text/plain' by default, or as an ag-ui Server-Sent Events stream ('text/event-stream') when the request sends the header 'Accept: text/event-stream'.

citationboolean

Whether to include citations in the response

groundingboolean

Whether to enable Gemini's Google Search grounding during answer generation. Only effective when the completion model is a Gemini model and the gemini_grounding feature is enabled on the server; otherwise this flag is ignored.

tagsstring[]

target tag names to be obtained

tag_idsstring[]

target tag IDs to be obtained

tag_filter_logic'AND' | 'OR'

Logical operator for combining filter conditions

source_typesSourceType[]
file_typesFileType[]
date_frominteger

Start date for content search (Unix timestamp in seconds)

date_tointeger

End date for content search (Unix timestamp in seconds)

domainsstring[]

Array of domains to search within (supports partial matching)

use_postfilterboolean

Whether to bypass LanceDB prefilter and apply WHERE after the vector search (IVF_PQ) returns top-K. Significantly faster for broad filters that cover most of the table, but may return fewer than limit results when the hit rate is low.

limitinteger

Maximum number of chunks to retrieve as context for completion

authz_policystring

(reserved for future use) Name of the registered authz policy to evaluate when retrieving context. Defaults to the reserved "default" policy when omitted. Ignored when authz is disabled.

principal_idstring

Identifier of the end-user (principal) on whose behalf this request is made. Used to look up the principal's authz subject attributes for policy evaluation. When omitted, subject attributes are empty (most restrictive). Ignored when authz is disabled.

conversation_idstring

Conversation to append this turn to. When omitted, a new conversation is created server-side and its id is returned (response body for JSON, the X-Conversation-Id header for streaming). Pass it back on subsequent turns so the answer is persisted into the same conversation history tree. When set, the past conversation is rebuilt server-side from the stored tree, so only the latest user message in messages is used as the new input (earlier messages entries are ignored). Retrieval uses that latest question.

parent_message_idstring

Id of the message node to branch this turn from (the parent of the new user message). When omitted, the turn continues from the conversation's current active leaf. Set it to fork a branch (e.g. editing an earlier question). Must belong to conversation_id.

regenerateboolean

When true, the latest user message is NOT persisted again; instead a new assistant answer is created as a sibling under parent_message_id (which must reference an existing user message). Used to regenerate an answer.

truncation'auto' | 'disabled'

How to handle a reconstructed conversation that exceeds the model's context budget (only relevant when conversation_id is set, i.e. history is rebuilt server-side). "auto": drop the oldest turns until it fits (the latest question is always kept). "disabled": return 400 if it does not fit.

Example request

{
  "authz_policy": "external-chatbot",
  "principal_id": "user-123",
  "conversation_id": "550e8400-e29b-41d4-a716-446655440000",
  "parent_message_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479"
}

Response

Successful completion. If the 'stream' parameter is false, the response is returned as application/json. If the 'stream' parameter is true, the response is returned as a stream of plain text (text/plain) by default, or as an ag-ui Server-Sent Events stream (text/event-stream) when the request sends 'Accept: text/event-stream'.

createdinteger required

The Unix timestamp (in seconds) of when the completion was created

conversation_idstring nullable

Target conversation id. For an existing conversation (conversation_id was supplied) it is always returned, even if this turn's answer was not persisted. For a new conversation it is null when nothing was persisted (e.g. no answer generated, or the history write failed).

user_message_idstring nullable

Id of the persisted user message node for this turn. Null when the turn was not persisted.

assistant_message_idstring nullable

Id of the persisted assistant message node (the new active leaf). Null when no assistant node was persisted.

Example response

{
  "created": 1677649420,
  "conversation_id": "550e8400-e29b-41d4-a716-446655440000",
  "user_message_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479",
  "assistant_message_id": "9b2e6f1a-3c4d-4e5f-8a9b-0c1d2e3f4a5b",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?"
      },
      "finish_reason": "stop"
    }
  ]
}

Changes

Changed in 19 of the 71 revisions of this API.103068

  • 09c7c8b9121d1See the full diff
    • ○

      added the new optional request property

      new-optional-request-property

  • b849852eb53b7See the full diff
    • ○

      added the new optional request property

      new-optional-request-property

    • ○

      added the new optional request property

      new-optional-request-property

    • ○

      added the new optional request property

      new-optional-request-property

    • ○

      added the new optional request property

      new-optional-request-property

    • ○

      added the optional property to the response with the status

      response-optional-property-added

    • ○

      added the optional property to the response with the status

      response-optional-property-added

    • ○

      added the optional property to the response with the status

      response-optional-property-added

  • dda59d58227c1See the full diff
    • ○

      added the optional property //// to the response with the status

      response-optional-property-added

  • 5b90237b6d514See the full diff
    • ○

      added the new optional request property

      new-optional-request-property

    • ○

      added the optional property // to the response with the status

      response-optional-property-added

    • ○

      added the optional property // to the response with the status

      response-optional-property-added

    • ○

      added the optional property // to the response with the status

      response-optional-property-added

  • 05add33422b22See the full diff
    • ○

      added the new optional request property

      new-optional-request-property

    • ○

      added the new optional request property

      new-optional-request-property

  • 406f64767f061See the full diff
    • ○

      added the new optional request property

      new-optional-request-property

  • 02104c44c0991See the full diff
    • ○

      added the new optional request property

      new-optional-request-property

  • 995a53b75a921See the full diff
    • ○

      added the new optional request property

      new-optional-request-property

  • 756b9db552592See the full diff
    • ○

      added the new datetime enum value to the request property ///

      request-property-enum-value-added

    • ○

      added the new datetime enum value to the request property ///

      request-property-enum-value-added

  • 288bf03b951a1See the full diff
    • ○

      added the new optional request property

      new-optional-request-property

Of the 71 revisions, 18 have no diff computed.