Sign in
AI Chat

Streaming AI Chat

Streaming AI chat with real-time response chunks via Server-Sent Events (SSE).

The response is a stream of events with the following types:

  • encoding_complete: PII encoding finished
  • llm_chunk: Partial LLM response text
  • llm_complete: Full LLM response assembled
  • complete: Full processing finished (includes decoded response)
  • error: An error occurred

SDK Usage:

const result = await client.chatWithAIStream(
  { prompt: 'Summarize account for Jane Smith', model: 'gemini-2.0-flash-exp' },
  (update) => {
    if (update.type === 'llm_chunk') {
      process.stdout.write(update.data.chunk);
    }
  }
);
post/api/ai/chat/stream

Request body

promptstring required

User prompt (will be automatically encoded to remove PII before reaching the LLM)

sessionIdstring

Session tracking ID for multi-turn conversations

sessionContinuationIdsstring[]

Continuation IDs from prior conversation turns (enables cross-turn decoding)

modelstring

LLM model name

provider'vertex' | 'aws' | 'mock'

LLM provider

temperaturenumber double

Generation temperature (higher = more creative)

maxTokensinteger

Maximum tokens in the LLM response

Example request

{
  "prompt": "Analyze credit history for John Doe at john@company.com",
  "model": "gemini-2.0-flash-exp",
  "provider": "vertex"
}

Response

Server-Sent Events stream

Changes