---
title: "Stream transcription (WebSocket)"
method: GET
path: "/v1/audio/transcriptions/stream"
tags: ["Speech To Text (Hear)"]
---

# Stream transcription (WebSocket)

`GET /v1/audio/transcriptions/stream`

## Request

- Base URL: `https://api.pyai.com`
- URL: `https://api.pyai.com/v1/audio/transcriptions/stream`
- Auth: one of:
  - HTTP bearer
  - API key in header `x-api-key`

Upgrade to a WebSocket for streaming speech-to-text with eager partials (PyAI Hear first partial measured at about 200 ms in-region. It is revisable and not an SLA). Requires the `hear:stream` scope. The only frame protocol is `pyai-hear-v1`, the published frame catalog below.

**Auth:** browsers can't set `Authorization` on a WebSocket, so send the key as a subprotocol, together with the `pyai.v1` marker: `Sec-WebSocket-Protocol: pyai.v1, pyai-key.<API_KEY>` (server clients may use `?api_key=` instead). Offer both values: PyAI echoes only the marker and never reflects your key, and a 101 can only select a value the client offered. The key is validated and swapped for the internal upstream credential on the upgrade.

Set `vocabulary` only when the session has known names, brands, products, or other distinctive terms. This biases known vocabulary only when supplied for the session or enabled through the stored `hear_stream` profile. Request terms take priority and the effective list is fixed when the session opens.

Knowledge-base retrieval is off by default for plain transcription. Set `grounding=true` on the connect URL to opt in when the service has retrieval configured; it does not enable an unconfigured service. Omission or `grounding=false` skips retrieval. Retrieval may add a `grounding` list to `speech_final` and delays delivery while the bounded lookup completes. Opting out removes that lookup and its grounding add-on usage, while preserving transcript text, event order, and Hear usage. The option is fixed for the connection; only one lowercase `true` or `false` value is accepted. Invalid or repeated values return `400 invalid_grounding` before the WebSocket opens.

**Client -> server:** stream mono little-endian PCM16 binary audio frames continuously at `sample_rate=8000` or `sample_rate=16000` (default). The service converts 8 kHz input before recognition and endpointing. Other formats are rejected with `400 unsupported_audio_format` before the WebSocket opens. Send a JSON `{"type":"commit"}` text frame to force-finalize the current utterance (e.g. when your VAD detects end-of-turn). Completion frames may have empty text when the utterance contains no recognized speech; retain these acknowledgments when draining pending commits. Closing the socket also flushes a final for any buffered audio.

Set `endpointing_ms` on the connect URL or send `{"type":"config","endpointing_ms":800}` mid-session. It is the minimum trailing-pause length before an utterance may end. Turn detection may wait longer; an utterance always completes within `max(endpointing_ms, 1500 ms)` of streamed silence. Keep sending audio frames, including silence, while the caller pauses. Pausing the stream pauses the frame-driven clock.

**Server -> client (JSON text frames):**

| `type` | When | Payload |
| --- | --- | --- |
| `config_ack` | after connect-time or mid-session endpointing config | `{endpointing_ms, effective_floor_ms, effective_ceiling_ms, score_interval_ms, warnings}`, the applied values and any validation warnings |
| `partial` | every eager tick | `{text, stable_text, active_text, utterance_id, t_ms}`, the live hypothesis for that `utterance_id` |
| `partial_stable` | when a prefix locks in | `{text, utterance_id, t_ms}`, the portion the recognizer no longer expects to revise |
| `speech_final` | on endpoint/commit | `{text, utterance_id, t_ms, audio_ms, endpoint_reason}`, stable, end of an utterance |
| `final` | follows `speech_final` | `{text, utterance_id, t_ms, audio_ms, endpoint_reason}`, corrected full-context transcript |
| `usage` | just before a graceful close | `{product, meter, audio_seconds, minutes}`, the session's billed active-audio, so you can reconcile realtime spend in-band (a realtime WS carries no `x-pyai-units` response header). Best-effort; absent if the session had no billable audio or closed abnormally. |
| `error` | on fault | `{code, message}` |

`t_ms` is the audio-timeline position of the hypothesis; `audio_ms` is the utterance's active-speech length (the billed signal); `utterance_id` groups partials/finals for one utterance. Assert that `config_ack.warnings` is empty instead of inferring that a setting applied. Warning reasons include `clamped_to_range`, `not_a_number`, and `unknown_config_field`; invalid config leaves the session open. `endpoint_reason` reports why automatic endpointing fired, including `peak_te_early` for the high-confidence path and `silence_backstop` for the bounded fallback. Log it when tuning conversational turns. Cue grounding configuration frames remain reserved and cannot change the connect-time `grounding` option.

If recognition loses its connection or cannot deliver or finish recognizing audio after caller PCM was accepted, the server emits `error` with `code: recognition_interrupted` and closes with `1011`. Treat that session as incomplete and start a new stream; a reconnect does not recover missing audio. Internal connection retries are limited to before caller audio is accepted. No further transcript results or automatic Recap are produced from the interrupted session.

**Close codes:** `1000` normal · `1008` auth/policy (bad key, scope, revoked token) · `1011` engine error · `4429` over concurrency cap.

**Billing:** metered active audio at the Hear rate ($0.001/min), speech time derived from transcript timing rather than connection wall-clock.

## Query parameters

- `protocol` 'pyai-hear-v1'
- `model` string
- `grounding` boolean
- `language` 'auto' | 'en' | 'es' | 'fr' | 'de' | 'hi' | 'it' | 'pt' | 'nl' | 'zh' | 'yue' | 'ar' | 'id' | 'ko' | 'th' | 'vi' | 'ja' | 'tr' | 'ms' | 'sv' | 'da' | 'fi' | 'pl' | 'cs' | 'fil' | 'fa' | 'el' | 'hu' | 'mk' | 'ro'
- `sample_rate` 8000 | 16000
- `encoding` 'pcm16'
- `interim_results` boolean
- `numerals` boolean
- `smart_format` boolean
- `dictation` boolean
- `drop_fillers` boolean
- `vocabulary` string
- `seed` integer
- `temperature` number
- `endpointing_ms` integer
- `call_id` string
- `pack_id` string
- `call_direction` 'inbound' | 'outbound'

## Other responses

- `101`: Switching Protocols, the streaming transcription WebSocket is open.
- `401`: Missing or invalid API key (`code: unauthorized`)
- `402`: Billing gate: org out of prepaid credit, per-key budget hit, or plan quota exhausted (`code: credit_exhausted | key_budget_exceeded | insufficient_quota`). Do not retry; add credit or raise the limit. A brand-new key may see this on its first call until the account is funded.
- `403`: Key lacks the required scope or the origin is not allow-listed (`code: forbidden | origin_not_allowed`)
- `429`: Too many requests or too many concurrent realtime sessions; see Retry-After header (`code: rate_limit_exceeded | concurrency_limit_exceeded | daily_cap_exceeded`)

---

[API](https://skmtc.dev/pyai/apis/pyai-api.md) · [All operations](https://skmtc.dev/pyai/apis/pyai-api/llms.txt) · [OpenAPI document](https://skmtc.dev/pyai/apis/pyai-api/revisions/819d8e7c4d09?raw)