Stream transcription (WebSocket)
Upgrade to a WebSocket for streaming speech-to-text with eager partials (PyAI Hear first partial measured at about 200 ms in-region. It is revisable and not an SLA). Requires the hear:stream scope. The only frame protocol is pyai-hear-v1, the published frame catalog below.
Auth: browsers can't set Authorization on a WebSocket, so send the key as a subprotocol, together with the pyai.v1 marker: Sec-WebSocket-Protocol: pyai.v1, pyai-key.<API_KEY> (server clients may use ?api_key= instead). Offer both values: PyAI echoes only the marker and never reflects your key, and a 101 can only select a value the client offered. The key is validated and swapped for the internal upstream credential on the upgrade.
Set vocabulary only when the session has known names, brands, products, or other distinctive terms. This biases known vocabulary only when supplied for the session or enabled through the stored hear_stream profile. Request terms take priority and the effective list is fixed when the session opens.
Knowledge-base retrieval is off by default for plain transcription. Set grounding=true on the connect URL to opt in when the service has retrieval configured; it does not enable an unconfigured service. Omission or grounding=false skips retrieval. Retrieval may add a grounding list to speech_final and delays delivery while the bounded lookup completes. Opting out removes that lookup and its grounding add-on usage, while preserving transcript text, event order, and Hear usage. The option is fixed for the connection; only one lowercase true or false value is accepted. Invalid or repeated values return 400 invalid_grounding before the WebSocket opens.
Client -> server: stream mono little-endian PCM16 binary audio frames continuously at sample_rate=8000 or sample_rate=16000 (default). The service converts 8 kHz input before recognition and endpointing. Other formats are rejected with 400 unsupported_audio_format before the WebSocket opens. Send a JSON {"type":"commit"} text frame to force-finalize the current utterance (e.g. when your VAD detects end-of-turn). Completion frames may have empty text when the utterance contains no recognized speech; retain these acknowledgments when draining pending commits. Closing the socket also flushes a final for any buffered audio.
Set endpointing_ms on the connect URL or send {"type":"config","endpointing_ms":800} mid-session. It is the minimum trailing-pause length before an utterance may end. Turn detection may wait longer; an utterance always completes within max(endpointing_ms, 1500 ms) of streamed silence. Keep sending audio frames, including silence, while the caller pauses. Pausing the stream pauses the frame-driven clock.
Server -> client (JSON text frames):
| type | When | Payload |
|---|---|---|
| config_ack | after connect-time or mid-session endpointing config | {endpointing_ms, effective_floor_ms, effective_ceiling_ms, score_interval_ms, warnings}, the applied values and any validation warnings |
| partial | every eager tick | {text, stable_text, active_text, utterance_id, t_ms}, the live hypothesis for that utterance_id |
| partial_stable | when a prefix locks in | {text, utterance_id, t_ms}, the portion the recognizer no longer expects to revise |
| speech_final | on endpoint/commit | {text, utterance_id, t_ms, audio_ms, endpoint_reason}, stable, end of an utterance |
| final | follows speech_final | {text, utterance_id, t_ms, audio_ms, endpoint_reason}, corrected full-context transcript |
| usage | just before a graceful close | {product, meter, audio_seconds, minutes}, the session's billed active-audio, so you can reconcile realtime spend in-band (a realtime WS carries no x-pyai-units response header). Best-effort; absent if the session had no billable audio or closed abnormally. |
| error | on fault | {code, message} |
t_ms is the audio-timeline position of the hypothesis; audio_ms is the utterance's active-speech length (the billed signal); utterance_id groups partials/finals for one utterance. Assert that config_ack.warnings is empty instead of inferring that a setting applied. Warning reasons include clamped_to_range, not_a_number, and unknown_config_field; invalid config leaves the session open. endpoint_reason reports why automatic endpointing fired, including peak_te_early for the high-confidence path and silence_backstop for the bounded fallback. Log it when tuning conversational turns. Cue grounding configuration frames remain reserved and cannot change the connect-time grounding option.
If recognition loses its connection or cannot deliver or finish recognizing audio after caller PCM was accepted, the server emits error with code: recognition_interrupted and closes with 1011. Treat that session as incomplete and start a new stream; a reconnect does not recover missing audio. Internal connection retries are limited to before caller audio is accepted. No further transcript results or automatic Recap are produced from the interrupted session.
Close codes: 1000 normal · 1008 auth/policy (bad key, scope, revoked token) · 1011 engine error · 4429 over concurrency cap.
Billing: metered active audio at the Hear rate ($0.001/min), speech time derived from transcript timing rather than connection wall-clock.
Request
- Base URL: https://api.pyai.com
- URL: https://api.pyai.com/v1/audio/transcriptions/stream
- Auth: one of:
- HTTP bearer
- API key in header x-api-key
Query parameters
Canonical frame protocol. Omit to use the same pyai-hear-v1 default.
Streaming STT model.
Knowledge-base retrieval and grounding add-on usage are off by default. Set true to opt in when the service has retrieval enabled; omission or false skips retrieval. Fixed at connection start; send a single lowercase true or false. Invalid or repeated values return 400 invalid_grounding before upgrade. Transcript text, event order, and Hear usage are unchanged.
Omit or use auto to detect the spoken language automatically. An explicit supported language code pins recognition for the session. Unsupported explicit values are rejected at the upgrade with 400 unsupported_language.
Input PCM sample rate in Hz. Send 8000 for 8 kHz telephony PCM; the service converts it before recognition and endpointing.
Mono little-endian 16-bit PCM. Compressed audio is not supported on this stream.
Emit eager partial hypotheses.
Tri-state inverse-text normalization for English final transcripts (never interim partials). true renders spoken numbers as digits (phones, currency, dates, ordinals). false keeps those spans in spoken form. Omitted keeps the live engine default: number formatting is ON for finals. Independent of smart_format.
Opt-in English punctuation and sentence capitalization on final transcripts only. Interim partials are never formatted. May change only case and punctuation; any failure returns the unformatted final. Default false. Independent of numerals. Non-English requests are unchanged.
Opt-in spoken punctuation commands on English final transcripts only: period, comma, new paragraph, and question mark. Separate from smart_format and off by default. Interim partials are never rewritten.
Opt-in stripping of filled pauses (um, uh, umm, uhh, er) on English final transcripts. Off by default. Do not enable on legal or compliance audio by default. Interim partials are never rewritten.
Optional per-session terms for known names, brands, products, and other distinctive phrases. PyAI trims entries, removes case-insensitive duplicates, and keeps the first spelling and order. Entries shorter than 4 characters, longer than 64 characters, longer than 5 words, or made only of common words are ignored. At most 5 effective terms are used. When stored vocabulary is enabled for hear_stream, request terms come first and stored suggestions fill any remaining slots. The effective list is fixed for the session and does not select transcription language. Send a comma-separated list or a JSON array string.
Optional determinism seed for reproducible eval runs. Forwarded to the engine and honored once the engine supports it; no effect when omitted.
Optional sampling temperature for reproducible eval runs. Forwarded to the engine and honored once the engine supports it; no effect when omitted.
Minimum trailing-pause length in milliseconds (50-5000, clamped) before an utterance may end. Turn detection may wait longer; an utterance always completes within max(endpointing_ms, 1500 ms) of streamed silence. The same setting can be changed mid-session with {"type":"config","endpointing_ms":800}. Both paths emit config_ack; assert that warnings is empty. A non-empty warnings means a value was not applied verbatim, but the session remains open. Timing counts only audio you stream, including silence, so pausing the stream pauses the clock. {"type":"commit"} still forces immediate end-of-turn on the STT socket.
Stable call identifier used for the post-call Recap when Recap is enabled. Omit to use the stream session id.
Optional Recap pack for the post-call record.
Optional call direction attached to the post-call Recap.
Changes
No recorded changes to this operation across all 1 revision of this API.