audio

Stream Speech With Timestamps

Synthesize speech and stream it back together with word-level speech marks, for text highlighting, captions and audio-text synchronization while the audio is still arriving.

The response is a Server-Sent Events stream. Each speech.chunk event carries a Base64-encoded run of audio, the speech marks that became final with it, or both - a chunk may carry only one of the two, and the last chunk of a stream is often marks-only. A terminal speech.done event ends the stream; there is no [DONE] sentinel. Ignore any event type you do not recognize, so that new event types do not break your integration.

Speech-mark times are absolute milliseconds from the start of the synthesis, so concatenate the audio chunks into one stream and apply the marks against that single timeline. Which chunk a mark arrives on is a delivery detail and carries no meaning. Times stay correct for every output_format: changing the codec or sample rate does not change the duration.

Every model this route accepts produces speech marks here. The default simba-3.0 and simba-3.2 stream marks word by word alongside the audio. The legacy simba-english and simba-multilingual models, still selectable on a workspace pinned before API version 2026-09-21, synthesize a sentence at a time, so their marks arrive a sentence at a time with that sentence's audio; from that version on they return 400 model_retired on every synthesis route, and from 2026-11-21 both ids are served by our current models. For Base64-encoded audio and speech marks in one non-streamed JSON response, use POST /v1/audio/speech.

post/v1/audio/stream/with-timestamps

Request

  • Base URL: https://api.speechify.ai
  • URL: https://api.speechify.ai/v1/audio/stream/with-timestamps
  • Auth: HTTP bearer

Headers

Speechify-Versionstring
Accept'audio/mpeg' | 'audio/ogg' | 'audio/aac' | 'audio/pcm'

Selects the audio container/codec carried inside the events when output_format is not set in the request body. The selected media type is echoed on the Speechify-Audio-Content-Type response header, since the response's own Content-Type is text/event-stream.

Request body

inputstring required

Plain text or SSML to be synthesized to speech. Up to 20,000 characters as sent, SSML markup included. Refer to https://docs.speechify.ai/docs/api-limits for the input size limits. SSML sets pauses (<break>), speaking rate (<prosody rate>) and pronunciation (<sub alias>): see https://docs.speechify.ai/build/guides/text-to-speech/ssml for each tag. Emotion styles, pitch, volume and emphasis are applied only by the Simba 1.6 models, until 2026-11-21; simba-3.2 and simba-3.0 accept those tags without applying them.

languagestring

Language of the input. Follow the format of an ISO 639-1 language code and an ISO 3166-1 region code, separated by a hyphen, e.g. en-US. Please refer to the list of the supported languages and recommendations regarding this parameter: https://docs.speechify.ai/docs/language-support.

model'simba-3.0' | 'simba-3.2'

Model used for audio synthesis. Defaults to simba-3.0, which is streaming-native and multilingual: it officially supports English plus de-DE, es-ES, es-MX, fr-FR, it-IT and pt-BR, and routes each request to its English or its multilingual training based on language (falling back to the voice's locale when language is omitted). simba-3.2 is the streaming-native model with the lowest TTFB and richest expressivity, and the recommended Simba 3 model; it is English only, so a non-English voice returns 400.

The legacy Simba 1.6 models simba-english and simba-multilingual are retired from API version 2026-09-21: naming one returns 400 model_retired. Pinning your API version to a date before 2026-09-21 keeps them on their Simba 1.6 training until 2026-11-21; from then both ids are served by our current models on every API version that can still name them. Migrate to simba-3.2 (English) or simba-3.0 before then; call GET /v1/audio/models to see the set your workspace can select today.

output_format'pcm_8000' | 'pcm_16000' | 'pcm_22050' | 'pcm_24000' | 'pcm_44100' | 'pcm_48000' | 'mp3_22050_32' | 'mp3_22050_64' | 'mp3_22050_96' | 'mp3_22050_128' | 'mp3_22050_160' | 'mp3_22050_192' | 'mp3_24000_32' | 'mp3_24000_64' | 'mp3_24000_96' | 'mp3_24000_128' | 'mp3_24000_160' | 'mp3_24000_192' | 'ulaw_8000' | 'ogg_24000' | 'aac_24000'

Audio output format for the streaming endpoint (POST /v1/audio/stream), as a codec_sampleRate_bitrate string. Same as AudioOutputFormat minus the wav_* formats: wav is only available on POST /v1/audio/speech. pcm_* and ulaw_8000 are headerless raw audio; pcm_16000 and ulaw_8000 are the telephony formats Twilio/LiveKit SIP expect. From API version 2026-09-30, pcm_16000 is 16 kHz audio; a workspace pinned to an earlier version keeps receiving 24 kHz audio for pcm_16000, as it always has.

160 kbps is the highest bitrate an mp3 can carry at 22.05 and 24 kHz, so mp3_22050_160 and mp3_24000_160 are the maximum-fidelity mp3 formats; a request for mp3_*_192 is encoded at 160 kbps. The two mp3_*_160 formats are served by the Simba 3 models only.

safety_identifierstring

A stable, opaque identifier for the end user of your application on whose behalf this request is made. Use it when many of your own users share one Speechify workspace, for example behind a gateway, a reseller or a consumer app.

Send a hash of your internal user id (for example a hex or base64 SHA-256), or an opaque id that means nothing outside your system. Never send an email address, a name, a phone number or any other personal data: the field accepts only letters, digits and . _ - : | + / =, so an email address is refused with 400 validation_failed.

The identifier is request metadata, not content. It is recorded with the request even when your workspace has zero data retention, so abuse can be attributed to one end user rather than to your whole workspace. When your workspace has per-end-user limits, it also has its own rate and concurrency allowance inside the workspace's (429 with a message naming the safety_identifier), and a blocked identifier is refused with 403 safety_identifier_blocked while your other end users are unaffected. Omitting it changes nothing. See https://docs.speechify.ai/build/guides/concepts/safety-identifiers.

voice_idstring required

Id of the voice to be used for synthesizing speech. Refer to /v1/voices endpoint for available voices

Response

A Server-Sent Events stream of speech.chunk events followed by one terminal speech.done event. A failure after the stream has started, including a content_policy_violation refusal, is delivered as a speech.error event carrying the standard error envelope, because the status code is already committed.

The transport is text/event-stream: each event is an event:/data: pair whose data is one JSON payload matching the schema below. The payload's type field mirrors the event: name, so the stream is also parseable from data: lines alone. Ignore event types you do not recognize.

Changes

No recorded changes to this operation across all 1 revision of this API.