Sign in
v1 audio

Generate Audio

Dedicated audio generation endpoint using the Universal Schema with flat parameters.

Supports three audio types:
- **speech**: Multi-speaker TTS (auto-detects characters, designs unique voices per entity)
- **music**: AI-generated music
- **sfx**: AI-generated sound effects

**Universal Base Schema:**
- user_id (str, required): The end-user ID
- project_id (str, required): The project ID
- persona_id (str, optional): The specific system persona/voice to use
- disabled_learning (bool, optional): If true, request is ignored by long-term memory

**Input:**
- text_input (str, required): Text to speak or audio prompt
- context (str, optional): Additional context

**Audio Params (Flat):**
- model (str, required): Model ID (e.g., eleven-turbo)
- voice (str, required): Voice ID for TTS
- audio_type (str, optional): 'speech', 'music', or 'sfx'
- speed (float, optional): Playback speed (0.5-2.0)
- duration (float, optional): Max duration in seconds
- seed (int, optional): Random seed for reproducibility

**Reference inputs:**
- image_base64 (str, optional): Base64 encoded reference image for context
- video_base64 (str, optional): Base64 encoded reference video for context
- audio_base64 (str, optional): Base64 encoded reference audio for context

**Authentication**: Requires valid API key or JWT token

Note: Reasoning is not currently supported for audio generation.
post/v1/audio/generations

Request body

user_idstring required

The end-user ID

project_idstring nullable

The project ID

persona_idstring nullable

The specific system persona/voice to use

disabled_learningboolean

If true, this request is ignored by long-term memory

text_inputstring required

The prompt/description for audio generation

session_idstring nullable

Session ID for conversation context

use_reasoningboolean

Enable Chain-of-Thought/Reasoning steps before answering

max_reasoning_iterationsinteger

Max reasoning steps if reasoning is enabled

image_base64string nullable

Base64 encoded reference image for context

video_base64string nullable

Base64 encoded reference video for context

audio_base64string nullable

Base64 encoded reference audio for context

modelstring

Audio generation model: 'lyria-2' (Google Lyria 2 on Vertex AI, default for music — 30s 48kHz WAV), 'audiocraft' (MusicGen/AudioGen on Cloud Run), or 'eleven-turbo' (ElevenLabs TTS for speech)

voicestring

Voice ID for TTS (alloy, echo, fable, onyx, nova, shimmer)

audio_type'speech' | 'sfx' | 'music'

Audio type: 'speech', 'sfx', or 'music'

speednumber

Playback speed (0.5-2.0), only for speech

durationnumber nullable

Max duration in seconds for music/sfx (Lyria 2 always generates 30s)

seedinteger nullable

Random seed for deterministic generation (Lyria 2 only, cannot be combined with sample_count)

Example request

{
  "audio_type": "speech",
  "model": "eleven-turbo",
  "project_id": "proj_ABC",
  "speed": 1,
  "text_input": "Hello world, this is a test.",
  "user_id": "user_123",
  "voice": "rachel"
}

Response

Successful Response

audio_base64string nullable

Base64 encoded audio content. Present when the payload is under ~30 MB. May be absent for very large outputs.

audio_urlstring nullable

Signed GCS URL to download the audio (expires after 24 h). Always present when the upload succeeds.

output_typestring

Delivery method: 'both' (base64 + url), 'url' (url only, base64 omitted due to size), or 'base64' (GCS upload failed).

audio_formatstring required

Audio format, e.g. mp3, wav

duration_secondsnumber nullable

Duration of generated audio in seconds

audio_typestring required

Type of audio generated

successboolean

Whether the request succeeded

Changes

Changed in 4 of the 29 revisions of this API.27

  • 62c9ee5530333See the full diff
    • ○

      added the new optional request property max_reasoning_iterations

      new-optional-request-property

    • ○

      added the new optional request property use_reasoning

      new-optional-request-property

    • ○

      the model request property default value changed from eleven-turbo to lyria-2

      request-property-default-value-changed

  • 8c335444d3041See the full diff
    • ○

      the output_type response's property default value changed from base64 to both for the status 200

      response-property-default-value-changed

  • 9cc8f20f40c822See the full diff
    • ▲

      the response property audio_base64 became optional for the status 200

      response-property-became-optional

    • ▲

      response property audio_base64 list-of-types was widened by adding types null to media type application/json of response 200

      response-property-list-of-types-widened

    • ○

      added the optional property audio_url to the response with the 200 status

      response-optional-property-added

    • ○

      added the optional property output_type to the response with the 200 status

      response-optional-property-added

  • bb6f89ac9cd31See the full diff
    • ○

      endpoint added

      endpoint-added

    This revision also has 20 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog