---
title: "Rtvi Connect"
method: POST
path: "/connect"
tags: ["app"]
---

# Rtvi Connect

`POST /connect`

## Request

- Base URL: `https://realtime-api.convai.com` (the document declares no server; this is the origin it was published from)
- URL: `https://realtime-api.convai.com/connect`
- Auth: none declared

## Headers

- `X-API-Key` string, nullable
- `API-AUTH-TOKEN` string, nullable

## Request body

- ConnectRequest
  - `character_id` string, nullable: Single-character UUID, optionally followed by -draft, -latest, or a major.minor[.patch] version. Roster entries currently accept bare UUIDs only.
  - `characters` ConnectCharacterRequest[], nullable
    - `character_id` string, uuid, required
    - `character_session_id` union
      - string
      - string, uuid
    - `action_config` unknown
    - `aliases` unknown
  - `connection_type` 'audio' | 'video' | 'text'
  - `llm_provider` 'gemini-live' | 'gemini-live-beta' | 'gemini-baml' | 'dynamic'
  - `blendshape_provider` 'not_provided' | 'ovr' | 'neurosync'
  - `blendshape_config` NeurosyncConfig: Configuration for NeuroSync blendshape provider.
    - `provider` 'neurosync'
    - `enable_chunking` boolean
    - `chunk_size` integer
    - `deliver_chunks_ahead` boolean
    - `output_fps` integer
    - `max_ahead_chunk_lead_seconds` number
    - `frames_buffer_duration` number
    - `format` 'mha' | 'arkit' | 'cc4_extended' | 'cc5_hd' | 'visemes'
  - `emotion_config` union
    - NRCLexEmotionConfig: Configuration for NRCLex emotion detection provider.
      - `provider` 'nrclex'
      - `min_word_threshold` integer: Minimum words required for analysis (used when use_sentence_detection=False)
      - `low_intensity_threshold` number: Threshold for low intensity emotions (0-1 scale)
      - `high_intensity_threshold` number: Threshold for high intensity emotions (0-1 scale)
      - `use_sentence_detection` boolean: Use sentence detection instead of word count threshold
      - `min_sentence_threshold` integer: Minimum complete sentences required for analysis (used when use_sentence_detection=True)
    - LLMEmotionConfig: Configuration for LLM emotion detection provider.
      - `provider` 'llm'
  - `state_of_mind` string, nullable
  - `dynamic_info` DynamicInfo
    - `text` string
    - `keep_in_context` boolean: Whether to keep this dynamic info static in the context for the session.
  - `turn_detection_config` TurnDetectionConfig
    - `type` 'smart_turn' | 'not_provided'
    - `params` SmartTurnParams: Configuration parameters for smart turn analysis. Parameters: stop_secs: Maximum silence duration in seconds before ending turn. pre_speech_ms: Milliseconds of audio to include before speech starts. max_duration_secs: Maximum duration in seconds for audio segments.
      - `stop_secs` number
      - `pre_speech_ms` number
      - `max_duration_secs` number
  - `narrative_template_keys` object, nullable
  - `video_track_name` string
  - `vision_prompt_mode` 'auto' | 'disabled': Controls Core-owned vision instructions and visual-input tooling for this session. 'auto' preserves legacy runtime behavior; 'disabled' omits the vision prompt and suppresses visual input while retaining scene metadata and other text context.
  - `vision_input_config` VisionInputConfig: Vision context injection settings for discrete multimodal LLMs. Frames are sampled from the video channel, buffered silently, and attached to the next LLM-triggering turn. max_resolution=None uses the provider-aware default (Gemini 384px for the single-tile token rate, others 768px).
    - `enabled` boolean
    - `sample_interval_secs` number
    - `frames_per_turn` integer
    - `buffer_frames` integer, nullable
    - `sampling_windows` VisionSamplingWindowConfig[], nullable
      - `count` integer
      - `interval_ms` integer
    - `staleness_seconds` number
    - `max_resolution` integer, nullable
    - `replace_previous_vision_context` boolean
  - `respond_modes` RespondModesConfig: Connect-time respond-mode defaults by modality. User text and audio keep the product floor from the design doc: they may only be set to must_respond.
    - `text` 'must_respond' | 'auto' | 'silent', nullable
    - `audio` 'must_respond' | 'auto' | 'silent', nullable
    - `vision` 'must_respond' | 'auto' | 'silent', nullable
    - `context_update` 'must_respond' | 'auto' | 'silent', nullable
    - `trigger` 'must_respond' | 'auto' | 'silent', nullable
    - `scene_metadata` 'must_respond' | 'auto' | 'silent', nullable
  - `thinking_mode` boolean
  - `action_config` ActionConfig
    - `actions` Action[], nullable
      - `value` string, required
    - `characters` ActionCharacterConfig[], nullable
      - `name` string, required
      - `bio` string, required
    - `objects` ActionObjectConfig[], nullable
      - `name` string, required
      - `description` string, required
    - `current_attention_object` ActionObjectConfig
      - `name` string, required
      - `description` string, required
    - `tools` ClientToolDeclaration[], nullable
      - `name` string, required
      - `description` string, required
      - `inputSchema` object, required
  - `capabilities` ActionProtocolCapabilities: Client-requested or server-selected realtime protocol capabilities. ``model_output_version`` remains absent for legacy clients so the serialized response shape is compatible with clients that use strict capability decoders.
    - `action_protocol_version` integer
    - `model_output_version` integer, nullable
  - `scene_description` ActionObjectConfig[], nullable
    - `name` string, required
    - `description` string, required
  - `character_session_id` union
    - string
    - string, uuid
  - `vad_params` VADParams
    - `confidence` number
    - `start_secs` number
    - `stop_secs` number
    - `min_volume` number
  - `room_name` string, nullable
  - `room_session_id` string, uuid, nullable
  - `room_brief` string, nullable: Moderator's brief for a multi-character room; rendered into every character's static room context
  - `group_chat` boolean: Opt a multi-character room into moderated group chat. Only a room created with this flag receives server-authored room context, roster republishes and directed group-address turns; a roster room without it behaves exactly as multi-character V0 does. Set at create; a join inherits whatever the room was created with.
  - `group_conversation` ModelsGroupConversationGroupConversationConfig1
    - `version` string, required
    - `player` GroupConversationPlayer
      - `display_name` string
    - `scene` GroupConversationScene, required
      - `brief` string, required
      - `topics` GroupConversationTopic[]
        - `id` string, required
        - `text` string, required
        - `starter_character_id` string, uuid, nullable
    - `spatial` GroupConversationSpatial
      - `hearing_range_m` number
      - `pause_hysteresis_m` number
      - `address_radius_m` number
      - `group_radius_m` number
      - `gaze_dwell_threshold` number
    - `timing` GroupConversationTiming
      - `lull_seconds` number
      - `player_reply_wait_seconds` number
      - `reply_to_question_window_seconds` number
      - `empty_transcript_timeout_seconds` number
      - `greeting_reentry_seconds` number
      - `line_timeout_seconds` number
    - `debug_options` GroupConversationDebugOptions: Debug-only knobs, honoured only when ``ConnectRequest.debug`` is true. Not part of the Unity contract.
      - `bid_policy` 'auto' | 'llm' | 'rules' | 'silent'
  - `workspace_id` string, uuid, nullable: Shared Workspace whose characters fill a multi-character room created by a member with their own API key. Core authorizes every seat with a Workspace preview grant issued for that member, runs the room on the Workspace runtime credential and records the member as its actor. The member is the speaker, so end_user_id and end_user_metadata must be absent. Only valid with characters and mode='create'.
  - `mode` 'create' | 'join'
  - `spawn_agent` boolean
  - `shared_session_key` string, nullable: Client-provided grouping key used to deterministically place participants into the same shared session/room.
  - `connect_attempt_id` string, nullable: Client-generated idempotency key for one logical connection attempt. Retries with the same key and payload replay the original response.
  - `max_num_participants` integer
  - `end_user_id` string, nullable
  - `end_user_metadata` object, nullable
  - `default_tts_enabled` boolean
  - `default_stt_enabled` boolean
  - `disable_live_fallback` boolean
  - `stt_provider` string, nullable
  - `debug` boolean
  - `debug_row_cap` integer, nullable
  - `use_response_lifecycle_contract` boolean, nullable
  - `preserve_links_in_output` boolean, nullable
  - `invocation_metadata` InvocationMetadata
    - `source` 'not_provided' | 'ue_sdk' | 'unity_sdk' | 'web_sdk' | 'internal_testing' | 'published_chat'
    - `client_version` string, nullable
    - `extra_metadata` object, nullable
  - `audio_config` ModelsApiAudioConfig1: Configuration for audio behavior.
    - `output` OutputAudioConfig: Configuration for audio output behavior (LiveKit-specific).
      - `add_wav_header` boolean: Whether to add WAV header to audio output
      - `sample_rate` integer, nullable: Audio sample rate (None for auto-detect)
      - `max_chunk_duration_ms` integer: Maximum duration of audio chunks in milliseconds (default: 100ms)
      - `audio_routing` 'audio_only' | 'data_only' | 'both': Audio routing mode for LiveKit audio output configuration.

## Response `200`

Successful Response

- ConnectResponse: Response model for /connect endpoint
  - `session_id` string, required: Unique session token
  - `request_trace_id` string, required: Server-side request trace ID for correlating client events, logs, telemetry, and session records for this /connect request
  - `character_session_id` string, required: Interaction session id
  - `room_url` string, required: URL of the room to connect to
  - `room_name` string, nullable: Room name (only for LiveKit transport)
  - `token` string, required: Authentication token for the room
  - `end_user_id` string, nullable: The end_user_id associated with this session when provided
  - `end_user_metadata` object, nullable: Metadata associated with the end user
  - `memory_status` string, nullable: What long-term memory will do for this session: 'disabled', 'active', or 'skipped_no_end_user' when the character has memory enabled but no end_user_id was supplied, so nothing will be remembered. Reported positively rather than left to be inferred from an absent end_user_id.
  - `room_session_id` string, uuid, nullable
  - `active_membership_id` string, uuid, nullable
  - `route_epoch` integer, nullable
  - `roster_epoch` integer, nullable
  - `partial_dispatch` boolean
  - `characters` ConnectCharacterResponse[], nullable
    - `membership_id` string, uuid, required
    - `character_id` string, uuid, required
    - `session_id` string, required
    - `character_session_id` string, required
    - `participant_identity` string, required
    - `is_initial` boolean, required
    - `provisioning_status` 'dispatch_queued' | 'dispatch_accepted' | 'dispatch_failed', required
    - `failure_code` string, nullable
    - `display_name` string, nullable: Character display name for roster UIs; None when unknown
    - `description` string, nullable: Character description; None when unknown
  - `capabilities` ActionProtocolCapabilities: Client-requested or server-selected realtime protocol capabilities. ``model_output_version`` remains absent for legacy clients so the serialized response shape is compatible with clients that use strict capability decoders.
    - `action_protocol_version` integer
    - `model_output_version` integer, nullable

## Other responses

- `422`: Validation Error

## Changes

- **2026-10-23** `4361c107ef49`: 3 info
  - added the new optional request property `characters/anyOf[subschema #1]/items/action_config`
  - added the new optional request property `characters/anyOf[subschema #1]/items/aliases`
  - added the new optional request property `group_conversation`
- **2026-10-01** `e755e2e1e546`: 1 info
  - added the new optional request property `workspace_id`
- **2026-09-22** `7c9b85cfea11`: 1 info
  - added the optional property `memory_status` to the response with the `200` status
- **2026-09-17** `3c8632f2dc7b`: 1 info
  - added the new optional request property `preserve_links_in_output`
- **2026-09-09** `00d82882f86f`: 1 info
  - added the new `published_chat` enum value to the request property `invocation_metadata/anyOf[subschema #1: InvocationMetadata]/source`

[Operation history](https://skmtc.dev/convai/apis/convai-api/changes/connect/post.md)

---

[API](https://skmtc.dev/convai/apis/convai-api.md) - [All operations](https://skmtc.dev/convai/apis/convai-api/llms.txt) - [OpenAPI document](https://skmtc.dev/convai/apis/convai-api/revisions/4361c107ef49?raw)