v1 modals

Unified Contextual Memory Search (Text + Multimodal)

Retrieves relevant memories based on text conversation context and/or multimodal inputs.

You can provide **text messages**, **images**, **video**, **audio**, or any combination.
The system finds memories semantically relevant to the provided inputs.

**Universal Base Params:**
- user_id (str, required): Restrict search to this user
- project_id (str, required): Restrict search to this project
- persona_id (str, optional): Use persona's context if provided

**Input — at least one required:**
- messages (array, optional): Text conversation context
- video_base64 (str, optional): Base64 encoded video
- image_base64 (str, optional): Base64 encoded image
- audio_base64 (str, optional): Base64 encoded audio

**Search Config:**
- include_modalities (array, optional): Filter results by type: ["text", "image", "video"]

**Response:**
- new_prompt (str): Enhanced prompt with retrieved memory context
- raw_results (dict): Structured memory data from retrieval
- entity_images (dict, optional): Reference images for matched entities
- success (bool): True if query succeeded

Returns 200 OK with memory data. Requires authentication.

post/v1/modal/query

Request body

user_idstring required

Restrict search to this user (required)

project_idstring nullable

Restrict search to this project (required)

persona_idstring nullable

Optional persona ID. If provided, query uses persona's context

text_inputstring nullable

Text input to search against. The system finds memories relevant to this text.

video_base64string nullable

Base64 encoded video content

image_base64string nullable

Base64 encoded image content

audio_base64string nullable

Base64 encoded audio content (supports webm, wav, mp3, mp4, and other formats)

include_modalitiesstring[] nullable

Filter results by type: ['text', 'image', 'video']

session_idstring nullable

Optional session identifier for conversation context

Example request

{
  "include_modalities": [
    "text",
    "video"
  ],
  "messages": [
    {
      "content": "What did we decide about the animation speed last time?",
      "role": "user"
    }
  ],
  "project_id": "proj_ABC",
  "user_id": "user_123"
}

Response

Successful Response

new_promptstring required

Enhanced prompt with retrieved memory context

raw_resultsobject

Raw results from the retrieval process

entity_imagesobject nullable

Reference images for matched entities (entity_name -> base64 image)

successboolean

Whether the query was processed successfully

Example response

{
  "new_prompt": "User preferences: likes morning coffee...",
  "raw_results": {
    "episodic": [],
    "identity": [],
    "preferences": []
  },
  "success": true
}

Changes

Changed in 6 of the 29 revisions of this API.2222

Feb 9, 2026bb6f89ac9cd3128See the full diff
- ▲
  the response property raw_results became optional for the status 200
  response-property-became-optional
- ●
  removed the request property filter_memory_types
  request-property-removed
- ●
  removed the request property question
  request-property-removed
- ○
  api tag v1 modals added
  api-tag-added
- ○
  api tag v1 modal removed
  api-tag-removed
- ○
  added the new optional request property audio_base64
  new-optional-request-property
- ○
  added the new optional request property image_base64
  new-optional-request-property
- ○
  added the new optional request property include_modalities
  new-optional-request-property
- ○
  added the new optional request property text_input
  new-optional-request-property
- ○
  added the new optional request property video_base64
  new-optional-request-property
- ○
  added the optional property entity_images to the response with the 200 status
  response-optional-property-added
This revision also has 20 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog
Jan 21, 20261660d0b2cbd92See the full diff
- ○
  added the new optional request property persona_id
  new-optional-request-property
- ○
  added the new optional request property project_id
  new-optional-request-property
Dec 10, 20255a9a6b9cb07a10See the full diff
- ○
  added the non-success response with the status 404
  response-non-success-status-added
- ○
  added the optional property details to the response with the 400 status
  response-optional-property-added
- ○
  added the optional property details to the response with the 401 status
  response-optional-property-added
- ○
  added the optional property details to the response with the 500 status
  response-optional-property-added
- ○
  added the optional property request_id to the response with the 400 status
  response-optional-property-added
- ○
  added the optional property request_id to the response with the 401 status
  response-optional-property-added
- ○
  added the optional property request_id to the response with the 500 status
  response-optional-property-added
- ○
  added the required property error_code to the response with the 400 status
  response-required-property-added
- ○
  added the required property error_code to the response with the 401 status
  response-required-property-added
- ○
  added the required property error_code to the response with the 500 status
  response-required-property-added
This revision also has 7 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog
Nov 17, 2025826a61027c591See the full diff
- ○
  endpoint added
  endpoint-added
This revision also has 6 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog
Nov 17, 202593aa65273d4d1See the full diff
- ▲
  api path removed without deprecation
  api-path-removed-without-deprecation
This revision also has 5 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog
Nov 5, 202583e4b50f22361See the full diff
- ○
  endpoint added
  endpoint-added
This revision also has 16 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

Full history