---
title: "Unified Contextual Memory Search (Text + Multimodal)"
method: POST
path: "/v1/modal/query"
tags: ["v1 modals"]
---

# Unified Contextual Memory Search (Text + Multimodal)

`POST /v1/modal/query`

Retrieves relevant memories based on text conversation context and/or multimodal inputs.
    
    You can provide **text messages**, **images**, **video**, **audio**, or any combination.
    The system finds memories semantically relevant to the provided inputs.
    
    **Universal Base Params:**
    - user_id (str, required): Restrict search to this user
    - project_id (str, required): Restrict search to this project
    - persona_id (str, optional): Use persona's context if provided
    
    **Input — at least one required:**
    - messages (array, optional): Text conversation context
    - video_base64 (str, optional): Base64 encoded video
    - image_base64 (str, optional): Base64 encoded image
    - audio_base64 (str, optional): Base64 encoded audio
    
    **Search Config:**
    - include_modalities (array, optional): Filter results by type: ["text", "image", "video"]

    **Response:**
    - new_prompt (str): Enhanced prompt with retrieved memory context
    - raw_results (dict): Structured memory data from retrieval
    - entity_images (dict, optional): Reference images for matched entities
    - success (bool): True if query succeeded

    Returns 200 OK with memory data. Requires authentication.

## Request body

- QueryRequest — Unified request model for contextual memory search (text + multimodal)
  - `user_id` string, required — Restrict search to this user (required)
  - `project_id` string, nullable — Restrict search to this project (required)
  - `persona_id` string, nullable — Optional persona ID. If provided, query uses persona's context
  - `text_input` string, nullable — Text input to search against. The system finds memories *relevant* to this text.
  - `video_base64` string, nullable — Base64 encoded video content
  - `image_base64` string, nullable — Base64 encoded image content
  - `audio_base64` string, nullable — Base64 encoded audio content (supports webm, wav, mp3, mp4, and other formats)
  - `include_modalities` string[], nullable — Filter results by type: ['text', 'image', 'video']
  - `session_id` string, nullable — Optional session identifier for conversation context

## Response `200`

Successful Response

- QueryResponse — Unified response model for memory query (text + multimodal)
  - `new_prompt` string, required — Enhanced prompt with retrieved memory context
  - `raw_results` object — Raw results from the retrieval process
  - `entity_images` object, nullable — Reference images for matched entities (entity_name -> base64 image)
  - `success` boolean — Whether the query was processed successfully

## Other responses

- `400` — Bad Request
- `401` — Unauthorized
- `404` — User/Persona/Project Not Found
- `422` — Validation Error
- `500` — Internal Server Error

## Changes

- **2026-02-09** `bb6f89ac9cd3` — 1 breaking, 2 warning, 8 info
  - the response property `raw_results` became optional for the status `200`
  - removed the request property `filter_memory_types`
  - removed the request property `question`
  - api tag `v1 modals` added
  - …7 more
- **2026-01-21** `1660d0b2cbd9` — 2 info
  - added the new optional request property `persona_id`
  - added the new optional request property `project_id`
- **2025-12-10** `5a9a6b9cb07a` — 10 info
  - added the non-success response with the status `404`
  - added the optional property `details` to the response with the `400` status
  - added the optional property `details` to the response with the `401` status
  - added the optional property `details` to the response with the `500` status
  - …6 more
- **2025-11-17** `826a61027c59` — 1 info
  - endpoint added
- **2025-11-17** `93aa65273d4d` — 1 breaking
  - api path removed without deprecation

[Full history](https://skmtc.dev/elicitlabs/apis/elicit-labs-api/changes/v1/modal/query/post.md)

---

[API](https://skmtc.dev/elicitlabs/apis/elicit-labs-api.md) · [All operations](https://skmtc.dev/elicitlabs/apis/elicit-labs-api/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/elicitlabs/elicit-labs-api/revisions/27701839a070/schema)