embed > v2

Create sync embeddings

Changed on

This method synchronously creates embeddings for multimodal content and returns the results immediately in the response.

Use this method to embed a query for retrieving matching content. With Marengo 3.5, audio and video can be up to 30 seconds. With Marengo 3.0, they can be up to 10 minutes. For longer content, use the POST method of the /embed-v2/tasks endpoint instead.

The content this method accepts depends on the model. With Marengo 3.5, this method accepts only the multi_input input type; provide text, images, audio, video, or documents as media sources. With Marengo 3.0, use the individual input types. For the formats, resolutions, file sizes, and duration limits each model accepts, see the input requirements for Marengo 3.5 or Marengo 3.0.

<Note title="Note"> This method is rate-limited. With Marengo 3.5, the platform counts input tokens for each type of content. A request can exceed a limit before you see an error. For details, see [Input token limits for embedding](/v1.3/docs/get-started/plans-and-billing/rate-limits/new-billing#input-token-limits-for-embedding). </Note>
post/embed-v2

Request

  • Base URL: https://api.twelvelabs.io/v1.3
  • URL: https://api.twelvelabs.io/v1.3/embed-v2
  • Auth: API key in header x-api-key

Request body

input_type'text' | 'image' | 'text_image' | 'audio' | 'video' | 'multi_input' required

The type of content for the embeddings.

Values:

  • multi_input: Text and up to 10 media sources, combined into a single embedding. To reference a specific media source from your text, use a placeholder in the following format: <@name>, where name matches the name field of a media source. Marengo 3.5 accepts images, video, audio, and documents as media sources. Marengo 3.0 accepts images.
  • audio: An audio file. Requires Marengo 3.0.
  • video: A video file. Requires Marengo 3.0.
  • image: An image file. Requires Marengo 3.0.
  • text: Text input. Requires Marengo 3.0.
  • text_image: Text and an image. Requires Marengo 3.0.
model_name'marengo3.0' | 'marengo3.5' required

The embedding model to use.

Values:

  • marengo3.5: For details about this version, see the Marengo 3.5 page.
  • marengo3.0: For details about this version, see the Marengo 3.0 page.
auto_truncateboolean

Controls the behavior of the platform when the text in your request exceeds 2,000 tokens. Requires Marengo 3.5.

Values:

  • false: The platform returns a 400 error.
  • true: Truncate your text to fit the limit, and set the usage.truncated field to true in the response.
embedding_uncertaintyboolean

Set this parameter to true to include a per-dimension uncertainty vector in the data[].embedding_uncertainty field of the response. The vector has the same length as the embedding array. A higher value indicates lower confidence in that dimension. Requires Marengo 3.5.

Requirements:

  • Set this parameter to true only for a text-only or media-only request. If you combine text with media sources, the platform returns a 400 error.
  • The platform returns a 400 error if your request includes a document, whether PDF, plain text, or Markdown.
embedding_dimension'128' | '256' | '512'

The number of dimensions for each embedding in the response, including the data[].embedding_uncertainty vector.

Marengo 3.5 produces Matryoshka embeddings: a shorter embedding consists of the first values of the full-length embedding. A 256-dimension embedding, for example, is the first 256 values of a 512-dimension embedding of the same content. Shorter embeddings reduce index size and speed up similarity search; longer embeddings produce higher retrieval quality.

Requirements:

  • Requires Marengo 3.5. Setting this parameter with model_name: marengo3.0 returns a 400 error.
  • Applies to the entire request: you cannot set it for a single input type or embedding.
  • Use the same value across an index.

Default: 512

Response

The embeddings have been successfully created.

Changes

  • 69c273061729Revision changes
    • breaking

      added subschema #3 subschema #4 subschema #5 to the response property oneOf list for the response status

    • warning

      added the new chunk enum value to the /// response property for the response status

    • info

      added the new optional request property

    • info

      added the new document enum value to the request property ///

    • info

      added the non-success response with the status

    • info

      added the optional property // to the response with the status

    • info

      added the optional property // to the response with the status

    • info

      added the optional property / to the response with the status

    • info

      removed subschema #3 subschema #4 subschema #5 from the response property oneOf list for the response status

  • b8591e05c0baRevision changes
    • warning

      deleted the header request parameter x-api-key

    • info

      the endpoint scheme security x-api-key was added to the API