---
title: "RAG use case"
method: POST
path: "/ai/prediction/rag/{MODEL_ID}"
tags: ["Get predictions"]
---

# RAG use case

`POST /ai/prediction/rag/{MODEL_ID}`

The rag use case uses candidate documents that are inserted into a LLM’s context to ground the generated response to those documents instead of generating an answer from details stored in the LLM’s trained weights. This type of search adds guardrails so the LLM can search private data collections.

The RAG search can perform queries against external documents passed in as part of the request.

## Headers

- `Authorization` string, required
- `Content-Type` string

## Request body

- RagExtDocRequest
  - `batch` object[]
    - `text` string — Content for the model to analyze. Multiple instances of text can be sent in the request.
    - `documents` Document[]
      - `body` string — The contents of the document.
      - `source` string — The URL that identifies the source of the document.
      - `title` string — The title of the document.
      - `date` string, date-time — The date and time the document was created, displayed in the required ISO-8601 format of `yyyy-mm-ddThh:mm:ssZ`.
  - `useCaseConfig` UseCaseConfigRagExtDoc
    - `memoryUuid` string — The universal unique identifier (UUID) stored in the trained set of data in the model that is used in the model request. This parameter is optional, and is used when previous chat history reference information is available.
    - `answerNotFoundMessage` string — This parameter is optional, and can be passed to change the response if the LLM cannot answer the request. The default is "Not possible to answer given this content."
  - `modelConfig` ModelConfig — Provides fields and values that specify ranges for tokens.
    - `vectorQuantizationMethod` string — Vector quantization compresses data size, as well as reducing memory usage. The methods are: * `min-max` - Creates tensors of the text and converts it to uint8 by normalizing it to the range [0, 255]. * `max-scale` - Finds the maximum absolute value for the encoded text, normalizes it by scaling the text to a range of -127 to 127, and then returns the quantized text as an 8-bit integer tensor.
    - `dimReductionSize` integer — Used to reduce vector size while maintaining good quality. This field allows any integer above 0, but less than or equal to the vector dimension of the model. If you send a vector dimension larger than the model, a 400 Bad Request error is returned. Not every model is designed to support this parameter. In this scenario, a warning message is generated that indicates quality can decrease.

## Response `200`

OK

- RagExtDocResponse
  - `predictions` RagExtDocResponseTokens[]
    - `response` string — The unparsed response returned from the request.
    - `tokensUsed` Token
      - `promptTokens` integer — The number of tokens generated to prompt the model to continue generating results.
      - `completionTokens` integer — The number of tokens used until the model completes.
      - `totalTokens` integer — The sum of the prompt and completion tokens used in the model.
    - `answer` string — The parsed response text from the document.
    - `sources` string[] — One or more URLs that identify the source of the document returned in the response.
    - `memoryUuid` string — The universal unique identifier (UUID) stored in the trained set of data in the model that is used in the model request. This parameter is optional, and is used when previous chat history reference information is available.
    - `answerNotFoundMessage` string — This parameter is optional, and can be passed to change the response if the LLM cannot answer the request. The default is "Not possible to answer given this content."
    - `answerFound` boolean — This parameter is false if the value in the answerNotFoundMessage field is used in the response. If an answer is returned for the request, this parameter is true.

---

[API](https://skmtc.dev/lucidworks/apis/rules-rewrites-api.md) · [All operations](https://skmtc.dev/lucidworks/apis/rules-rewrites-api/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/lucidworks/rules-rewrites-api/revisions/f2d3747848e8/schema)