---
title: "Passthrough use case"
method: POST
path: "/ai/prediction/passthrough/{MODEL_ID}"
tags: ["Get predictions"]
---

# Passthrough use case

`POST /ai/prediction/passthrough/{MODEL_ID}`

The passthrough use case lets you use the service as a proxy to the large language model (LLM). The service sends text (no additional prompts or other information) to the LLM and returns a response.

## Headers

- `Authorization` string, required
- `Content-Type` string

## Request body

- PassthroughRequest
  - `batch` object[] — The content sent in the request.
    - `text` string — The content sent in the request.
  - `useCaseConfig` UseCaseConfigPassthrough — NOTE: If both `useSystemPrompt` and `dataType` are present, the value in `dataType` is used.
    - `useSystemPrompt` boolean — This optional parameter contains a default value of true. If set to false, the `batch.text` value serves as the prompt for the model. The prompt must be in a specific format the model can comprehend.
    - `dataType` string — This optional parameter enables model-specific handling in the API to help improve model accuracy. Use the most applicable fields based on available dataTypes and the dataType value that best aligns with the text sent to the API. The values for `dataType` in the Passthrough use case are: * `"dataType": "text"` - This value is equivalent to `"useSystemPrompt": true` and is a pre-defined, generic prompt. * `"dataType": "raw_prompt"` - This value is equivalent to `"useSystemPrompt": false` and is passed directly to the model or third-party API. * `"dataType": "json_prompt"` - This value follows the generics that allow three roles: - `system` - `user` - Only the last user message is truncated. - If the API does not support system prompts, the user role is substituted for the system role. - `assistant` - If the last message role is `assistant`, it is used as a pre-fill for generation and is the first generated token the model uses. The pre-fill is prepended to the model output, which makes models less verbose and helps enforce specific outputs such as YAML. - The Google Vertex AI does not support generation pre-fills, so an exception error is generated. - This follows the HuggingFace template contraints at https://huggingface.co/docs/transformers/main/en/chat_templating. - Additional `json_prompt` information: - Consecutive messages for the same role are merged. - You can paste the information for a hosted model into the `json_prompt` value and change the model name in the stage.
  - `modelConfig` ModelConfig — Provides fields and values that specify ranges for tokens.
    - `vectorQuantizationMethod` string — Vector quantization compresses data size, as well as reducing memory usage. The methods are: * `min-max` - Creates tensors of the text and converts it to uint8 by normalizing it to the range [0, 255]. * `max-scale` - Finds the maximum absolute value for the encoded text, normalizes it by scaling the text to a range of -127 to 127, and then returns the quantized text as an 8-bit integer tensor.
    - `dimReductionSize` integer — Used to reduce vector size while maintaining good quality. This field allows any integer above 0, but less than or equal to the vector dimension of the model. If you send a vector dimension larger than the model, a 400 Bad Request error is returned. Not every model is designed to support this parameter. In this scenario, a warning message is generated that indicates quality can decrease.

## Response `200`

OK

- PassthroughResponse
  - `predictions` PassthroughResponseTokens[]
    - `response` string — The results returned from the request.
    - `tokensUsed` Token
      - `promptTokens` integer — The number of tokens generated to prompt the model to continue generating results.
      - `completionTokens` integer — The number of tokens used until the model completes.
      - `totalTokens` integer — The sum of the prompt and completion tokens used in the model.

---

[API](https://skmtc.dev/lucidworks/apis/rules-rewrites-api.md) · [All operations](https://skmtc.dev/lucidworks/apis/rules-rewrites-api/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/lucidworks/rules-rewrites-api/revisions/f2d3747848e8/schema)