Cache

Create Cache

Create a persistent cache for low-latency completions.

Args: name: Unique identifier for the cache. model: The model name to use when generating completions. gguf_file: Path to the gguf weights file to load. filters: Optional metadata filters used to select documents. docs: Explicit list of document IDs to include in the cache. auth: Authentication context used for permission checks.

Returns: A dictionary describing the created cache.

post/cache/create

Query parameters

namestring required
modelstring required
gguf_filestring required

Headers

authorizationstring

Request body

filtersobject nullable
docsstring[] nullable

Response

Successful Response

object required

Changes