Execute raw inference with provider+model or custom plugin.

This endpoint provides direct access to inference services without the retriever framework overhead. Supports two modes:

1. Provider + Model: Use standard providers (openai, google, anthropic). gemini is accepted as an alias for google.
2. Custom Plugin: Use your custom inference plugins by inference_name

Supported Providers

• openai: GPT models, embeddings, Whisper transcription
• google (alias gemini): Gemini models, Vertex multimodal embeddings (1408D)
• anthropic: Claude models

Examples

Custom Plugin (by inference_name)

{
    "inference_name": "my_text_embedder_1_0_0",
    "inputs": {"text": "hello world"},
    "parameters": {}
}

Custom Plugin (by feature_uri)

{
    "feature_uri": "mixpeek://my_custom_embedder@1.0.0/embedding",
    "inputs": {"text": "hello world"},
    "parameters": {}
}

Builtin Embedder (by feature_uri)

{
    "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
    "inputs": {"text": "hello world"},
    "parameters": {}
}

Chat Completion

{
    "provider": "openai",
    "model": "gpt-4o-mini",
    "inputs": {"prompts": ["What is AI?"]},
    "parameters": {"temperature": 0.7, "max_tokens": 500}
}

Text Embedding (OpenAI)

{
    "provider": "openai",
    "model": "text-embedding-3-large",
    "inputs": {"text": "machine learning"},
    "parameters": {}
}

Text Embedding (Google Vertex Multimodal - 1408D)

{
    "provider": "google",
    "model": "multimodalembedding",
    "inputs": {"text": "machine learning"},
    "parameters": {}
}

Image Embedding (Google Vertex Multimodal - 1408D)

{
    "provider": "google",
    "model": "multimodalembedding",
    "inputs": {"image_url": "https://example.com/image.jpg"},
    "parameters": {}
}

Image Embedding from Base64

{
    "provider": "google",
    "model": "multimodalembedding",
    "inputs": {"image_base64": "<base64-encoded-image>"},
    "parameters": {}
}

Video Embedding (Google Vertex Multimodal - 1408D)

{
    "provider": "google",
    "model": "multimodalembedding",
    "inputs": {"video_url": "https://example.com/video.mp4"},
    "parameters": {}
}

Video Embedding from Base64

{
    "provider": "google",
    "model": "multimodalembedding",
    "inputs": {"video_base64": "<base64-encoded-video>"},
    "parameters": {}
}

Audio Transcription

{
    "provider": "openai",
    "model": "whisper-1",
    "inputs": {"audio_url": "https://example.com/audio.mp3"},
    "parameters": {}
}

Vision (Multimodal LLM)

{
    "provider": "openai",
    "model": "gpt-4o",
    "inputs": {
        "prompts": ["Describe this image"],
        "image_url": "https://example.com/image.jpg"
    },
    "parameters": {"temperature": 0.5}
}

Args: request: FastAPI request object (populated by middleware) payload: Raw inference request

Returns: Inference response with results and metadata

Raises: 400 Bad Request: Invalid provider, model, or inputs 422 Unprocessable Entity: inputs carries no key the provider call reads (for example prompt instead of prompts) 401 Unauthorized: Missing or invalid API key 429 Too Many Requests: Rate limit exceeded 500 Internal Server Error: Inference execution failed

# Text embedding (provider + model)
{
    "provider": "openai",
    "model": "text-embedding-3-large",
    "inputs": {"text": "machine learning"},
    "parameters": {}
}

# Custom plugin (inference_name)
{
    "inference_name": "my_text_embedder_1_0_0",
    "inputs": {"text": "hello world"},
    "parameters": {}
}

# Audio transcription
{
    "provider": "openai",
    "model": "whisper-1",
    "inputs": {"audio_url": "https://example.com/audio.mp3"},
    "parameters": {}
}

# Vision (multimodal)
{
    "provider": "openai",
    "model": "gpt-4o",
    "inputs": {
        "prompts": ["Describe this image"],
        "image_url": "https://example.com/image.jpg"
    },
    "parameters": {"temperature": 0.5}
}

{
  "provider": "openai",
  "model": "gpt-4o-mini",
  "inference_name": "my_text_embedder_1_0_0",
  "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
  "inputs": {
    "prompts": [
      "What is the capital of France?"
    ]
  },
  "parameters": {
    "max_tokens": 500,
    "temperature": 0.7
  }
}

{
  "tokens_used": {
    "completion": 120,
    "prompt": 15,
    "total": 135
  }
}