Rerank documents based on semantic relevance to a query.

This endpoint computes semantic relevance scores for a list of documents against a query, returning them in descending order of relevance. Supports multiple reranking methods from fast bi-encoders to advanced hybrid approaches. Ideal for improving search results and retrieval-augmented generation (RAG) pipelines.

Request Body:

• model: Model ID (required). Use your aligned model id (e.g. model_01kmqm4nrn9fw6r); the same aligned id serves chat completions, completions and embeddings too
• query: Search query or question text (required)
• documents: Array of document texts or objects to rerank (required) - Can be strings or dictionaries

Returns:

• results: Array of reranked documents with relevance scores, each containing:
  • index: Original position in input array
  • relevance_score: Computed relevance score (higher = more relevant)
  • document (optional): Document text or object (if requested)
• meta: Operation metadata containing:
  • method: Reranking method used
  • deployed_model_id: Model ID that was used
  • model_cross (optional): Cross-encoder model used (if applicable)
  • stage1_top_k (optional): Stage 1 top-k for hybrid methods

Example Request (Fast Method):

POST /api/v3/inference/reranker
Headers: {"Authorization": "Bearer <api_key>"}

{
  "model": "model_01kmqm4nrn9fw6r",
  "query": "How do I rotate API keys?",
  "documents": [
    "Rotate keys from the admin console; the old key stays valid.",
    "Python is a popular programming language for web development.",
    "Expired keys are purged after a ninety day grace.",
    "The canteen menu changes on Mondays.",
    "Key rotation requires the security admin role."
  ],
  "method": "fast",
  "top_n": 3
}

Example Response:

{
  "results": [
    {
      "index": 0,
      "relevance_score": 0.95,
      "document": "Rotate keys from the admin console; the old key stays valid."
    },
    {
      "index": 4,
      "relevance_score": 0.82,
      "document": "Key rotation requires the security admin role."
    },
    {
      "index": 2,
      "relevance_score": 0.76,
      "document": "Expired keys are purged after a ninety day grace."
    }
  ],
  "meta": {
    "method": "fast",
    "deployed_model_id": "cross-encoder/ms-marco-MiniLM-L-6-v2"
  }
}

Example Request (Hybrid Method):

POST /api/v3/inference/reranker
Headers: {"Authorization": "Bearer <api_key>"}

{
  "model": "model_01kmqm4nrn9fw6r",
  "query": "How is regional failover tested?",
  "documents": [
    "Failover drills run each quarter against the standby region.",
    "The stock market is volatile today.",
    "Rollback restores the previous release build."
  ],
}

Example Response (Hybrid Method):

{
  "results": [
    {
      "index": 0,
      "relevance_score": 0.92,
      "document": "Failover drills run each quarter against the standby region."
    },
    {
      "index": 2,
      "relevance_score": 0.88,
      "document": "Rollback restores the previous release build."
    }
  ],
  "meta": {
    "deployed_model_id": "model_01kmqm4nrn9fw6r",
  }
}

Use Cases:

• Search Enhancement: Improve search result ranking by semantic relevance
• RAG Pipelines: Select most relevant context chunks for LLM prompts
• Semantic Filtering: Find documents most related to a specific topic
• Question Answering: Rank candidate answers by relevance to the question

Notes:

• Documents can be plain strings or structured objects (dictionaries)
• Documents are returned in descending order of relevance (most relevant first)
• Results are always sorted by relevance_score in descending order (highest first)
• Relevance scores typically range from 0.0 to 1.0

{
  "model": "model_01kmqm4nrn9fw6r",
  "query": "What is Python?",
  "documents": [
    "Python is a programming language",
    "Java is a language"
  ]
}