Upload a pre-prepared benchmark file of instruction and response samples.

This endpoint allows you to upload a JSON file containing benchmark samples, associating it with a specific document for evaluation purposes.

Request Body (multipart/form-data):

• file: JSON file containing benchmark samples (required)
• benchmark_name: Display name for the benchmark (required)
• document_id: Document ID that this benchmark is based on (required)
• description (optional): Description of the benchmark

File Format:

The uploaded JSON file should contain an array of sample objects. Each sample should have:

[
  {
    "sample_num": 1,
    "instruction": "What is...",
    "response": "The response is..."
  }
]

Returns:

• benchmark_id: Unique identifier for the uploaded benchmark
• benchmark_name: Name of the benchmark
• status: Upload status (PROCESSING, READY, FAILED)
• samples: List of benchmark sample objects, each containing:
  • sample_num: Sample number
  • instruction: Instruction text
  • response: Expected response text
• n_samples: Total number of samples in the benchmark

Raises:

• 400: If file format is invalid, JSON structure is incorrect, or document_id is not found
• 413: If file size exceeds limit

Example Request:

POST /api/v3/benchmarks/upload
Headers: {"Authorization": "Bearer <api_key>"}
Content-Type: multipart/form-data

Form Data:
  file: benchmark_questions.json
  name: "Contract Clause Review"
  document_id: "document_01k4x9m2p7q3r5s8"
  description: "Benchmark for contract clause extraction"

Example Response:

{
  "benchmark_id": "benchmark_01k4x9m2p7q3r6a1",
  "benchmark_name": "My Benchmark",
  "status": "READY",
  "samples": [
    {
      "sample_num": 1,
      "instruction": "How do I reset my password?",
      "response": "You can reset your password by clicking the 'Forgot Password' link on the login page."
    },
    {
      "sample_num": 2,
      "instruction": "What are the system requirements?",
      "response": "The system requires a modern web browser with JavaScript enabled."
    }
  ],
  "n_samples": 2
}

Notes:

• File must be in valid JSON format with proper sample structure
• Each sample must include sample_num, instruction, and response fields
• The benchmark is immediately available for use in evaluations after upload
• The document_id provided in the request is stored internally and will appear as a single-element documents array (e.g., ["document_01k4x9m2p7q3r5s8"]) when retrieving the benchmark via /benchmarks/list or /benchmarks/{id}/data endpoints

{
  "file": "benchmark_questions.json",
  "name": "Contract Clause Review",
  "document_id": "document_01k4x9m2p7q3r5s8",
  "description": "Benchmark for contract clause extraction"
}

{
  "benchmark_id": "benchmark_01k4x9m2p7q3r6a1",
  "benchmark_name": "My Benchmark",
  "n_samples": 10
}