Create a new benchmark generation run from documents.

This endpoint starts a benchmark generation run over the documents you name. The generated samplions can be used for model evaluation and testing.

Request Body:

• document_ids: List of document IDs to generate benchmarks from (required)
• n_samples: Number of benchmark samples to generate (required)
• benchmark_name (optional): Display name. Defaults to the benchmark id if omitted

Returns:

• benchmark_id: Unique identifier for tracking generation progress
• status: Initial status (always QUEUED)

Raises:

• 400: If no documents provided
• 400: If document validation fails (missing documents or unauthorized access)

Example Request:

POST /api/v3/benchmarks/create
Headers: {"Authorization": "Bearer <api_key>"}

{
  "document_ids": ["document_01k4x9m2p7q3r5s8", "document_01k4x9m2p7q3r5sc", "document_01k4x9m2p7q3r5sd"],
  "n_samples": 10
}

Example Response:

{
  "benchmark_id": "benchmark_01k4x9m2p7q3r6a1",
  "status": "QUEUED"
}

Notes:

• All document IDs are validated before creation to ensure they exist and belong to the user
• Use the returned benchmark_id with /benchmarks/{id}/status endpoint to track generation progress
• Once completed, the benchmark will be available for use in model evaluations

{
  "document_ids": [
    "document_01k4x9m2p7q3r5s8",
    "document_01k4x9m2p7q3r5s9"
  ],
  "benchmark_name": "Contract Clause Review",
  "n_samples": 10
}

{
  "benchmark_id": "benchmark_01k4x9m2p7q3r6a1",
  "benchmark_name": "Contract Clause Review"
}