Evaluations

Get Evaluation Results

Retrieve the complete results of a finished evaluation.

This endpoint returns detailed evaluation metrics and scores for a completed evaluation. Works for both single model evaluations and comparison mode evaluations.

Path Parameters:

  • evaluation_id: Unique evaluation identifier

Returns:

  • evaluation_id: The evaluation identifier
  • raw_responses_count: Number of raw responses generated during evaluation
  • method (optional): Evaluation method (eval for single model, eval-compare for comparison)
  • metrics (optional): Evaluation metrics and scores (single model only)
  • baseline_model_id (optional): Baseline model ID (comparison mode only)
  • model (optional): Aligned model results (comparison mode only)
  • baseline (optional): Baseline model results (comparison mode only)
  • comparison (optional): Comparison results between models (comparison mode only)

Identity, status, and timestamps are served by GET /{evaluation_id}.

Raises:

  • 404: If evaluation not found or doesn't belong to the authenticated user
  • 400: If evaluation is not yet completed, or FAILED with no results

Example Request:

GET /api/v3/evaluations/evaluation_01k4x9m2p7q3r7b1/results
Headers: {"Authorization": "Bearer <api_key>"}

Example Response (Single Model):

{
  "evaluation_id": "evaluation_01k4x9m2p7q3r7b1",
  "method": "eval",
  "raw_responses_count": 10,
  "metrics": {
    "accuracy": 0.92,
    "relevance": 0.88,
    "average_score": 0.90,
    "n_samples": 10,
    "correct_answers": 9
  }
}

Example Response (Comparison Mode):

{
  "evaluation_id": "evaluation_01k4x9m2p7q3r7b1",
  "baseline_model_id": "model_01kmqm4nrn9fw6r",
  "method": "eval-compare",
  "raw_responses_count": 20,
  "model": {
    "model_id": "model_01kmqm4nrn9fw6r",
    "average_score": 0.92,
    "n_samples": 10
  },
  "baseline": {
    "model_id": "Qwen/Qwen2.5-0.5B-Instruct",
    "average_score": 0.85,
    "n_samples": 10
  },
  "comparison": {
    "winner": "Qwen/Qwen2.5-0.5B-Instruct",
    "score_difference": 0.07,
    "statistical_significance": true
  }
}
get/api/v3/evaluations/{evaluation_id}/results

Request

  • Base URL
    https://api.nugen.in
  • Auth
    HTTP bearer

Path parameters

evaluation_idstring required

ID of the evaluation, returned when it was created

Response

Returns detailed evaluation metrics and scores for a completed evaluation. Identity, status, and timestamps are served by GET /{evaluation_id}.

evaluation_idstring required

Unique identifier for the evaluation

metricsobject nullable

Metric names and scores. binary_correctness_mean is the fraction of responses whose key facts match the benchmark's expected response (0 to 1). response_relevance_mean rates how relevant each response is to its instruction (0 to 1) and ignores correctness; read the pair together. The metric set evolves with our evaluation research.

raw_responses_countinteger required

Number of raw responses generated; read metric means relative to this size

method'eval' | 'eval-compare' nullable

Evaluation method: 'eval' or 'eval-compare'

baseline_model_idstring nullable

ID of baseline model (for comparison)

modelobject nullable

Aligned model results (comparison mode)

baselineobject nullable

Baseline model results (comparison mode)

comparisonobject nullable

Comparison results between models

Example response

{
  "evaluation_id": "evaluation_01k4x9m2p7q3r7b1",
  "raw_responses_count": 100,
  "method": "eval-compare",
  "baseline_model_id": "model_01k4x9m2p7q3r9d2",
  "comparison": {
    "accuracy_difference": 0.05
  }
}

Changes

No recorded changes to this operation across all 1 revision of this API.