ab-tests

Get A/B test comparison

Compare control and variant performance for an A/B test.

Required permission: ab_tests.view. With judges.view, the response also includes judge names.

get/v1/projects/{project_id}/ab-tests/{test_id}/comparison

Request

  • Base URL: https://api.connic.co
  • URL: https://api.connic.co/v1/projects/{project_id}/ab-tests/{test_id}/comparison
  • Auth: HTTP bearer

Path parameters

project_idstring uuid required

ID of the project the API key belongs to.

test_idstring uuid required

A/B test ID from GET /ab-tests.

Query parameters

convert_costs_to'EUR' | 'USD' nullable

Convert token costs to EUR or USD at the daily ECB reference rate of each model call

Response

Successful Response

has_enough_databoolean required

True when both arms have at least min_sample_size completed runs.

min_sample_sizeinteger required

Minimum completed runs per arm set on the test.

selected_judge_idstring uuid nullable

Judge whose scores fill avg_score: the primary-metric judge when the primary metric is judge_score, otherwise the quality judge. Present only when the caller can view judges.

selected_judge_namestring nullable

Name of the judge in selected_judge_id. Present only when the caller can view judges.

quality_judge_namestring nullable

Name of the test's quality judge. Present only when the caller can view judges.

Changes

No recorded changes to this operation across all 2 revisions of this API.