---
title: "List A/B tests"
method: GET
path: "/v1/projects/{project_id}/ab-tests"
tags: ["ab-tests"]
---

# List A/B tests

`GET /v1/projects/{project_id}/ab-tests`

## Request

- Base URL: `https://api.connic.co`
- URL: `https://api.connic.co/v1/projects/{project_id}/ab-tests`
- Auth: HTTP bearer

List an agent's A/B tests in an environment, newest first.

Required permission: `ab_tests.view`. With `judges.view`, the response also includes judge names.

## Path parameters

- `project_id` string, uuid, required

## Query parameters

- `environment_id` string, uuid, required: Environment to list tests from.
- `agent_name` string, required: Base agent whose tests to list.

## Response `200`

Successful Response

- ABTestListResponse
  - `tests` ABTestResponse[], required: A/B tests for the agent, newest first.
    - `id` string, uuid, required
    - `project_id` string, uuid, required: Project the test belongs to.
    - `environment_id` string, uuid, required: Environment the test runs in.
    - `agent_name` string, required: Base (control) agent.
    - `test_name` string, required: Variant suffix.
    - `variant_agent_name` string, required: Variant agent name, `{agent_name}-test-{test_name}`.
    - `traffic_percentage` integer, required: Percentage of the agent's traffic routed to the variant.
    - `status` 'draft' | 'running' | 'paused' | 'concluded' | 'rolled_back' | 'failed', required: A/B test state: `draft` until started, `running` while routing traffic, `paused`, `concluded`, or `failed` when the variant is missing from the active deployment. `rolled_back` is no longer set.
    - `analysis_mode` 'exploratory' | 'confidence': `exploratory` compares descriptive metrics; `confidence` follows a locked statistical plan with planned decision looks.
    - `analysis_plan` ABTestAnalysisPlanResponse: Statistical plan of a confidence-mode A/B test, locked when the test first starts.
      - `primary_metric` 'success_rate' | 'judge_score', required: `success_rate` compares the share of completed runs; `judge_score` compares the normalized score from `judge_id`.
      - `objective` 'superiority' | 'noninferiority': `superiority` tests whether the variant beats control; `noninferiority` tests that it is not worse by more than a margin.
      - `direction` 'increase' | 'decrease': Which change in the primary metric counts as better: `increase` or `decrease`.
      - `baseline_value` number, required: Expected control value: a success rate between 0 and 1 (exclusive), or a judge score from 0 to 1.
      - `baseline_stddev` number, nullable: Expected standard deviation of the normalized judge score. Required and positive for `judge_score`.
      - `minimum_detectable_effect` number, required: Smallest absolute difference worth detecting, for example 0.05 for 5 percentage points.
      - `noninferiority_margin` number, nullable: Largest acceptable absolute shortfall versus control. Required for `noninferiority`.
      - `alpha` number: Two-sided significance level.
      - `power` number: Probability of detecting the planned effect.
      - `randomization_unit` 'run' | 'session': `run` assigns each run independently; `session` keeps all runs of one session in the same arm.
      - `judge_id` string, uuid, nullable: Judge that scores runs for `judge_score`. It must belong to the same agent and environment, be enabled and automatic, evaluate every run and have no filter. Must be null for `success_rate`.
      - `futility_threshold` number: Conditional power below which a look is flagged as unlikely to succeed. Advisory only; it does not stop the test.
      - `id` string, uuid, required
      - `ab_test_id` string, uuid, required: A/B test the plan belongs to.
      - `control_allocation` number, required: Share of assigned traffic that goes to control.
      - `variant_allocation` number, required: Share of assigned traffic that goes to the variant.
      - `judge_snapshot` object: Metric judge configuration captured at the first start. Empty for `success_rate` plans or before the start; only `redacted: true` without `judges.view`.
      - `quality_judge_snapshot` object: Quality guardrail judge configuration captured at the first start. Empty when the guardrail is off; only `redacted: true` without `judges.view`.
      - `planned_control_samples` integer, required: Maximum number of control units the plan collects.
      - `planned_variant_samples` integer, required: Maximum number of variant units the plan collects.
      - `look_fractions` number[], required: Shares of the planned sample at which analysis looks happen.
      - `boundaries` object[], required: Decision boundary for each planned look, in the same shape as the plan preview `boundaries`.
      - `design_version` string, required: Identifier of the sequential design, such as `two-sided-obf-v1`.
      - `srm_threshold` number, required: p-value below which a sample-ratio mismatch between the planned and observed split stops the test.
      - `outcome_policy` string, required: Identifier of the rule that turns each assigned unit into an outcome.
      - `locked_at` string, date-time, nullable: When the plan was locked by the test's first start. null while the test is a draft.
      - `created_at` string, date-time, required
      - `updated_at` string, date-time, required
    - `min_sample_size` integer, required: Completed runs needed in each group before the comparison reports enough data.
    - `auto_rollback_enabled` boolean, required: Whether the failure guardrail is on.
    - `rollback_threshold` number, required: Variant failure rate (0 to 1) above which the failure guardrail pauses the test.
    - `rollback_window` integer, required: Number of most recent finished variant runs the failure guardrail evaluates.
    - `quality_guardrail_enabled` boolean: Whether the quality guardrail is on.
    - `quality_judge_id` string, uuid, nullable: Judge for the quality guardrail. null when unset or without `judges.view`.
    - `quality_threshold` number: Lowest acceptable average normalized score (0 to 1) for the variant.
    - `quality_window` integer: Number of most recent evaluated variant runs the quality guardrail averages.
    - `quality_min_evaluations` integer: Evaluations needed before the quality guardrail can act.
    - `winner` 'control' | 'variant', nullable: Manually chosen winner of an exploratory test. null when none.
    - `recommendation` 'collecting' | 'continue' | 'variant_recommended' | 'control_recommended' | 'inconclusive' | 'invalid_srm' | 'safety_stopped': Test verdict: `collecting` before the first planned look, `continue` after a look without a decision, `variant_recommended` or `control_recommended` when a boundary was crossed, `inconclusive`, `invalid_srm` after a sample-ratio mismatch, or `safety_stopped` when the test was stopped automatically.
    - `recommendation_reason` string, nullable: Explanation of `recommendation`.
    - `error` string, nullable: Why the test was stopped automatically. Cleared when the test resumes.
    - `stop_reason` 'manual' | 'failure_guardrail' | 'quality_guardrail' | 'efficacy' | 'futility' | 'srm' | 'analysis_error' | 'variant_unavailable': Why a test stopped: `manual`, a guardrail (`failure_guardrail`, `quality_guardrail`), a planned look (`efficacy` when a boundary was crossed, `futility` when the final look was inconclusive), `srm` (sample-ratio mismatch), `analysis_error` (metric judge evaluations failed), or `variant_unavailable`.
    - `stop_details` object: Details for `stop_reason`, such as the observed failure rate and window for `failure_guardrail`.
    - `created_at` string, date-time, required
    - `updated_at` string, date-time, required
    - `started_at` string, date-time, nullable: When the test first started. null for drafts.
    - `stopped_at` string, date-time, nullable: When the test was last paused or stopped. Cleared when it resumes.
    - `concluded_at` string, date-time, nullable: When the test was concluded.
    - `control_runs` integer: Completed control runs. Confidence tests exclude forced runs.
    - `variant_runs` integer: Completed variant runs. Confidence tests exclude forced runs.
    - `control_avg_cost` number, nullable: Always null. Cost statistics are returned by the comparison endpoint.
    - `variant_avg_cost` number, nullable: Always null. Cost statistics are returned by the comparison endpoint.
    - `control_avg_duration` number, nullable: Average duration of completed control runs, in milliseconds.
    - `variant_avg_duration` number, nullable: Average duration of completed variant runs, in milliseconds.
    - `control_avg_score` number, nullable: Average normalized judge score (0 to 1) of control runs, from the metric or quality judge when set. null without evaluations.
    - `variant_avg_score` number, nullable: Average normalized judge score (0 to 1) of variant runs, from the metric or quality judge when set. null without evaluations.
  - `total` integer, required: Number of tests returned.

## Other responses

- `401`: Missing or invalid API key.
- `403`: The API key lacks the required permission or access to the environment.
- `422`: Validation Error
- `429`: Rate limit exceeded.

---

[API](https://skmtc.dev/connic/apis/connic-api.md) · [All operations](https://skmtc.dev/connic/apis/connic-api/llms.txt) · [OpenAPI document](https://skmtc.dev/connic/apis/connic-api/revisions/482932da5fbe?raw)