---
title: "Preview A/B test plan"
method: POST
path: "/v1/projects/{project_id}/ab-tests/plan-preview"
tags: ["ab-tests"]
---

# Preview A/B test plan

`POST /v1/projects/{project_id}/ab-tests/plan-preview`

## Request

- Base URL: `https://api.connic.co`
- URL: `https://api.connic.co/v1/projects/{project_id}/ab-tests/plan-preview`
- Auth: HTTP bearer

Calculate the sample targets and analysis boundaries for a confidence-mode test without creating it.

Required permission: `ab_tests.create`.

## Path parameters

- `project_id` string, uuid, required

## Query parameters

- `environment_id` string, uuid, required: Environment the test would run in.

## Request body

- ABTestPlanPreviewRequest: Inputs for previewing a confidence-mode plan.
  - `traffic_percentage` integer, required: Percentage of traffic routed to the variant, 1 to 99.
  - `analysis_plan` ABTestAnalysisPlanInput, required: Assumptions for a confidence-mode A/B test plan. Metric values use a 0 to 1 scale.
    - `primary_metric` 'success_rate' | 'judge_score', required: `success_rate` compares the share of completed runs; `judge_score` compares the normalized score from `judge_id`.
    - `objective` 'superiority' | 'noninferiority': `superiority` tests whether the variant beats control; `noninferiority` tests that it is not worse by more than a margin.
    - `direction` 'increase' | 'decrease': Which change in the primary metric counts as better: `increase` or `decrease`.
    - `baseline_value` number, required: Expected control value: a success rate between 0 and 1 (exclusive), or a judge score from 0 to 1.
    - `baseline_stddev` number, nullable: Expected standard deviation of the normalized judge score. Required and positive for `judge_score`.
    - `minimum_detectable_effect` number, required: Smallest absolute difference worth detecting, for example 0.05 for 5 percentage points.
    - `noninferiority_margin` number, nullable: Largest acceptable absolute shortfall versus control. Required for `noninferiority`.
    - `alpha` number: Two-sided significance level.
    - `power` number: Probability of detecting the planned effect.
    - `randomization_unit` 'run' | 'session': `run` assigns each run independently; `session` keeps all runs of one session in the same arm.
    - `judge_id` string, uuid, nullable: Judge that scores runs for `judge_score`. It must belong to the same agent and environment, be enabled and automatic, evaluate every run and have no filter. Must be null for `success_rate`.
    - `futility_threshold` number: Conditional power below which a look is flagged as unlikely to succeed. Advisory only; it does not stop the test.

## Response `200`

Successful Response

- ABTestPlanPreviewResponse: Sample sizes and decision boundaries calculated for a confidence-mode plan.
  - `planned_control_samples` integer, required: Maximum control sample, including the sequential design's inflation and a minimum of 80 per arm.
  - `planned_variant_samples` integer, required: Maximum variant sample, including the sequential design's inflation and a minimum of 80 per arm.
  - `total_samples` integer, required: Sum of the planned control and variant samples.
  - `fixed_horizon_control_samples` integer, required: Control sample a single analysis at the end would need.
  - `fixed_horizon_variant_samples` integer, required: Variant sample a single analysis at the end would need.
  - `max_information_inflation` number, required: Factor by which the sequential design increases the fixed-horizon sample.
  - `look_fractions` number[], required: Shares of the planned sample at which analysis looks happen.
  - `boundaries` ABTestSequentialBoundaryResponse[], required: Decision boundary for each planned look.
    - `look_number` integer, required: 1-based position of the look.
    - `information_fraction` number, required: Share of the planned sample reached at this look.
    - `critical_z` number, required: Absolute z-score the effect must exceed to stop at this look.
    - `nominal_two_sided_alpha` number, required: Two-sided significance level that corresponds to `critical_z`.
    - `cumulative_alpha_spent` number, required: Type I error spent through this look.
    - `final_critical_z` number, required: Critical z-score at the final look.
    - `total_looks` integer, required: Number of planned looks.

## Other responses

- `401`: Missing or invalid API key.
- `403`: The API key lacks the required permission or access to the environment.
- `422`: Validation Error
- `429`: Rate limit exceeded.

---

[API](https://skmtc.dev/connic/apis/connic-api.md) · [All operations](https://skmtc.dev/connic/apis/connic-api/llms.txt) · [OpenAPI document](https://skmtc.dev/connic/apis/connic-api/revisions/482932da5fbe?raw)