test_insights

Get what Fail First did on a repository

Changed on

What Fail First did on the repository over the last 30 days. When a batch fails, the merge queue retests smaller splits of it, called bisection steps, to find the pull request that broke it. In each step, Fail First asks a job to run the tests that already failed before the others, and the job stops early when one of them fails again. The response counts the steps where a job was asked and those where it stopped early, and estimates the test time saved.

get/ci/{owner}/repositories/{repository}/test-selection/failure-first

Request

Path parameters

repositorystring required

The name of the repository

ownerstring required

The owner of the repository

Response

Successful Response

state'not_served' | 'active' required

active: since since, Mergify asked at least one job of a bisection step to run its failed tests first. It does not say the job did: a job on a Mergify plugin release that does not apply Fail First keeps its usual order. not_served: no job was asked. Possible reasons: the merge queue split no failed batch, the jobs of its steps had no failed tests known by name, no job uses the Mergify plugin for its test framework, or Fail First is not enabled for the organization. See https://docs.mergify.com/test-engine/fail-first for what a job needs, and contact Mergify Support if your jobs have it and a failed batch still gets not_served.

sincestring date-time required

The start of the period every figure covers.

stepsinteger required

Bisection steps in which at least one job was asked to run first the tests it had already failed, in the failed batch or an earlier step. A job on a Mergify plugin release that does not apply Fail First counts too, though it runs its tests in their usual order.

stopped_early_stepsinteger required

Of steps, those in which such a job stopped before running the rest of its tests. Any early stop counts, including one caused by the job's own failure limit (-x, --bail).

estimated_test_time_saved_msinteger required

An estimate, in milliseconds, of the test time the jobs that stopped early did not spend. For each of them: the median test time of the same job (same pipeline, name and matrix leg) over the period's bisection steps where it ran all its tests, minus its own test time, and never below zero. A stop caused by the job's own failure limit counts too, so this can include time that limit would have saved anyway. It adds up test durations, so with parallel workers the time gained on the clock is smaller. A job with no such run to compare with adds nothing here and counts in unmeasured_stopped_early_jobs instead.

unmeasured_stopped_early_jobsinteger required

Early stops with no estimated saving, because the same job (same pipeline, name and matrix leg) never ran all its tests in a bisection step of the period. Each leg that stopped early counts once per step.

Changes