---
title: "Execute a test suite"
method: POST
path: "/api/projects/{projectId}/test-suites/{suiteId}/execute"
tags: ["CI & Test Runs"]
---

# Execute a test suite

`POST /api/projects/{projectId}/test-suites/{suiteId}/execute`

## Request

- Base URL: `https://api.routebase.dev`
- URL: `https://api.routebase.dev/api/projects/{projectId}/test-suites/{suiteId}/execute`
- Auth: API key in header `X-API-Key`

Runs every test case in a suite in order, carrying extracted variables from one
case into the next. This is the full result shape, so it also carries the
per-case detail, the seed results and the performance metrics that
`/api/cli/run` leaves out.

Set `persistResults` to false for a dry run that leaves no test run behind.

## Path parameters

- `ProjectId` string, uuid, required
- `suiteId` string, uuid, required

## Headers

- `X-RB-Region` string

## Request body

- ExecuteTestSuiteRequest: Options for a suite run. Every field has a default, so the body may be omitted entirely.
  - `environmentId` string, uuid, nullable: Environment whose variables the run substitutes.
  - `persistResults` boolean, nullable: Whether to store the run and its results. Defaults to true.
  - `iterations` integer, nullable: How often to run the suite, between 1 and 10000. Defaults to 1.
  - `delayMs` integer, nullable: Pause between iterations in milliseconds, up to 60000. Defaults to 0.
  - `onError` 'ignore' | 'stop' | 'abort': What a run does when a test case errors. `ignore` continues, `stop` ends the current iteration but starts the next one, and `abort` ends the whole run.
  - `dataSetId` string, uuid, nullable: Data set for data-driven execution, which runs every case once per row.
  - `threads` integer, nullable: Parallel threads for performance mode, between 1 and 50. Defaults to 1.
  - `progressRunId` string, uuid, nullable: Client-generated id for live progress. Join the test-runner hub group for this id before sending the request, then follow the run case by case.
  - `caseIds` string[], nullable: Run only these test cases. Null or empty runs the whole suite.

## Response `200`

The run completed.

- TestSuiteExecutionResult: The full result of a suite run, including per-case detail, seeds and performance figures.
  - `results` TestCaseExecutionResult[], required: One entry per test case execution, in run order.
    - `statusCode` integer, required: Status code from the target server, or 0 when the request never went out.
    - `statusText` string, nullable: Reason phrase from the target server.
    - `headers` object, required: Response headers from the target server.
    - `body` string, nullable: Response body, which may be null for an empty response.
    - `elapsedMilliseconds` integer, required: Request duration in milliseconds.
    - `error` string, nullable: Set when the request itself failed, for example on a timeout.
    - `resolvedUrl` string, required: The URL after variable substitution.
    - `resolvedMethod` string, required: The HTTP method used.
    - `unresolvedVariables` string[], nullable: Variables that could not be resolved, which is usually a missing environment value.
    - `passed` boolean, nullable: Whether every assertion held. Null when the case has no assertions or never ran.
    - `assertionResults` AssertionResult[], nullable: One entry per assertion on the case. Null when the case carries none or never ran.
      - `assertionId` string, uuid, required: Public id of the assertion, stable across runs.
      - `type` string, required: What the assertion checks, for example the status code or a JSON path.
      - `operator` string, required: How expected and actual are compared.
      - `target` string, nullable: What the assertion points at, for example a JSON path.
      - `expectedValue` string, nullable: The value the assertion demanded, rendered as text.
      - `actualValue` string, nullable: The value the response actually carried, rendered as text.
      - `passed` boolean, required: Whether this single assertion held.
      - `errorMessage` string, nullable: Why the assertion failed.
      - `checkedVersionNumber` string, nullable: Specification version the contract assertion checked against.
      - `checkedVersionSource` string, nullable: Where that version came from.
      - `driftedFromVersionNumber` string, nullable: Set when the response no longer matches the version it used to.
    - `testCaseId` string, uuid, nullable: Public id of the test case.
    - `testCaseName` string, nullable: Name of the test case.
    - `extractedVariables` object, nullable: Variables this response contributed to later cases.
    - `consumedVariables` object, nullable: Variables this case read.
    - `resolvedHeaders` object, nullable: Request headers after variable substitution.
    - `resolvedBody` string, nullable: Request body after variable substitution.
    - `responseSizeBytes` integer, nullable: Size of the response body.
    - `preRequestScriptOutput` string[], nullable: Console output of the pre-request script.
    - `preRequestScriptError` string, nullable: Set when the pre-request script failed.
    - `postResponseScriptOutput` string[], nullable: Console output of the post-response script.
    - `postResponseScriptError` string, nullable: Set when the post-response script failed.
    - `timingBreakdown` TimingBreakdown: Where the time inside one request went, from name resolution to the last byte.
      - `dnsMs` integer, required: Time spent resolving the hostname. Zero on a reused connection.
      - `connectMs` integer, required: Time spent opening the TCP connection.
      - `tlsMs` integer, required: Time spent on the TLS handshake.
      - `firstByteMs` integer, required: Time from request sent to the first response byte, which is where server work shows up.
      - `downloadMs` integer, required: Time spent reading the response body.
      - `totalMs` integer, required: Total wall time of the request. It covers the phases above plus scheduling overhead.
    - `isPending` boolean, required: The case has not run yet.
    - `isBlocked` boolean, required: The case was never sent, because it writes and the environment is read-only. This is neither a pass nor a failure, so do not fail a build on it alone.
  - `extractedVariables` object, required: Variables extracted during the run and carried between cases.
  - `allPassed` boolean, required: Whether every case passed.
  - `totalElapsedMs` integer, required: Total run time in milliseconds.
  - `testRunPublicId` string, uuid, nullable: Public id of the stored run, or null when results were not persisted.
  - `iterations` integer, required: Number of iterations executed.
  - `iterationResults` IterationResult[], nullable: Per-iteration breakdown, present only when more than one iteration ran.
    - `iterationNumber` integer, required: The 1-based iteration number.
    - `results` TestCaseExecutionResult[], required: The result of every test case in this iteration, in run order.
      - `statusCode` integer, required: Status code from the target server, or 0 when the request never went out.
      - `statusText` string, nullable: Reason phrase from the target server.
      - `headers` object, required: Response headers from the target server.
      - `body` string, nullable: Response body, which may be null for an empty response.
      - `elapsedMilliseconds` integer, required: Request duration in milliseconds.
      - `error` string, nullable: Set when the request itself failed, for example on a timeout.
      - `resolvedUrl` string, required: The URL after variable substitution.
      - `resolvedMethod` string, required: The HTTP method used.
      - `unresolvedVariables` string[], nullable: Variables that could not be resolved, which is usually a missing environment value.
      - `passed` boolean, nullable: Whether every assertion held. Null when the case has no assertions or never ran.
      - `assertionResults` AssertionResult[], nullable: One entry per assertion on the case. Null when the case carries none or never ran.
        - `assertionId` string, uuid, required: Public id of the assertion, stable across runs.
        - `type` string, required: What the assertion checks, for example the status code or a JSON path.
        - `operator` string, required: How expected and actual are compared.
        - `target` string, nullable: What the assertion points at, for example a JSON path.
        - `expectedValue` string, nullable: The value the assertion demanded, rendered as text.
        - `actualValue` string, nullable: The value the response actually carried, rendered as text.
        - `passed` boolean, required: Whether this single assertion held.
        - `errorMessage` string, nullable: Why the assertion failed.
        - `checkedVersionNumber` string, nullable: Specification version the contract assertion checked against.
        - `checkedVersionSource` string, nullable: Where that version came from.
        - `driftedFromVersionNumber` string, nullable: Set when the response no longer matches the version it used to.
      - `testCaseId` string, uuid, nullable: Public id of the test case.
      - `testCaseName` string, nullable: Name of the test case.
      - `extractedVariables` object, nullable: Variables this response contributed to later cases.
      - `consumedVariables` object, nullable: Variables this case read.
      - `resolvedHeaders` object, nullable: Request headers after variable substitution.
      - `resolvedBody` string, nullable: Request body after variable substitution.
      - `responseSizeBytes` integer, nullable: Size of the response body.
      - `preRequestScriptOutput` string[], nullable: Console output of the pre-request script.
      - `preRequestScriptError` string, nullable: Set when the pre-request script failed.
      - `postResponseScriptOutput` string[], nullable: Console output of the post-response script.
      - `postResponseScriptError` string, nullable: Set when the post-response script failed.
      - `timingBreakdown` TimingBreakdown: Where the time inside one request went, from name resolution to the last byte.
        - `dnsMs` integer, required: Time spent resolving the hostname. Zero on a reused connection.
        - `connectMs` integer, required: Time spent opening the TCP connection.
        - `tlsMs` integer, required: Time spent on the TLS handshake.
        - `firstByteMs` integer, required: Time from request sent to the first response byte, which is where server work shows up.
        - `downloadMs` integer, required: Time spent reading the response body.
        - `totalMs` integer, required: Total wall time of the request. It covers the phases above plus scheduling overhead.
      - `isPending` boolean, required: The case has not run yet.
      - `isBlocked` boolean, required: The case was never sent, because it writes and the environment is read-only. This is neither a pass nor a failure, so do not fail a build on it alone.
    - `allPassed` boolean, required: Whether every case passed in this iteration.
    - `elapsedMs` integer, required: Duration of this iteration.
    - `wasAborted` boolean, required: Whether the iteration stopped early because of the error behaviour.
    - `dataRowIndex` integer, nullable: The 0-based data row, for a data-driven run.
    - `dataRowValues` object, nullable: The data row values used, for a data-driven run.
  - `dataSetName` string, nullable: Name of the data set, for a data-driven run.
  - `dataRowCount` integer, nullable: Number of data rows used.
  - `threads` integer, required: Threads the run used.
  - `performanceMetrics` PerformanceMetrics: Latency distribution and throughput of a run. Present once a run used parallel threads or several iterations.
    - `averageResponseTimeMs` number, double: Mean response time across every request in the run.
    - `minResponseTimeMs` integer: Fastest single response in the run.
    - `maxResponseTimeMs` integer: Slowest single response in the run, which is the number a timeout budget has to beat.
    - `medianResponseTimeMs` integer: Median response time, less distorted by outliers than the mean.
    - `p95ResponseTimeMs` integer: 95th percentile, so one request in twenty was slower than this.
    - `p99ResponseTimeMs` integer: 99th percentile, the usual number behind a latency commitment.
    - `throughputPerSecond` number, double: Completed requests per second across the run.
    - `errorRatePercent` number, double: Share of requests that failed, from 0 to 100.
    - `totalRequests` integer: Requests sent across all threads and iterations.
    - `successfulRequests` integer: Requests that completed and passed their assertions.
    - `failedRequests` integer: Requests that errored or failed an assertion.
    - `perTestCase` TestCasePerformance[]: The same figures broken down per test case, which is how you find the one slow call.
      - `testCaseId` string, uuid: Public id of the test case these figures belong to.
      - `testCaseName` string: Display name of the test case.
      - `averageResponseTimeMs` number, double: Mean response time of this case across all its executions.
      - `minResponseTimeMs` integer: Fastest execution of this case.
      - `maxResponseTimeMs` integer: Slowest execution of this case.
      - `p95ResponseTimeMs` integer: 95th percentile response time of this case.
      - `errorRatePercent` number, double: Share of this case executions that failed, from 0 to 100.
      - `requestCount` integer: How often this case ran, which is iterations multiplied by threads.
  - `preRunSeedResult` SeedExecutionResult: The outcome of a seed sequence, which prepares data before a suite or cleans up after it.
    - `success` boolean, required: Whether every step of the seed completed. A failed seed does not stop the suite by itself.
    - `elapsedMilliseconds` integer, required: Duration of the whole seed sequence.
    - `stepResults` SeedStepResult[], required: One entry per seed step, in execution order.
      - `order` integer: Position of the step in the sequence, starting at 1.
      - `type` string: What the step does.
      - `success` boolean: Whether this step completed.
      - `elapsedMilliseconds` integer: Duration of this step.
      - `statusCode` integer, nullable: HTTP status the step received. Null for steps that send no request.
      - `error` string, nullable: Why the step failed. Null on success.
      - `resolvedUrl` string, nullable: The URL after variable substitution.
      - `resolvedMethod` string, nullable: The HTTP method the step used.
      - `iterationIndex` integer, nullable: Which repetition this is, for a step configured to loop.
      - `capturedVariables` object, nullable: Variables this step captured for the steps and cases that follow.
    - `capturedVariables` object, required: Variables the seed captured for the suite that follows.
    - `error` string, nullable: Why the seed failed. Null on success.
  - `postRunSeedResult` SeedExecutionResult: The outcome of a seed sequence, which prepares data before a suite or cleans up after it.
    - `success` boolean, required: Whether every step of the seed completed. A failed seed does not stop the suite by itself.
    - `elapsedMilliseconds` integer, required: Duration of the whole seed sequence.
    - `stepResults` SeedStepResult[], required: One entry per seed step, in execution order.
      - `order` integer: Position of the step in the sequence, starting at 1.
      - `type` string: What the step does.
      - `success` boolean: Whether this step completed.
      - `elapsedMilliseconds` integer: Duration of this step.
      - `statusCode` integer, nullable: HTTP status the step received. Null for steps that send no request.
      - `error` string, nullable: Why the step failed. Null on success.
      - `resolvedUrl` string, nullable: The URL after variable substitution.
      - `resolvedMethod` string, nullable: The HTTP method the step used.
      - `iterationIndex` integer, nullable: Which repetition this is, for a step configured to loop.
      - `capturedVariables` object, nullable: Variables this step captured for the steps and cases that follow.
    - `capturedVariables` object, required: Variables the seed captured for the suite that follows.
    - `error` string, nullable: Why the seed failed. Null on success.
  - `resetSnapshotResult` SnapshotRestoreResult: The outcome of restoring an environment to a known state before a run.
    - `success` boolean, required: Whether the environment was restored to the snapshot state.
    - `phase` string, required: Which phase the restore reached.
    - `elapsedMilliseconds` integer, required: Duration of the whole restore.
    - `cleanupResults` SnapshotCleanupStepResult[], required: One entry per cleanup call, in execution order.
      - `order` integer: Position of the cleanup call in the sequence, starting at 1.
      - `method` string: HTTP method of the cleanup call, usually DELETE.
      - `url` string: The URL the cleanup call went to.
      - `success` boolean: Whether the cleanup call succeeded.
      - `statusCode` integer, nullable: HTTP status the cleanup call received.
      - `error` string, nullable: Why the cleanup call failed. Null on success.
      - `elapsedMilliseconds` integer: Duration of the cleanup call.
    - `reSeedResult` SeedExecutionResult: The outcome of a seed sequence, which prepares data before a suite or cleans up after it.
      - `success` boolean, required: Whether every step of the seed completed. A failed seed does not stop the suite by itself.
      - `elapsedMilliseconds` integer, required: Duration of the whole seed sequence.
      - `stepResults` SeedStepResult[], required: One entry per seed step, in execution order.
        - `order` integer: Position of the step in the sequence, starting at 1.
        - `type` string: What the step does.
        - `success` boolean: Whether this step completed.
        - `elapsedMilliseconds` integer: Duration of this step.
        - `statusCode` integer, nullable: HTTP status the step received. Null for steps that send no request.
        - `error` string, nullable: Why the step failed. Null on success.
        - `resolvedUrl` string, nullable: The URL after variable substitution.
        - `resolvedMethod` string, nullable: The HTTP method the step used.
        - `iterationIndex` integer, nullable: Which repetition this is, for a step configured to loop.
        - `capturedVariables` object, nullable: Variables this step captured for the steps and cases that follow.
      - `capturedVariables` object, required: Variables the seed captured for the suite that follows.
      - `error` string, nullable: Why the seed failed. Null on success.
    - `warnings` string[], required: Non-fatal problems, such as a cleanup call that found nothing to delete.
    - `error` string, nullable: Why the restore failed. Null on success.

## Other responses

- `400`: The request was malformed or failed validation.
- `401`: The API key is missing, invalid, expired or revoked. A US organization calling without `X-RB-Region: us` also lands here, because the request reached the wrong region.
- `403`: The key is valid but lacks the permission or the project scope for this call. A scoped key is also refused on organization level operations by design.
- `404`: The resource does not exist, or it belongs to another organization or project. Both cases answer the same way on purpose, so the API cannot be used to probe for foreign identifiers.
- `429`: The rate limit was exceeded. The `Retry-After` header gives the seconds to wait.

---

[API](https://skmtc.dev/routebase/apis/routebase-public-api.md) · [All operations](https://skmtc.dev/routebase/apis/routebase-public-api/llms.txt) · [OpenAPI document](https://skmtc.dev/routebase/apis/routebase-public-api/revisions/7187df489f27?raw)