Sign in
Metrics & Debugging

List Pipeline Events

Required role: read or higher.

Retrieve monitoring events in reverse chronological order.

Pipeline health is monitored regularly every several seconds. Not every monitoring action results in a pipeline monitor event being constructed and inserted into the database. This happens if:

  • Any status changed
  • Only the status details changed, and it has been 10s since the last event
  • Nothing has changed for more than 10 minutes

This endpoint returns the most recent persisted events, up to by default approximately 720.

get/v0/pipelines/{pipeline_name}/events

Request

  • The document declares no server URL.
  • Auth: HTTP bearer

Path parameters

pipeline_namestring required

Unique pipeline name

Query parameters

selector'all' | 'status'

The selector parameter limits which fields are returned. Limiting which fields is particularly handy for instance when frequently monitoring over low bandwidth connections while being only interested in status.

Response

deployment_has_errorboolean required
deployment_resources_desired_status'Stopped' | 'Provisioned' required
deployment_resources_status'Stopped' | 'Provisioning' | 'Provisioned' | 'Stopping' required

Pipeline resources status.

/start (early start or provision check failed)
┌───┐
│   ▼
Stopped ◄────────── Stopping
/start │                   ▲
│                   │ /stop?force=true
│                   │ OR: timeout (from Provisioning)
▼                   │ OR: fatal runtime or resource error
⌛Provisioning ────────────│ OR: runtime status is Suspended
│                   │
│                   │
▼                   │
Provisioned ─────────────┘

Desired and actual status

We use the desired state model to manage the lifecycle of a pipeline. In this model, the pipeline has two status attributes associated with it: the desired status, which represents what the user would like the pipeline to do, and the current status, which represents the actual (last observed) status of the pipeline. The pipeline runner service continuously monitors the desired status field to decide where to steer the pipeline towards.

There are two desired statuses:

  • Provisioned (set by invoking /start)
  • Stopped (set by invoking /stop?force=true)

The user can monitor the current status of the pipeline via the GET /v0/pipelines/{name} endpoint. In a typical scenario, the user first sets the desired status, e.g., by invoking the /start endpoint, and then polls the GET /v0/pipelines/{name} endpoint to monitor the actual status of the pipeline until its deployment_resources_status attribute changes to Provisioned indicating that the pipeline has been successfully provisioned, or Stopped with deployment_error being set.

deployment_resources_status_detailsunknown
deployment_runtime_desired_status'Unavailable' | 'Coordination' | 'Standby' | 'Paused' | 'Running' | 'Suspended'
deployment_runtime_status'Unavailable' | 'Coordination' | 'Standby' | 'Initializing' | 'AwaitingApproval' | 'Bootstrapping' | 'Replaying' | 'Paused' | 'Running' | 'Suspended' | 'ConcurrentBootstrapping' | 'Synchronizing'

Runtime status of the pipeline.

Of the statuses, only Unavailable is determined by the runner. All other statuses are determined by the pipeline and taken over by the runner.

deployment_runtime_status_detailsunknown
event_idstring uuid required

Pipeline monitor event identifier.

program_status'Pending' | 'CompilingSql' | 'SqlCompiled' | 'CompilingRust' | 'Success' | 'SqlError' | 'RustError' | 'SystemError' required

Program compilation status.

recorded_atstring date-time required
storage_status'Cleared' | 'InUse' | 'Clearing' required

Storage status.

The storage status can only transition when the resources status is Stopped.

Cleared ───┐
▲       │
/clear │       │
│       │
Clearing   │
▲       │
│       │
InUse ◄───┘
storage_status_detailsunknown

Example response

[
  {
    "deployment_error": {
      "error_code": "CodeSpecifyingErrorType",
      "message": "Explanation of the error that occurred."
    }
  }
]

Changes

No recorded changes to this operation across all 51 revisions of this API.