DemoAcme Cloud · Blueprint template

Use this template

Acme Cloud

Partial outage

Live status for the Acme Cloud API, dashboard and data pipeline. Updates land here first, or read the incident handbook.

Title
Acme Cloud
Dwg no.
ACME-CLOUD
Rev
3
Drawn by
Crontap
Sheet
1 of 1
Scale
90 D
Status
Deviation
Date

Revisions

RevDescriptionDate
2Last released issuedate unknown
3Deviation on Backupsdate unknown

Incidents

  1. Note 1.

    Minor impact · Identified

    Nightly backups running late

    Started · Affects H1 Backups

    Last night's backup has not checked in yet. Data is safe; we are looking into the storage queue.

    1. The backup is running again and should finish within the hour.

    Earlier updates (1)
    1. A stuck storage queue worker held the job. We restarted it.

Subsystem A · Core platform

  • U1Operational

    API

    Uptime 99.84 %

    Lat 190 ms

  • U2Operational

    Dashboard

    Uptime 99.84 %

    Lat 334 ms

  • U3Operational

    Auth

    Uptime 99.84 %

    Lat 157 ms

Subsystem B · Data

  • U4Operational

    Postgres primary

    Uptime 99.84 %

    Lat 41 ms

  • TOL

    H1Degraded

    Backups

    Uptime 96.70 %

    Last ping

    See note 1

Subsystem C · Edge

  • U5Operational

    CDN

    Uptime 99.84 %

    Lat 60 ms

  • U6Operational

    Webhooks

    Uptime 99.81 %worsening

    Lat 273 ms

History

RevDateDescription
2
Nightly backups running late · Minor impact · open
1
Elevated error rates · Major impact · 50 min
2 automatic events
  • API recovered
  • API went down
Details

A bad deploy raised error rates for about 45 minutes. We rolled it back and added a canary stage.

  1. Error rates are back to normal. This incident is resolved.

  2. We found a faulty deploy and are rolling it back.