DemoAcme Cloud · Retro Desktop template

Use this template

Acme Cloud

Live status for the Acme Cloud API, dashboard and data pipeline. Updates land here first, or read the incident handbook.

Some programs are slow (Partial outage)

System Monitor: Partial outage
ProgramStatusHistory (90 days)Uptime (90 days)CPU (response time)Trend
Core platform
APIResponding (Operational)99.84%, response 190 ms190 ms (fast)steady
DashboardResponding (Operational)99.84%, response 334 ms334 ms (ok)steady
AuthResponding (Operational)99.84%, response 157 ms157 ms (fast)steady
Data
Postgres primaryResponding (Operational)99.84%, response 41 ms41 ms (fast)steady
BackupsNot responding (Degraded)Minor incident96.70% reliability—steady
Edge
CDNResponding (Operational)99.84%, response 60 ms60 ms (fast)steady
WebhooksResponding (Operational)99.80%, response 273 ms273 ms (fast)worsening
Programs: 7Responding: 6

Messages

Nightly backups running late

IdentifiedMinor impact · Started

Affects Backups

Last night's backup has not checked in yet. Data is safe; we are looking into the storage queue.

  1. The backup is running again and should finish within the hour.

  2. A stuck storage queue worker held the job. We restarted it.

Event viewer

Event viewer
LevelTimeSourceEvent
IncidentStatus pageNightly backups running late [Identified]
IncidentStatus pageElevated error rates [Resolved · 50 min]
2 automatic events
  • API recovered
  • API went down
Details

A bad deploy raised error rates for about 45 minutes. We rolled it back and added a canary stage.

  1. Error rates are back to normal. This incident is resolved.

  2. We found a faulty deploy and are rolling it back.