Monitoring

Observatory. Use signals to narrow causes and protect users.
3 labs, easiest first0 / 3
22 minutes of failed labels, nobody pagedPM-2402Yesterday from 14:02 to 14:24 one in five label requests failed. Depot staff phoned it in. Monitoring stayed green, and nobody was paged.MediumFix 35 min · Linux + Docker
The alert that could not seePM-2417After the Waybill 2.4.4 deploy, labels failed for one in three requests for eleven minutes. The error alert exists, promtool says it is valid, and nobody was paged.HardIncident 35 min · Linux + Docker
The average says 90 millisecondsCX-311Since release 2.4.3, depot-3 staff say some label prints freeze for two seconds. The latency dashboard shows /labels at 90 ms.HardInvestigate 40 min · Linux + Docker