The assignment
Docklight receives parcel events and emits notifications. Add signals that distinguish queue backlog, slow dependencies, and worker failure; then write an alert that maps to user impact.
What you hand in
- Instrumented service and sample workload
- Prometheus rules and a focused Grafana dashboard
- Alert test cases for normal, backlog, and dependency-failure states
- On-call runbook with evidence-first diagnosis
How it gets reviewed
- A reviewer can identify which failure occurred from the supplied signals.
- A short transient spike does not page; sustained customer impact does.
- Metric labels have bounded cardinality.
- Runbook steps reproduce and verify recovery.