People
Each person can have their own email, phone, iMessage address, ntfy topic, Pushover key and PagerDuty key. Send a test to one person from their row.
DokkitAlerts
Alerts
Dokkit tells you when something needs you, on the channels you set up: paging for what cannot wait, a quiet push for today, and email for the morning.
How it decides
An issue alerts once, when it opens, and what it raised is resolved when it closes. Its state picks the channel.
The issueWhat decidesWhereWhat arrives
nightly-purge did not rundb-01 · a job · stopped · DOK-12
When something stops
PagePagerDuty
nightly-purge did not run (db-01, DOK-12)01:04It was due at 01:00 and no start was reported.
upload-portal is answering slowlyweb-02 · a service · struggling · DOK-09
When something is struggling
Pushntfy
upload-portal is answering slowly (web-02, DOK-09)09:40Answers are taking 4.2 s, over its 2 s limit.
report-builder is answering slowlyapp-01 · a service set not to wake you · struggling · DOK-14
Night, 22:00 to 07:00opened at 02:15
Pushheld until 07:00
report-builder is answering slowly (app-01, DOK-14)07:00Held through the night, then sent.
- Only the production environment alerts. Issues in other environments are listed, and stay quiet.
- If something recovers and fails again inside 30 minutes, it is the same issue, and it stays quiet.
- Tick more than one channel and each sends its own alert.
The channels
Each one is set up on Dokkit's Alerts page, in Settings, and has a Send a test button. Pick one to see its fields.
People and rules
Until you name people, an alert goes to the addresses set for each channel. Name them at four levels: the default, a rule, a service and a check. The most specific level that names someone decides.
Rules
A rule picks the channel and the people for a kind of issue, a state and a machine. The first rule that fits decides. A service's own routing comes first.
Night
Choose when night starts and ends, and a time zone. A service set not to wake you holds its alerts until the night ends.
Limits
The percent of a disk, or of memory for three minutes running, that needs a look. Leave either blank to switch it off.
When Dokkit is down
Dokkit cannot tell anyone that it is down. Give it an outside heartbeat: an https address it calls every minute, at a service that alerts when the calls stop.
- When Dokkit stops taking reports or cannot reach its database, it tells you on the loudest channel that is set up, unless you choose one.
- Lesser trouble, such as an archive folder it cannot write to or a certificate close to running out, has its own choice.