Cron and heartbeat monitor
The way heartbeat monitoring works is that you need to periodically make requests to a monitor's unique URL in order for the monitor NOT to create a new incident.
We have a background job that makes a daily database backup of our primary PostgreSQL database. We want to get alerted if the background job doesn't run successfully.
Creating a heartbeat monitor:
- Go to Integrations → Create heartbeat.
- Name your heartbeat — "Daily database backup".
- Change the Expect a heartbeat every selection to "1 day".
- Optionally configure the On-call escalation settings.
- Click Save heartbeat.
- Copy the secret URL on the Heartbeat detail page; you will need it later.
This heartbeat will expect us to make a GET or POST request to the URL provided to us every day after the first request.
Set up a CRON and a background job
Add a CRON task that will execute backup_database.sh background job every day at midnight:
0 0 * * * ruby /home/deploy/backup_database.sh >/dev/null 2>&1
Include the curl call to the heartbeat URL at the end of your backup script:
#!/usr/bin/env bash set -o errexit set -o xtrace date=`date "+%Y-%m-%d_%H:%M:%S"` file="/dumps/uptime.betterstack.$date.dump" time dokku postgres:export uptime > "$file" /usr/local/bin/aws s3 cp "$file" s3://uptime-dbbackups/ rm "$file" # you get this URL in the Uptime dashboard curl "https://uptime.betterstack.com/api/v1/heartbeat/XXXXXXXXXXXXXXXXXXXXXX"
What happens here is that the Heartbeat URL we've created above expects a GET or POST request every day since having made the first request.
If the code above fails, our background job won't make the request to the Heartbeat URL. In that case, the Heartbeat will alert the current on-call person and create an Incident.