Monitor uptime and latency with Elastic’s Heartbeat
This article covers the following topics:
- Introduction to Heartbeat
- Basic configuration and concepts of Heartbeat
- Running Heartbeat locally or via Docker
- Packaging your own Dockerfile
- Visualizing your data in Kibana
- Alerting downtimes using Elasticsearch Watchers and Slack
This is a two-parts series, check out the second part Alerting downtimes in Slack using Heartbeat and Elasticsearch Watchers.
Introduction
Heartbeat is a lightweight shipper for uptime monitoring. Elastic explains it well
Monitor services for their availability with active probing. Given a list of URLs, Heartbeat asks the simple question: Are you alive? Heartbeat ships this information and response time to the rest of the Elastic Stack for further analysis.
Heartbeat can easily be configured using YAML, pings your targets and sends the information to Logstash or straight to Elasticsearch.
Supported transfer protocols are ICMP, TCP, and HTTP. TLS, authentication and proxies are supported aswell.
On top of that, we get great visualization with Kibana alerting with Elasticsearch.
Configuration
As for configuration, a simple YAML file is used.
I will explain the most common/important parts of the configuration, there is a lot more to configure.
The heart of heartbeat are monitors (pun intended). A basic monitor has a type (transport protocol: http/icmp/tcp), a schedule (when to check) and a list of urls.
This monitor will do a HTTP call on http://elasticsearch:9200 and http://kibana:5601 every 5 seconds and by default check for a 200 status code.
This monitor will do a HTTP call on https://my-basic-auth-protected-rurl.com with the basic auth credentials foo:bar.
Issue a POST request every 5 seconds to http://localhost:8080/demo/add, with the specified header and body, expecting a 200 HTTP response code and a response body containg one of the given regular expressions.
To check that a specific field in the JSON response body matches something, you can use the check.response.json matcher.
Check further configuration options in the official docs.
Fields
Optional fields that you can specify to add additional information to the monitor output. Fields can be scalar values, arrays, dictionaries, or any nested combination of these.
Processors
Processors are used to reduce the number of fields in the exported event or to enhance the event with external metadata.
Check out the full documentation on processors.
Outputs
Heartbeat will send the collected data every X seconds to the configured output.
Supported outputs are:
- Elasticsearch
- Logstash
- Kafka
- Redis
- File
- Console
- Elastic Cloud
Using environment variables
You can use environment variable references in the config file to set values that need to be configurable during deployment. To do so, simply use: ${VAR}
Some examples
Running heartbeat
We’ll start with the most simple example. Check kevcodez.de every 10 seconds and print the results to the console.
We are setting a name here, otherwise it will get a cryptic name in the Kibana Uptime dashboard.
Locally
Download the appropriate package here. Heartbeat also has packages for yum, apt-get and homebrew.
After unpacking, we’ll place the configuration above in the same folder.
To execute heartbeat, simply run heartbeat -e.
After a few seconds, the first ping should be sent to the console.
Heartbeat is up and running. We may now change the monitors and output.
Docker
Elastic offers an official docker image in their own docker repository.
To run the docker image with your heartbeat.yml file, run:
Packaging your own docker image
To dockerize heartbeat with your custom heartbeat configuration, check out the following Dockerfile, it’s pretty self-explanatory:
Official Docs for running Heartbeat on Docker
Kibana
Kibana has an Uptime tab to visualize Heartbeat pings. On top of that, alerting can be configured to alert you in case of a downtime.
Visualization
The main dashboard for the uptime view shows you a quick overview of your services and their status.
If you select a service, you can see all the pings and the latencies for each.
Alerting
Check out the second part, Alerting downtimes in Slack using Heartbeat and Elasticsearch Watchers.
If you like this post, feel free to follow me on Twitter or leave a comment.
