Sunday, January 21, 2018

Monitoring Your PiCluster with Elasticsearch and Kibana

PiCluster is
a simple way to manage Docker containers on multiple hosts. It will
work on x86 or ARM CPU architectures that can run Docker and Node.js.
PiCluster 2.3 will be out soon and I wanted to share the exciting new
Elasticsearch integration currently in the dev branch.
Traditionally, PiCluster had the ability to store its log output in
Elasticsearch. Starting in PiCluster 2.3, node metrics such as disk,
cpu, and memory utilization is stored also. With this ability, users can
easily view their host metrics over time.

Requirements

PiCluster

Elasticsearch 6+

Kibana 6+

Requirements

If you want to quickly spin up an Elasticsearch cluster with Kibana in Docker, check this out.

PiCluster Configuration

PiCluster needs to be configured for Elasticsearch so it knows where
to send data. All you need to do is add the following config option in config.json that points to your Elasticsearch endpoint and restart the PiCluster server process:"elasticsearch": "http://my-es-endpoint:9200"

Kibana Configuration

Starting in PiCluster 2.3, there are pre-set graphs available for viewing host metrics (cpu, memory, and disk).

Creating the Index Pattern

Click on Management -> Index Patterns.

Click Create Index.

Under Index Pattern, type picluster-monitoring

Click Next Step

Under Time Filter field name, choose date.

Click Create.

Repeat steps 1-6 for the picluster-logging Index Pattern.

Importing the Graphs

To import the PiCluster Monitoring graphs in Kibana:

Click on Management -> Saved Objects.

Click import.

Navigate to the PiCluster Git directory and choose kibana-graphs.json

Click open.

Repeat steps 1-4 for kibana-dashboard.json

Viewing the Dashboard

Click on Dashboard

Choose PiCluster Monitoring

Viewing the Graphs

Click on Visualize.

Choose CPU, Disk, or Memory.

Viewing the PiCluster Logs

1. Click on Discover.
2. Change the index to picluster-logging

Feel like trying it out? Check out our dev branch to start playing with it now or wait for it to hit our main branch! To learn more about how PiCluster works, Check out our Wiki page.