ELK Stack for Kubernetes Metrics

Part of the work of the Platform Services team at Alfresco is around deployment, specifically Kubernetes.

Monitoring a Kubernetes (K8s) cluster is critical in production and there are many great tools out there like Prometheus and Grafana, but I wanted to ‘play’ with getting K8s metrics into the Elasticsearch, Logstash, and Kibana (ELK) stack, all deployed in my local Docker Desktop Kubernetes environment.

I found several resources on various parts of the solution, but didn’t find one that tied all of it together, so hopefully this post will be of use to someone.

Prerequisites

1. Deploy the ELK Stack

Helm provides excellent package management of K8s and is always my first choice for deployment so I started there for the ELK stack, and sure enough, there’s a Helm chart for that (elastic-stack).

You can easily override/merge the default values of a chart with your own values file so I created one that enabled filebeat and pointed it to logstash and the elasticsearch client and pointed logstash to the elasticsearch client:

4. Deploy Metricbeat

Metricbeat collects (easy guess) metrics and, through its Kubenetes modules, can get data from K8s nodes and the API. Sure enough, (you know where this is going…) there’s a chart for that (metricbeat).

In this case we have to override values to:

Use the open source version of the metricbeat Docker image (metricbeat-oss)