Actuator

Spring Actuator does most of the heavy lifting when it comes to instrumentation by providing a number features to help manage applications in production. Actuator features are exposed as Endpoints and it ships with quite a few out of the box. Installing Actuator is as simple as adding the Actuator Spring Boot Starter to the build.gradle file:

Prometheus

Prometheus is an open-source time series database that provides considerable capability in analyzing application metrics over time. Although prometheus provides many ways to integrate with it, the simplest way is configuring it to periodically poll the target application to gather metrics. Spring Actuator already provides a metrics endpoint at http://localhost:8080/metrics but it is not in a format that Prometheus can consume. Fortunately Spring makes it easy to add custom endpoints and this post will cover how to bridge the gap between Actuator and Prometheus.

Install the Prometheus client JAR by adding it to the build.gradle file:

Gauges & Counters

Metrics fall broadly into two main categories, counters and gauges. These metrics exist in both Actuator and Prometheus.Counters are cumulative metrics that represent a single numerical value which only ever increases. For example, it can be utilized to count the number HTTP 500 responses generated by the application.Gauges on the other hand are metrics that represent a single numerical value which can arbitrarily increase and decrease. They are typically used to measure things like average response time.

Parts of an Actuator Endpoint

Basically Spring Actuator provides three extension points for custom Endpoints as illustrated by the yellow classes. The core of the extension is the AbstractEndpoint, which is used to expose useful information about the application. AbstractEndpointMvcAdapter adapts Endpoints to the MVC layer thereby allowing access to Endpoints over HTTP. The first two mentioned classes are concerned with exposing information, the MetricWriter is used to collect the information. MetricWriters are periodically notified of snapshot Metric values, which provide the ideal opportunity to export the metrics. The red CollectorRegistry is provided by Prometheus client API and is used to register and update gauges and counters for Prometheus. To allow Prometheus to understand Actuator Metrics we need to implement the green classes.

PrometheusMetricWriter

The PrometheusMetricWriter is periodically called with metric updates and it in turn simply updates the corresponding Counter or Gauge in the PrometheusCollectorRegistry. It also takes care of creating the Counter or Gauge if it does not yet exist in the CollectorRegistry. Unfortunately Prometheus only allows certain characters to be used for metric names, sanitizeName takes care of converting the Actuator metric name into a format Prometheus will accept. Consider your metric naming convention when planning to monitor a large number of metrics.

Limitations

At the time of writing, I found that there is no obvious way to get Actuator to report distinct metrics for every HTTP method supported by @RepositoryRestResource. It only captures the gauge.response.repository metric, which is a mixture of GET, POST, PUT and DELETE response times even though these HTTP methods will likely have different performance characteristics.