SignalFx is the only real-time cloud monitoring platform for infrastructure, microservices, and applications. The platform collects metrics and traces across every component in your cloud environment, replacing traditional point tools with a single integrated solution that works across the stack.

This article explains how to send Spring Boot and Netflix Servo metrics to AWS CloudWatch. Moreover, it describes mechanisms for making it happen. It also mentions problems I run into trying to do this with Spring Boot and Spectator.

If you are not interested in how and why and just want to make Spring Boot work with AWS CloudWatch, do the following:

To find providers for presented metrics, check org.springframework.boot.actuate.endpoint.PublicMetricsinterface and its class hierarchy.

CounterService & GaugeService

Of the PublicMetrics providers, MetricReaderPublicMetrics is special as it reads all metrics registered by CounterService and GaugeService. CounterService - registers tracks just increasing a counter (so can be used for number of requests, pages visits, etc).

GaugeService can store any value (it’s not incremental, just set), so its application may vary from measuring time to showing info about threads, connections, etc.

counter.status.200 means number of hits to this particular endpoint with response code = 200

gauge.response - is the last response time

The inner workings are simple. With spring-boot-starter-actuator by MetricFilterAutoConfiguration we added MetricsFilter, which basically is javax.servlet.Filter submiting metrics by CounterService and CounterServicefor every request.

CounterService and GaugeService may also be used to create metrics on our own, like in:

CounterService and GaugeService are interfaces, and by default, they use in-memory objects (CounterBuffers, GaugeBuffers) to store metrics.

To send it to the outside world, MetricExporters are used, which in a separate thread (MetricExporters.ExportRunner) will send the counter and gauge metrics wherever we want. MetricExportAutoConfiguration tells us that to make it work, at least one MetricWriter needs to be present.

None is registered but default, but even with spring-boot-starter-actuator we have the components to write metrics as MBeans, send them to StatsD (front-end proxy for the Graphite/Carbon metrics server), or store them in Redis (check different implementations of org.springframework.boot.actuate.metrics.writer.MetricWriter).

Nothing for CloudWatch here, but we can easily get there.

Sending Metrics to AWS CloudWatch

Now, to be able to write our metrics to CloudWatch, we need CloudWatchMetricWriter, and we get it by adding

we will see that one more piece is missing: cloud.aws.cloudwatch.namespace, so let’s set it in application.properties

cloud.aws.cloudwatch.namespace=m3trics

Now the application can be deployed to the AWS Cloud, or run locally if we have the proper AWS credentials configured (for different types of providing AWS credentials look at DefaultAWSCredentialsProviderChainclass). Moreover, if you are running the app locally, also add the following property:

cloud.aws.region.static=us-east-1

(or another region that you use), as Spring Cloud won’t be able to figure it out if not running in an EC2 instance.

Now access different endpoints in the app and, with few seconds delay, you will be able to observe our counter/gauge metrics in AWS CloudWatch.

Log to the AWS Console, and go to CloudWatch -> m3trics (which we set as the namespace).

Registered Metrics namespace in CloudWatch.

Spring Boot basic metrics in CloudWatch.

Netflix Servo Metrics

All of this is, of course, only half of the story. With Spring Cloud, we will use a lot of Netflix libraries, and these do not use Spring Cloud’s CounterService/GaugeService but generate metrics on their own using Netflix Servo and Spectator collection libraries (you will find some info about these here, the main point being that Spectator is newer and should replace the old Servo).

Let’s try with Hystrix and add this dependency to our project:

compile('org.springframework.cloud:spring-cloud-starter-hystrix')

With Hystrix, we also got the spring-cloud-netfix-core library and inside we find MetricsInterceptorConfiguration, which is loaded conditionally on servo package present, so let’s add it:

compile('com.netflix.servo:servo-core')

We want to gather metrics from RestTemplate as well, and MetricsInterceptorConfiguration specifies that this one is dependent on aspectjweaver, so let’s add it as well.

compile('org.aspectj:aspectjweaver')

With ServoMetricsAutoConfiguration we got ServoMetricServices, which is both CounterService and GaugeService implementation. So from now on, all gathered servo metrics will be sent to CloudWatch!

Let’s see some Hystrix in action. Just enable it by @EnableCircuitBreaker, and create some Hystrix traffic, like this:

Add Spectator as a dependency and ServoMetricsAutoConfiguration will no longer be loaded (@ConditionalOnMissingClass(“com.netflix.spectator.api.Registry”)), so no ServoMetricReader will be registered, so MetricsExporter will not work (there is SpectatorMetricReader in spring-cloud-netflix-spectator library, but for some reason, it’s not registered by SpectatorMetricsAutoConfiguration).

At the time of this writing, the newest Spring Cloud library is Finchley.M2 (and Finchley is the only version working with Spring Boot 2.0) - so maybe everything will work with the GA version.

SignalFx is built on a massively scalable streaming architecture that applies advanced predictive analytics for real-time problem detection. With its NoSample™ distributed tracing capabilities, SignalFx reliably monitors all transactions across microservices, accurately identifying all anomalies. And through data-science-powered directed troubleshooting SignalFx guides the operator to find the root cause of issues in seconds.