Give Codeship’s CI/CD Platform a Try

Want to learn more?

In the first blogpost of this two part series, we talked about setting up our Elastic Beanstalk application with CloudFormation. Additionally we configured our application to send logs to CloudWatch Logs.

Elastic Beanstalk exposes many different metrics through CloudWatch, but in addition to ELB or usage metrics, we also want to be able to add our own custom metrics. A great way to do this is through your log statement and configuring CloudWatch to parse your log and turn it into metrics.

There are many use cases for custom metrics. At Codeship, for example, an important metric is all of the incoming webhooks that trigger builds. We want to know as early as possible if there is an issue either in our system or with a partner that results in builds not coming in correctly.

Writing a log message every time a hook comes in and then turning those log messages into a metric allows us to get an alarm whenever the number of hooks that are coming in gets too low.

CloudWatch even allows you to turn numbers in your log line into graphs, so they actually parse your log and don’t just count when a log statement occurs. You can find more about their very flexible system in their examples, specifically in their Bytes Transferred Example

In this post, we’re going to set up a MetricFilter that turns “Hello World” from our logs into a metric. We’ll then set up an alert that sends an email through SNS whenever the Hello World log messages happens more than five times in a minute.

Metric Filter

To begin, we need to filter the Hello World log message that’s written to the log by the Sinatra application whenever we open up the website. Following is a snippet from the CloudFormation template that transforms “Hello World” into a metric value of 1. If it gets called several times, it will be summed up accordingly.

The metric gets a name and namespace to be easily identified. You can take a look at the metric filter documentation for all the details on a metric filter.

Or if you want to create your own logic to fix issues, you could trigger Lambda functions that will react on a specific problem in your infrastructure. Take a look at the Lambda through SNS documentation to get an idea about how to set this up.

It uses the same Namespace and Metric Name we’ve set before. Whenever the sum of all Hello World log statements is over five in one minute, the alarm will be triggered and, as an Action, sends the message to the SNS topic. Here’s a sample of the email you will receive once this alarm happens:

Conclusion

CloudFormation, Elastic Beanstalk, and CloudWatch are an incredibly powerful combination. You can build complex infrastructure, completely automate it, and make sure it handles outages and issues automatically while notifying you easily.

Let us know in the comments what other tools and services you’re using to manage your AWS infrastructure.

Subscribe via Email

Over 60,000 people from companies like Netflix, Apple, Spotify and O'Reilly are reading our articles. Subscribe to receive a weekly newsletter with articles around Continuous Integration, Docker, and software development best practices.

We promise that we won't spam you. You can unsubscribe any time.

Join the Discussion

Leave us some comments on what you think about this topic or if you like to add something.