Search

How to Configure Logstash to handle Glassfish’s server logs

Logstash is a nice tool for managing events and logs. After you finished the setup, the next step is the configuration of the agents/shippers. Since one of the strengths of logstash is the ability to query your logs and make some nice statistics about them, it is key to ‘convert’ your logs somehow in a more structured way. Here’s a little example on how to do this…
The configuration of a shipper is done through a config file (ex. logstash.conf). Below is an example to handle Glassfish’s server log.

Some remarks

Multiline filter

By default Logstash will treat every new line from your log as a new event. Since events often consists of multiple lines you have to tell Logstash how to group multiple lines into a single event. For Glassfish’s server logs the Logstash multiline filter from my example will start a new event for every line starting with [#| + 4 digits. You can easily test this here: http://rubular.com

Grok that log

One of the basic things you’ll do with your logs is applying a grok filter to them. This process is all about parsing the arbitrary text and structure it.
Here are two grok examples based on the above config.

A Little about SLF4J and Logback

Under the hood my java application is using SLF4J and Logback. Although Logback is not the most common choice, it has a nice advantage compared to other logging frameworks: currently it is the only framework implementing SLF4J Markers. Since my application contains both functional and technical log messages it was a requirement to have both types of messages marked properly in logstash. Although there are multiple solutions to handle this, I solved it through the use of an SLF4J Marker. Here’s an example.