Docker Logging with a Tomcat Container with the Native Graylog Driver

Matthew Close

August 28, 2015

Share +

Managing logs for cloud infrastructure can be a daunting task. Fortunately, we have tools like Graylog to help us out. Graylog is useful for storing, searching, and analyzing logs from a multitude of different sources. One common log source is a Tomcat app running in a Docker container. However, getting logs from containers isn't always simple. Oftentimes, you have to spend extra time configuring other services like syslog or logstash to deal with log aggregation.

Thankfully, there is a faster way. The most recent release of Docker, v1.8.0, makes collecting and sending logs to Graylog much easier. Additionally, the logs are significantly enhanced with extra data such as container ID, container command, and configurable tags. These extra fields can be invaluable when troubleshooting an application problem. In this tutorial I will cover how to set up Docker to natively send Tomcat logs to Graylog. Using the ideas explained in this tutorial should help simplify your existing Tomcat container deployments and logging requirements. As an added bonus, the techniques discussed here can also be generalized to other services running in Docker.

Requirements

Docker version: >=1.8.0

Instructions

Install Docker. Docker's native support for Graylog Extended Log Format (GELF) is new, and you will need to install the latest version of Docker for your host OS. For detailed instructions, please consult Docker Supported Installations. I'm using Ubuntu vivid and in order to install the latest version I simply run the following command:

$ curl -sSL https://get.docker.com/ | sh

Create a Java8 Dockerfile. We'll be using Dockerfiles to create two Docker images. The first is strictly Java8. In a later step, we'll use this image to build the Tomcat8 server. I like to create separate directories for each Dockerfile in case any helper scripts are needed during the image creation. First mkdir jdk8-oracle and then cd jdk8-oracle. Place the contents below into a file called Dockerfile. The Dockerfiles in this tutorial are available from GitHub.

Build the image. This step will take a few minutes to complete if you haven't already pulled down the Ubuntu vivid container image.

$ docker build -t mclose/jdk8-oracle .

Create a Tomcat8 image. Similar to above, first mdkir tomcat8 and cd tomcat8. Then place the contents below into a Dockerfile. In order to use the latest version of Tomcat8, I'm installing tomcat manually as opposed to using apt-get which will install an older version. Basically, this Dockerfile creates a Tomcat user, downloads and installs the latest version of Tomcat8, and finally runs Tomcat. In order to keep the container running we tail -F the output of the logs. More importantly, by using the tail command, Tomcat logs are sent to stdout. The GELF log driver then picks up both stdout and stderr and sends them to Graylog. By tailing the catalina.log files we are able to direct Tomcat logs to Graylog. To generalize this to other applications besides Tomcat is simply a matter of tailing your service's log file.

Build the image. This build should be much quicker than the previous one.

$ docker build -t mclose/tomcat8 .

Start a Graylog container. For the purpose of this tutorial, I'm going to use a Docker container to run Graylog. Since Docker is already installed this is the fastest way to test. However, if you already have a Graylog server configured, please feel free to use it. First pull then run the container. In this example, I'm exposing two ports. Port 9000 is used for Graylog administration. Port 12201/udp is used to receive GELF log entries. For more information, see Graylog and Docker.

$ docker run -t -p 9000:9000 -p 12201:12201/udp graylog2/allinone

Login into Graylog.http://localhost:9000. For the Docker container, the user is admin and the password is admin.

Create the GELF UDP input in Graylog. From the main screen select System -> Inputs. In the dropdown on the next screen, select GELF UDP and click Launch New Input.

Configure the GELF UDP input in Graylog. Simply give the input a Title and click the Launch button at the bottom of the window. There is no need to change any of the defaults for this tutorial.

Verify the Graylog input is running. Once launched, the input should now show as running in Graylog.

Start a Tomcat8 container. There are a couple of important things at work in the docker command below: First -p 8080:8080 exposes the default Tomcat port so we can validate it is running. Next --log-driver=gelf selects GELF for logs, --log-opt gelf-address=udp://localhost:12201 sets the destination of the logs, and --log-opt gelf-tag="tomcat8 example" will set a useful tag in the Graylog entries. If you are using your own server, change localhost to the correct IP. For a complete list of Docker GELF options, please see Docker's Configuring Logging Drivers.

Verify Tomcat is running. Tomcat should have successfully started during the previous step. To verify go to http://localhost:8080.

View Tomcat logs in Graylog. You should now see log entries for Tomcat within Graylog.

View log detail. Click on one of the entires and you can see the amount of extra detail you get from the GELF driver. Not only can you see the log entry, but now it includes a wealth of extra data about the Docker container that sent the log.

Conclusion

I hope this tutorial has convinced you that using the new Docker GELF drivers is well worth it. The extra data about Docker containers included in every log will help reduce troubleshooting time. Additionally, your Tomcat deployments should now be easier.