Give Codeship a try

Want to learn more?

Docker Compose is a great utility for anyone developing Dockerized applications. It’s a tool that I personally use daily. Recently I came across yet another powerful feature of Docker Compose: the ability to change Linux Kernel Parameters.

In today’s article, we will explore how to use this often overlooked but “useful when you need it” feature of Docker Compose.

To get started however, let’s first create a Redis service using Docker Compose.

Starting with a Simple Redis Service

Docker Compose is a tool that allows users to create Dockerized services with a simple YAML file. To get a better understanding of how this works, let’s go ahead and create an example docker-compose.yml.

version: '3'
services:
redis:
image: redis:latest

In the above example, we have a single “service” named redis. This service definition is how Docker Compose allows users to define different services. Docker Compose will take these services and run them within Docker containers.

In the example above, the redis service is provided by a container using the redis:latest image. To start this service, we can simply execute the docker-compose command followed by the up parameter.

$ docker-compose up
Creating network "rediscomposeexample_default" with the default driver
Creating rediscomposeexample_redis_1 ...
Creating rediscomposeexample_redis_1 ... done
Attaching to rediscomposeexample_redis_1
redis_1 | 1:C 12 Oct 03:57:02.915 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1 | 1:M 12 Oct 03:57:02.937 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1 | 1:M 12 Oct 03:57:02.937 # Server started, Redis version 3.2.6
redis_1 | 1:M 12 Oct 03:57:02.939 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1 | 1:M 12 Oct 03:57:02.940 * The server is now ready to accept connections on port 6379

In the above output, we can see that the docker-compose up command created a single container, named rediscomposeexample_redis_1.

As far as creating a single Redis container with Docker Compose, that is it. With only a few lines within a docker-compose.yml file and a single command, we have a running Redis container. However, this example doesn’t show the power of Docker Compose.

To get a better idea of how Docker Compose is useful, let’s add another container into the mix.

Multi-service Docker Compose

Since our first example used a Redis service, let’s make our next container a bit more interesting. We will go ahead and add a Redis Commander service to our docker-compose.yml file.

Redis Commander is an application that allows users to explore a Redis instance through a browser. What this means is not only do we have to have an instance of Redis Commander, we also need to be able to connect it to our redis service.

In the above example, we can see there are two Docker containers running: a redis container as well as a redis-commander container.

This is a much better example of why Docker Compose is a powerful tool. In this example, we quickly specified multiple services within a single YAML file. We then turn those services into linked Docker containers with a single command.

We can also copy this docker-compose.yml file to any server and start up these same services. So not only is Docker Compose useful for local development, it can also be useful for deployment.

!Sign up for a free Codeship Account

Modifying Kernel Parameters with Docker Compose

Now that we have a better idea of what Docker Composes is and how it works, let’s get to an uncommon feature: using Docker Compose to change Linux Kernel Parameters of our services.

If we look back at the output from our above example when we started the redis service, we can see a couple of warnings. One of those warnings is in regards to the somaxconn setting.

redis_1 | 1:M 16 Oct 20:56:51.618 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.

The somaxconn parameter is a Linux Kernel Parameter that specifies the maximum backlogged TCP/IP sockets. This parameter is a setting in Linux that by default is set to 128. This means that the kernel will only allow 128 connections to be “backlogged” at a time.

For highly accessed services like Redis, this may not be enough. The error above shows that Redis was attempting to use a value of 511.

Since the container’s somaxconn value is the default of 128, Redis was not able to use a value of 511.

In a non-containerized world, changing this kernel parameter would be as simple as placing the following into the /etc/sysctl.conf file.

net.core.somaxconn=1024

After adding the above, you would simply execute sysctl -p. However, in the container world this is a bit different.

We could perform the same tasks via a Dockerfile. However in our example, we are using public images from DockerHub. A simpler approach would be to specify this setting within the docker-compose.yml file itself.

In the above example, we executed docker-compose up. However, we also added redis to the command. This is a way of telling Docker Compose to only bring up the redis service.

If we look at the above, we can see that the warning around the somaxconn value is gone. This means with two simple lines, we were successful in changing the Redis container’s somaxconn kernel parameter.

Summary

In today’s articl, we went through a little refresher on Docker Compose. We explored how it is useful for launching multiple connected containers. Finally, we also learned an often overlooked feature, one we can use to easily change the Linux Kernel Parameters within our containers.

Subscribe via Email

Over 60,000 people from companies like Netflix, Apple, Spotify and O'Reilly are reading our articles. Subscribe to receive a weekly newsletter with articles around Continuous Integration, Docker, and software development best practices.

We promise that we won't spam you. You can unsubscribe any time.

Join the Discussion

Leave us some comments on what you think about this topic or if you like to add something.