How to use OpenVswitch with Docker

It is a known fact, that Docker uses linux bridge for container networking, by default.

When docker daemon starts, it creates a linux bridge named docker0, and assigns an IP address from the following range of private IP addresses. The containers gets assigned IP addresses from the same subnet.

The IP address, that gets assigned depends on IP addresses already in use by the docker host. For example, if none of the private IP addresses in the list above is in use by the docker host, then the docker daemon will assign docker0 the IP address 172.17.42.1, and the containers from same subnet (172.17.42.1/16). Note that IP allocation to the containers is handled by the docker daemon itself.

It’s also possible to use a different Linux bridge instead of the default docker0. The docker daemon needs to be started mentioning the same. All containers will then use that bridge for connectivity.

For normal usage Linux bridge is a good choice. However, there are cases, where OpenvSwitch (OVS) might be required instead of Linux bridge. For example, a single Linux bridge can only handle 1024 ports. This limits the scalability of docker as it won’t be possible to create more than 1024 containers, each having a single network interface. Another example is requirement of tunneling mechanism like GRE or VXLAN. But as of this writing, is not natively integrated with docker. There is an issue opened for the same- https://github.com/docker/docker/issues/8952

However, it is possible to use OVS for docker networking. Let’s see how.

First ,we need to understand what happens internally when a docker container is started.

This is what happens behind the scenes:

Creation of veth pair

Attaching one end of veth to the host bridge (docker0), and putting another end in container net namespace.

Rename the veth end-point in container net namespace as ethX.

IP address configuration for the containers

For Linux bridge, the steps are taken care by docker daemon itself, hence there is no user intervention required.

Logically, the relationship looks like the following:

Since OVS is not yet integrated natively with docker, we have to perform the above steps manually.

Let’s go through the steps for a specific scenario of using OVS bridge ovsbr0 for docker containers. The instructions mentioned here are not architecture specific and works for both Intel and Power archs. My setup is a mix of Ubuntu 14.10 and 15.04 on Intel and Power KVM.

Depending on your requirement, you might want to use DHCP. Accordingly, you might be required to setup a DHCP server. For example this is my DHCP server configuration on Ubuntu 15.04 LE (ppc64el) server.

If static IP is used, then you would be required to set an IP for the container ethX interface explicitly. Otherwise, the container image needs to have a DHCP client (like dhclient, udhcpc) to get IP from the DHCP server.

Hope this gives you an idea on how to use OVS with docker.

Beyond few containers, you will soon realize, that the manual steps are too tedious to follow practically. You’ll need some automation.

Fortunately there is a way out, courtesy by @jpetazzo, who has written an excellent script to automate the entire process.

Download the script from https://github.com/jpetazzo/pipework.git

[host]#git clone https://github.com/jpetazzo/pipework.git

The manual set of steps mentioned above boils down to the following single command