Pages

Tuesday, January 13, 2015

Flannel and Docker on Fedora - Getting Started

Lets set up 3 Fedora servers for the purposes of testing flannel on Fedora. These can be bare metal, VMs (on KVM, VMware, RHEV, etc...). Why do we want to test this? This is to demonstrate setting up the flannel overlay network and confirming connectivity. Specifically, I want to test container connectivity across hosts. I'd like to make sure that container A on host A can talk to container B on host B. I received quite a bit of guidance from Jeremy Eder of breakage.org - Thanks for the tips!

A few setup notes: I haven't looked at this on GCE or AWS. It helps to add the hosts to
/etc/hosts, or have some other DNS solution. In my case, I set up these
VM's in Vagrant on my laptop and modified /etc/hosts.

Check the interfaces on the host now. Notice there is now a flannel.1 interface.

# ip a

Now that fed-master is configured, let's configure the minions (fed-minion{1,2}).

From the minions:

Use curl to check firewall settings from the minion to the master. We need to ensure connectivity to the etcd service.

curl -L http://fed-master:4001/v2/keys/coreos.com/network/config

From the fed-master:

Copy over flannel configuration to the minions.

# for i in 1 2; do scp /etc/sysconfig/flanneld fed-minion$i:/etc/sysconfig/.; done

From master, restart services on the minions.

# for i in 1 2; do ssh root@fed-minion$i systemctl restart flanneld; done
# for i in 1 2; do ssh root@fed-minion$i systemctl enable flanneld; done

From master, check the new interface on the minions.

# for i in 1 2; do ssh root@fed-minion$i ip a l flannel.1; done

From any node in the cluster, check the cluster members by issuing a query to etcd via curl. You should see that three servers have consumed subnets. You can associate those subnets to each server by the MAC address that is listed in the output.

From all nodes, review the /run/flannel/subnet.env file. This file was generated automatically by flannel.

# cat /run/flannel/subnet.env

Configure Docker:

Configure the Docker daemon on each minion. The /usr/lib/systemd/system/docker.service unit file on each minion should look as follows, pay special attention to the items in bold. We are instructing systemd to import and read the /run/flannel/subnet.env file to set up the variables used in the ExecStart key below. Specifically setting the Docker bridge IP and the MTU for flannel.

Check the network on the minion. If Docker fails to load, or the flannel IP is not set correctly, reboot the system. A functioning configuration should look like the following; notice the docker0 and flannel.1 interfaces.

At this point the flannel cluster is set up and we can test it. We have etcd running on the fed-master node and flannel / Docker running on fed-minion{1,2} minions. Next steps are for testing cross-host container communication which will confirm that Docker and flannel are configured properly.

From each minion, pull a Docker image for testing. In our case, we'll use fedora:20.

You should have received a reply. That's it. flannel is set up on the two minions and you have cross host communication. Etcd is set up on the master node. Next step is to overlay the cluster with kubernetes.