Service discovery in the cloud using Avahi

I’m been writing a provisioning tool for OpenStack
recently, and I’ve put together a demo configuration that installs a
simple cluster consisting of three backend nodes and a front-end http
proxy. I needed a way for the backend servers to discover the ip
address of the frontend server. Since in my target environment
everything would be on the same layer-2 network segment, service
discovery with multicast DNS (mDNS) seemed like the way to go.

Avahi is the canonical mDNS implementation for Linux, and it comes
with command-line tools for interacting with the Avahi server. There
is also a DBUS interface and appropriate Python bindings for taking
advantage of it.

It is relatively simple to publish a service with Avahi; you can
simply drop an XML file into /etc/avahi/services and you’re done.
Discovering services on the client is a little bit more complicated.
Doing it right would involve a chunk of code that interacts with DBUS
in an event-driven environment with lots of callbacks. It seemed like
a big hammer for my little problem, so I ended up parsing the output
of avahi-browse instead. This turns out to be a little tricker than
you might think, since:

The master node might come up after the backend nodes, so we need
to wait for it to publish the service registration.

The registration process on the backends might run before the local
avahi-daemon has started.

The -t flag asks avahi-browse to terminate after receiving “all”
entries, and -f causes avahi-browse to retry a connection to
avahi-daemon rather than failing if the daemon is not available. Pu
tall together, we get:

$ avahi-browse -rptf _http._tcp

We can use awk to get the information we need, and timeout from
GNU coreutils to take care of the situation I encountered in which
avahi-browse would never exit.