IPv4 Anycast with Linux and Quagga

“DNS is down and nothing is working!” is not something anyone ever wants
to hear at 3am. Virtually every service on a modern network depends
on DNS to function. When DNS goes down, you can't send mail, you can't
get to the Web, you can't do much—hopefully, your coffeemaker is not
Web-enabled! Administrators do a lot of things to mitigate this risk.
The traditional safeguard is to establish multiple DNS servers for a
given site. Each DNS client on the network is configured with each
of those servers' IP addresses. The chances of all of those servers
failing in a catastrophic way are fairly small, so you have a margin
of safety.

On the other hand, many stub resolvers will take only two DNS
servers, making it nearly impossible to have any meaningful geographical
dispersion in your DNS topology. DNS stub resolvers generally use the
first of two configured DNS servers exclusively. Consequently, you end up
with one server taking the entire query load and one idling, waiting for
a failure. Not optimal, but hey, that's the price of redundancy...right?
It doesn't have to be.

DNS redundancy and failover is a classic use case for anycast. Anycast is
the concept of taking one IP address and sharing it between multiple
servers, each unaware of the others. The DNS root nameservers make
extensive use of anycast. There are currently 16 root nameserver
IP addresses, only eight of which make use of anycast. There are
167 servers that respond to those 16 IP addresses.

Of course,
anycast is not limited to DNS. It can be used to provide redundancy
and failover for any number of stateless protocols and applications.
Anycast might sound a little like multicast, but aside from the one-to-many,
IP-to-endpoint relationship, they have very little in common. Multicast
takes packets from one sender and delivers them to multiple endpoints,
all of which subscribe to a single multicast address using a number of
multicast-specific routing technologies. Anycast takes packets from
one sender and delivers those packets to the “closest” of a number of
possible endpoints using nothing more than standard unicast routing.

How Does It All Work?

Let's start with some terminology:

An endpoint (also known as a node) is a server that responds to an
anycast address and, by extension, provides services on that address.

An anycast address is an IP address that has multiple endpoints
associated with it. Anycast addresses can be from any part of the normal
IPv4 address space.

A service address is a unique IP address on a physical device on the
system. Service addresses are used for administrative or monitoring
access to anycast endpoints.

IGP anycast refers to an anycast scheme confined to a single network
(typically a larger network with multiple physical sites). I
cover IGP anycast in this article.

BGP anycast refers to an anycast scheme that spans multiple networks
and can span the entire Internet. The DNS root servers use BGP anycast.

Anycast endpoints participate in whatever internal routing protocol is
being run on your network. All endpoints for a given anycast IP advertise
a host route (also known as a /32) for the anycast IP to the router.
In other words, each endpoint announces that the anycast IP can be
reached through it. Your routers will see the advertisements coming
from the various servers and determine the best path to that IP address.
Therein lies the magic. Because the IP address is advertised from multiple
locations, your router ends up choosing the best path to that IP address,
according to the metric in use by that routing protocol—meaning
either the path with the fewest hops (RIP), the highest bandwidth path
(OSPF) or some other measurement of network goodness. When you send a
request to an anycast IP address, it will be routed to the single server
with the best metric according to the routers between you and the server.

What if that server fails? If the host fails, it will stop sending out
routing advertisements. The routing protocol will notice and remove
that route. Traffic then will flow along the next best path. Now,
the fact that the host is up does not necessarily mean that the service is up.
For that, you need some sort of service monitoring in place and the
capability to remove a host from the anycast scheme on the fly.

Naturally, myriad other details need to be worked out when
designing an anycast scheme. The general concept is pretty simple, and
small implementations are easy to set up. However, no matter what size
implementation you're dealing with, proper IP address architecture is
a must. Your anycast address should be on its own subnet, separate
from any other existing subnets. The anycast subnet must never, ever,
be included in a summary.

Trending Topics

Upcoming Webinar

Getting Started with DevOps - Including New Data on IT Performance from Puppet Labs 2015 State of DevOps Report

August 27, 2015
12:00 PM CDT

DevOps represents a profound change from the way most IT departments have traditionally worked: from siloed teams and high-anxiety releases to everyone collaborating on uneventful and more frequent releases of higher-quality code. It doesn't matter how large or small an organization is, or even whether it's historically slow moving or risk averse — there are ways to adopt DevOps sanely, and get measurable results in just weeks.