Thought Leadership

Load balancers? We don’t need no load balancers! (Part 2)

March 17, 2017

By Ben Gordon

In Part 1 of this blog, we discussed the value of load balancers in a network and the services that they provide at L4 and L7. We also covered the cost of these devices and the relative impact on a CAPEX budget. Here at nToggle, we had to design our network to be massively scalable and cost efficient. We knew we would be processing millions of transactions per second and in order to be cost effective we needed to get creative with our architecture.

When we started designing our network, we knew enough to be dangerous and had some ideas based on an old blog post by CloudFlare. We decided that their approach might work for us so we wanted to get some feedback from someone who actually builds enterprise networks. We had a buddy of ours who is a network engineer (and a Cisco shill) come to our office for beers and whiteboarding, where we presented the approach for a load-balancer-less network. The idea was to use our switches as load balancers by using IPv4 anycast network addressing as described in the blog post that we read. While our friend thought this was a cool idea and recommended taking a hard look at BGP, he promptly finished his beer and said, “good luck with that” and went on his way. It was time to get familiar with BGP and see if it fit our use case.

There are a few good books on BGP that we recommend reading, however, at this point, a 10,000 feet view of BGP and anycast is in order.

When using a traditional L4 or L7 load balancer, network nodes are typically seen as a single IP address by clients, and the load balancers see each network interface having a unique IP address. This unique IP address is advertised to the network by the local network device that acts as the default gateway.

Anycast can be used to advertise the same IP address from multiple nodes across a network. The network routing protocol will decide with which nodes to establish a connection. Anycast lends itself naturally to load balancing and fault tolerance as a cluster of machines can receive traffic on a single public IP address.

BGP is the routing protocol of the internet. The protocol sits on top of TCP and is sometimes referred to as a “path vector protocol”. BGP routes network traffic between domains or autonomous systems. An autonomous system number (ASN) is assigned to globally represent an organization, similar to an IP address block. BGP is traditionally used to connect networks to ISPs/transit providers and peering exchanges. When a network is connected to a BGP peer, a TCP connection is established and routing information is exchanged without the need for discovery, as is such in other routing protocols. Also unlike other routing protocols, BGP does not advertise its entire routing table, instead peers can be configured to provide routing table updates as they converge.

Back to our task at hand, how do we get our network to leverage anycast and BGP to load balance traffic to multiple clusters of servers?

To make BGP work, each server node must run a routing daemon and have a public IP address. The routing daemon is configured to announce a BGP route per server node to our TOR switches which in turn advertise to spine switches then to peering exchanges, edge routers and the Internet. We run Quagga on each server node, as it is the routing daemon used by our switch software vendor Cumulus Networks.

So how does the network balance load to individual clusters of machines? We will discuss this in more detail and the caveats of this approach in my next post.