I have 2 servers (S1, S2) at IP's1.1.1.1 and 1.1.1.2, and I would like to load balance traffic on www.example.com to them. I am thinking of having a machine at 1.1.1.3 that would be the load balancer : the dns example.com would point to 1.1.1.3 and the LB would redirect to 1.1.1.1 or 1.1.1.2.

Question : A web browser client sends a 1Mb file to example.com. Does the file go entirely through LB before going to S1 ? I mean do all the packets travel from the client to LB to S1 ? Or does it happen like that : the web browser asks for example.com, it is returned the IP 1.1.1.3 (LB) by DNS, then for the first packet the LB tells the client "Hey, talk to 1.1.1.1 instead", and so the web browser sends all his http packets to S1 1.1.1.1 and so the LB receives only 0.001% of the total traffic?

One other possibility would be that we assume the request is big in terms of CPU / database access etc and so as a load balancer would not handle the request (and just transfer it) it would still be of some use even though it absorbs all the traffic

I hope things happen as I say but I don't know enough about http protocol to answer ; I saw some commercial hardware that does this that's why I wonder if a pure software solution exists.

I'm not an English native, my question is very basic I think so if it looks complicated don't hesitate to ask me to reformulate :)

The load balancer would get all the traffic. The client has no idea it's communicating to a load balancer rather than a web server itself. Note that it's just passing traffic along so it won't impact resources as much as it would a web server.
–
Nathan CApr 1 '14 at 16:02

@NathanC Ah sh*t. so what's the maximum bandwidth it can get ?
–
ThomasApr 1 '14 at 16:03

4

Your maximum bandwdith is dependent on your connection, the quality of the load balancers and what kind of traffic is being served as well as the responsiveness of the actual servers being serviced by the load balancers.
–
RexApr 1 '14 at 16:04

1

Note that if you don't want to use a load balancer you have the option of having a DNS-based "balancer" by adding 2 DNS A record entries (one for each of your servers) pointing your web site www.example.com to 1.1.1.1 and 1.1.1.2. This will make some traffic go to one server and some to the other but it also has its issues (lack of control, cached DNS entry in users' browsers)
–
LinuxDevOpsApr 1 '14 at 16:40

1

@Thomas - for what it's worth, a common load balancer haproxy can push 10Gb/sec of load balanced data on very modest hardware. Over 100k requests per second.
–
Mark Henderson♦Apr 2 '14 at 2:59

3 Answers
3

There are different kinds of load-balancing. You can have multiple public IPs, which are visible in DNS records. Each of those IPs could point directly to a server. It will then be the client which choose among them, and the client can do fail-over between them. You should not rely too much on the fail-over between servers, if you leave that up to the client.

You can adjust the above scenario by not handing out all your public IPs in all DNS requests. There are multiple reasons for not handing out all your public IPs:

There may just be so many that the DNS reply would become too large.

You may want more control over where load goes.

You may want to direct users to servers that are geographically closer to them.

You may want to stop telling clients about public IPs, which are currently out of service.

The above methods are generally known as DNS based load-balancing.

At the next layer in the chain, your public IPs could be virtual IPs, which can be migrated between different hardware units. Each virtual IP can only be routed to one piece of hardware at a time, so it would not make sense to have many more boxes at this layer than you have public IP addresses.

Such virtual IPs is more often used for availability, they are not very flexible as a load-balancing solution.

At the next layer you can have a conventional load-balancer. The load-balancer receives the requests from the clients and forward them to a pool of servers. All the traffic from the client to the server has to go through the load-balancer, but the processing the load-balancer need to perform can be extremely light.

This layer of load-balancer can operate in two different modes. They can operate either in the conventional proxy mode, in which one TCP connection is used between client and load-balancer and another TCP connection is used between load-balancer and server, or in DSR mode in which TCP connections are terminated on the servers behind the load-balancer.

In proxy mode the load-balancer not only have to handle all the packets from the clients. It also has to handle all the packets from the servers back to the clients. And the load-balancer need a full TCP stack with buffering and retransmissions.

In DSR mode the load-balancer only need a simple connection tracking of each connection from clients. This reduces memory usage significantly on the load-balancer. It also means that packets from the server to the client does not have go through the load-balancer, it is send directly to the client (obviously going through routers on the way). This property is the reason this mode is called Direct Server Return.

The drawback of DSR mode is that the network configuration is a bit more complicated. The packets from the load-balancer to the server cannot rely on just ordinary routing. Since it does not rewrite the destination IP of packets from client to server, it needs to manipulate the destination address at a lower protocol layer to route the packet to the proper server or insert a tunneling layer in order to have a layer on which to put such a destination address.

All of the above methods can be layered in front of each other. That's how you can build a site scaling to hundreds of millions of users.

It just occurred to me that only mentioning proxy and DSR modes for the load-balancer is an incomplete list of modes. There is also NAT mode, which is somewhere between proxy and DSR modes. NAT mode is similar to the proxy mode, but avoids TCP buffers on the load-balancer. All traffic from server to client still passes through the load-balancer in NAT mode, but the load-balancer does not have to buffer it.
–
kasperdApr 1 '14 at 21:52

About what you say for DNS : if you have 1 server in Europe and 1 in Asia and you publish the two ips as public in the DNS, will clients automatically find the one geographically closer to them ?
–
ThomasApr 2 '14 at 8:00

@Thomas You shouldn't count on clients finding the closest IP in that case. They could theoretically do it by sending SYN packets to both at the same time and use the one which responds first. But in order to not cause unnecessary server load, the client should only send one SYN at first and then wait for 100 ms or so before sending another.
–
kasperdApr 2 '14 at 8:55

So how do big websites (google.com, ...) do to make their domain name point to their closest servers ?
–
ThomasApr 2 '14 at 10:34

@Thomas Google use anycast for DNS lookups. This means the DNS server IP address is not just hosted in one place, but on different servers all over the world. Anycast works great for simple query-response protocols over UDP like DNS. But it is not very suitable for TCP. The DNS server has a table mapping ranges of client IP address to preferred server IP addresses. Those tables can be updated within minutes, should there be a need.
–
kasperdApr 2 '14 at 11:32

A load balancer usually works similar to a reverse proxy.
The client initiates the connection with the load balancer and sends its request to it.
The load balancer takes the request and passes it on to the backend node and waits for the reply.
It gets the reply and forwards it to the original client.

So yes, the full traffic will flow over it, both request and responds.