Fair traffic shaping an ADSL line for a local network using Linux

I originally intended this to be a fully usable script to shape an ADSL line. However, given the amount of development work I have done to improve the script and the associated added complications, I now see this as a good tutorial instead. It can be used in its current form and should work quite well, but for production use I recommend the scripts located in the full web-portal instructions: here

Traffic shaping a standard ADSL link in order to share it with a couple of hundred users is a common problem. There are dozens of bits of software and firewall scripts out there already to do this. This particular script is one that I have written. It aims to be as simple as possible, is easily customised, and uses connlimit to identify P2P users. Although the latter is not 100% reliable, it seems to work pretty well and does not fall foul of any new/changed P2P software that happens to some of the other scripts.

Software required

Any recent Linux distribution should be able to be used for these scripts. The example shown here uses Debian.

The following packages are required:

Kernel 2.6.26

IPset

IPset is not yet in the stable kernel, so the easiest way to install it (in Debian Squeeze) is:

All traffic shaping is hashed by client, not by connection. This means that one user with several large download connections will get the same amount of total bandwidth as one user with one connection. The default within Linux and routers is normally to do it per connection.

ipset is used to generate a list of client IP addresses that are using P2P software.

connlimit is used to detect P2P software. The firewall rules below look for multiple connections to high port numbers from a client.

Assumptions

ppp0 is the internet connection

eth0 is the local network connection

Firewall rules required

I will first describe some of the rules that form the basis of the traffic shaping. Later in this page the full script is shown.

The first task is to create an ipset for storing the IP addresses of all our naughty users slurping up the bandwidth with constantly downloading P2P software. A timeout of 60 seconds is used, so that as soon as they turn off their software their IP address is removed. The current IP addresses in the IPset can be monitored with the 'ipset -L' comand from the bash prompt

# Create a set called p2p with 60 second timeout
ipset -N p2p iptree --timeout 60

Next we need to mark traffic as required, using the principles set out earlier. The following code contains some examples.

Now we need to look out for all those P2P connections. We're going to find these out by looking for a client on the network making lots of connections to high port numbers, which is generally what P2P software does. This isn't foolproof of course: I have seen P2P software start to use port 80, and there could be false negatives, but on the whole it seems to work better than any other solutions out there that I have tried.

The above rules just add the client IP address to the ipset. We now need to mark the traffic, which the following rules do. On one network, traffic became so slow that I marked ALL traffic to and from those clients as '60'. This certainly sped the network up, but of course the entire internet connection for that client became really slow. If you want to do that just remove the destination port parameters.

Shaping the traffic using HTB

Now that we have got to this stage, we have all our traffic nicely marked. All we need to do now is shape it. Unfortunately ingress shaping on Linux is somewhat limited, so instead we only do egress shaping. To limit traffic coming in from the internet, we shape the traffic as it leaves the server. Not ideal, but it seems to work okay.

In these scripts I limit the downlink at 3600 kbps and the uplink at 550 kpbs. These figures *must* be less that your connection is capable of, otherwise no shaping will take place as the buffer at the other end of the ADSL link will do the shaping instead. The generally accepted approach is to set the values to 90% the theoretical maximum, but I advise you to experiment.

Downlink

First, the downlink. Shaping is done on eth0 as the traffic leaves for the network.

We use HTB to do the shaping. It can be difficult to understand HTB and tc in general, but if you look hard enough there is some good documentation out there, it's just hard to find. I will place some links at the end of this document.

First, some basics. A qdisc is a whole set of shaping rules that we apply, normally to a whole interface. Here we add a HTB qdisc to the interface eth0. This is known as the root for the interface.

We don't set a default class; this is so that local eth0 traffic is not shaped. As already stated, we have to shape at eth0 and not ppp0 as we can only do egress shaping decently.

We now have one root HTB qdisc with a rate limit of 3600kbps. To this we add 6 further children. Note the numbers. 1: (or 1:0) is the root. 1:1 is the first child with the overall rate limit. Each child of this is 1:10, 1:20 and so on. To make things simpler, I have numbered the children below to align with the MARK numbers. Note that all the rates of the children should add up to the single rate limit of the parent.

We now have the HTB qdisc fully set up. However, no traffic will be sent to it yet, and the traffic will not be shaped within each class.

The next set of rules shape traffic in each class. If we don't do this, then all the traffic for a particular class (such as all the webbrowsing traffic - 30) will be piled into the class on a fifo basis. We want to be more intelligent than this. SFQ does some nice fair shaping.

So, everything is set up and ready to go. We just need to divert some traffic into each of the classes. We do this by attaching a filter to the class. A filter looks for traffic of a particular type and sucks it into the class. In this example, we use the MARK of the traffic (called flowid here).

Everything will be working nicely at this point. However, we have one more tweak to do. We want to share traffic between clients (by IP address) not by connection. This means that if one client has 4 downloads on the go, and another has only one, that traffic will be split 50/50, as opposed to the first client getting 80%. We do this by applying more filters to the existing ones.