Traffic Shaping

08/24/2000

Traffic shaping is the general term given to a broad range of techniques
designed to enforce prioritization policies on the transmission of data over
a network link. In this month's column, we'll look at some of the IP traffic
shaping tools available for Linux and a simple example of how to use them.

Most of us will at some time or another have experienced the effects of
network latency and queuing. A common experience is the sometimes frustrating
delay before characters are echoed when ssh- or Telnet-connected to a remote
host. Any of you who have been forced to use a low speed dial-up PPP or SLIP
connection will have seen the effect that a file transfer has on interactive
traffic over the connection. The file transfer easily consumes most of the
link bandwidth, forcing the Telnet or ssh data to be queued, waiting for a
free slot before being transmitted across the link.

This problem occurs because the datagrams containing the file transfer data
are given equal priority on the link to the Telnet or ssh datagrams. No
consideration is given to the type of data contained within the datagram when
queuing it for transmission; queuing is performed on a "First In, First Out" (FIFO) basis, and the datagrams are scheduled for transmission on a "First
Come, First Served" basis. When a new datagram arrives at the queue, it is added to the tail of the queue; when the link bandwidth becomes available, the datagram at the head of the queue is transmitted.

Traffic shaping allows us to implement a specific policy that alters the way
in which data is queued for transmission. Datagrams associated with file
transfers are generally quite large, often MTU sized, while datagrams
associated with interactive sessions like ssh or Telnet are often quite small.
All data takes time to transmit over a network connection; the larger the
datagram, the longer it takes. As a result, if you have to wait for a large
datagram associated with a file transfer to be transmitted before your Telnet
keystroke can be transmitted, you will perceive considerable delay.
Additionally, because file transfers don't need to wait for human input to
continue, you can be sure that at any time you hit a key in your Telnet session,
there are already not one, but a number of datagrams from the file transfer
session sitting in the queue and further compounding the delay. Ultimately, it will
take the same amount of time to transmit all of a set of datagrams across a
network link, no matter what order they are transmitted in, so at some point you
may decide that it is worth trading off a small amount of delay in completing
the file transfer for more normal response times for your interactive sessions
like ssh or Telnet by giving them higher priority.

Implicit here is a key principle:
Traffic shaping affects only data to be transmitted across a link;
it is mostly too late to shape data after it has been received. This has
deployment implications that we'll look at later.

The shaper device

For some time a simple IP shaping mechanism has been supported by the Linux
kernel. This is called the shaper device. The shaper was primarily designed to
limit the amount of bandwidth that nominated data paths could consume. The
shaper device is a virtual network device that is configured with two
important parameters, a bandwidth limit and a physical network device to which
it is attached. The sum total of traffic routed via the shaper device will be
limited to meet the bandwidth cap by simply discarding any datagrams that would
cause the cap to be exceeded.

The shaper device is included in your kernel if you select the
Traffic Shaper (EXPERIMENTAL) option when compiling it.
It may be built as either a loadable module or an inbuilt device. The shaper
uses a special configuration tool called /usr/sbin/shapecfg
and is included in most Linux distributions.

To use the shaper, you first attach a physical network device to the master
shaper device: shaper0. You then configure the shaper rate limit.
Finally you route the traffic you wish to be rate limited via the shaper
network device.

Let's assume we have a simple network that looks something like that depicted
in Figure 1. We have three Ethernet-based IP networks: 192.168.1.0/24, 192.168.2.0/24, and 192.168.3.0/24. Internetworking these we have two routers, unimaginatively named A and B.

Figure 1. A simple network.

Further, imagine that we wish to shape all traffic routed from the
192.168.1 network to the 192.168.3 network to 5 Mbps while allowing all other traffic to be unshaped.

Remembering that shaping applies only in the forward direction, it should be
clear that it makes sense to configure our shaping on the A
router. The network configuration on the A router might look
something like:

This configuration would cause all traffic routed via the shaper0 network device of router A to be rate limited to 5 Mbps. Any data received by the shaper device that would allow it to stay within the set bandwidth limit is sent to the associated physical device for transmission. Any data that would cause the limit to be exceeded is dropped (thrown away).

All traffic routed directly via the Ethernet device is unshaped. In our example any data routed via our default route (most likely destined for the
192.168.2 network!) will be unshaped.

If we wanted to bandwidth limit the data flowing in the reverse direction,
i.e., from the 192.168.3 network to the 192.168.1 network, we would use an almost identical configuration on the B router (remember that shaping works in the forward direction only), substituting
appropriate IP addresses to reflect the data flowing from right to left on our
diagram.

If you carefully design your routing, you can produce a variety of simple
and functional IP traffic shaping designs using the shaper device. Exploiting
the ip policy routing tool, you could for example cause
shaping to be applied only to data sourced from chosen hosts. You can shape
anything you can route.

QoS Packet Scheduling, more sophisticated shaping

While the shaper device is useful in many circumstances, many real-world
applications demand more flexible and sophisticated shaping techniques.
A number of standards exist to implement "Quality of Service" (QoS) in
an IP environment. The best known of these are the "Resource ReSerVation
Protocol" (RSVP) described in RFC-2205, "Integrated Services" described in
RFC-1633, and "Differentiated Services" described in RFC-2475. These each
rely heavily on the ability of routers, especially at the edge of a network,
to be able to shape the traffic flows they are carrying.

Related software

If you're interested in locating some of the related software, you can find it at:

Linux supports a number of the most important QoS mechanisms in quite a flexible and powerful way. Packet classification methods supported include RSVP,
firewall marking, route-based, and Ugly 32-bit key. Some of the supported
scheduling disciplines are class-based queues (CBQ), token bucket flows (TBF),
3-band priority queues, random early drop (RED), stochastic fairness
queueing (SFQ), and Clark-Shenker-Zhang (CSZ). Kernel support for these is
provided in the QoS and/or fair queueing kernel
configuration menu of the Networking options menu. You should
select each of the datagram classifiers and schedulers that you will require.
They may be compiled as loadable modules, and this is probably a good idea.
Each of the classifiers and schedulers may be combined in a multitude of ways
to meet a wide variety of QoS and IP traffic requirements.

The primary tool for configuration of the Linux QoS features is the /sbin/tc (Traffic Control) tool included in the iproute package. To use the tc tool, you must have
support for the Kernel/User netlink socket in your kernel.

This is a very rich and complex topic. There is not enough space in this
column to adequately cover all you will need to know to deploy IP QoS in your
Linux-based networking environment. Fortunately, some good documentation is
available on-line that will help you. Saravanan Radhakrishnan has a published
document entitled the Linux-IP-QoS-HOWTO; this document provides a good description of the way that the various QoS mechanisms have been implemented in Linux and worked examples of how to use the tc tool.

In a future column, I'll present a worked example of a QoS deployment. In the
meantime, have fun exploring the features available and experiment!

Terry Dawson
is the author of a number of network-related HOWTO documents for the Linux Documentation Project, a co-author of the 2nd edition of O'Reilly's Linux Network Administrators Guide, and is an active participant in a number of other Linux projects.