NISTNet: Emulating Networks on Your Own LAN

06/22/2000

The role of the network administrator is broad and varied. While most of our
time is spent with day-to-day operational tasks such as assigning IP addresses
and configuring hosts and applications to use an existing network, we'll
occasionally be called upon to do some more interesting things.

One of the more interesting and rewarding activities of the network
administrator involves the planning and design of a brand-new network
and the selection and configuration of network applications to run across it.
Network design inevitably involves compromise, trading off one characteristic
against another. Common trade-offs are link bandwidth against cost, or
latency against loss, or bandwidth against latency.

Consider the case of a remotely located site that you wish to network-connect
to a central site. A satellite-based connection might offer high bandwidth at
a reasonable cost but offer latency times of over a second or more, compared
to a terrestrial link which might offer lower bandwidth but with latency of
the order of hundreds of milliseconds instead for much the same price. A
dedicated point-to-point link might offer a virtually loss-less connection but
at a higher cost than a frame relay connection that exhibits some datagram
loss.

Definitions

network bandwidth - The volume of data a network can carry over a period of time. Usually expressed in terms of bits, kilobits or megabits per second for network links.

network latency - The total time taken to carry a unit of data from one point on a network to another point on the network. Sometimes called transit delay.

latency variation - The change in network latency from moment to moment. This often results from buffering caused by sudden bursts or lapses of data being carried by the network.

Choosing the best solution requires a detailed understanding of the end-user
requirements and the network applications they will be using. Sometimes the
best or only way of knowing what is the best network design is to actually try
a few different ones and see which works best. Doing that is difficult and expensive
unless you have a way of emulating the behavior of each of the options with
your existing network. This is just one use of NISTNet.

NISTNet

NISTNet is a software package developed by Mark Carson of the North American
National Institute of Standards and Technology (NIST) that allows network
designers, application developers, and network researchers to pretty accurately
emulate the performance conditions presented by a variety of TCP/IP networks
and network paths.

NISTNet replaces the normal Linux IP forwarding mechanism with one that allows
the network administrator to set and control the levels of each of a number of
key network behaviors. These behaviors include datagram loss, datagram
delay, delay variation, and the maximum available bandwidth in the forward
and backward directions. Real networks display each of these characteristics.
Consider the following examples:

Busy or congested IP networks or network links with lots of errors often
result in datagram loss.

Carriage of IP datagrams across low speed or, in the case of satellite,
long distance links results in high latency.

The "bursty" nature of most IP traffic results in varying amounts of
buffering of data occurring for a network connection over time, and this
results in variation in the latency times.

Network technologies such as ADSL and satellite provide more bandwidth in one
direction than the other. Many high bandwidth LANs are connected via lower
bandwidth links.

NISTNet allows each of these conditions to be intentionally introduced into an
otherwise healthy network connection to allow administrators to emulate
larger, more complex networks and observe the resulting behavior of network protocols
and applications.

NISTNet provides both command line and X11-based userspace clients to control
the operation of the kernel code.

Finding, compiling and installing NISTNet

The NISTNet software package is public domain software and so is freely
available for all to use and abuse as they please. The software provides
a number of loadable kernel modules and corresponding userspace applications
to control the behavior of the modules. Kernel patch files are supplied that
provide replacement timer code that allows finer clock resolution, which
is essential for accurate emulation.

The most recent version of the NISTNet software is version 2.0 alpha,
revision 3. This version compiles against the most current 2.0 and 2.2 version
kernels. The NISTNet source code is available from the
NISTNet web site.

The software is fairly easy to compile. No special libraries are required.
There are only two small tricks. First, make sure that your kernel source is
either in the /usr/src/linux directory or that there is a symlink
pointing to where you have your kernel source. There are scripts that automate
the application of the fast-timer patches that assume this to be the case.
Second, be sure to remember to inspect the monitor/Imakefile file and
make sure the OURXAWLIB define is pointing to the version of the
Athena Widgets library you wish to use; the default is probably OK.

To compile the software you should run the Patch.Kernel script to
patch the kernel. Recompile the kernel with the patches applied using the
usual process. Be sure to select both of the new configuration options
presented:

Kernel hacking --->
[*] Fast timer
[*] Packet routine hacking

When the kernel is compiled and installed, return to the top level of the
NISTNet source and run:

make
make install

This will compile both the text and X11 versions of the userspace programs as well as
the new kernel modules, and will install them in fairly sensible locations
on your machine. Debian users note: the kernel modules are installed in
the /lib/modules/misc/, which is probably not what you want. You
can easily move them into the appropriate misc/ subdirectory of the
kernel you've compiled. The installation step will also create some special
device files in the /dev/ directory that provide the interface that
the userspace programs will use to talk to the kernel modules.