This chapter should provide enough knowledge on networking to enable a systems
administrator to connect a Linux server to a network and troubleshoot basic
network-related problems. First, we will go over the basics of the 7-layer Open
Systems Interconnection (OSI) model, which is a standard framework with which to
describe communication system logical layers. Next, we will delve into each layer of the OSI
model in more detail as it applies to the role of systems administration.

Before any discussion of networking, however, it’s important to have a
working knowledge of the numbered Request for Comments (RFC) documents
and how they apply to computer networking. These documents describe the technical
specifics for every protocol you will run into (e.g., TCP, IP, HTTP, SMTP), and
as such are the authoritative source for how computers communicate
with one another.

Starting in 1969, the RFC document series describes standards for how computers
communicate. The series gets its name from the RFC process, wherein industry
experts publish documents for the community at large and solicit comments on
them. If the Internet community finds errors in a document, a new, revised
version is published. This new version obsoletes the prior versions. Some
documents, such as the document specifying email messages, have had several
revisions.

The RFC Editor manages the RFC archive, as well
as associated standards. New documents go to the RFC Editor for publication
[1], whether revision or new standard. These documents go through a
standardization process, eventually becoming Internet-wide standards. Many of
the networking protocols discussed in later chapters have RFCs governing their
behavior, and each section should provide information on the relevant RFCs.

There are a number of RFCs which don’t pertain to any specific technology but
which are nevertheless seminal. These documents establish procedure or policy
which have shaped everything after, and as such have a timeless quality.
In some cases, later documents make references to them. This list is given in
increasing numerical order, though is not exhaustive.

This document, referenced in many following RFCs, presents a common vocabulary
for specifying the relationship between a standard and implementations of that
standard. It provides keywords that specify how closely an implementation
needs to follow the standard for it to be compliant.

This document provides an overview of the current standards documented by the
RFCs and which RFC is the most recent for each standard. This document is
regularly updated with the current standards document status.

The early history of computer networking was marked by various proprietary
implementations of networking hardware and the protocols that ran on them (such
as for example, IBM’s “Systems Network Architecture” (SNA) and Digital Equipment
Corp’s “DECnet”.) In the mid-1970’s, two standards organizations (the
International Organization for Standardization, known as the “ISO”, and the
International Telecommunications Union, known as the “ITU”) led a push to define
a set of open interconnection standards, which became known as the “Open Systems
Interconnection” (OSI) standard. While the actual protocols they developed did
not become popular and are not in wide use today, the model they came up with
has achieved wide popularity as a way of thinking about the network protocol
stack. It’s important to understand that the OSI model is a conceptual model,
and therefore more of a mental framework than a technical framework. It is useful
primarily for troubleshooting and explanation purposes. While the OSI model may
seem esoteric as you’re learning it, it will eventually prove invaluable to your
daily work once you have a grasp of the concepts.

The OSI model describes seven layers of abstraction that enable software
programs to communicate with each other on separate systems. The seven layers
are designed to allow communication to occur between systems at a given level,
without concern for how the lower levels are implemented. In this way,
innovation on the protocols in the various layers can be achieved without
modifying any other layer’s design and code. The job of each layer is to provide
some service to the layer above by using the services provided by the layer
below.

Layer 1 - Physical layer

The physical layer describes the physical connections between devices. Most
enterprise networks today implement Ethernet at the physical layer, described
in IEEE 802.3 for wired connections and IEEE 802.11 for wireless networks.
Other Layer 1 protocols that are no longer in wide use today are Token-Ring, and FDDI.

Layer 2 - Data link layer

The data link layer defines the basic protocol for communicating between two
points on a network, that may consist of many intermediate devices and cables,
possibly spanning a large geographic area. The IEEE 802.3 specification defines
the data link layer, which includes the Media Access Control (MAC)
addresses that allow hosts to address their data to one or more systems on the
same Ethernet segment. The MAC address is a flat (non-hierarchical) 48-bit
address, and therefore when a node sends out a MAC broadcast frame (usually
written as FF:FF:FF:FF:FF:FF), all stations on the Layer 2 segment receive it.
Therefore, a layer 2 segment is also known as a “broadcast domain”.

Layer 3 - Network layer

The network layer is what allows many flat Layer 2 segments to be
interconnected, and separates broadcast domains. It is this layer of
the OSI model that enables the Internet to exist, using Internet Protocol
(IP) addressing. Version 4 of the Internet Protocol (IPv4), most commonly found in
today’s production networks, is described in RFC 791. Its successor, IP version
6 (IPv6) is described in RFC 2460. Both protocols utilize addresses that are
hierarchical in nature, providing for a network portion of the address space,
and also a host portion within the network.

There are also two control protocols included in this layer; namely, the
“Internet Control Message Protocol” (“ICMP”) as described in RFC 792, and
“Internet Group Management Protocol” (“IGMP”), as described in RFC 1112. The
ICMP protocol provides for Layer 3 device control messaging, and among other
uses, allows small test packets to be sent to a destination for troubleshooting
purposes, such as those used by the ubiquitous ping utility. The IGMP
protocol is used to manage multicast groups, which implements “one-to-many”
packet sending between interested systems.

Layer 4 - Transport layer

The transport layer is where things really start to get interesting for the
systems administrator. It is at the transport layer that the Transmission
Control Protocol (TCP), and the User Datagram Protocol (UDP) are defined.
The TCP and UDP protocols allow data to be sent from one system to another using
simple “socket” APIs that make it just as easy to send text across the globe as it
is to write to a file on a local disk - a technological miracle that is often
taken for granted.

The job of the presentation layer is to handle data encoding and decoding as
required by the application. An example of this function is the Multipurpose
Internet Mail Extensions (MIME) protocol, used to encode things other than
unformatted ASCII text into email messages. Both the session layer and the
presentation layer are often neglected when discussing TCP/IP because many
application-layer protocols implement the functionality of these layers
internally.

Layer 7 - Application layer

The application layer is where most of the interesting work gets done,
standing on the shoulders of the layers below. It is at the application layer
that we see protocols such as Domain Name System (DNS), HyperText Transfer
Protocol (HTTP), Simple Mail Transfer Protocol (SMTP), and Secure SHell
(SSH). The various application-layer protocols are at the core of a good
systems administrator’s knowledge base.

When the ARPAnet project spawned TCP/IP, which became the dominant network
protocol stack used on the Internet, it was envisioned with a much simpler
4-layer stack (the ISO layers roughly map to the ARPA model as indicated below.)
The elements of this stack from the lowest to highest are as follows:

Link Layer

Basically a combination of the ISO Layer 1 (physical) and Layer 2 (data link) layers.
This layer covers the physical network card in the computer (and the transmission
medium between the computers) as well as the device driver in the operating system
of the computer. As an example, both Ethernet and MAC are covered in this layer.

Network Layer

This layer maps to the ISO’s Layer 3, and covers the movement of packets between
networks. For this reason, it is also called the “Internet Layer”, and the main
protocol used in this layer is named the “Internet Protocol”, or “IP” as it is commonly
referred to. As discussed above, the ICMP and IGMP protocols are also included in
this layer.

Transport Layer

This layer maps to the ISO’s Layer 4, and covers the creation, management and teardown
of “virtual circuits” or “flows” between end hosts. There are two different protocols
in use at this layer as discussed above in the ISO Layer 4 section, namely, TCP and
UDP.

Application Layer

This layer maps to the ISO’s Layer 5 through Layer 7, and covers the application
processes that use the network to communicate.

Internet Protocol Version 4 (IPv4) is the fourth version of the Internet
protocol, the first version to be widely deployed. This is the version of the
protocol you’re most likely to encounter, and the default version of the IP
protocol in Linux.

IPv4 uses a 32-bit address space most typically represented in 4 dotted decimal
notation, each octet contains a value between 0-255, and is separated by a dot.
An example address is below:

10.199.0.5

There are several other representations, like dotted hexadecimal, dotted octal,
hexadecimal, decimal, and octal. These are infrequently used, and will be
covered in later sections.

The Internet has experienced tremendous growth from its beginnings in research
at universities to its use in e-commerce and other applications today.
As a result, more and more IPv4 addresses were given out to enable users to connect
to the Internet.
However, despite the number of IPv4 addresses being a large number (4294967296), they
are slowly running out.

Internet Protocol Version 6 (IPv6) is the latest version of the Internet protocol
which aims to address the IPv4 address exhaustion problem.

Probably the most obvious difference of IPv6 to IPv4 is the representation of addresses.
Unlike IPV4, IPv6 uses a hexadecimal format to represent a 128-bit address. The format is
grouped into 8 groups of four hexadecimal digits separated by a colon.
For example:

2001:0DB8:0000:CBAD:4321:0000:0000:1234

Leading zeroes may be left off of a group in the address. Whole groups of zeros may be
left out entirely, but only once. Generally, the longest run of zeros is dropped, but
any run may be. Using these rules, the above address can also be represented in either
of these ways:

Both TCP RFC 793 and UDP RFC 768 provide data transfer between processes
through ports. These process ports can be on the same computer or separate
computers connected by a network. TCP provides the following: reliability,
flow control, and connections (see Example Difference 1 below). UDP is less
feature-rich, it does its work with a header that only contains a source port,
destination port, a length, and a checksum. TCP provides its capabilities
by sending more header data, more packets between ports and performing more
processing. UDP requires less header data in the individual
packets and requires fewer packets on the network to do its work. UDP does no
bookkeeping about the fate of the packets sent from a source. They could be
dropped because of a full buffer at a random router between the source and
destination and UDP wouldn’t account for it in itself (other monitoring systems
can be put in place to do the accounting, however that is beyond the UDP
protocol).

The choice of protocols to use is often based on whether the risk of losing
packets in real-time without immediate alerting is acceptable. In some cases
UDP may be acceptable, such as video or audio streaming where programs can
interpolate over missing packets. However, TCP will be required due to its
reliable delivery guarantee in systems that support banking or healthcare.

Example 1

The TCP protocol requires upfront communication and the UDP protocol does
not. TCP requires an initial connection, known as the “three way handshake”,
in order to begin sending data. That amounts to one initial packet sent
between ports from initiator of the communication to the receiver, then
another packet sent back, and then a final packet sent from the initiator
to the receiver again. All that happens before sending the first byte of
data. In UDP the first packet sent contains the first byte of data.

Example 2

TCP and UDP differ in the size of their packet headers. The TCP header is
20 bytes and the UDP header is 8 bytes. For programs that send a lot of
packets with very little data, the header length can be a large percentage of
overhead data (e.g. games that send small packets about player position and state).

A subnet is a logical division of an IP network, and allows the host system to
identify which other hosts can be reached on the local network. The host system
determines this by the application of a routing prefix. There are two typical
representations of this prefix: a netmask and Classless Inter-Domain Routing (CIDR).

Netmasks typically appear in the dotted decimal notation, with values between
0-255 in each octet. These are applied as bitmasks, and numbers at 255 mean that
this host is not reachable. Netmask can also be referred to as a Subnet Mask and
these terms are often used interchangeably. An example IPv4 address with a typical
netmask is below:

IP Address

Netmask

192.168.1.1

255.255.255.0

CIDR notation is a two-digit representation of this routing prefix. Its value can range
between 0 and 32. While having long been a staple in network equipment and used mainly by
network engineers, CIDR notation can now be found in Linux (iproute’s ip command uses
CIDR notation). In addition, CIDR notation is often encountered in discussions, as it’s
quicker and simpler to reference than a netmask. Below is the same example as above
with CIDR notation:

IP Address

CIDR

192.168.1.1

/24

As a server administrator, it is helpful to understand subnetting so as to communicate more
effectively with network engineers, and to design networks yourself. There are many guides
online for learning how to subnet, each with different approaches and tricks. A full subnetting
guide is out of the scope of this document.

You may hear people refer to networks as “Class A”, “Class B”, or “Class C”. This is known
as “Classful Addressing” and has been deprecated for decades, thanks to the introduction
of CIDR in 1993 (RFC 1519). Classful addressing has the drawback of assuming that a
network is drawn on very large boundaries. For example, in the table in the next section,
each block would be a single network. One can see how using an entire /16 (65534 hosts)
when only five hosts are needed would be wasteful. As such, CIDR was created, allowing
people to create subnets only as large as required. For these reasons, one should not refer
to any particular network block as a “Class X” (eg, 10.0.0.0/28 should not be called a Class A
network) as it is misleading at best, and incorrect at worst. You should always use CIDR
notation to ensure accuracy.

Network Address Translation, or NAT, is a technology that allows multiple
internet-connected devices to share one common public IP address, while
still retaining unique, individual private IP addresses. The distinction
between public and private is integral to the understanding of the service
that NAT provides and how it works.

In our Sample Network we can see that two firewall
machines sit between the Internet and the other hosts in the network; traffic
going in and out of the network will pass through these firewall machines. The
addresses assigned to the firewalls (10.10.10.1 and 10.10.10.2) are private IPs
and are visible to just the hosts within the network. A device on the external
Internet will, instead, see the public IP address for each firewall. It’s
important to note that none of the hosts within the network will have a public
IP address, except for the firewalls and the DNS servers, since they are the
only parts of the network that directly communicate with the external Internet.

When a device behind a NAT-enabled router sends a packet, the source IP address
on the packet is the device’s local, private IP address. If the packet is going
outside the local network, it will pass through a router, which will modify the
source IP address to its own public IP address. When a response for the packet
is received by the router, it needs to ensure that the response can be forwarded
to the host that sent the packet in the first place. To do this, the router
maintains a Translation Table. This table maps a device’s IP address and
port to a port on the router itself. The router’s public IP address and
the correct port number from the table are used as the source IP and port on
the packet and then sent to the destination.

These maps are temporary and exist on a per-connection basis. This means that each
connection opened by a device will have a unique port number on the device and a
unique port number on the router as well. This port on the router is used as the
public port for that connection. Once the connection terminates, the router is
free to assign that port to another connection. However, the total number of
available ports is limited to 65,536, so it is entirely possible that a router
has no more free ports and cannot assign a new NAT address. This is commonly
referred to as port exhaustion.

Similar to port exhaustion, timeouts can also affect the router’s ability to
assign new NAT addresses. Each entry in the translation table has a timeout
value which refers to the amount of time for which that entry can remain
inactive and still keep its place in the table. An entry that has remained
inactive for a period of time longer than the timeout will automatically be
removed, freeing up space for a new one.

The most common type of network cables are what is known as “unshielded twisted
pair” cables. They use 4 sets of twisted pairs of copper, relying on the twist
with differential signaling to prevent noise and signal propagation between the
pairs. The four pairs of twisted copper wires are encased in a plastic sheath.

There are different standards for copper network cables set by the
Telecommunications Industry Association (TIA) and the International Organization
for Standardization (ISO). Both organizations use the same naming convention
(“Category __”) for the components, but unfortunately differ on the naming for
the cable standards. The most common reference is the TIA’s, and the category
designation is usually shortened to “Cat”, so you’ll hear references to “Cat5”
or “Cat6” cable.

Single-mode fiber has a small core diameter, which only allows one (a single)
mode of light to be transmitted through the fiber. Using a single mode of light
completely eliminates the possibility of light dispersion and associated signal
loss, and so is used mainly for long-haul runs, such as the cables that run
between buildings and cities. However, since single-mode fiber can only transmit
one wavelength of light at a time, it typically involves much more expensive
light generation sources (i.e., laser diode transmitters) and is very expensive
to produce.

Multimode fiber has a larger core diameter (either 50u or 62.5u) and can
therefore carry multiple modes (“multimode”) of light, which can be used to
transmit much more information during a given timeslice. The drawback is that
carrying multimode lightwaves causes light dispersion and associated signal
loss, which limits its effective distance. Multimode is a less expensive fiber
optic cable, that is typically useable with lower cost optical components. It is
very common to see it used for building intra-building backbones, and
system/switch to switch applications.

LC and SC connectors are the two most common type of fiber connectors you will
use.

LC stands for “Lucent Connector”, but is also referred to as “Little Connector”.
They are typically used for high-density applications, and are the type of
connector used on SFPs or XFPs. Typically the connector is packaged in a duplex
configuration with each cable side by side, and have a latch mechanism for
locking.

SC stands for “Subscriber Connector”, but are also known as “Square Connector”,
or “Standard Connector”. This is the type of connector typically used in the
telecom industry. They have a larger form factor than the LC connectors, and can
be found in single and duplex configurations. SC connectors have a push/pull
locking mechanism, and because of this, are also colloquially known as
“Stab-and-Click” connectors.

The variety in optical fiber makes for a correspondingly large variety in
optical fiber interface standards. Different interface types will impose
different requirements on the fiber used and the length of the connection.

If optical fiber interfaces were incorporated directly into network equipment,
the number of models made by the manufacturer would have to be multiplied by
the number of interface standards in existence. For this reason, modern network
hardware rarely incorporates such interfaces directly. Instead, pluggable
transceiver modules are used as a layer of indirection between medium-dependent
and medium-independent interfaces. This allows a transceiver slot to be
provided supporting any desired interface standard, whether copper or fiber.
There are some limitations to this, detailed below.

Various module types have been introduced over the years:

Name

Introduced

Speed

Size

GBIC

1995

1 Gb/s

Large

SFP

2001

1 Gb/s

Small

XENPAK

2001

10 Gb/s

Large

XFP

2002

10 Gb/s

Small

SFP+

2006

10 Gb/s

Small

QSFP

2006

40 Gb/s

Small

CFP

2009

100 Gb/s

Large

There are a large number of compatibility issues with such modules. Some
compatibility issues cause problems between two ends of a link; others cause
problems between a module and its host device.

Transceivers are not generally compatible with lower speed versions of the
same standard. A 1000BASE-T Ethernet card can interface with a 10BASE-T card,
but a 1 Gb/s fiber transceiver cannot interface with a 10 or 100 Mb/s
transceiver. In the case of fiber, this is generally due to the different
wavelengths used; but even many copper transceivers do not support lower
speeds, although some do. You should assume that any transceiver will only
support the exact interface for which it is designed unless specified
otherwise.

Modules are only made for the speed targeted by a format. For example, SFP+
modules are only made for 10 Gb/s standards, and not for lower speeds.

Some equipment may accept SFP modules in SFP+ slots, but this is not
universal.

Vendor lock-in is widely practiced. Equipment may refuse to operate with the
modules made by a different manufacturer. Workarounds are generally
available, but this may complicate support or warranty arrangements.

These issues can create pathological cases. Suppose you have two switches which
should be connected to one another. One is connected via a 1 Gb/s transceiver
to fiber. The other only has SFP+ slots. If these slots also support SFP
modules, then a 1 Gb/s SFP transceiver can be used, but if they do not,
interconnection is impossible: all SFP+ modules target 10 Gb/s, and fiber
transceivers do not support lower speeds.