This file describes the ixgb Linux Base Driver for the 10 Gigabit Intel(R)
Network Connection. This driver includes support for Itanium(R)2-based
systems.

For questions related to hardware requirements, refer to the documentation
supplied with your 10 Gigabit adapter. All hardware requirements listed apply
to use with Linux.

The following features are available in this kernel:

Native VLANs

Channel Bonding (teaming)

SNMP

Channel Bonding documentation can be found in the Linux kernel source:
/Documentation/networking/bonding.txt

The driver information previously displayed in the /proc filesystem is not
supported in this release. Alternatively, you can use ethtool (version 1.6
or later), lspci, and iproute2 to obtain the same information.

Instructions on updating ethtool can be found in the section “Additional
Configurations” later in this document.

This value is the number of receive descriptors allocated by the driver.
Increasing this value allows the driver to buffer more incoming packets.
Each descriptor is 16 bytes. A receive buffer is also allocated for
each descriptor and can be either 2048, 4056, 8192, or 16384 bytes,
depending on the MTU setting. When the MTU size is 1500 or less, the
receive buffer size is 2048 bytes. When the MTU is greater than 1500 the
receive buffer size will be either 4056, 8192, or 16384 bytes. The
maximum MTU size is 16114.

This value delays the generation of receive interrupts in units of
0.8192 microseconds. Receive interrupt reduction can improve CPU
efficiency if properly tuned for specific network traffic. Increasing
this value adds extra latency to frame reception and can end up
decreasing the throughput of TCP traffic. If the system is reporting
dropped receives, this value may be set too high, causing the driver to
run out of available receive descriptors.

This value delays the generation of transmit interrupts in units of
0.8192 microseconds. Transmit interrupt reduction can improve CPU
efficiency if properly tuned for specific network traffic. Increasing
this value adds extra latency to frame transmission and can end up
decreasing the throughput of TCP traffic. If this value is set too high,
it will cause the driver to run out of available transmit descriptors.

With the 10 Gigabit server adapters, the default Linux configuration will
very likely limit the total available throughput artificially. There is a set
of configuration changes that, when applied together, will increase the ability
of Linux to transmit and receive data. The following enhancements were
originally acquired from settings published at http://www.spec.org/web99/ for
various submitted results using Linux.

NOTE:

These changes are only suggestions, and serve as a starting point for
tuning your network performance.

The changes are made in three major ways, listed in order of greatest effect:

Use ip link to modify the mtu (maximum transmission unit) and the txqueuelen
parameter.

Use sysctl to modify /proc parameters (essentially kernel tuning)

Use setpci to modify the MMRBC field in PCI-X configuration space to increase
transmit burst lengths on the bus.

NOTE:

setpci modifies the adapter’s configuration registers to allow it to read
up to 4k bytes at a time (for transmits). However, for some systems the
behavior after modifying this register may be undefined (possibly errors of
some kind). A power-cycle, hard reset or explicitly setting the e6 register
back to 22 (setpci -d 8086:1a48 e6.b=22) may be required to get back to a
stable configuration.

If your server does not seem to be able to receive UDP traffic as fast as it
can receive TCP traffic, it could be because Linux, by default, does not set
the network stack buffers as large as they need to be to support high UDP
transfer rates. One way to alleviate this problem is to allow more memory to
be used by the IP stack to store incoming data.

For instance, use the commands:

sysctl -w net.core.rmem_max=262143

and:

sysctl -w net.core.rmem_default=262143

to increase the read buffer memory max and default to 262143 (256k - 1) from
defaults of max=131071 (128k - 1) and default=65535 (64k - 1). These variables
will increase the amount of memory used by the network stack for receives, and
can be increased significantly more if necessary for your application.

Configuring a network driver to load properly when the system is started is
distribution dependent. Typically, the configuration process involves adding
an alias line to /etc/modprobe.conf as well as editing other system startup
scripts and/or configuration files. Many popular Linux distributions ship
with tools to make these changes for you. To learn the proper way to
configure a network device for your system, refer to your distribution
documentation. If during this process you are asked for the driver or module
name, the name for the Linux Base Driver for the Intel 10GbE Family of
Adapters is ixgb.

Link messages will not be displayed to the console if the distribution is
restricting system messages. In order to see network driver link messages on
your console, set dmesg to eight by entering the following:

The driver supports Jumbo Frames for all adapters. Jumbo Frames support is
enabled by changing the MTU to a value larger than the default of 1500.
The maximum value for the MTU is 16114. Use the ip command to
increase the MTU size. For example:

ip li set dev ethx mtu 9000

The maximum MTU setting for Jumbo Frames is 16114. This value coincides
with the maximum Jumbo Frames size of 16128.

Excessive CRC errors may be observed if the Intel(R) PRO/10GbE CX4
Server adapter is connected to a Fujitsu XENPAK CX4 module in a SmartBits
chassis using 15 m/24AWG cable assemblies manufactured by Fujitsu or Leoni.
The CRC errors may be received either by the Intel(R) PRO/10GbE CX4
Server adapter or the SmartBits. If this situation occurs using a different
cable assembly may resolve the issue.

Excessive CRC errors may be observed if the Intel(R) PRO/10GbE CX4 Server
adapter is connected to an HP Procurve 3400cl switch port using short cables
(1 m or shorter). If this situation occurs, using a longer cable may resolve
the issue.

Excessive CRC errors may be observed using Fujitsu 24AWG cable assemblies that
Are 10 m or longer or where using a Leoni 15 m/24AWG cable assembly. The CRC
errors may be received either by the CX4 Server adapter or at the switch. If
this situation occurs, using a different cable assembly may resolve the issue.

Memory allocation failures have been observed on Linux systems with 64 MB
of RAM or less that are running Jumbo Frames. If you are using Jumbo
Frames, your system may require more than the advertised minimum
requirement of 64 MB of system memory.

Degradation in throughput performance may be observed in some Jumbo frames
environments. If this is observed, increasing the application’s socket buffer
size and/or increasing the /proc/sys/net/ipv4/tcp_*mem entry values may help.
See the specific application manual and /usr/src/linux*/Documentation/
networking/ip-sysctl.txt for more details.

Allocating Rx buffers when using Jumbo Frames on 2.6.x kernels may fail if
the available memory is heavily fragmented. This issue may be seen with PCI-X
adapters or with packet split disabled. This can be reduced or eliminated
by changing the amount of available memory for receive buffer allocation, by
increasing /proc/sys/vm/min_free_kbytes.

Due to the default ARP behavior on Linux, it is not possible to have
one system on two IP networks in the same Ethernet broadcast domain
(non-partitioned switch) behave as expected. All Ethernet interfaces
will respond to IP traffic for any IP address assigned to the system.
This results in unbalanced receive traffic.

If you have multiple interfaces in a server, do either of the following:

Turn on ARP filtering by entering:

echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter

Install the interfaces in separate broadcast domains - either in
different switches or in a switch partitioned to VLANs.

Under small packets UDP stress test with 10GbE driver, the Linux system
may drop UDP packets due to the fullness of socket buffers. You may want
to change the driver’s Flow Control variables to the minimum value for
controlling packet reception.