This article describes how to get the greatest benefits from your Ethernet NIC interface. It includes information on the tools that can help you achieve the best results from that interface, as well as a section on troubleshooting. This article targets an advanced reader.

Like this article? We recommend

Like this article? We recommend

This article describes how to get the greatest benefits from your Sun Gigabit
Ethernet network interface card (NIC) interface and a few valuable tools to help
you achieve that.

Gigabit Ethernet connections create the greatest stress by far to Sun
systems. Therefore, to get the maximum benefit from your gigabit Ethernet NIC
interface you need to be aware of the added complications of Auto-negotiation as
well as the new ways to ensure that you get the maximum performance from both
the gigabit Ethernet interface and the system.

There are two parts to getting the maximum performance from your gigabit
Ethernet NIC and the system: first, you need to understand the system itself;
second, you need to know the traffic profile through the gigabit Ethernet
NIC.

Two key parameters to Sun systems are important for maximizing gigabit
Ethernet performance: the number of CPUs in the system and the access time for
memory. Establishing the number of CPUs is relatively simple. The memory access
time is often hidden, but a simple rule is the larger the system the longer the
memory access time. These factors become important for tuning transmitting (Tx)
DMA thresholds and deciding how much load balancing of incoming receiving (Rx)
traffic is meaningful.

The traffic profile has many dimensions also, including any one of the
following characteristics or any combination of them: Rx intensive, Tx intensive
or Equal, small packets, large packets, or latency sensitive.

The combination of system parameters and the traffic profile makes it very
difficult to enumerate all the possibilities and provide one set of tuning
parameters that will address every combination equally and fairly.

Therefore, we can only take the alternative approach of listing the readily
available tunable parameters along with an explanation of how and when to use
them to get the best results based on your system and application needs.

Each NIC has kernel statistics that provide a means of measuring the traffic
profile. You can use this information to adjust ndd and /etc/system parameters
to get the best performance from the NIC.

For more sophisticated features like CPU load balancing, there are some other
tools that allow you to look at the system behavior and determine if tuning can
better utilize the system as well as the NIC, given the system and the
application providing the traffic profile.

"Network Driver Configuration Parameters" describes the details of the three methods you can use for configuring the
driver parameters.

"Ethernet Physical Layer Troubleshooting" discusses the physical layer because that layer is the most important with
respect to creating the link between two systems.

The ndd method is excellent for adjusting parameters during normal operation,
but the configuration is lost once the system is rebooted. You can avoid this
configuration loss by applying the chosen parameter in the driver.conf file of
the driver you want to configure.

The driver.conf file for the device being configured must reside next to the
driver being configured in the file system. For example, the Gigabit Ethernet
driver has the following path:

/kernel/drv/ge
/kernel/drv/ge.conf

The following example shows the path for the GigaSwift driver in two
different platforms:

Modifying the parameters in the driver.conf file can be done with two goals
in mind: configuring parameters in a global manner, where all interface
instances in the machine using the same driver get the same parameter value, or
on a per instance basis, where a parameter value applies to only one
instance.

The global configuration method for the ge.conf file will appear as follows,
applying the ndd configuration previously shown:

adv_autoneg_cap = 1;

Note that the previously shown ndd example only applied instance 1, so the
global configuration may be overkill. Determine whether per instance or the
global method is more appropriate for your needs.

The per instance method does require you to get the 'parent' and
'unit-address' properties associated with the instance your
configuring. This can be achieved by looking at the lines associated with that
instance in the path_to_inst file.

The /etc/system configuration method allows you to initialize global
variables in the device driver. It has no direct association with ndd and
driver.conf setting unless explicitly implemented in the driver. In cases where
a driver parameter has been defined for use in either /etc/system or
driver.conf, you should choose to use the preferred driver.conf method
instead.

Parameters set in the /etc/system always require a system reboot to take
effect, the following example shows how an /etc/system variable is set up.

hostname# vi /etc/system
...
set ge:ge_intr_mode = 1
....

The remainder of the document will discuss ndd, so you should assume that any
of the following parameters described as an ndd parameter should only be
initialized using the driver.conf file. Any of following parameters described as
/etc/system parameters get initialized using only modifications to /etc/system
parameters and do require a reboot.