Stack backtraces from the mind of Garrett. Symbolic debugger not included.

Saturday, March 31, 2007

Not All GigE Are Equal

As a consequence of work I've been doing lately since I joined Sun, I've learned some things that folks who care a great deal about performance might like to know.

The most important of these is that not all gigabit cards are created equal. And even among those that are, some of them get preferential treatment at Sun.

One surprise: the gigE device that gets the most preferential treatment is not a Sun branded NIC. In fact, its a device that you can readily find at your local computer retailer.

I speak of bge.

The bge (Broadcom) NIC has some very, very sophisticated logic on it, that Crossbow is going to be able to take advantage of to get you some very nice performance acceleration, plus some greatly added support for QoS and stack virtualization. If you're thinking about a NIC, my first choice would be a Broadcom NIC.

The Cassini (Sun Gigaswift) has many of the same features, but costs more, and is harder to find. And, the Cassini isn't supported by some of the crossbow features -- yet. This issue will of course get resolved, but for the immediate now, your best bet is a broadcom NIC, especially if you want to run Solaris.

The other commodity NICs (RealTek 8169, Intel Pro/G, etc.) are certainly nice enough, but the features on these nics are an incremental update over similar 100 Mbit hardware, and don't hold a candle to the separate hardware rings, advanced classification engines, and similar features present in the broadcom and cassini hardware. (Noteably, these features will be more important with 10G NICs, and devices like Neptune -- Sun's 10G offering, will be featuring them prominently.) And finally, now that 10G andstack virtualizationneed these features, Solaris is going to start taking advantage of them. Some of this is already in Nevada, and more is on the way soon.

I wouldn't be surprised if other high-end NIC developers (Intel? Marvell?) start offering these features in future updates, although I expect some players (such as RealTek) will continue to focus on much simpler (and hence cheaper) devices.