Work Related Sun / Solaris Stuff that I pick up from newsgroups and web sites

Friday, October 14, 2005

Effective throughput can I expect on a Gigabit Ethernet link with UltraSparc-3 CPU

As there were many various interesting answers, I will quote all of them.Basically, I could expect near 100 Mbytes/s with Jumbo Frames under SunFire(This does not take the disk backend into account, just memory-to-memorytransfer speaking...).Thank you all.

--------------------------------------------------------------Joe Fletcher's answers:On a V880 8x900 we did some basic tests using ftp which gave us about45MB/s.This put about a 10-15% overhead on the machine (ie it takes about a wholeUltraIII cpu to drive the card in any serious sense). This is dumping datafrom an FC array down separate HBAs to another array volume....Just checked some old results from another site I used to run. Probably notvery interesting to you but on an Alpha ES40 4xEV6 serving a group of Intelclients we managed to get about 80MB/s. The Alpha was linked into a 3COMswitch via gigabit with the clients each on a 100FD port on the same switch.Each client was tranferring a different set of files, some via ftp, somevia the SMB server software (ASU). We could get similar results using twoAlphas with memory filesystems mounted which allowed us to get the storageout of the picture. Not representative of real world particularly but wejust wanted to see how fast it was capable of going. I suspect the filecaching helped quite a lot where the PC clients were concerned.--------------------------------------------------------------Christophe Dupre's answer:What ethernet card do you have in your server ? Sun has at least twochipsets used in gigabit cards: GEM (with interface as ge0) and Cassini (sungigaswift, ce0 interface).The GEM is older and pretty much all the processing is done by the CPU, andthe throughput isn't that great. The Cassini is much better and offload someprocessing (IP CRC and TCP CRC) to the card, yielding much betterthroughput.Note that GEM is only 1000BaseSX, while Cassini does both fiber and copper.What do you use to compute the throughput ? I use iperf and between twoservers (both ultrasparc -2 400MHz, both dual CPU), both having GEM-basedcards connected to a Cisco 4506 switch I get 85Mbit/s for a singleconnection, and an aggregate of 94Mbit/s with about 40% kernel timeaccording to top. This is using an MTU of 1500 (the GEM and Cisco switchdon't do jumbo frames). The TCP Window size was 64KByte.By comparison, iperf runs between a Sun ultraSPARC3 with a Sun gigaswift anda Dell PowerEdge 2650 with a Broadcom 1000TX card connected using the sameCisco catalyst and 48KByte TCP windows yield 480MBit/s.So before upgrading the CPU you should make sure you have a card thatoffloads the CPU like the gigaswift. Next, jumbo frames don't matter much -support is not standardized, not much equipment supports it, and you can getpretty good performance without.I'm not sure how much the CPU speed is needed, though. I'll install agigaswift in an ultraSPARC2 soon, I can tell you the performance differencethen.--------------------------------------------------------------Jason Santos's answer:I would suspect that your bottleneck on the E10K would be the SBUSinterface, not CPU speed. With a gem or GigaSwift PCI card in a 750MHz6800, we get about 60MB/s over NFS with a single thread. Raw UDP or TCPthroughtput would be much higher, although I never tested it.Let me test now, stand by...This is a quick test from a 4x750MHz 6800 to a 4x1200MHz V880 (no networktuning, single thread):ttcp-t: buflen2768, nbuf 48, align384/0, portP01 tcp ->nbmasterttcp-t: socketttcp-t: connectttcp-t: 1073741824 bytes in 23.59 real seconds = 44441.28 KB/sec +++ttcp-t: 1073741824 bytes in 23.16 CPU seconds = 45275.30 KB/cpu secttcp-t: 32768 I/O calls, msec/call = 0.74, calls/sec = 1388.79ttcp-t: 0.1user 23.0sys 0:23real 98% 0i+0d 0maxrss 0+0pf 3756+261cswttcp-t: buffer address 0x74000

The fastest Gigabit transfers I have ever seen were from an IBM x345 (dualIntel Xeon 2.4GHz) over NFS to a NetApp FAS960, I was able to get over100MB/sec, which is 80% of the theoretical max of 125MB/sec.--------------------------------------------------------------Paul Theodoropoulos's answer:Sun's 'Rule of Thumb' from the UltraSPARC II era was that you should have300Mhz of ultraSPARC II horsepower per gigabit adapter. That's 'dedicated'horsepower - if you had one 300Mhz cpu and one gigabit adapter, you'd haveno horsepower to spare for your applications. In practice of course, thegigabit gets throttled down and the horsepower shared. But i would expectapproximately the same performance requirements with ultrasparc III,frankly.--------------------------------------------------------------Alex Madden's answer:http://www.sun.com/blueprints/0203/817-1657.pdf<http://www.sun.com/blueprints/0203/817-1657.pdf>--------------------------------------------------------------JV's answer:

#2) Throughput may depend more on the underlying storage architecture'sability to READ. You will get better with Hardware RAID 0/1 than softwareRAID like Disksuite or VXVM.#3) copper or optical gigE? I use optical, but I just got v240s last monthso I am beginning to experiment with their ce interfaces. #4) On opticalge, with 14 column Veritas stripes, on large-ish dbf files (1.5-2GB),6x336Mhz cpus, I can get 45 MB/sec with 35% sys. I haven't had a chance totune and test my 10-12 cpu UltraS-II (optical) or 2 cpu UltraS-III v240(copper ce) boxes.

--------------------------------------------------------------Tim Chipman's answer:You might want to use " ttcp " utility to test tcp bandwidth throughput. Itis more likely to represent " best case scenario " throughput that isin-keeping with statements like " gig-ether can do 100Mbytes/sec " :-)we did a bit of testing here a while back, and I'm appending the info belowas a general reference, for what use it may be.test boxes were,

Basic question is : What effective throughput can I expect on a GigabitEthernet link with UltraSparc-3 CPU, with or without Jumbo Frame support,with or without multithreaded transfer ?

I ask this because with UltraSparc-2 CPU (E10K) and GE link (without JumboFrame support) we couldn't get more than :- 15 Mbytes/s with monothreaded transfer- 55 Mbytes/s with multithreaded transfer (the best rate was reachedwith 10 threads)

(We measured application throughput, that is to say TCP throughput).

As you see the CPU overhead with 1500 MTU was so high (truss showed 80%kernel), that we had to multithread the transfer to reach the bestthroughput (55 Mbytes/s).Unfortunately we were far from the theoretical limit (100 Mbytes/s ?), evenif there were still CPU resources free (50%), and I can't determine if itwas caused by the small MTU, the poor US-2 throughput or both ?

I think that Jumbo Frame could increase the throughput and lower the CPUoverhead, but how much ?Will the US-3 throughput help much ?Is there any chance to reach the 100 Mbytes/s limit ?

5 Comments:

Make no mistake: Our mission at Tip Top Equities is to sift through the thousands of underperforming companies out there to find the golden needle in the haystack. A stock worthy of your investment. A stock with the potential for big returns. More often than not, the stocks we profile show a significant increase in stock price, sometimes in days, not months or years. We have come across what we feel is one of those rare deals that the public has not heard about yet. Read on to find out more.

Nano Superlattice Technology Inc. (OTCBB Symbol: NSLT) is a nanotechnology company engaged in the coating of tools and components with nano structured PVD coatings for high-tech industries.

Nano utilizes Arc Bond Sputtering and Superlattice technology to apply multi-layers of super-hard elemental coatings on an array of precision products to achieve a variety of physical properties. The application of the coating on industrial products is designed to change their physical properties, improving a product's durability, resistance, chemical and physical characteristics as well as performance. Nano's super-hard alloy coating materials were especially developed for printed circuit board drills in response to special market requirements

The cutting of circuit boards causes severe wear on the cutting edge of drills and routers. With the increased miniaturization of personal electronics devices the dimensions of holes and cut aways are currently less than 0.2 mm. Nano coats tools with an ultra thin coating (only a few nanometers in thickness) of nitrides which can have a hardness of up to half that of diamond. This has proven to increase tool life by almost ten times. Nano plans to continue research and development into these techniques due to the vast application range for this type of nanotechnology

We believe that Nano is a company on the move. With today�s steady move towards miniaturization we feel that Nano is a company with the right product at the right time. It is our opinion that an investment in Nano will produce great returns for our readers.

Online Stock trading, in the New York Stock Exchange, and Toronto Stock Exchange, or any other stock market requires many hours of stock research. Always consult a stock broker for stock prices of penny stocks, and always seek proper free stock advice, as well as read a stock chart. This is not encouragement to buy stock, but merely a possible hot stock pick. Get a live stock market quote, before making a stock investment or participating in the stock market game or buying or selling a stock option.

Are you stuck in a job that is leading you on the path to no where?We can help you obtain a College Degree with classes, books, and examsGet a Genuine College Degree in 2 Weeks!Well now you can get them!

Call this number now 24 hours a day 7 days a week (413) 208-3069

Get these Degrees NOW!!!

BA, BSc, MA, MSc, MBA, PHD,

Within 2 weeks!No Study Required!100% Verifiable

Call this number now 24 hours a day 7 days a week (413) 208-3069

These are real, genuine, They are verifiable and student records andtranscripts are also available. This little known secret has beenkept quiet for years. The opportunity exists due to a legal loopholeallowing some established colleges to award degrees at their discretion.

With all of the attention that this news has been generating, I wouldn't be surprised to see this loophole closed very soon.

Get yours now, you will thank me later!Call this number now (413) 208-3069 We accept calls 24 hours a day 7 days a week.

While you read this, YOU start to BECOME aware of your surroundings, CERTIAN things that you were not aware of such as the temperature of the room, and sounds may make YOU realize you WANT a real college degree.

Call this number now, (413) 208-3069

Get an unexplained feeling of joy, Make it last longer by getting your COLLEGE DEGREE. Just as sure as the sun is coming up tomorrow, these College Degree's come complete with transcripts, and are VERIFIABLE.

You know THAT Corporate America takes advantage of loopholes in the system. ITS now YOUR turn to take advantage of this specific opportunity, Take a second, Get a BETTER FEELING of joy and a better future BY CALLING this number 24 hours a day. (413) 208-3069