> I'm having trouble with getting the right performance out of my software> raid 5 system. I've installed Red Hat 9.0 with kernel 2.4.20 compiled> myself to match my harware (had the same problem with the default> kernel). When I test the raid device's speed using 'hdparm -Tt /dev/hdx'> I get this:>> /dev/md0:> Timing buffer-cache reads: 128 MB in 1.14 seconds =112.28 MB/sec> Timing buffered disk reads: 64 MB in 2.39 seconds = 26.78 MB/sec>> This is really low for a raid system and I can't figure out what is> causing this. I use rounded cables but I also tried the original Promise> cables and there was no difference in performance.

The ONLY reason that I can think of to use round cables would be forlooks. From a performance or reliability standpoint, they are a waste ofmoney. I routinely build systems with dual 8-channel IDE RAID cards(3Ware 7500-8) and 16 disks, and ONLY use flat cables.

Your network probably belongs in this list somewhere, but you didn'tprovide enough information for me to determine where it belongs. Thenecessary information would be the network type, switch/hub, and the typeand quantity of processing being performed prior to putting the data onthe net.

> 4 of the raid HD's are connected to the Promise controller and the other> raid HD is master on the second onboard IDE channel (udma2/ATA33)

I've NEVER heard that the FastTrak runs fast (I have been told that theyrun VERY slowly). I've benchmarked RAW disk speeds with an Ultra-100 andWD1200JB drives, and gotten 50MB/S from each disk, as opposed to your26MB/S (there should be almost no difference for the BB drives). Mysuggestion would be to remove the FastTrak, and get two Ultra-100s and anUltra-133 (or one Ultra-100 and two Ultra-133s). Use one disk per channel(Master only), and move the system disk onto one of the Promise cards.This change should get you up to the performance of your friend's oldsystem.

If you switch to a dual bus system, make sure that the two controllerswith two RAID drives are on separate buses. In a triple bus system, eachof the three controllers would be on a different bus (put the NIC with thesystem drive).

An inexpensive P4 Celeron with FAST memory would probably run faster.FYI, The primary reason that the large file servers use the dual Xeonboards is to get the high memory and I/O bandwidth. The high CPUbandwidth is also useful, primarily with TCP/IP encapsulation.

<SNIP>

> As you see /dev/hdc is on the onboard IDE channel. Tos be certain that> this was not the bottleneck, I removed it from the raid so that is runs> in degraded mode. This did not change the performance much. Rebuild> speed is laso very slow, it is round 6MB/sec.>> The drives originally came from a friend's file server where they were> also employed in a raid configuration. I've compared my results in> Bonny++ to his results:>> My raid:

<SNIP>

14MB/S block write, 30MB/S block read

> His raid:

<SNIP>

99MB/S block write, 178MB/S block read

This MUST have had a better disk controller system AND multiple PCI buses.The limit of one PCI bus (32-bit/33MHz, as used by the Promisecontrollers) is 133MB/S, and they are exceeding that.

> His machine has dual P Xeon 2000Mhz processors but that shouldn't be the> reason that the results are so different. My processor isn't at 100%> while testing. Even his older system which had 80gig drives and dual> 500Mhz processors is much faster:

<SNIP>

35MB/S block write, 72MB/S block read

As mentioned above, the CPU isn't the primary limit in I/O. You have toconsider I/O bandwidth (number, width and clock of PCI buses), and memorybandwidth.