Rock solid storage unleashedminutes to build, seconds to boot, a lifetime to store

Saturday, December 4, 2010

Benchmarking EON ZFS NAS performance (Part I)

At some point you may want to know if your EON ZFS storage is performing on par with your hardware's specifications/setup or if it is doing otherwise. Using various (prerequisite) binary kit tools we can identify if network and I/O performance are optimal or working within expectations. We will use the following tools for our testing
Iperf (Mac, Win)
IOMeter [Part II]
In most cases your storage is accessed/utilized across a network using a client PC. So testing will involve a source, destination and the network that connects them. The source will be your client PC and the destination will be your EON ZFS storage. This will help identify what the network is capable of, is it working properly or if it is a bottleneck to your storage.

On the EON ZFS storage (destination), connect 2 terminals using ssh or putty sessions from your client (Win XP in this case). In one terminal, run

iperf -s

In the other terminal, run (substitute your interface name for bge0)

dladm show-link bge0 -s -i 1

On the source or client PC we will run repetitions of below, changing TCP window size

iperf -c IP_of_EON -P 1 -i 1 -w 8k -f m -t 20

The above says, run 1 process thread P=1 for a test duration of t=20secs with interval i=1sec updates and a TCP window size w=8 in kilobytes. The results will be formatted in (m) megabytes.

This will produce an output showing the upper bounds of your network between the source and destination. It may also help identify the optimal TCP window size for your network. The output below will show the different transfer rates with each change of the TCP window size from 8k to 64k, on the client on a gigabit network. So lets begin.

iperf performace fluctuates on the higher tcp window size (56k-64k) for my tests. There'd be times when iperf only shows 785Mbit/s and then the next test will jump to 845Mbit/s. I don't really understand much of network tuning, so I don't really know how it'll affect my connections. I'm happy with getting at least 84% of my gigE connection.

Btw Andre, really doin' a great job here. I've played around with other distros for opensolaris but I've always come back to your EON. Nothing compares when it comes to it's reliability and stableness.

Also, thanks for adding mediatomb. I've installed it and was able to get on the web UI in 10mins! Now I just gotta start loading media files in my storage and fire up my PS3. Good job! and Thank you!

I finally hit a snag with the server. I think this is one of the problems I had before with my NIC card. My post above describes what I get with the rge driver. I was having problems with transferring large files so I changed the driver to gani. I had problems figuring that out before, but now I finally managed to get it working properly. I did the benchmark again, and I am not getting the same results as I had with the rge driver. It doesn't really matter. Because as of now, my transfer is still running, compared to when it was still the rge driver, I would lose my connection within a minute or two. Transfer is half-way thru, and still no hiccup with the connection. Getting a consistent 42-50MB/s on CIFS. I'm a happy camper now!

That's good to hear. The rge driver has always been hit/miss. It's either the opensolaris or gani driver. Good to hear you got it worked out. It would be great and benefit others if you could share a gani driver howto.

I finally got me some Intel PCI GIGe NIC. I've reinstalled everything starting from the beginning. I'm not running any production right now, so it doesn't really matter right now. I haven't tested transfers yet with the Intel. I did have it aggregated as I bought 2 NICs.

As for the gani driver, I had to install a Virtualbox opensolaris. Compilation for the gani driver needs sunstudio12u1 or gcc. So I had to get a full opensolaris box for that. I followed the README on the zip archive. After I got everything set up, I just copied the binary at /kernel/drv/amd64/gani to my eon box. I also copied the add_drive.sh from the zip archive to the eon box and ran it. Not all rge entries are updated so I also edited all the rge entries in /etc/name_to_major and /etc/driver_aliases to point to gani. Add all the proper .backup entries, updimg.sh, then reboot. Everything works after that. Also, don't forget to remove the /etc/hostname.rge and /etc/dhcp.rge entries before running updimg.sh.

As of right now, I'm trying to get napp-it to work on this new build. I'm having trouble with the webserver not executing the perl files. Even test-cgi on the cgi-bin directory won't execute. I get (13)Permission Denied on the error log trying to execute test-cgi. Any ideas?