4 Answers
4

ZFS works optimally if it knows as much as possible about the underlying physical disk structure. A hardware RAID 1 will present a single volume to the OS which ZFS will then treat as a single disk. You'll therefore miss out on the benefits of error correction / fault detection / checksumming etc.

There's another benefit to using ZFS over HW RAID - if your hardware dies, be it the motherboard, RAID card, CPU etc, you can pull the disks and use them in any other system that is running ZFS, and the entire filesystem is readable. If you use HW RAID, you may have a dependency on that specific RAID card, so if it fails you might need to find a similar model before you can read the disks again. You can mitigate this by having dual RAID cards, but this usually starts to become expensive and complex.

In my experience, for the majority of use-cases that I've used a mirrored disk configuration, the performance difference between ZFS and HW RAID wasn't great. If performance is your #1 priority, HW RAID will offer the best solution. In all other cases, I believe ZFS is the better choice.

How can having to write data twice immediately over the bus come close to the performance of writing once immediately to RAM, regardless of how much RAM and CPU you've got?
–
wombleAug 11 '11 at 8:59

Not sure what you mean - both HW RAID and ZFS will do write caching...? In fact, ZFS lets you specify an entire set of disks as write cache devices if you want to.
–
growseAug 11 '11 at 9:06

1

If write latency is that important to you, then use HW RAID. In all other cases, I believe that the advantages of ZFS outweigh those of HW RAID. YMMV. Obviously.
–
growseAug 11 '11 at 9:22

4

@AndyM: If you want availability then ZFS is a better option, hardly any HW RAID solution checks both mirrors when reading let alone adds checksums to written data.
–
Hubert KarioAug 11 '11 at 11:12

2

@growse: You no longer HAVE to mirror the ZIL disks. In the more recent Zpool vers (26+ IIRC) you can safely remove a ZIL disk from a pool. The pool simply reverts to using the pool disks itself for the ZIL. If you're doing a lot of sync writes then you'd take a performance hit until you replace the ZIL disk.
–
TimAug 11 '11 at 12:42

Although I love ZFS if you're doing a LOT of writing then take into consideration that any software RAID 1/10 will need to send every write twice over the bus and through the controller - meaning you can get into congestion and queue issues.

This is a good point - from a performance point of view, HW RAID has some significant advantages in certain situations.
–
growseAug 11 '11 at 9:14

2

On the other hand, if you have close to 150MiB/s random writes (it's a db server) you're a madman for running it on a single bus. Interface bandwidth is hardly a problem.
–
Hubert KarioAug 11 '11 at 11:15

ZFS has lots of built in optimizations that work best when used without Raid controllers. On a ZFS Raid1 you get read performance that is usually superior to a hardware Raid, also with RaidZ you can get superior performance to Hardware Raid5
Just take a look at Anadtech.

What you absolutely need to take into consideration is how ZFS caches though. You should basically have 4GB of Ram to use it at all, 8GB to get some decent performance, but more is always good. There are lots of cases where more Ram helps with ZFS. The best ways to improve performance though are seperate devices (SSDs) for the ZIL and L2ARC, as those generally tend to be bottlenecking performance. This explains the idea behind L2ARC a little.

The principal problem with Hardware Raid here is, that ZFS does, what the Raid Controller does, so when using a Hardware Raid below ZFS, there is a Raid Controller that expects itself and only itself to be doing Queueing, Caching, IO Optimization and so on and there is ZFS on top of it also expecting the same of itself. This has side effects of course and they aren't necessarily pretty and will most likely have negative performance effects due to the way ZFS does I/O Parallelization. Also its predictive "self healing" features will in part be kind of useless that way. Therefor it is specifically recommended that when you use a Raid controller, you should expose the Disks JBOD are as Single-Disk Raid0 devices, so that ZFS can do the housekeeping by itself.