Intriguingly, ZFS-FUSE is much faster running on top of Linux software RAID than it is using RAID-Z over raw disc devices. This suggests that a lot of its slowness is not just due to FUSE overhead. Another performance quirk that I’ve noticed was that the latency is absolutely horrible when multitasking. It wouldn’t surprise me if this one might be a FUSE limitation (e.g. if FUSE required requests to be dealt with in the order they were received). As I wrote on the ZFS-FUSE mailing list – “According to bonnie, I see 125 MB/s reads on ext3 RAID5, 65 MB/s on ZFS RAID5 (using Linux’s software RAID) and 20 MB/s on ZFS raidz (using the same raw drives). Writes are also proportionally slower.”

It does look like ZFS-FUSE scales to around half the available disk bandwidth of a single underlying drive for some reason. So on the above server that’s a 10MB/s to 4.xMB/s, for you that’s 128MB/s to 65 MB/s and on my desktop box it’s 39MB/s to 18MB/s (on MD RAID-1).

What I’d love to know is what do you get for ext3 on a single drive ? My prediction would be that you would get around 35-45MB/s based on that theory.