My filer currently has two basic md-type software raid 5 sets. One 8 x 500 GB and one 4 x 1TB. To make the configuration a little more flexible I'm considering adding LVM. Before I do, I figured I should also look into ZFS.

More recently, there is now a PPA for ZFS on Ubuntu. It sidesteps the licensing issue by using DKMS to compile the kernel module on your own system, so no distribution of the module itself ever occurs.
–
Ryan ThompsonMay 13 '11 at 1:07

ZFS on Ubuntu works with the "file system in userspace" (FUSE) system, rather than being a kernel module. I believe this means you will not be able to boot from ZFS, but you should be able to keep user data in it. Here is the hope page for the ZFS on FUSE project: http://www.wizy.org/wiki/ZFS_on_FUSE

ZFS has its own ways of being redundant, so yes it replaces md-raid and lvm.

Note that Btrfs is coming soon, and it will be the native Linux answer to ZFS. It is available now, but I would not advise you trusting important data to it. I expect that Btrfs will be solid in perhaps another year.

I will stick with ext4 on RAID until Btrfs is out of beta status. I would also be willing to install OpenSolaris on a spare computer and use it as a ZFS file server with NFS. I am reluctant to trust important data to beta software, and ZFS on FUSE is still considered beta.

EDIT: I'm updating this to recommend ext4 instead of ext3. Sadly, I think that BTRFS is still at least a year away from being ready, so I'm not editing that part. (BTRFS still doesn't have a working fsck that can fix errors yet!)

It is possible to pivot_root to a FUSE mount on boot, however it requires a lot of initrd hackery.
–
Tim Post♦Oct 11 '09 at 7:01

What the system drive is running doesn't really matter, it's the data volumes that need both redundancy and flexibility. Snapshots, flexible volumes and thin provisioning are all very tempting. However, I'm not sure I'm willing to run a user space file system, even if it was out of beta. LVM it is then :) Thanks for the informative answer.
–
RoyOct 11 '09 at 17:04

Happened randomly across this post, you get bonus points for referencing my blog! ZFS has been steady so far, but as with you, I am a bit nervous about using a "beta" filesystem.
–
Dave DragerOct 14 '09 at 13:56

I would not use ZFS on Linux through FUSE. Development mostly stopped on it as far as I know and it's not close to out of beta, nowhere near ZFS on Solaris. Btrfs should be a great replacement once it gets out of development, but it's very much immature software that's hardly ready for...well...anything. I'll be sticking to ext4/Reiser4 for my Linux storage needs.

No, ZFS is not available for Linux (as a kernel-integrated filesystem). The main reason is the incompatible license. I would not recommend the user space solution with FUSE for any serious use.

The thing that comes very close to your requirements is Nexenta. It combines the OpenSolaris kernel with the Ubuntu userland. You should give it a try because ZFS is (unfortunately) miles ahead of Linux's md-raid and lvm.

Other ZFS options are:

OpenSolaris which also comes with a GNOME desktop and looks like Linux on the first sight. But as soon as you take a closer look you will find many differences and it would take some time to learn the new environment. Many things in OpenSolaris are better than in Linux but there are definitely also many disadvantages...

I'm familiar with both Solaris and FreeBSD, but I'm pretty much bound to Linux at this point. I wouldn't feel comfortable using fuse on a filer, even if the ZFS project was out of beta, so I'll go with LVM for now. Thanks for the informative replies.
–
RoyOct 11 '09 at 17:01

Thanks for pointing this out. My understanding is that zfsonlinux.org was primarily created to serve the Lustre & HPC community, and since Oracle has Kicked Lustre to the Curb, this project has lost some of it's primary impetus. But I see that the lead (Brian Behlendorf from llnl.gov) is still working on it.
–
Stefan LasiewskiNov 23 '11 at 20:09

Yes it created to serve the Lustre, and by-product is ZFS for Linux boxes as kernel module :)
–
BBKNov 23 '11 at 20:18

It's not hand-wavy. The license incompatibility doesn't make it illegal to build or use a ZFS kernel module for Linux, it just makes such a module illegal to distribute. At least on Ubuntu, the ZFS kernel modules are compiled from source right on your system using DKMS, so they are never distributed.
–
Ryan ThompsonOct 28 '11 at 19:33