What do we want? Data integrity! When do we want it? Now! —

ZFS-loving Mac users demand support in OS X 10.9

You can sign an online petition, but it may be too late to sway Apple.

Some Mac users are demanding that Apple add modern file system support in the next major version of OS X. An online petition has been started to let Apple know that its aging HFS+ file system just won't cut it any more, and the company should include ZFS in OS X 10.9, expected later this year.

HFS+ is the current file system used by OS X (and iOS). It was originally developed as HFS, or "Hierarchical File System," for the original Mac OS in the early '80s. A team at Apple, led by engineer Don Brady, adapted HFS for 32-bit systems in the mid-1990s. Brady later adapted HFS+ to work with the UNIX environment that OS X was built on, and over time he and other Apple engineers added additional features, including the extensible metadata used by Mac OS X's Spotlight search, live partition resizing used for Boot Camp, and the Adaptive Hot File Clustering used to reduce seek times for oft-used system files.

Despite all the features Apple has managed to tack on to HFS+, though, its design certainly isn't modern. "The initial HFS+ was primarily about addressing the block count problem," Brady told Ars in 2011. "Since we believed it was only a stop-gap solution, we just went from 16 to 32 bits. Had we known that it would still be in use 15 years later with multi-terabyte drives, we probably would have done more design changes!"

ZFS, on the other hand, was designed from the ground up to address the ever-increasing needs for large amounts of storage, as well as the need to protect data as it is written to and read from disk. As a 128-bit native file system, ZFS can address up to a theoretical 256 quadrillion zettabytes. One zettabyte alone is equivalent to over a billion terabytes; 256 quadrillion billion terabytes is more storage space than could practically be used on Earth. It also includes several features designed to ensure the integrity of data on the disk, including checksumming every block of data so the system knows if a block goes "bad" and RAID-like features that allow disks to "heal" themselves if data corruption is detected.

Apple actually flirted with ZFS early on in its development. Brady was involved in a "skunkworks" project to port ZFS to OS X that started in 2005. Some of the code shipped in Leopard (10.5) and was expected to be a major feature of Snow Leopard (10.6). But due to licensing issues with Sun (and perhaps other reasons), Apple dropped all support for ZFS in Snow Leopard and cancelled the open source project that had served as official support for ZFS on OS X. (That project was forked and still exists as MacZFS.)

Brady later left Apple and started his own company to build a commercial version of ZFS for OS X. He got as far as releasing a command line "community" version called ZEVO before his company was acquired by enterprise software maker GreenBytes in June 2012. GreenBytes still offers the free community version of ZEVO while Brady continues to work on a GUI version that can integrate with OS X (though restrictions like sandboxing have proven difficult to work around).

Still, some users want official support for ZFS "or its equivalent" from Apple, and they want it soon. Mac user Thomas Monte started an online petition practically demanding that Apple add modern filesystem support to OS X 10.9.

(The petition also asks Apple to update the ancient OpenGL support in OS X, which still lags Windows significantly, from version 3.2 to the latest 4.3. Previous sources have indicated that improved OpenGL support is indeed coming.)

Unfortunately, the demand for ZFS support may fall on deaf ears. Aside from the fact that Apple hasn't shown any indication that it will support anything other than HFS+ for the time being, OS X 10.9 is already showing signs that it is being widely tested internally at Apple. It is also likely to get its first public showing in the next several weeks, and if Apple keeps to its projected one-year development cycle, it could be released this summer.

Still, it's been 30 years since Apple originally developed the basis for the file system currently used in OS X. Whether Apple adopts ZFS, Oracle's BtrFS, or is secretly rolling its own modern file system, OS X is long overdue for something new.

Full data checksum verfication, snapshotting, and atomic writes justify ZFS even for single drive computers.

A lot of the features that make a modern filesystem great and useful for single-disk / regular users consume disk space (deduplication excluded, I guess), but we are in a weird transitional period with SSDs, where most macs (laptops) don't really have disk space to spare. It might be another couple of generations before we are back the the affordable terabyte region where normal folks are not constantly running into the limits of their small SSD, at which point the fact that ZFS snapshots can be exposed as a fully local Time Machine view will be awesome.

I use ZEVO and like it. I have a 12TB array using ZFS RAID-Z2 and it works fine (even over USB 3.0 to an external array). While HFS+ is old and outdated, I don't think we need a full-on replacement - ZFS is overkill for the average Mac user. If you're a power user, just use ZEVO.

It's a good thing that ZFS supports case sensitivity or case insensitivity, depending on options specified at pool creation; because Adobe certainly won't be fixing that little issue before the heat death of the universe, much less before Apple moves from HFS+ to something else...

Licensing issues can be dealt with contracts and money, and even if that doesn’t work out Apple can fork the last free version of ZFS which was a very competent production quality piece of code.

But that was always the case; the real reason that ZFS was dropped was Jobs getting pissy at Sun’s CEO for crowing about ZFS being integrated into OS X. With Apple products, Apple is the one to debut new features, not the component vendor. As we all know, Jobs held on to grudges forever. That’s why we all know that once he got upset with Sun, ZFS as the primary FS for OS X would never happen while he was alive.

I can't imagine that internally Apple is oblivious of the need for a more modern file system. I just hope it's been given the necessary resources to see it through to completion in the not-too-distant future.

Back when 10.5 came out I never would have imagined we'd still be on HFS+ with 10.9.

ZFS will never replace HFS+. The licensing issues would be enormous and Apple would be crazy to hand over control of something as fundamental as the filesystem to another company or group. As modern as ZFS is, that doesn't mean it's a magical silver bullet - it's designed for the datacenter, not the consumer space. The majority of its features mean nothing for a laptop or tablet with single drives, which is where the vast majority of Apple's products fit.

HFS+ may be long in the tooth, but that doesn't mean it needs to be thrown out. What exactly is so horrible about it? It performs well, it's managed to scale into multi-terabytes without issue, it supports all of the technologies that Apple has shipped over the last decade (perhaps not elegantly - lots of dot folders at the root of the volume and all), and it's a known quantity from a stability and recovery perspective.

Change for the sake of change is not a good thing, especially with your data.

As much as I love ZFS (built my own home server with about 18TB of storage running on OpenIndiana and ESXi), to me it really shines on large disk arrays. With Apple focusing more an more on light mobile clients, which usually only have a single physical drive and relatively limited RAM, the argument for a significant investment in ZFS seems less compelling to me.

From what I've seen of the development cycles of ZFS, BtrFS, and even ReFS, creating a new advanced filesystem is a HUGE undertaking requiring years of testing and a very slow, conservative roadmap for introduction of features. So if they do have something of their own in development, I would expect it to be a significant step down from ZFS featurewise.

While official support would be nice, ZEVO works well enough that it's extremely functional already, so long as you're willing to use the command line for most management functions, and can deal with a separate boot drive vs data pool. The entire way they handled marketing and release of it was really too bad, but startups have it very hard. ZFS as a whole though is in something of a state of flux right now, and that has to weigh into Apple's decision making too. After Oracle acquired Sun and closed ZFS off, a new open source fork was supposed to be getting organized by illumos, but it doesn't seem to have yet really gotten going. Improvements from Delphix and the like remain isolated, block-pointer rewrite (and, related, features like FDE or RAID-Z shrinking) hasn't happened, etc.

I have no intention to switch away from using ZFS, it's fantastic and works extremely well under OS X and FreeBSD, but it's clearly not in "general user" territory right now either and it'd be harder to pick up then if there was a clear central project. I guess from that perspective I'd be happy to see Apple get involved, but that ship has probably sailed already.

ZFS can address up to a theoretical 256 quadrillion zettabytes—equivalent to over a billion terabytes

(emphasis added)

That is an understatement. According to google 256 quadrillion zettabytes = 2.74877907 × 10^26 terabytes. That's a whole heck of a lot more than a billion. According to Wikipedia no computer storage system has achieved a single zettabyte of storage capacity.

I'm not just trying to be pedantic, I want people to appreciate how much information we are theoretically talking about.

I want people to appreciate how much information we are theoretically talking about.

Yes, but how much of that matters? While it's unwise to bet against the progress of technology, I have a hard time seeing such capabilities being anything more than a theoretical bullet point for the next few decades.

Licensing issues can be dealt with contracts and money, and even if that doesn’t work out Apple can fork the last free version of ZFS which was a very competent production quality piece of code.

...

Now that Cook is in charge, there as [sic] least the possibility.

If ZFS is out of bounds because of licensing issues, would the Linux alternative BTRFS ever be in the running for inclusion? I know the GPL license would complicate things with the requirement to release the source code of a fork.

I love my Mac but I feel like OS X is starting to stagnate. I bought it when Leopard came out and it felt like the Mac was really picking up the pace and would be a great platform with some interesting developments. It is a great platform and it's still my OS of choice (and will be for the foreseeable future), but unfortunately all that seems to have happened is Apple poking it with a stick until it awkwardly shuffles a bit closer to iOS with every release. Stuff like Notification Centre: good. Stuff like Launchpad: wholly unnecessary. Speaking of which, iOS has the same feel. Here's hoping Apple can continue to improve their platforms with something new and fresh.Oh and stop making stuff like the Library folder hidden. People who shouldn't be messing with it don't even know it's there, it just makes it a pain in the arse when I need to go there.

Licensing issues can be dealt with contracts and money, and even if that doesn’t work out Apple can fork the last free version of ZFS which was a very competent production quality piece of code.

But that was always the case; the real reason that ZFS was dropped was Jobs getting pissy at Sun’s CEO for crowing about ZFS being integrated into OS X. With Apple products, Apple is the one to debut new features, not the component vendor. As we all know, Jobs held on to grudges forever. That’s why we all know that once he got upset with Sun, ZFS as the primary FS for OS X would never happen while he was alive.

Now that Cook is in charge, there as least the possibility.

Oh, stop it! It happened because of the licensing issues, mostly. No doubt there were some technical uses as well.

Full data checksum verfication, snapshotting, and atomic writes justify ZFS even for single drive computers.

How much of that is needed on a consumer system? Seriously. I live and breathe this stuff in the datacenter (and it's critical there), but this is a solution in search of a problem consumer-wise.

These nice-to-have features don't justify swapping out a filesystem. If you want to change filesystems, you have to:* Figure out how to migrate existing users* Provide cross-system compatibility (both between older Macs and newer, and some semblance of cross-platform)* Ensure all of your OS features are reworked for the new filesystem (given how much is tied into it - Spotlight, Time Machine, etc)* Validate for stability and recoverability, which can take years* Cover all of the corner cases (and in filesystems, the are enormous numbers of such "corner cases"). Apple can't easily say "well, it doesn't do that - but since I don't use that feature or use case, it doesn't matter".

Unless Apple are planning on releasing +4 HDD NAS server there's no point in ZFS. I'm not even sure that ZFS is even suitable for NAS servers as it was designed for huge datacenters. And yes, I have three disk NAS server in raidz1 at home.

i've just tried zfs this past week-end (and suffered two kernel panic since then), the fact that it doesn't support Time Machine is a no go. I can live without spotlight for a set of files, but not time machine.

I yearn for the day the last resource fork is gone from my system. I'm kind of shocked that anything still has them today - what the heck do you even create them with? Does Resourcerer still exist?

Bundles > Resource forks, in every way.

I'm not aware of any apps that still use resource forks, but some (like iTunes) have .rsrc files in their bundles which are literally just the old ResEdit data structures saved into the data fork of a dedicated file.

i've just tried zfs this past week-end (and suffered two kernel panic since then), the fact that it doesn't support Time Machine is a no go. I can live without spotlight for a set of files, but not time machine.

I don't trust timemachine. I've tried restoring from backup with it and had issue's. Thankfully I made a mirror copy of my drive beforehand.

Full data checksum verfication, snapshotting, and atomic writes justify ZFS even for single drive computers.

How much of that is needed on a consumer system? Seriously. I live and breathe this stuff in the datacenter (and it's critical there), but this is a solution in search of a problem consumer-wise.

Ever increasing drive sizes means data rot is becoming a legit problem even for users that only have a single 2 or 3TB drive.

Time Machine is exactly the sort of thing that enormously benefits from snapshots, both from a time to backup standpoint and a backup consistency standpoint. Snapshotting and then spinning the snapshot out to the capsule would work out very very well indeed. Especially because the capsule can keep many backup copies with delta diffs.

Then there’s also system patches. In some OSs, Solaris for instance, the patch tool will make a system snapshot (it’s near instantaneous and takes very little disk space) before applying the patch. If the patch install fails, it’s far better to roll back the whole system then to try to back out a broken patch.

There may not even be a need for end users to snapshot (maybe just through the command line) but the system infrastructure can greatly benefit.

diamondsw wrote:

These nice-to-have features don't justify swapping out a filesystem. If you want to change filesystems, you have to:* Figure out how to migrate existing users* Provide cross-system compatibility (both between older Macs and newer, and some semblance of cross-platform)* Ensure all of your OS features are reworked for the new filesystem (given how much is tied into it - Spotlight, Time Machine, etc)* Validate for stability and recoverability, which can take years* Cover all of the corner cases (and in filesystems, the are enormous numbers of such "corner cases"). Apple can't easily say "well, it doesn't do that - but since I don't use that feature or use case, it doesn't matter".

But they're looking at that anyway. HFS+ needs to be replaced. Whether the replacement is ZFS or not, those issues will need to be dealt with, and Apple has never been afraid of breaking compatibility.

As for stablity, that's the whole reason to use ZFS. It's already used in numerous production systems. This isn’t some obscure FS that nobody uses. To me, that’s the argument of using it over btrfs. It’s had more development. That said, btrfs is a far better choice than some secret Apple FS project that’s never seen the light of day until it starts shipping.

I use ZEVO and like it. I have a 12TB array using ZFS RAID-Z2 and it works fine (even over USB 3.0 to an external array). While HFS+ is old and outdated, I don't think we need a full-on replacement - ZFS is overkill for the average Mac user. If you're a power user, just use ZEVO.

I could not disagree more. The fact that a corrupted file in HFS+ will overwrite a good backup on a Time Machine drive is horrible. I use CrashPlan for offsite backup and it maintains every version of every file indefinitely (until I tell it to explicitly remove a file). CrashPlan is my only way to guard against files getting corrupted under HFS+.

However, with ZFS a Mac could automatically detected a corrupted file under HFS+ and then restore it from the most recent backup automatically. It would be seamless and beautiful.

There may be features of ZFS that are overkill for most users, but the basic feature of CRC checksums at the block level is a must for OS X.

the real reason that ZFS was dropped was Jobs getting pissy at Sun’s CEO for crowing about ZFS being integrated into OS X. With Apple products, Apple is the one to debut new features, not the component vendor. As we all know, Jobs held on to grudges forever. That’s why we all know that once he got upset with Sun, ZFS as the primary FS for OS X would never happen while he was alive.

Now that Cook is in charge, there as least the possibility.

Wow, conspiracy theory much? Are you trying to imply that Jobs thought ZFS would radically improve the Mac platform, usher in a utopia for users, and make him billions of dollars with no risk to Apple, but then shelved it because one guy pissed him off? Seems more likely that either it cost too much, didn't produce the gains that would justify the potential compatibility issues and transition problems, or Apple isn't willing to give control of a key part of its product to an external company. Seeing as Apple always tries to control or at least have an in house alternative for strategically important software- Safari, iWork, Apple Maps, etc- I think it's the third option.

Were in not for case sensitive issues, XFS would be a good option to use. Responding to an earlier commenter, I thought one of the features of ZFS was the ability to lost disks in a pool and the system keep on running. I remember a demo of ZFS from Sun where some one smashed a disk with a hammer that was connected and the system kept going even while doing a large file copy.

While official support would be nice, ZEVO works well enough that it's extremely functional already, so long as you're willing to use the command line for most management functions…

I'm pretty sure that I perform “management functions” in Disk Utility when I initialize a disk but I'm having a hard time guessing what functions would be performed by 90% of Mac users.

So it it's just about setting up a big pool to hang off a mini in the basement? Command-line == little or no problem. A new option or two in disk utility? OK, make a limited Push Here Dummy version of the manager. But otherwise, something else is holding Apple back.

The bigger problem may be that Apple has more revenue coming in from iTunes than from Macs.

Wow, conspiracy theory much? Are you trying to imply that Jobs thought ZFS would radically improve the Mac platform, usher in a utopia for users, and make him billions of dollars with no risk to Apple, but then shelved it because one guy pissed him off?

Did you miss the recent episode (GMaps) where Apple damaged the iOS platform, ushered in a hell for many users, and lost a bunch of money because one company pissed him off?