Main navigation

APFS tools suck

APFS is now over two years old, its official release having been in March 2017 with iOS 10.3. Look at its supporting tools today, though, and you could be forgiven for thinking that it’s still in beta.

A couple of days ago, I looked at its limitations in terms of minimum size of containers and volumes, mainly using Disk Utility, and came away completely confused. I resolved to return and this time tackle the problem at the command line, using the all-powerful diskutil. This article is a report on the complications that result from trying to manage APFS using diskutil.

If you’ve used diskutil to manage HFS+ disks and volumes, you’ll be disappointed to discover that, rather than extend its standard set of verbs, Apple has effectively shoehorned in a second tool with its own set of verbs which are prefaced bydiskutil apfs
and using man diskutil is more like opening a disjoint library.

Erasing a disk and creating a single APFS container with a single APFS volume is simple using the eraseDisk verb, but what you can’t do is create any more than a single APFS container. For example, repartitioning an existing disk withdiskutil partitionDisk disk5 GPT apfs test1 125g apfs test2 0
which you’d expect to create one container of 125 GB and a second occupying the remainder of the disk, returns the errorYou may not specify more than one APFS Container per partition map

Similarly, trying to cheat and splitting a second non-APFS partition into one APFS container and a journalled HFS+ volume usingdiskutil splitPartition disk5s3 apfs test2 50g jhfs+ remains R
results in the doom-laden errorYou may not cause more than one APFS Container to exist per partition map
which could make you assume that disks can only have a maximum of one APFS container after all.

Being persistent if not dogged, I then tried creating a single APFS container, shrinking it to the new size of 125 GB, then adding a second. I started withdiskutil apfs resizeContainer disk6 125g
which works fine, and leaves the remainder of the disk unallocated. But you can’t then use diskutil apfs createContainer to add the second container, nor can you allocate the remainder using a plain diskutil command.

One interesting feature of the createContainer option is that it allows you to make APFS ‘Fusion Drives’. If you have two APFS disks, each with single containers, you can link them together to function as one using a command likediskutil apfs createContainer -main disk6 -secondary disk7
where disk6 becomes the primary caching store, and disk7 the secondary store, assumed to be the slower of the pair.

At this stage, I decided that my only option was to cheat, and see how Disk Utility manages to add a new container when you repartition an APFS disk. Its sequence runs:

resize the existing container to its new size and check it;

create a case-insensitive journalled HFS+ volume in the unallocated space;

convert the HFS+ volume to a single APFS container with a single volume.

which is a workaround, not a solution.

Taking the hint, I found that I could create a disk with three APFS containers as follows. I created the first APFS container usingdiskutil partitionDisk disk5 GPT apfs test1 125g jhfs+ test2 50g jhfs+ test3 0
then converted each of the two HFS+ partitions into APFS containers usingdiskutil apfs convert disk5s3

If you want to add a container to an existing disk, though, things get more complicated, as you’ll first have to shrink an existing APFS container using something likediskutil apfs resizeContainer disk6 125g
You then have to unmount the disk and use the gpt command to add the HFS+ partition, which is detailed in klanomath’s excellent answer here. I’m afraid that at that stage I chickened out and looked at adding volumes, which is consummately easy by comparison:diskutil apfs addVolume disk6 APFS test1a
and so on.

I had finally divided my single external SSD into three APFS containers, each of which contains two volumes.

Returning to my point of entry, it perhaps isn’t surprising that there is little correlation between the minimum container size given in Apple’s APFS documentation and what actually happens, given the circuitous means by which containers have to be created. If Apple’s tools for working with APFS are anything to go by, we’re not supposed to be using multiple containers at all, which seems odd given that it was Apple which designed them into APFS in the first place.

Given that containers are a valuable feature in APFS, why, more than two years after its release, has Apple not provided command tools which fully support its features? Nor does there appear to be any sign of change coming in Catalina, as far as I can gather.

Should you know of any better solutions to these problems, I’d be delighted to learn of them, please.

Thanks.
I don’t think the evidence points to APFS only being intended for iOS – a lot of its features and limits are clearly intended for macOS. Although snapshots may have some value in iOS, they really come into their own in macOS, as do ‘clone’ copies, sparse files, and containers themselves. Sure, the engineering priorities were for iOS, but macOS has not been a late kludge by any means.
The APFS team has already had showings at WWDC, and one of the most successful roll-outs of any file system across such a huge number of devices and computers. If anything it has been their purism which has been most difficult – the issue of Unicode normalisation was a good example of engineers who had made a decision and would only provide workarounds when Apple products suffered.
Where Apple has skimped is providing support for developers and advanced users – SDK, tool, and documentation support – which don’t affect its own engineering or development. I’m impressed with the engineering which has gone into the file system, but deeply disappointed in how Apple has effectively prevented anyone else from making the best out of that. Features like fast directory sizing are an excellent example. And that hits macOS most all, of course.
Howard.

A good case can be made that Apple is simply dropping the ball… again. macOS development under Tim Cook, IMHO, has been anything but stellar. It’s more buggy than ever. The user interface has devolved. Security changes are poorly thought out and implemented. Functionality is less discoverable. I could go on and give specific examples even; but you get the point. I’ve used every version of the Mac OS starting with System 6.0.4. I’ve always been passionate for the Mac until recent years, excited and champing at the bits for the next release of whatever. Today, I’m meh at best; my passion long faded since Jobs passed. Why? In short, I demand excellence and value from Apple. And it does not deliver.

I agree with this entirely. I’ve owned a Mac since 1985 and have been enthusiastic about every single Mac OS release– until Lion. Jobs carefully released a new version of the OS every couple of years. Why isn’t Tim Cook following Jobs?

Great article. Thanks. The best approach would have been Apple releasing APFS only when it was a finished product. Surprisingly, AFPS is a moving unfinished target. Why? Probably, Apple wants to finish Time Machine 2 supporting APFS disks, before releasing full APFS documentation.

Actually, documentation about how to write to APFS disks, since the one about how to read them was published in September 2018, after macOS 10.14 Mojave was released. So, the big question is how is Time Machine 2 development progressing? When will it be released (even as beta)? Next Apple WWDC by June 2020? Hopefully. Because then jewels like DiskWarrior will be also released for APFS.

I don’t know any of the answers, sadly.
What shocks me is that you expect the tools to have rough edges at first release, but more than two years later they haven’t advanced much, as far as I can see. Disk Utility still has to use a kludge to add more containers, which just seems crazy. It’s not as if Apple can’t afford the resources, but almost as if they’re being deliberately withheld to make life in macOS more difficult, even for its own engineers.
Howard.

It is obvious that Apple is mostly focused on iOS than macOS. But I guess that they have encountered huge unexpected problems when trying to make Time Machine 2. So much, that they would required to change APFS fundamentals. That could explain why they have not yet released full APFS documentation and Time Machine 2.

But Apple has also huge funding resources that should have allowed to address all that years ago. Hopefully, they will focus more on practical productivity goals and less on emojis-like stuff in the future. There is nothing like the Mac (intuitive and easy to use, yet powerful and flexible), and I hope that will continue in the future.

I see no evidence that Apple is changing any APFS fundamentals, rather that it’s steadily developing TM2. Indeed, it seems to be building on what we already have in Mojave, features like snapshots, which are already being used by TM in 10.14 (and were in 10.13 too).
None of us knows what Apple’s roadmap is for TM2. At present, the great majority of users back up to rotating disks anyway, where using APFS would be more of a mixed blessing.
Howard.

If that is what is happening, then Apple is being lazy with macOS, APFS and Time Machine 2, since, they have huge financial resources and there are also clever people out there to bring that out in time. On the other hand, why is that a “mixed” blessing?

Because APFS wasn’t designed for hard drives, and doesn’t perform particularly well on them. I think most users with hard drives are happy staying with HFS+ for the moment. I also suspect that few RAID HD arrays would be good with APFS.
Howard

Being tagged the “Mac guru” in our circle (not a term I would use to describe myself, certainly!) it fell to me to decommission a number of iMacs that had reached “end-of-the-road” status with High Sierra and re-purpose them for use by our church – still stuck in El Capitan! Until now I hadn’t had cause to screw around with file systems beyond that allowed by Disk Utility, which never seemed much of the toolkit for serious drive setup and maintenance that it could/should be. It always reminded me of that dull pocket knife your granddad gave you as a kid. But its shortcomings really hit home when I discovered that it lacked a secure erase tool for the SSDs I had installed in the optical bays of these machines. I had originally formatted them as APFS (unencrypted) and the factory spinning metal as HFS+ for use as time machines. So the thought of creating fusion drives never occurred to me until discovering this post, while researching the secureErase cmd in diskUtil vs doing a “cryptographic erase” of the SSDs using FileVault and Disk Utility to “throw away” the keys. What I discovered is that creating a fusion drive as hfs+ using coreStorage is easy and that using apfs should be even easier…but its not! The process fails at the “addVolume” step with the cryptic “Error: -69624: Unable to add a new APFS Volume to an APFS Container”. The answer may be obvious to someone with more experience, but not me, nor anyone I’ve spoken with at Apple …”so surprising!” Lol

Thank you.
Secure erase is a highly controversial area, and there isn’t really a good answer. Apple recommends FileVault encryption rather then secure erase, but I think there are still issues in ensuring complete coverage of an SSD.
High Sierra doesn’t support ‘Fusion Drives’ in APFS as far as I’m aware, but Mojave does. If you want to create a ‘Fusion Drive’ for High Sierra, you have to continue using HFS+ on them as far as I know.
I hope that helps.
Howard.

I came to that same realization after attempting to create the APFS volume on 2 fused 256G USB sticks on a H Sierra machine and failing, plugged them into a MB running Mojave then running the addVolume command and presto, 500G APFS Fusion Drive. But TBH, with the prices of SS dropping and the convenience of an internal backup drive, I find greater utility in the 2 drive configuration than the one big FD. IMO fusion drives are an intermediate tech, akin to those transitional periods in history after which everything changes. Thank you Howard. Sometimes all we need is confirmation.

Hi,
I’m an amateur user, really, but have been delving into apfs recently after deciding to upgrade from my trusted El Capitan. One of my motives was to upgrade a 13′ Macbook Pro (late 2013) I received, one of those that is saddled by a small 256 GB SSD that’s hard to change.
I added a 256 GB µSD card in an adapter that fits flush in the SD slot (Transcend also sells integrated “Jetdrive” adapters). The downside is that you get separate volumes, which can get inconvenient. Then I saw this post form Econ Tech’s blog, explaining how to create a fusion drive: https://www.econtechnologies.com/chronosync/TN-CS-APFS-Fusion-Drive.html
I went ahead and created that, with the additional constraint that I had to work on the system drive. Proceeding with Terminal in recovery mode, it worked.
I am now starting to use this machine with its hybrid SSD/µSD fusion APFS drive in a single container. It’s fairly cheap, operational and practical. Reading the various posts here does make me a bit wary, however. Time will tell.

Thank you.
APFS ‘fusion drives’ use the primary disk as a cache for the second. This generally works best when the primary drive is smaller than the second, and the second is slower than the primary.
In your case, the second of those is met, as your µSD card should be slower than the SSD. However, as the two disks are the same size, the caching isn’t likely to work to your advantage.
I’d be inclined to use them as two separate drives, putting the larger and more static files, like music and photo libraries, on the µSD, which is what I did when I had a similar setup in a MacBook Air.
Howard.

Hi,
I’d better post an update: I had to abandon this setup, as it created severe problems and made the computer unusable.
Actually it seemed to work at first, for a few days. The first hitch was when installing Microsoft Onedrive, which refused installation on the grounds that it couldn’t install on removable media (how did the installer find out?).
Anyway, the next day the computer was unable to boot up normally: it got stuck in loops at various stages in the bootup process.
So I scrapped the whole configuration and reinstalled from start, with standard separate volumes for the internal SSD and the SD card drive. No problem since then.
My hypothesis is that the SD card doesn’t mount reliably (even now as a separated volume, it sometimes vanishes when waking from sleep). And when that happens, the whole fused container is compromised.

So indeed it’s back to your recommended setup. Having separate volumes is less conveninent but more reliable.