Main navigation

How big is that folder: what happened to APFS Fast Directory Sizing?

It’s been another of those long-standing issues in macOS, and Mac OS before it: select a folder and Get Info. If you happened to choose a large high-level folder such as Applications, you might have some time to wait before your Mac can tell you its size. Surely there must be a better way?

APFS promised one: Fast Directory Sizing. At least that was in the original brief documentation which Apple teased us with.

The first paragraph there was full of promise, but by the third, doubt had set in. This seemed ideal for many folders, including ~/Documents as cited, and /Applications too, so the question is whether it has been implemented yet.

I set out to see if Mojave 10.14.3, now running the third release of APFS and maturing well, could return folder sizes more quickly. The answer, for major higher-level folders at least, seems to be no. This is made more complex by the fact that Finder seems to cache folder sizes. The first time that I used Get Info on /Applications, it took several minutes to generate its size. A few minutes later, the result was instant, and couldn’t have been thoroughly recalculated. And don’t be mislead by performance on a complete volume, which is quite a different question.

I turned then to the latest and much fuller APFS documentation, where there is no mention of Fast Directory Sizing at all. But there are some tantalising clues.

First, each directory information record has a data structure j_dir_stats_val_t, which contains a numerical value for total_size, which is described as:“The total size, in bytes, of all the files stored in this directory and all of this directoryʼs descendants.”
That’s just what we want. But it doesn’t seem that is always up to date. In each inode, there are j_inode_flags which include INODE_MAINTAIN_DIR_STATS. When set for a directory, that inode tracks the size of all its children. And all those child subdirectories must also have this flag set.

So for some folders at least, where this feature is enabled, it should be possible to obtain an immediate estimate of the folder’s total size, provided that they’re on an APFS volume.

The next part of the problem is discovering which folders this Fast Directory Sizing is active on. Here it all gets more difficult, because macOS doesn’t appear to provide high level languages such as Swift or Objective-C with ready access to the j_inode_flags which can tell whether it is enabled, nor does it provide any single call to return the size of a folder which might take advantage of this new feature of APFS.

A little time spent with Mojave’s Finder suggests that nothing has changed there either. If the Finder happens to have it cached, getting folder size is blazingly quick, if probably inaccurate; if it has to calculate it afresh, then you’re in for a long wait as it chunders through the entire folder’s contents doing its arithmetic.

It’s time for FileManager to get a function to return a folder’s total size (and total number of items) making best use of APFS Fast Directory Sizing when it’s available.

That’s interesting, I had wondered why directory sizing didn’t seem blisteringly fast as had (I thought) been promised for APFS.

I have been having doubts about the Finder’s ability to count items in a directory too. I have simple examples where the Get Info number of items (cited after the size) is only correct if I include DS_Store, which seems an odd thing to do if I don’t have invisible items set to appear. More complex examples seem to be either inconsistent or simply wrong but I haven’t had time to pin them down.

The item count includes sub-directories but DS_Store doesn’t always seem to be present, which complicates checking (I had previously thought that every folder had a DS_Store but I may simply be wrong about that. I have also seen DS_Store appear and then disappear as I copied (might be significant) a package into a folder and then later deleted it.