Lion’s FileVault 2 and disk restore: caveat encryptor

There are some subtle incompatibilities between Lion's FileVault 2 full disk …

When Mac OS X 10.7 Lion introduced full disk encryption, called FileVault 2, it was a huge improvement over the original FileVault, which only encrypts a user's home folder. And because of the "creative" way FileVault was implemented, there were numerous incompatibilities, gotchas, and caveats to its use. FileVault 2 on the other hand, encrypts individual disk blocks so the encryption is invisible to the file system and really doesn't get in the way of normal use. Adding to this is another new feature in Lion: the recovery partition and the network recovery system introduced in last year's hardware.

In almost all cases, these new features will work together without trouble. Still, there are a few things you should know, especially if you use encrypted Time Machine backups and if you can't depend on fast Internet connectivity to be available should your boot drive fail.

My tale of woe

But first, let me tell you the sad story of how I repaired some disk errors on my MacBook Air's SSD. After reading the section called "What's wrong with HFS+" in John Siracusa's Lion review, I'm surprised that the Mac's file system actually works most of the time. But invariably, after several months of use, some errors manage to accumulate, and it's necessary to use Disk Utility to repair them. I reached that state a few days ago when emptying the trash would hang for no apparent reason.

Disk Utility told me to boot from another drive and run itself in repair mode. This is where the recovery partition came in handy—and it's not like I have a Lion DVD lying around—so I rebooted my Air while holding the option key in order to be presented with the list of possible boot drives. But the recovery option wasn't listed. There was a list of wireless networks, which I used to start my Lion Internet recovery in August. But selecting my WiFi network didn't give the option to recover over the network, either.

Fortunately, I had written a recovery partition to my Time Machine drive for just such an eventuality with the Lion Disk Recovery Assistant. This one did show up in the list of boot drives, so I started Disk Utility—only to find out that my Air's boot drive was grayed out so I couldn't repair it. To add insult to injury, it was also impossible to restore from my encrypted Time Machine backup!

Look for the small but important difference.

I was able to resolve this situation by booting up my computer normally and then using the security settings in the System Preferences to decrypt my hard drive. (It's also possible to do this using the command line, of course.) After this, the recovery partition magically showed up as a boot option and I was able to repair the drive, which was just suffering from a minor case of miscounted free blocks. It could have been much, much worse.

Lessons learned

I later found out that I could have avoided all of these issues by just using command+R to get into the recovery system, as Apple tells us to. (In my defense, all these cryptic startup keyboard shortcuts are impossible to remember.) Despite the fact that it doesn't show up as a boot option when FileVault 2 is enabled, the recovery partition still works if you invoke it using command+R. Even more surprising, the Disk Utility that I started that way does know how to unlock FileVault 2 drives so you can repair an encrypted drive or restore from an encrypted Time Machine backup. (You simply type the password for one of the accounts that is allowed to boot the system.)

It seems the Internet recovery option is hidden if the recovery partition is present, but Ars reader @Sacrilicious told us via Twitter that command option+R will force the Internet recovery.

But what's the deal with the different behavior of the two recovery partitions? Although the GUI completely hides these partitions, you can mount them with the diskutil command and then peruse them using the command line. It turns out that my two partitions held different versions of the BaseSystem.dmg file. At some point—probably the 10.7.2 update—there must have been a change which made it possible for the recovery system to unlock encrypted drives. These modifications were of course written to the recovery partition on the internal drive, but not to a USB drive that is only connected once in a while.

The previously mentioned Lion Disk Recovery Assistant will happily write a new copy of the recovery partition to a drive that already has it—and without messing with other partitions on that drive. Which leads me to the next surprise. If you run the Lion Disk Recovery Assistant with FileVault 2 enabled, it creates recovery partitions that don't work. This makes some sense because FileVault 2 uses the recovery partition during its boot process. Once again, a decrypt > run the utility > re-encrypt cycle fixes the problem.

So where does this leave us?

If you're a FileVault 2 user, remember two things: use command+R when booting to go into recovery mode, and you can always decrypt your drive if FileVault 2 gets in the way of fixing a file system issue. Of course decryption won't do anything for physical drive problems, so always back up first and make sure you don't overwrite backups from before the disk trouble started.

If, like me, you are the owner of a post-Lion Mac, you probably won't have any reinstall media like an original Lion DVD. So, if your internal drive gets wiped or replaced, there are only two options to reinstall Lion: over the Internet or from a Time Machine backup. If you find yourself in a situation where downloading half a gigabyte for the recovery image or even four gigabytes for Lion could be problematic, then it all comes down to that Time Machine backup. And if that backup is encrypted—as it really should be—you really want to have a copy of the new version of the recovery partition on that Time Machine drive, or on a separate flash drive. So, for your own sanity, make sure you create this recovery partition on a system that doesn't have FileVault 2 enabled.

While the author's issue was just a matter of learning the correct key combo, it's clear that FileVault should only ever be used by the power-est of power users. Almost wish Apple would remove it from System Preferences and make it a Utility you have to dig up if you want it.

I was initially excited about FileVault 2 but, like many other Lion features, excitement waned and reality set in. What Apple SHOULD be doing is supporting the ATA security command-set. SSDs and higher end HDs have encryption built-in, so why not use it? At least give us folks that want to use SandForce based SSDs an option that doesn't suck.

I have seen several computers bricked because of the original FileVault. FileVault 2 does nothing to assuage my concerns. My next question is *why* anyone would need full disk encryption unless carrying sensitive data. I agree with icon master above - it really should be a utility for power users.

I was initially excited about FileVault 2 but, like many other Lion features, excitement waned and reality set in. What Apple SHOULD be doing is supporting the ATA security command-set. SSDs and higher end HDs have encryption built-in, so why not use it? At least give us folks that want to use SandForce based SSDs an option that doesn't suck.

I use FileVault2 on my MacBook Air and it hardly sucks. In fact, it works great. It benchmarks very fast, and in day to day use there is no perception of lag or slowness.

The only noticeable difference is that you type in your password before boot, not after.

I have seen several computers bricked because of the original FileVault. FileVault 2 does nothing to assuage my concerns. My next question is *why* anyone would need full disk encryption unless carrying sensitive data. I agree with icon master above - it really should be a utility for power users.

Pray tell, what do you mean by the word "bricked?" Because the original FileVault could not perform full disk encryption. It only encrypted user files. It was sometimes glitchy, yes. I'd seen problems, and chose not to use it.

But if a user loses all their data, is that considered "bricking" the computer? When it boots fine and other users have no problems?

What next, Microsoft Office won't run VBA, so the computer is considered "Bricked?"

You've got to be kidding. Even wiping the operating system shouldn't consider the computer bricked.

I have seen several computers bricked because of the original FileVault. FileVault 2 does nothing to assuage my concerns. My next question is *why* anyone would need full disk encryption unless carrying sensitive data. I agree with icon master above - it really should be a utility for power users.

Bricked? My arse, stop trolling! It simply encrypted user's files into large container files, the O/S is still intact to be booted from, if the Mac "bricked" it certainly wasn't FileVault.

Why do you need it? When come scumbag breaks into you house and nicks your kit, your laptop/desktop if they can carry it out, with all your stored online account passwords and finance records for example?

The world does not consist of black'n'white but a billion shades of grey, we're all different and all have different needs. Apple wrote Filevault as there was an obvious need for it. When they are giving the O/S away they are not going to waste research time and money on an add-on that no ones going to use, they need to ensure every R&D dollar is accounted for.

So I'm on Lion using FileVault2 and also encrypting my Time Machine... if I have copies of the OS X Lion disk image on various other drives and on a DVD, does this issue effect me at all? I can just install Lion from the image and then restore to the encrypted Time Machine backup normally, yes?

So I'm on Lion using FileVault2 and also encrypting my Time Machine... if I have copies of the OS X Lion disk image on various other drives and on a DVD, does this issue effect me at all? I can just install Lion from the image and then restore to the encrypted Time Machine backup normally, yes?

Unfortunately, the "issue" described in this article affects you and all other users. It is a global, worldwide issue. Apple has no plans to patch this issue.

You will be forced to read the user manual for your computer and discover that holding down Command-R is required to boot from the restore partition.

I was initially excited about FileVault 2 but, like many other Lion features, excitement waned and reality set in. What Apple SHOULD be doing is supporting the ATA security command-set. SSDs and higher end HDs have encryption built-in, so why not use it? At least give us folks that want to use SandForce based SSDs an option that doesn't suck.

I use FileVault2 on my MacBook Air and it hardly sucks. In fact, it works great. It benchmarks very fast, and in day to day use there is no perception of lag or slowness.

The only noticeable difference is that you type in your password before boot, not after.

I don't think Airs use SandForce controllers? Either way, it bypasses built-in hardware encryption that already works well on SSDs. I no longer use FileVault on my laptop since moving to an SSD, but the benchmarks I saw online indicate that FileVault cannot keep up with hardware level encryption. Some assumptions are made there since, of course, we can't directly benchmark hardware level drive encryption on OS X because Apple doesn't support the command-set.

I liked FileVault2 when I used it and it seemed stable, so I'm not necessarily advocating that it be removed. However, I do think Apple should support the Secure-ATA command set. No reason not to.

That and they could bring back the original spaces. Every-time I use a machine running Snow Leopard and use Spaces I'm both relieved and sad at the same time.

You will be forced to read the user manual for your computer and discover that holding down Command-R is required to boot from the restore partition.

That is the extent of the "issue."

That's ridiculous. Apple's famous, and rightfully so, for its intuitive interfaces. In particular, FileVault, Time Machine, etc. are absurdly easy to use, and thus a lot of computer "newbies" likely use it. Having to read a manual to discover a hidden key combo is the complete opposite of intuitive. Hardly anyone would do that in advance, and the last thing you want to do when your computer's having problems minor enough to recover itself is read a manual.

While the author's issue was just a matter of learning the correct key combo, it's clear that FileVault should only ever be used by the power-est of power users. Almost wish Apple would remove it from System Preferences and make it a Utility you have to dig up if you want it.

Having spent days recovering my mother-in-law's borked FileVault (v1) system, I heartily agree. Unless you're in a corporate setting that mandates it, you don't *need* encryption, and especially not system-wide. You might get a kick from the security, but said security is cold comfort when a disk error takes all your data with it because your disk repair utilities can't function on an encrypted drive.

As for the original issue, I bet secure erase of trash was set. I ran into this "freezing" problem when deleting an old VM directory. Shouldn't take hours to delete a single file, and puzzled out the secure erase. Given that I'm already fully encrypted, secure erase of trash is pointless.

The link is not broken, it's just that Apple's support site has been having trouble loading sometimes recently.

The worlds richest megacorp cant maintain a simple support website? is it running osx server or something?

The Ugly wrote:

You will be forced to read the user manual for your computer and discover that holding down Command-R is required to boot from the restore partition.

That is the extent of the "issue."

But But.... It Just Works!!?

newwb wrote:

That's ridiculous. Apple's famous, and rightfully so, for its intuitive interfaces. In particular, FileVault, Time Machine, etc. are absurdly easy to use, and thus a lot of computer "newbies" likely use it. Having to read a manual to discover a hidden key combo is the complete opposite of intuitive. Hardly anyone would do that in advance, and the last thing you want to do when your computer's having problems minor enough to recover itself is read a manual.

Apples UI's are an utter pain until you've adjusted to doing things "The Apple Way™" aka non-intuative.(osx also seems to have an unusual amount of keyboard-modifier 'shortcuts', a side effect of that old retarded one button mouse thing?)

There's one more thing iljitsch should do/have/try: Get an 8GB USB stick and image the current LION ESD installer to it. I've only had to use this 'boot the stick to get DiskUtil' solution ONCE and that was to repair a Disk/Recovery Partition gone bad. I've had no problems with accessing encrypted partitions, TM or otherwise because:

Using a USB stick imaged with the current update of Lion gets you the CURRENT Disk Utility should the recovery partition fail. Also, I'm (personally) dubious of having a recovery partition on a Time Machine HD. Especially since you can now also encrypt the Time Machine HD as well and (if you don't know about Cmd-R to get to the root one) compound your problems.

Since I live in the USA, and because I like "New Year's Resolutions" from the EFF, any field machine in my possession gets FDE. It's just good policy.

There's one more thing iljitsch should do/have/try: Get an 8GB USB stick and image the current LION ESD installer to it.

I never bought Lion (the Air was expensive enough, thankyouverymuch, and updating ALL your machines to the latest and greatest is risky, to say the least) so I don't have the installer.

leoofborg wrote:

Using a USB stick imaged with the current update of Lion gets you the CURRENT Disk Utility should the recovery partition fail.

I prefer having this on the backup drive so that's the only thing I need to restore my system even if the drive has been replaced with a completely empty one. (I guess Apple installs the OS for drive replacements under warranty / Apple Care?) I don't see how, with the latest version of the recovery image, having TM encrypted makes a difference.

It's not the Disk Utility version that leads to the different behavior, all recovery partitions had version 12 (346), the same one as on my regular system.

While the author's issue was just a matter of learning the correct key combo, it's clear that FileVault should only ever be used by the power-est of power users. Almost wish Apple would remove it from System Preferences and make it a Utility you have to dig up if you want it.

Having spent days recovering my mother-in-law's borked FileVault (v1) system, I heartily agree. Unless you're in a corporate setting that mandates it, you don't *need* encryption, and especially not system-wide. You might get a kick from the security, but said security is cold comfort when a disk error takes all your data with it because your disk repair utilities can't function on an encrypted drive.

"Caveat encryptor", indeed.

Personally? I'd say that if your entire data integrity strategy is "hope Disk Utility can fix it", then you deserve everything you get! Encryption or no, you need to have a proper backup strategy.

Been using Filevault 2 here on my laptop without any major issues. There's a couple of things that kind of bother me about it (doesn't it lock the drive when the computer is hibernating? It doesn't seem to! You can't apply it to a RAID set so I can't use it on my desktop machine, etc) but otherwise I'm happy with it.

Been using Filevault 2 here on my laptop without any major issues. There's a couple of things that kind of bother me about it (doesn't it lock the drive when the computer is hibernating? It doesn't seem to!

What gives you that idea?

There is nothing that "unlocks" the drive, really, except that the filesystem driver gets the key to the drive. The drive itself is not changed in any way by entering its password or by ejecting it. It remains encrypted all the time.

When you are waking the machine from hibernation you have to enter the drive password so it can access the hibernation image file at all. Then when the system has been restored you need to enter your system account login password.

This is all exactly as it is supposed to be with an encrypted drive as far as I can see.

There's one more thing iljitsch should do/have/try: Get an 8GB USB stick and image the current LION ESD installer to it.

I never bought Lion (the Air was expensive enough, thankyouverymuch, and updating ALL your machines to the latest and greatest is risky, to say the least) so I don't have the installer.

Eh, that being the case I don't really think this

Quote:

So, if your internal drive gets wiped or replaced, there are only two options to reinstall Lion: over the Internet or from a Time Machine backup.

is justified. It's $29 + $8 for a USB stick. It just isn't that big a deal, and while upgrading straight off is always risky 6 months down the line it's time to get cracking, assuming the machine supports it at all of course. It also means not having to bother with a recovery partition at all, which is useful when using and SSD and not wanting to waste valuable space. Even if you don't feel like doing that, not mentioning it in the article does a disservice to readers who don't know any better. $40 is significantly cheaper then past updates and running of a USB stick is massively faster then optical media ever was, and for the vast majority of us is faster then our net connections too. People should know it's an option.

Having to read a manual to discover a hidden key combo is the complete opposite of intuitive. Hardly anyone would do that in advance, and the last thing you want to do when your computer's having problems minor enough to recover itself is read a manual.

It would be nice if Apple had one hot-key or action that would load a menu of all of the things you can do with all of their start-up shortcuts. Then we could all just remember one key combo. It would become iconic and show up on geek trendy t-shirts at MacWorlds. I can think of at least 6 start-up key combos without any thought and I know there's probably 3 or 4 times that many more that I don't know.

"In my defense, all these cryptic startup keyboard shortcuts are impossible to remember."

Forgive me, but your "defense" is a lace tissue of damp toilet paper flapping in the howling wind of your desperation for a scapegoat. But really, it's 2012 and the world is your audience… Why say "Oops!" and get on with your life when you can whine your way through an entire post?

I prefer having this on the backup drive so that's the only thing I need to restore my system even if the drive has been replaced with a completely empty one. (I guess Apple installs the OS for drive replacements under warranty / Apple Care?) I don't see how, with the latest version of the recovery image, having TM encrypted makes a difference.

It's nice to have a Lion USB drive available in a pinch if all else fails. It fits on most 4GB drives and you can create the drive yourself easily. I used Lion DiskMaker and it worked perfectly: http://blog.gete.net/lion-diskmaker-us/

deet: what I prefer is what the updated version of the recovery image gives you: that you can get access to encrypted drives after entering the password.

Constructor: I think what passivesmoking is getting at is that apparently, at least sometimes and possibly always, the disk encryption keys are kept in memory while the computer is sleeping. This allows for cold boot attacks. I've set my system to ask for a password after one hour of sleep or screen saving. (Interestingly, there doesn't seem to be an option to turn this off or use more than one hour.) So during the first hour of sleeping the keys are still in RAM. It's likely that the keys are always retained in RAM otherwise stuff like wake on LAN can't do anything useful.

xoa: it honestly never occurred to me to buy a copy of Lion and extract the installer to a USB drive as you describe. Obviously Apple feels you shouldn't need to do this, and when I open the app store Lion is listed as "installed".

And I guess I'm old school in the sense that I don't like depending on remote servers for important stuff such as restoring the OS after a drive failure, so that leaves the recovery partition as the most obvious solution. As for your valuable space: as far as I can tell without actually trying to do it, I think the only way to reclaim that 650 MB used by the recovery partition on your boot drive would be by partitioning the drive from the command line. "Not for the faint of heart" seems an understatement for such a procedure.

Been using Filevault 2 here on my laptop without any major issues. There's a couple of things that kind of bother me about it (doesn't it lock the drive when the computer is hibernating? It doesn't seem to!

What gives you that idea?

There is nothing that "unlocks" the drive, really, except that the filesystem driver gets the key to the drive. The drive itself is not changed in any way by entering its password or by ejecting it. It remains encrypted all the time.

When you are waking the machine from hibernation you have to enter the drive password so it can access the hibernation image file at all. Then when the system has been restored you need to enter your system account login password.

This is all exactly as it is supposed to be with an encrypted drive as far as I can see.

OK, by hibernate I meant sleep (Where the CPU is in standby, but the contents of memory is preserved by battery/mains power and the power light is blinking). When you power back up the drive already seems to be accessible to the operating system, because tasks pick up where they left off without you logging in.

xoa: it honestly never occurred to me to buy a copy of Lion and extract the installer to a USB drive as you describe. Obviously Apple feels you shouldn't need to do this, and when I open the app store Lion is listed as "installed".

Haha, yeah but sometimes Apple gets a little ahead of the rest of the world .

Quote:

And I guess I'm old school in the sense that I don't like depending on remote servers for important stuff such as restoring the OS after a drive failure, so that leaves the recovery partition as the most obvious solution.

Well, I made a USB stick straight off for precisely that reason, although in my case unlike most people I have a local server available to NetBoot/NetInstall off of. Nevertheless, I consider a local copy on a completely different piece of media to be better then either remote servers or a recovery partition, as I dislike counting on the very device that might be experiencing trouble. That it's tiny, fast and solid state is a nice bonus.

I don't blame you for not considering it, it's not like Apple goes out of their way to make it obvious despite it being very straight forward and requiring no third party software. But that's why we've got the Mac Ach .

Quote:

As for your valuable space: as far as I can tell without actually trying to do it, I think the only way to reclaim that 650 MB used by the recovery partition on your boot drive would be by partitioning the drive from the command line. "Not for the faint of heart" seems an understatement for such a procedure.

A number of ways to take care of it actually. Someone could even stumble straight into in the form of a drive upgrade, which I don't think is that uncommon. It is pretty standard procedure for a drive upgrade to just format the new drive, then use CCC or SuperDuper or whatever to simply dupe your existing one onto the new one, and then install the new one. This is not only straight forward but inherently leaves one with an instant fallback in case of error, but it will also result in no recovery partition unless you go to extra trouble to specifically make one. Even the MacBook Airs have SSD upgrades available, so I expect that will come up for some people even on the newest machines.

I consider a local copy on a completely different piece of media to be better then either remote servers or a recovery partition, as I dislike counting on the very device that might be experiencing trouble.

Hence my efforts to install a recovery partition on my Time Machine drive that can handle encrypted volumes. This way you boot from the recovery partition on the TM drive and reinstall everything to a blank internal drive when push comes to shove.

By the way, in the release notes of 10.7.2 Apple says "Enable booting into Lion Recovery from a locally connected Time Machine backup drive." I have absolutely no idea what they're trying to say here, though.

By the way, in the release notes of 10.7.2 Apple says "Enable booting into Lion Recovery from a locally connected Time Machine backup drive." I have absolutely no idea what they're trying to say here, though.

Can't help there, frankly I've never trusted TM and have therefore avoided it, and now that ZFS is available I'm pretty sure that will remain the case. That said it sounds like they're saying that if there's a TM volume attached via Firewire/USB or whatever it should be possible to just boot directly off of it as well as anything else you might have available. One nice thing about OS X has always been that it is extraordinarily forgiving about what it can boot from, so it sounds like this was just expanding that even further (perhaps by automatically tossing a recovery partition onto any updated TM or something).

xoa: it honestly never occurred to me to buy a copy of Lion and extract the installer to a USB drive as you describe. Obviously Apple feels you shouldn't need to do this, and when I open the app store Lion is listed as "installed".

hold down option when you click on "Purchases". doesn't always work the first time, but that'll get you the "Install" button eventually.

In reply to previous comments:>I have seen several computers bricked because of the original FileVault.

I don't think you know what "bricked" means. It means that the device is damaged in such a way that you can't restore it's functionality through software - i.e. it has to be sent back to the factory. This can happen with some devices if they are interrupted while updating the firmware, but it's not normally an issue for computers. At the very worse, they could have re-installed the OS to solve any problems FileVault 1 could have caused.

Also, since FileVault 1 only encrypts a DMG file of the user's home directory, so it couldn't cause problems for the OS. They could simply have logged in as another user.

>FileVault 2 does nothing to assuage my concerns.

Well, it's designed from the ground up using totally different technology, so if there are concerns, they will be totally different ones. It doesn't rely on huge files (like the DMG approach), and it can work gradually in the background, meaning there is no long waiting, and if it's interrupted half way through encrypting/decrypting, there is no problem. Also, since it's done at the block level, it can work with any filesystem, not just HFS+.

Iljitsch van Beijnum / Iljitsch is a contributing writer at Ars Technica, where he contributes articles about network protocols as well as Apple topics. He is currently finishing his Ph.D work at the telematics department at Universidad Carlos III de Madrid (UC3M) in Spain.