Search This Blog

Revisiting Android disk encryption

In iOS 8, Apple has expanded the scope of data encryption and now mixes in the user's passcode with an unextractable hardware UID when deriving an encryption key, making it harder to extract data from iOS 8 devices. This has been somewhat of a hot topic lately, with opinions ranging from praise for Apple's new focus on serious security, to demands for "golden keys" to mobile devices to be magically conjured up. Naturally, the debate has spread to other OS's, and Google has announced that the upcoming Android L release will also have disk encryption enabled by default. Consequently, questions and speculation about the usefulness and strength of Android's disk encryption have sprung up on multiple forums, so this seems like a good time to take another look at its implementation. While Android L still hasn't been released yet, some of the improvements to disk encryption it introduces are apparent in the preview release, so this post will briefly introduce them as well.

This post will focus on the security level of disk encryption, for more details on its integration with the platform, see Chapter 10 of my book -- 'Android Security Internals' (early access full PDF is available now, print books should ship by end of October).

Android 3.0-4.3

Full disk encryption (FDE) for Android was introduced in version 3.0 (Honeycomb) and didn't change much until version 4.4 (discussed in the next section). Android's FDE uses the dm-crypt target of Linux's device mapper framework to implement transparent disk encryption for the userdata (mounted as /data) partition. Once encryption is enabled, all writes to disk automatically encrypt data before committing it to disk and all reads automatically decrypt data before returning it to the calling process. The disk encryption key (128-bit, called the 'master key') is randomly generated and protected by the lockscreen password. Individual disk sectors are encrypted by the master key using AES in CBC mode, with ESSIV:SHA256 to derive sector IVs.

Android uses a so called 'crypto footer' structure to store encryption parameters. It is very similar to the encrypted partition header used by LUKS (Linux Unified Key Setup), but is simpler and omits several LUKS features. While LUKS supports multiple key slots, allowing for decryption using multiple passphrases, Android's crypto footer only stores a single copy of the encrypted master key and thus supports a single decryption passphrase. Additionally, while LUKS splits the encrypted key in multiple 'stripes' in order to reduce the probability of recovering the full key after it has been deleted from disk, Android has no such feature. Finally, LUKS includes a master key checksum (derived by running the master key through PBKDF2), which allows to check whether the entered passphrase is correct without decrypting any of the disk data. Android's crypto footer doesn't include a master key checksum, so the only way to check whether the entered passphrase is correct is to try and mount the encrypted partition. If the mount succeeds, the passphrase is considered correct.

The structure includes the version of the FDE scheme, the key size, some flags and the name of the actual disk encryption cipher mode (aes-cbc-essiv:sha256). The crypto footer is immediately followed by the encrypted key and a 16-bit random salt value. In this initial version, a lot of the parameters are implicit and are therefore not included in the crypto footer. The master key is encrypted using an 128-bit AES key (key encryption key, or KEK) derived from an user-supplied passphrase using 2000 iteration of PBKDF2. The derivation process also generates an IV, which is used to encrypt the master key in CBC mode. When an encrypted device is booted, Android takes the passphrase the user has entered, runs it through PBKDF2, decrypts the encrypted master key and passes it to dm-crypt in order to mount the encrypted userdata partition.

Bruteforcing FDE 1.0

The encryption scheme described in the previous section is considered relatively secure, but because it is implemented entirely in software, it's security depends entirely on the complexity of the disk encryption passphrase. If it is sufficiently long and complex, bruteforcing the encrypted master key could take years. However, because Android has chosen to reuse the losckreen PIN or password (maximum length 16 characters), in practice most people are likely to end up with a relatively short or low-entropy disk encryption password. While the PBKDF2 key derivation algorithm has been designed to work with low-entropy input, and requires considerable computational effort to bruteforce, 2000 iterations are not a significant hurdle even to current off-the-shelf hardware. Let's see how hard it is to bruteforce Android FDE 1.0 in practice.

Bruteforcing on the device is obviously impractical due to the limited processing resources of Android devices and the built-in rate limiting after several unsuccessful attempts. A much more practical approach is to obtain a copy of the crypto footer and the encrypted userdata partition and try to guess the passphrase offline, using much more powerful hardware. Obtaining a raw copy of a disk partition is usually not possible on most commercial devices, but can be achieved by booting a specialized data acquisition boot image signed by the device manufacturer, exploiting a flaw in the bootloader that allows unsigned images to be booted (such as this one), or simply by booting a custom recovery image on devices with an unlocked bootloader (a typical first step to 'rooting').

Once the device has been booted, obtaining a copy of the userdata partition is straightforward. The crypto footer however, despite its name, typically resides on a dedicated partition on recent devices. The name of the partition is specified using the encryptable flag in the device's fstab file. For example, on the Galaxy Nexus, the footer is on the metadata partition as shown below.

Once we know the name of the partition that stores the crypto footer it can be copied simply by using the dd command.

Very short passcodes (for example a 4-digit PIN) can be successfully bruteforced using a script (this particular one is included in Santoku Linux) that runs on a desktop CPU. However, much better performance can be achieved on a GPU, which has been specifically designed to execute multiple tasks in parallel. PBKDF2 is an iterative algorithm based on SHA-1 (SHA-2 can also be used) that requires very little memory for execution and lends itself to paralellization. One GPU-based, high-performance PBKDF2 implementation is found in the popular password recovery tool hashcat. Version 1.30 comes with a built-in Android FDE module, so recovering an Android disk encryption key is as simple as parsing the crypto footer and feeding the encrypted key, salt, and the first several sectors of the encrypted partition to hashcat. As we noted in the previous section, the crypto footer does not include any checksum of the master key, so the only way to check whether the decrypted master key is the correct one is to try to decrypt the disk partition and look for some known data. Because most current Android devices use the ext4 filesystem, hashcat (and other similar tools) look for patterns in the ext4 superblock in order to confirm whether the tried passphrase is correct.

The Android FDE input for hashcat includes the salt, encrypted master key and the first 3 sectors of the encrypted partition (which contain a copy of the 1024-byte ext4 superblock). The hashcat input file might look like this (taken from the hashcat example hash):

Even when run on the GPU of a mobile computer (NVIDIA GeForce 730M), hashcat can achieve more than 20,000 PBKDF2 hashes per second, and recovering a 6 digit PIN takes less than 10 seconds. On the same hardware, a 6-letter (lowercase only) password takes about 4 hours.

As you can see, bruteforcing a simple PIN or password is very much feasible, so choosing a strong lockscreen password is vital. Lockscreen password strength can be enforced by installing a device administrator that sets password complexity requirements. Alternatively, a dedicated disk encryption password can be set on rooted devices using the shell or a dedicated application. CyanogenMod 11 supports setting a dedicated disk encryption password out of the box, and one can be set via system Settings, as shown below.

Android 4.4

Android 4.4 adds several improvements to disk encryption, but the most important one is replacing the PBKDF2 key derivation function (KDF) with scrypt. scrypt has been specifically designed to be hard to crack on GPUs by requiring a large (and configurable) amount of memory. Because GPUs have a limited amount of memory, executing multiple scrypt tasks in parallel is no longer feasible, and thus cracking scrypt is much slower than PBKDF2 (or similar hash-based KDFs). As part of the upgrade process to 4.4, Android automatically updates the crypto footer to use scrypt and re-encrypts the master key. Thus every device running Android 4.4 (devices using a vendor-proprietary FDE scheme excluded) should have its FDE master key protected using an scrypt-derived key.

As you can see, the footer now includes an explicit kdf_type which specifies the KDF used to derive the master key KEK. The values of the scrypt initialization parameters (N, r and p) are also included. The master key size (128-bit) and disk sector encryption mode (aes-cbc-essiv:sha256) are the same as in 4.3.

Bruteforcing the master key now requires parsing the crypto footer, initializing scrypt and generating all target PIN or password combinations. As the 1.2 crypto footer still does not include a master key checksum, checking whether the tried PIN or password is correct again requires looking for known plaintext in the ext4 superblock.

While hashcat does support scrypt since version 1.30, it is not much more efficient (and in fact can be slower) than running scrypt on a CPU. Additionally, the Android 4.4 crypto footer format is not supported, so hashcat cannot be used to recover Android 4.4 disk encryption passphrases as is.

Instead, the Santoku Linux FDE bruteforcer Python script can be extended to support the 1.2 crypto footer format and the scrypt KDF. A sample (and not particularly efficient) implementation can be found here. It might produce the following output when run on a 3.50GHz Intel Core i7 CPU:

As you can see, trying 1200 PIN combinations requires almost 5 minutes, so recovering a simple PIN is no longer instantaneous. That said, cracking a short PIN or password is still very much feasible, so choosing a strong locksreen password (or a dedicated disk encryption password, when possible) is still very important.

Android L

A preview release of the upcoming Android version (referred to as 'L') has been available for several months now, so we can observe some of expected changes to disk encryption. If we run the crypto footer obtained from an encrypted Android L device through the script introduced in the previous section, we may get the following output:

As you can see above, the crypto footer version has been upped to 1.3, but the disk encryption cipher mode and key size have not changed. However, version 1.3 uses a new, unknown KDF specified with the constant 3 (1 is PBKDF2, 2 is scrypt). Additionally, encrypting a device no longer requires setting a lockscreen PIN or password, which suggests that the master key KEK is no longer directly derived from the lockscreen password. Starting the encryption process produces the following logcat output:

As discussed in a previous post, 'QSEE' stands for Qualcomm Secure Execution Environment, which is an ARM TrustZone-based implementation of a TEE. QSEE provides the hardware-backed credential store on most devices that use recent Qualcomm SoCs. From the log above, it appears that Android's keymasterHAL module has been extended to store the disk encryption key KEK in hardware-backed storage (Cf. 'Using scrypt with keymaster for cryptfs KDF' in the log above). The log also mentions scrypt, so it is possible that the lockscreen password (if present) along with some key (or seed) stored in the TEE are fed to the KDF to produce the final master key KEK. However, since no source code is currently available, we cannot confirm this. That said, setting an unlock pattern on an encrypted Android L device produces the following output, which suggests that the pattern is indeed used when generating the encryption key:

As you can be see in the listing above, the cryptfs changepw command, which is used to send instructions to Android's vold daemon, has been extended to support a pattern, in addition to the previously supported PIN/password. Additionally, the amount of time the password change takes (6 seconds) suggests that the KDF (scrypt) is indeed being executed to generate a new encryption key. Once we've set a lockscreen unlock pattern, booting the device now requires entering the pattern, as can be seen in the screenshot below. Another subtle change introduced in Android L, is that when booting an encrypted device the lockscreen pattern, PIN or password needs to be entered only once (at boot time), and not twice (once more on the lockscreen, after Android boots) as it was in previous versions.

While no definitive details are available, it is fairly certain that (at least on high-end devices), Android's disk encryption key(s) will have some hardware protection in Android L. Assuming that the implementation is similar to that of the hardware-backed credential store, disk encryption keys should be encrypted by an unextractable key encryption key stored in the SoC, so obtaining a copy of the crypto footer and the encrypted userdata partition, and bruteforcing the lockscreen passphrase should no longer be sufficient to decrypt disk contents. Disk encryption in the Android L preview (at least on a Nexus 7 2013) feels significantly faster (encrypting the 16GB data partition takes about 10 minutes), so it is most probably hardware-accelerated as well (or the initial encryption is only encrypting disk blocks that are actually in use, and not every single block as in previous versions). However, it remains to be seen whether high-end Android L devices will include a dedicated crypto co-processor akin to Apple's 'Secure Enclave'. While the current TrustZone-based key protection is much better than the software only implementation found in previous versions, a flaw in the secure TEE OS or any of the trusted TEE applications could lead to extracting hardware-protected keys or otherwise compromising the integrity of the system.

Update 2014/11/4: The official documentation about disk encryption has been updated, including details about KEK protection. Quote:

The encrypted key is stored in the crypto metadata. Hardware backing is implemented by using Trusted Execution Environment’s (TEE) signing capability. Previously, we encrypted the master key with a key generated by applying scrypt to the user's password and the stored salt. In order to make the key resilient against off-box attacks, we extend this algorithm by signing the resultant key with a stored TEE key. The resultant signature is then turned into an appropriate length key by one more application of scrypt. This key is then used to encrypt and decrypt the master key. To store this key:

Generate random 16-byte disk encryption key (DEK) and 16-byte salt.

Apply scrypt to the user password and the salt to produce 32-byte intermediate key 1 (IK1).

Summary

Android has included full disk encryption (FDE) support since version 3.0, but versions prior to 4.4 used a fairly easy to bruteforce key derivation function (PBKDF2 with 2000 iterations). Additionally, because the disk encryption password is the same as the lockscreen one, most users tend to use simple PINs or passwords (unless a device administrator enforces password complexity rules), which further facilitates bruteforcing. Android 4.4 replaced the disk encryption KDF with scrypt, which is much harder to crack and cannot be implemented efficiently on off-the-shelf GPU hardware. In addition to enabling FDE out of the box, Android L is expected to include hardware protection for disk encryption keys, as well as hardware acceleration for encrypted disk access. These two features should make FDE on Android both more secure and much faster.

[Note that the discussion in this post is based on "stock Android" as released by Google (references source code is from AOSP). Some device vendors implement slightly different encryption schemes, and hardware-backed key storage and/or hardware acceleration are already available via vendor extensions on some high-end devices.]

Comments

You took an example with a platform that uses Trustzone and HW-based crypto module.What will happen is Release L runs on top of a platform that does not support these features? Will it run the same FDE that can be found in Android 4.4?

I read your blog post but to be honest, most of it I don't understand.

However, on Linux I use LUKS/dm-crypt which allows me to use a password for the disk encryption and a seperate one for my user account. So, I chose a long password for the disk encryption and a less long one for my user account.

As you pointed out, up to 4.4 android used the same password for encryption as for screen unlock. So I really welcomed the change in 4.4.

However now in Android L this seems to have changed again. The screen unlock password seems to be part of the disk encryption password again - sorry, maybe I just don't really understand what it does.

So, I wonder, how secure is L encryption then? For me having two seperate passwords, one for encryption which is really long, seems to be way more secure. HOwever you said something about the screen unlock password being padded to 16 characters. Does that mean that also a "short" password provides good security?

There is only one password in Android, this hasn't changed. It is not padded to 16 characters, those are implementations details of the key derivation algorithm. What has changed in Lollipop (Android 5.0) is that you cannot decrypt a disk image without the actual device. Therefore, even a short PIN/password is more secure than 4.4 and earlier. Longer is, of course, better.

Very well written article, thank you! One thing I wanted to point out is that AOSP up to 4.4 did not *necessarily* use the same password for protecting the key as for the lockscreen/PIN. The framework supports different passwords, but the UI simply does not. If you have root access, you can fix it yourself using vdc directly or (more comfortably) using this open source app:

I would welcome it if you could post some instruction how to access the data on the data partition you dumped off your Android device (up to 4.4), that is using i.e. cryptsetup and scrypt. If you could enhance your python script for this task this would be even cooler!

Actually not true. In 4.4 you can have different passwords for encryption and screen unlock and you can alter them seperately: http://images.sjau.ch/img/4c5b005b.png . Well, not sure if that's just the case in CM though.

I'm curious if you've had a chance to revisit this since the sources were made available? Even after reading the encryption docs, I'm still a bit confused about where the DEK is actually stored now that they are calling the encryption key "hardware-backed". It seems like the key isn't actually resident in the TEE, but rather they've just included the TEE-private key as an additional step in the encryption of the key that is stored.

From what I can tell, the encrypted key is still persisted to the crypto footer partition, and is stored fully decrypted in the RAM space of vold once booted. I was hopeful the TEE backing meant the decrypted key would be stored there instead of RAM. Does that seem correct to you?

I'm also curious if you saw any direct references to vold now requiring (or at least leveraging when available) hardware crypto functions to speed up dm-crypt?

You are correct on the first two points, see the diagram I added, hopefully it makes the process clearer (I should really revise the whole part about L...). As for hardware acceleration, vold only decrypts the DEK, the actual disk encryption is done by dm-crypt, which is in the kernel. There is a hardware-accelerated version for Qualcomm devices (used in the One PlusOne) which takes advantage of the AES accelerator. The AOSP version only uses software. If you have a recent kernel and a ARMv8 device, dm-crypt *should* take advantage of ARMv8's native AES instructions (via the kernel crypto provider).

No, it is stored in the crypto footer (see diagram). The key that encrypts it is accessed via a TrustZone trusted app, and is most probably stored in the SoC (hence 'hardware-bound'. You probably need to sign an NDA with Qualcomm, etc. to get the details

Rate limiting has been there for a while, it's not directly related to the hardware-bound key. IIRC, you have to wait 30 secs after 10 unsuccessful tries. Android will wipe after 30 tries by default, you can change this with a device admin.

I mean rate limiting in the case where the attacker has full control over the flash. (which is where full-disk encryption really matters, right?) So, they can run whatever they want on the CPU in this case and bypass the rate limits imposed by software that normally runs on the Android side. Will access to the hardware-bound key slow down the attacker beyond just whatever the throughput of the encryption/decryption engine is?

I don't think this is part of the threat model. The hardware-bound key prevents offline attacks on faster hardware. I don't think there is any rate limiting when accessing HW-bound keys, but that part is proprietary, so you'd have to analyze timing, etc. to be sure.

"Rate limiting has been there for a while, it's not directly related to the hardware-bound key. IIRC, you have to wait 30 secs after 10 unsuccessful tries. Android will wipe after 30 tries by default, you can change this with a device admin."After the 2 x 30 second wait and the count down from 10 to reset to factory starts, is there a way to re-set the failed attempts?

If you can boot into recovery and modify the crytpfooter, you can reset to 0. Currently (6.0) there is no TEE or dm-verity involved. For this you need manufacturer keys or unlocked bootloader, of course. If the bootloder is locked, there is not much you can do, short of a bootloader exploit.

Thanks for your quick reply.I can get into recovery mode and from ADB can start a shell, from there I can mount /system BUT NOT /data as it's encrypted.Unfortunately unlike your example above where you see "encryptable=/dev/block/platform/omap/omap_hsmmc.0/by-name/metadata"I only have "encryptable=footer", if I understand correctly it should be the last 16Kb of the undecrypted userdata-partition (but I cannot mount /data).How can I view or chance the crytpfooter to reset failed attempts to 0. I am on the last attempt, my thinking is if I can reset it back to 10 or 9 attempts (0 or 1 failed) I will be able to try a few regular p/w's and get back into the phone without loosing anything. I am trying this on a Samsung S5.

Yes, if it says 'footer', it's an actual footer, and is right after the filesystem. Your best bet is to take an image of the partition on a USB drive, etc. (IIRC, TWRP allows you to do this). DO NOT mount /data when the recovery boots, just find out the partition name and dump it to USB. If you can't dump the whole thing, things get a bit more tricky -- you need to parse the ext4 header to find out where the filesystem ends, and then dump just the footer with dd, specifying the end of the FS as the starting offset.

Once you have a backup image, you can play with the footer. Alternatively, just use a recovery that supports your device. I don't *think* the recovery will auto-wipe, but you have to check the source to be sure.

Is it possible to have absolute security of the content in an encrypted device today with Android 6.0.1 o 7? Will it be possible someday? Is it possible in Apple? How does Android stands against Apple in terms of security (encryption) now?

Popular posts from this blog

One of the new features Android M introduces is adoptable storage. This feature allows external storage devices such as SD cards or USB drives to be 'adopted' and used in the same manner as internal storage. What this means in practice is that both apps and their private data can be moved to the adopted storage device. In other words, this is another take on everyone's (except for widget authors...) favorite 2010 feature -- AppsOnSD. There are, of course, a few differences, the major one being that while AppsOnSD (just like app Android 4.1 app encryption) creates per-app encrypted containers, adoptable storage encrypts the whole device. This short post will look at how adoptable storage encryption is implemented, and show how to decrypt and use adopted drives on any Linux machine.Adopting an USB driveIn order to enable adoptable storage for devices connected via USB you need to execute the following command in the Android shell (presumably, this is not needed if your devic…

One of the less known new features introduced in ICS is the ability to backup a device to a file on your computer via USB. All you have to do is enable USB debugging, connect your phone to a computer and type the adb backup command in a shell. That will show a confirmation dialog on the phone prompting you to authorize the backup and optionally specify a backup encryption password. It looks something like this:

This doesn't require rooting your phone and lets you backup application data, both user installed and system applications (APK's), as well as shared storage (SD card) contents. There are some limitations though: it won't backup apps that have explicitly forbidden backups in their manifest, it won't backup protected (with DRM) apps and it won't backup some system settings such as APN's and WiFi access points. The transfer speed is limited by ADB channel speed (less than 1MB/s), so full backups can take quite some time. There is also a rather annoying bug …

It seems we somehow managed to let two months slip by without a single post. Time to get back on track, and the recently unveiled Android maintenance release provides a nice opportunity to jump start things. Official release notes for Android 4.2.2 don't seem to be available at this time, but it made its way into AOSP quite promptly, so you can easily compile your own changelog based on git log messages. Or, you can simply check the now traditional one over at Funky Android. As you can see, there are quite a few changes, and if you want a higher level overview your time would probably be better spent reading some of the related posts by the usual suspects. Deviating from our usually somewhat obscure topics, we will focus on a new security feature that is quite visible and has received a fair bit of attention already. It was even introduced on the official Android Developers Blog, fortunately for us only in brief. As usual, we like to dig a little deeper, so if you are interested i…