Search This Blog

Password storage in Android M

While Android has received a number of security enhancements in the last few releases, the lockscreen (also know as the keyguard) and password storage have remained virtually unchanged since the 2.x days, save for adding multi-user support. Android M is finally changing this with official support for fingerprint authentication. While the code related to biometric support is currently unavailable, some of the new code responsible for password storage and user authentication is partially available in AOSP's master branch. Examining the runtime behaviour and files used by the current Android M preview reveals that some password storage changes have already been deployed. This post will briefly review how password storage has been implemented in pre-M Android versions, and then introduce the changes brought about by Android M.

Keyguard unlock methods

Stock Android provides three keyguard unlock methods: pattern, PIN and password (Face Unlock has been rebranded to 'Trusted face' and moved to the proprietary Smart Lock extension, part of Google Play Services). The pattern unlock is the original Android unlock method, while PIN and password (which are essentially equivalent under the hood) were added in version 2.2. The following sections will discuss how credentials are registered, stored and verified for the pattern and PIN/password unlock methods.

Pattern unlock

Android's pattern unlock is entered by joining at least four points on a 3×3 matrix (some custom ROMs allow a bigger matrix). Each point can be used only once (crossed points are disregarded) and the maximum number of points is nine. The pattern is internally converted to a byte sequence, with each point represented by its index, where 0 is top left and 8 is bottom right. Thus the pattern is similar to a PIN with a minimum of four and maximum of nine digits which uses only nine distinct digits (0 to 8). However, because points cannot be repeated, the number of variations in an unlock pattern is considerably lower compared to those of a nine-digit PIN. As pattern unlock is the original and initially sole unlock method supported by Android, a fair amount of research has been done about it's (in)security. It has been shown that patterns can be guessed quite reliably using the so called smudge attack, and that the total number of possible combinations is less than 400 thousand, with only 1624 combinations for 4-dot (the default) patterns.

Android stores an unsalted SHA-1 hash of the unlock pattern in /data/system/gesture.key or /data/system/users/<user ID>/gesture.key on multi-user devices. It may look like this for the 'Z' pattern shown in the screenshot above.

Because the hash is unsalted, it is easy to precompute the hashes of all possible combinations and recover the original pattern instantaneously. As the number of combinations is fairly small, no special indexing or file format optimizations are required for the hash table, and the grep and xxd commands are all you need to recover the pattern once you have the gesture.key file.

PIN/password unlock

The PIN/password unlock method also relies on a stored hash of the user's credential, however it also uses a 64-bit random, per-user salt. The salt is stored in the locksettings.db SQLite database, along with other settings related to the lockscreen. The password hash is kept in the /data/system/password.key file, which contains a concatenation of the password's SHA-1 and MD5 hash values. The file's contents may look like this:

Note that the hashes are not nested, but their values are simply concatenated, so if you were to bruteforce the password, you only need to attack the weaker hash -- MD5. Another helpful fact is that in order to enable password auditing, Android stores details about the current PIN/password's format in the device_policies.xml file, which might look like this:

If you were able to obtain the password.key file, chances are that you would also have the device_policies.xml file. This file gives you enough information to narrow down the search space considerably when recovering the password by specifying a mask or password rules. For example, we can easily recover the following 6-digit pin using John the Ripper (JtR) in about a second by specifying the ?d?d?d?d?d?d mask and using the 'dynamic' MD5 hash format (hashcat has a dedicated Android PIN hash mode), as shown below . An 8-character (?l?l?l?l?l?l?l?l), lower case only password takes a couple of hours on the same hardware.

Android's lockscreen password can be easily reset by simply deleting the gesture.key and password.key files, so you might be wondering what is the point in trying to bruteforce it. As discussed in previous posts, the lockscreen password is used to derive keys that protect the keystore (if not hardware-backed), VPN profile passwords, backups, as well as the disk encryption key, so it might be valuable if trying to extract data from any of these services. And of course, the chance that a particular user is using the same pattern, PIN or password on all of their devices is quite high.

Gatekeeper password storage

We briefly introduced Android M's gatekeeper daemon in the keystore redesign post in relation to per-key authorization tokens. It turns out the gatekeeper does much more than that and is also responsible for registering (called 'enrolling') and verifying user passwords. Enrolling turns a plaintext password into a so called 'password handle', which is an opaque, implementation-dependent byte string. The password handle can then be stored on disk and used to check whether a user-supplied password matches the currently registered handle. While the gatekeeper HAL does not specify the format of password handles, the default software implementation uses the following format:

Here secure_id_t is randomly generated, 64-bit secure user ID, which is persisted in the /data/misc/gatekeeper directory in a file named after the user's Android user ID (*not* Linux UID; 0 for the primary user). The signature format is left to the implementation, but AOSP's commit log reveals that it is most probably scrypt for the current default implementation. Other gatekeeper implementations might opt to use a hardware-protected symmetric or asymmetric key to produce a 'real' signature (or HMAC).

Neither the HAL, nor the currently available AOSP source code specifies where password handles are to be stored, but looking through the /data/system directory reveals the following files, one of which happens to be the same size as the password_handle_t structure. This implies that it likely contains a serialized password_handle_t instance.

That's quite a few assumptions though, so time to verify them by parsing the gatekeeper.gesture.key file and checking if the signature field matches the scrypt value of our lockscreen pattern (00010204060708 in binary representation). We can do so with the following Python code:

The program output above leads us to believe that the 'signature' stored in the password handle file is indeed the scrypt value of the blob's version, the 64-bit secure user ID, and the blob's flags field, concatenated with the plaintext pattern value. The scrypt hash value is calculated using the stored 64-bit salt and the scrypt parameters N=16384, r=8, p=1. Password handles for PINs or passwords are calculated in the same way, using the PIN/password string value as input.

With this new hashing scheme patterns and passwords are treated in the same way, and thus patterns are no longer easier to bruteforce. That said, with the help of the device_policies.xml file which gives us the length of the pattern and a pre-computed pattern table, one can drastically reduce the number of patterns to try, as most users are likely to use 4-6 step patterns (about 35,000 total combinations) .

Because Androd M's password hashing scheme doesn't directly use the plaintext password when calculating the scrypt value, optimized password recovery tools such as hashcat or JtR cannot be used directly to evaluate bruteforce cost. It is however fairly easy to build our own tool in order to check how a simple PIN holds against a brute force attack, assuming both the device_policies.xml and gatekeeper.password.key files have been obtained. As can be seen below, a simple Python script that tries all PINs from 0000 to 9999 in order takes about 10 minutes, when run on the same hardware as our previous JtR example (a 6-digit PIN would take about 17 hours with the same program). Compare this to less than a second for bruteforcing a 6-digit PIN for Android 5.1 (and earlier), and it is pretty obvious that the new hashing scheme Android M introduces greatly improves password storage security, even for simple PINs. Of course, as we mentioned earlier, the gatekeeper daemon is part of Android's HAL, so vendors are free to employ even more (or less...) secure gatekeeper implementations.

Framework API

Android M is still in preview, so framework APIs are hardly stable, but we'll show the gatekeeper's AIDL interface for completeness. In the current preview release it is called IGateKeeperService and look likes this:

As you can see, the interface provides methods for generating/getting and clearing the secure user ID for a particular user, as well as the enroll(), verify() and verifyChallenge() methods whose parameters closely match the lower level HAL interface. To verify that there is a live service that implements this interface, we can try to call the getSecureUserId() method using the service command line utility like so:

The actual storage of password hashes (handles) is carried out by the LockSettingsService (interface ILockSettings), as in previous versions. The service has been extended to support the new gatekeeper password handle format, as well as to migrate legacy hashes to the new format. It is easy to verify this by calling the checkPassword(String password, int userId) method which returns true if the password matches:

Summary

Android M introduces a new system service -- gatekeeper, which is responsible for converting plain text passwords to opaque binary blobs (called password handles) which can be safely stored on disk. The gatekeeper is part of Android's HAL, so it can be modified to take advantage of the device's native security features, such as secure storage or TEE, without modifying the core platform. The default implementation shipped with the current Android M preview release uses scrypt to hash unlock patterns, PINs or passwords, and provides much better protection against bruteforceing than the previously used single-round MD5 and SHA-1 hashes.

Comments

Hello Nikolay,Are you aware of any hardware backed lock screen implementations with TrustZone that enforce a delay between authentication attempts? For example, on iOS, SecureEnclave forces a 5 second delay between authentication attempts. It's not very clear whether HAL provides this.Thanks,Lumus

How can i store application credentials on the device and use a fingerprint scanner to retrieve them and send to the API? http://stackoverflow.com/questions/34448221/android-keychain-like-storage-for-username-password?noredirect=1#comment56638397_34448221

I added following imports to m-pass-hash.pyimport structimport binascii

But I get following error when executing the script. Could you please let me know what is the library you imported scrypt. Better if you can share the whole script. hash = scrypt.hash(to_hash, salt, N, r, p)NameError: name 'scrypt' is not defined

I cant unlock my coolpad note 3 by right pattern,only unlocking using fingerprint is possible.if i replace my gatekeeper.pattern.key with another would it change successfully? Or is there any other file do i have to modify? Please help.. give me a default gatekeeper.pattern.key file if possible

Hey, I 've read this things, and it is very cool...Can you tell me how have you copied the gatekeeper.password.key? I want copy it with ADBm but anytime I failed. I do this script:adb shellsucd /data/system/chmod 666 gatekeeper.password.keycp gatekeeper.password.key /storage/sdcard0/And when i try to open the flie on my pc...The file doesn't exist. But if I check on my phone (with Root Explorer) it exist...How I can see it on my PC. What's I've wronged? Thanks for your reply(And sorry for my english XD)

i try the way,They are not equal。devices is nexus5 android6.0;key have 58 byte;Is this method available?signature f483f6f5e0c60c40db4d4e0725ec2667a3e3534ec0e3d688b21b9fc3eecc9ef5Hash: e6fc75fec85f29842c3d594e57a146ab45e86874b0f9811adb9ab102b7de6077Equal: False-------by the way , can dynamic debugging trace the encryption function in dvices。My native language is not English ，，sorry！

Popular posts from this blog

One of the new features Android M introduces is adoptable storage. This feature allows external storage devices such as SD cards or USB drives to be 'adopted' and used in the same manner as internal storage. What this means in practice is that both apps and their private data can be moved to the adopted storage device. In other words, this is another take on everyone's (except for widget authors...) favorite 2010 feature -- AppsOnSD. There are, of course, a few differences, the major one being that while AppsOnSD (just like app Android 4.1 app encryption) creates per-app encrypted containers, adoptable storage encrypts the whole device. This short post will look at how adoptable storage encryption is implemented, and show how to decrypt and use adopted drives on any Linux machine.Adopting an USB driveIn order to enable adoptable storage for devices connected via USB you need to execute the following command in the Android shell (presumably, this is not needed if your devic…

One of the less known new features introduced in ICS is the ability to backup a device to a file on your computer via USB. All you have to do is enable USB debugging, connect your phone to a computer and type the adb backup command in a shell. That will show a confirmation dialog on the phone prompting you to authorize the backup and optionally specify a backup encryption password. It looks something like this:

This doesn't require rooting your phone and lets you backup application data, both user installed and system applications (APK's), as well as shared storage (SD card) contents. There are some limitations though: it won't backup apps that have explicitly forbidden backups in their manifest, it won't backup protected (with DRM) apps and it won't backup some system settings such as APN's and WiFi access points. The transfer speed is limited by ADB channel speed (less than 1MB/s), so full backups can take quite some time. There is also a rather annoying bug …

It seems we somehow managed to let two months slip by without a single post. Time to get back on track, and the recently unveiled Android maintenance release provides a nice opportunity to jump start things. Official release notes for Android 4.2.2 don't seem to be available at this time, but it made its way into AOSP quite promptly, so you can easily compile your own changelog based on git log messages. Or, you can simply check the now traditional one over at Funky Android. As you can see, there are quite a few changes, and if you want a higher level overview your time would probably be better spent reading some of the related posts by the usual suspects. Deviating from our usually somewhat obscure topics, we will focus on a new security feature that is quite visible and has received a fair bit of attention already. It was even introduced on the official Android Developers Blog, fortunately for us only in brief. As usual, we like to dig a little deeper, so if you are interested i…