Search This Blog

Decrypting Android M adopted storage

One of the new features Android M introduces is adoptable storage. This feature allows external storage devices such as SD cards or USB drives to be 'adopted' and used in the same manner as internal storage. What this means in practice is that both apps and their private data can be moved to the adopted storage device. In other words, this is another take on everyone's (except for widget authors...) favorite 2010 feature -- AppsOnSD. There are, of course, a few differences, the major one being that while AppsOnSD (just like app Android 4.1 app encryption) creates per-app encrypted containers, adoptable storage encrypts the whole device. This short post will look at how adoptable storage encryption is implemented, and show how to decrypt and use adopted drives on any Linux machine.

Adopting an USB drive

In order to enable adoptable storage for devices connected via USB you need to execute the following command in the Android shell (presumably, this is not needed if your device has an internal SD card slot; however there are no such devices that run Android M at present):

$ adb shell sm set-force-adoptable true

Now, if you connect a USB drive to the device's micro USB slot (you can also use an USB OTG cable), Android will give you an option to set it up as 'internal' storage, which requires reformatting and encryption. 'Portable' storage is formatted using VFAT, as before.

After the drive is formatted, it shows up under Device storage in the Storage screen of system Settings. You can now migrate media and application data to the newly added drive, but it appears that there is no option in the system UI that allows you to move applications (APKs).

Adopted devices are mounted via Linux's device-mapper under /mnt/expand/ as can be seen below, and can be directly accessed only by system apps.

You can safely eject an adopted drive by tapping on it in the Storage screen, and the choosing Eject from the overflow menu. Android will show a persistent notification that prompts you to reinsert the device once it's removed. Alternatively, you also can 'forget' the drive, which removes it from the system, and should presumably delete the associated encryption key (which doesn't seem to be the case in the current preview build).

Inspecting the drive

Once you've ejected the drive, you can connect it to any Linux box in order to inspect it. Somewhat surprisingly, the drive will be automatically mounted on most modern Linux distributions, which suggests that there is at least one readable partition. If you look at the partition table with fdisk or a similar tool, you may see something like this:

As you can see, there is a tiny android_meta partition, but the bulk of the device has been assigned to the android_expand partition. Unfortunately, the full source code of Android M is not available, so we cannot be sure how exactly this partition table is created, or what the contents of each partition is. However, we know that most of Android's storage management functionality is implemented in the vold system daemon, so we check if there is any mention of android_expand inside vold with the following command:

Here expand_%s.key suspiciously looks like a key filename template, and we already know that adopted drives are encrypted, so our next step is to look for any similar files in the device's /data partition (you'll need a custom recovery or root access for this). Unsurprisingly, there is a matching file in /data/misc/vold which looks like this:

Decrypting the drive

That's exactly 16 bytes, enough for a 128-bit key. As we know, Android's FDE implementation uses an AES 128-bit key, so it's a good bet that adoptable storage uses a similar (or identical) implementation. Looking at the start and end of our android_expand partition doesn't reveal any readable info, nor is it similar to Android's crypto footer, or LUKS's header. Therefore, we need to guess the encryption mode and/or any related parameters. Looking once again at Android's FDE implementation (which is based on the dm-crypt target of Linux's device-mapper), we see that the encryption mode used is aes-cbc-essiv:sha256. After consulting dm-crypt's mapping table reference, we see that the remaining parameters we need are the IV offset and the starting offset of encrypted data. Since the IV offset is usually zero, and most probably the entire android_expand partition (offset 0) is encrypted, the command we need to map the encrypted partition becomes the following:

It completes with error, so we can now try to mount the mapped device, again guessing that the file system is most probably ext4 (or you can inspect the mapped device and find the superblock first, if you want to be extra diligent).

This reveals a very familiar Android /data layout, and you should see any media and app data you've moved to the adopted device. If you copy any files to the mounted device, they should be visible when you mount the drive again in Android.

Storage manager commands

Back in Android, you can use the sm command (probably short for 'storage manager') we showed in the first section to list disks and volumes as shown below:

Most features are also available from the system UI, but sm allows you to customize the ratio of the android_meta and android_expand partitions, as well as to create 'mixed' volumes.

Summary

Android M allows for adoptable storage, which is implemented similarly to internal storage FDE -- using dm-crypt with a per-volume, static 128-bit AES key, stored in /data/misc/vold/. Once the key is extracted from the device, adopted storage can be mounted and read/written on any Linux machine. Adoptable storage encryption is done purely in software (at least in the current preview build), so its performance is likely comparable to encrypted internal storage on devices that don't support hardware-accelerated FDE.

Comments

Could you possibly elaborate a bit more on how adoptable storage is exposed to applications? The preview SDK docs are unfortunately not very helpful.

- Is an encrypted card usable for media files without migrating just like an unencrypted card, i.e. is migration decoupled from encryption?- Where is the primary shared storage mounted, and does that change after migrating application data? What does Context.getExternalFilesDirs() return in each case?- After migration, is the internal (emulated) shared storage still accessible (via file managers and/or the external storage APIs), or is it completely shadowed by the external shared storage?- Where are APK expansion files (OBBs) stored, before and after migration?

Encryption is separate from migration. You can have multiple encrypted volumes mounted and migrate to only one (or none) of them. External storage is implemented the same way as before, i.e. emulated using FUSE on top of the media/ directory of the 'real' volume. It does look you can only have one 'primary external storage' at a time, but not 100% about this without the code.

Many people have been stuck with adopted internal storage, because they then have no SD card to which Google Play Music can store the music; and as Google Play Music won't move to the adopted SD card, all music is stored in the internal memory. For me, I have more music than Apps, so this just makes things worse.

However your reference to "sm" and "mixed" gave me something to work with.

My adopted memory card stopped working. So I pressed 'Forget the card'.Now I want to use the card as portable in my phone.Can't seem to format it either as portable card or internal card.Will this technique help? If yes, then great I'm gonna try it and let you know.If no please guide me through the way which may help me.

Hi, I configured my micro sd as internal storage in my moto g3 and I foolishly reseted the phone. So, now I can't access to all my photos because android tells me that I have to format the sd. The problem is that I really need to recover all my information. Is there any way in which I could unlocked the sd?I'll apreciate your help since I am really desperate and nobody can give me any solution.Thanks.

Bought a new Samsung 16GB SD Card. When i try to set-it up and format it as internal memory, settings app crashes and later when i open, it says SD card Corrupted. Im running on CM13 CONDOR NIGHTLY. Thanks in advance.

Thank you sir!!! You saved my life. Android wouldn't mount my microsd card anymore due to some corrupt directories. (I used adb shell and ran dmesg to see what's going on). I was able to backup over 18GB of precious family pictures by decrypting it from my ubuntu desktop. You are awesome!

hello I have a problem with my sd card, I had configured in android marshmallow like internal storage but I had to reinstall the rom and when I put the micro sd card I get as damaged and asks me to format but have important files , some form of they rescue these files or lost forever

Hi thanks for your contribution. I would like to consult you something because in my case I have a little more complicated. It has happened the same case as you, but additionally I did a hard reset the cell phone, which, I think I do not have that .key file. Without the key file there is no possibility that can decrypt the file system?

Hello all,I have recently upgraded my phone to Android 6.0Everything worked fine until i formatted my SD card to Internal.After the conversion, it was working well. I was pretty much satisfied. With that confidence, i moved all my apps from Internal to SD card.After some minutes my phone got struck, so i restarted by removing and placing the battery in the back.

When it booted up it didn't detect my SD card and kept showing an error that, i need to Forget my SD card.i googled the condition and found a post explaining the issue that it is Android 6.0's bug, and it's highly recommended to not move apps from Internal to SD card without a reboot, after converting the SD card to Internal.

The post also explained about wiping the SD card, as the only option left, and instructed to fastboot the device.

For some unknown reasons, the default "hold press Power button and press Volume Down" didn't work in my phone and it took me to a meta mode, where nothing is displayed apart from saying that it is a meta mode.

Then i took a wrong turn by repeating the same by pressing the Power and Volume Up button. It took me to factory mode, where i was displayed with 6 options in Chinese.

I can't read Chinese, and one of the option had an English word "eMMC", i thought this was the option to wipe out the SD card and selected it. It restored my phone to factory settings by erasing the Internal memory instead of the SD card.

Now, my phone detects my SD card, but shows it as "Unsupported" under Storage and USB option. And it asks me to Set Up the SD card by formatting it.There are two options, "Use as portable storage" and "Use as Internal Storage".If i select the first option, it says SD card is ready. But it doesn't format or fix the errors . And if i go for the 2nd option, it doesn't format and it displays the following error as a Toast.

Hi, my adobted card stopped working. Is is possible to reveal the encrypting key on my phone? It is not rooted. For rooting it needs to have an opened bootloader, but if I want to open it the phone is completely factory resettled - and the key deleted?

HiSome months ago, I had to do a hard reset on my phone, since then I have my 32gb sd card encrypted, it was originally set as adopted storage.After a lot of search, I'm still trying to recover such important files that I have in this sd card, after I fone The hard reset on my phone, is there any way to save the data on The sd? If I lost the encryption key, is there any chance to recover my sd data?

Can you tell me, please.Is it possible to recover data from the old card, if I clicked "forget" the disk and replaced it with a new one. Whether the result of this operation, a new encryption key is created and with it it is impossible to do?

When i m trying command sm string vold ] grep ~i expand i m gettin error string not found When set adoptable is done to true afyer has adoptable command it again shows false Unable to set it to true Please help hard earned money sdcard waste Tried cmd prompt and all formattersAdb shell method also fail

Hello runner, can you please tell me how you were able to decrypt and backup data from SD card. I am stuck in same kind of situation and I have pressed forget button as I wasn't able to access any other SD card on my phone. Now it is recognizing my SD card which was adopted as internal storage but asks to reformat it to be used as portable media or internal storage again. Please replyme on rakesh19900406@gmail.com

Wonderful, and very informative! It helps me understand the internal storage file system and encryption, however my problem is that my phone is destroyed so I'm unable to retrieve my encryption key. Is there a way to find out how the encryption key is derived, or if there is another way to retrieve it?

Hello and thank you for this, I'm finally hopping to get back the photos of my sd card adopted as internal storage on CM13... But I still get an error when I try the command : dmsetup create crypt1 --table "0 `blockdev --getsize /dev/mmcblk0` crypt \aes-cbc-essiv:sha256 0180e00...170fe15b73b4f 0 /dev/mmcblk0 0"device-mapper: reload ioctl on crypt1 failed: No such file or directoryCommand failed

Hi I was using a 64 GB card as internal storage on lenovo k4 note. Had given the phone for some repairs and removed the card. However after receiving the phone back which was factory reset the card was not recognized. Even the card was formatted . Have important data on it so please help me recover the data. I am not that computer savvy so plz explain in detail..

Thanks for these instructions. I'm hoping to test them out tomorrow. After reading your instructions carefully, I noticed that the encryption key you pass to the dmsetup command has what appears to be an extra "01" byte after the "09" byte that isn't in the od command above?

Popular posts from this blog

One of the less known new features introduced in ICS is the ability to backup a device to a file on your computer via USB. All you have to do is enable USB debugging, connect your phone to a computer and type the adb backup command in a shell. That will show a confirmation dialog on the phone prompting you to authorize the backup and optionally specify a backup encryption password. It looks something like this:

This doesn't require rooting your phone and lets you backup application data, both user installed and system applications (APK's), as well as shared storage (SD card) contents. There are some limitations though: it won't backup apps that have explicitly forbidden backups in their manifest, it won't backup protected (with DRM) apps and it won't backup some system settings such as APN's and WiFi access points. The transfer speed is limited by ADB channel speed (less than 1MB/s), so full backups can take quite some time. There is also a rather annoying bug …

It seems we somehow managed to let two months slip by without a single post. Time to get back on track, and the recently unveiled Android maintenance release provides a nice opportunity to jump start things. Official release notes for Android 4.2.2 don't seem to be available at this time, but it made its way into AOSP quite promptly, so you can easily compile your own changelog based on git log messages. Or, you can simply check the now traditional one over at Funky Android. As you can see, there are quite a few changes, and if you want a higher level overview your time would probably be better spent reading some of the related posts by the usual suspects. Deviating from our usually somewhat obscure topics, we will focus on a new security feature that is quite visible and has received a fair bit of attention already. It was even introduced on the official Android Developers Blog, fortunately for us only in brief. As usual, we like to dig a little deeper, so if you are interested i…