I really like Sony products, and when I approximately a year ago bought myself a Sony Xperia SP, I expected Sony to deliver a top-notch phone. This device is actually my 2nd Android phone, and was bought to substitute my beloved HTC Desire, which, with a little help from XDA Developers, had faithfully served me for almost four years. The first half year I was really satisfied with the performance of the device and the regular flow of OTA (Over The Air) updates. At one point, however, the updates stopped and I was left with Android 4.3 Jellybean and a message on Sony's website, stating that they were "investigating the possibility of updating the device to KitKat". After almost another half a year it was officially confirmed, that no further updates were planned for the phone, which in the mainstream phone market effectively means end-of-life for the device. This announcement really disappointed me, as I find the modern consumer electronics use-and-throw-away-a-perfectly-capable-device-after-half-a-year culture disgusting. As my phone gradually had become slower and slower: waiting time for the icons to refresh when you hit the "Home" button, constant warning about the phone running out of space and slow keyboard reactions, I decided it was time for a change.

Xperia SP should be perfectly capable of running Android 4.4 KitKat, as one of the main features of the system is its optimized design for a broader range of devices, including devices with small amounts of memory. When I started researching what custom ROMs there were available for the device, I quickly found out that an official CyanogenMod 11 ROM has been available for the phone since early June. CyanogenMod is one of the most popular after-market ROMs available for a very broad range of devices in "official" and "unofficial" versions, thanks to its open source nature. CyanogenMod adds a lot of additional functionality, theming and customization options to the phone, compared to official ROMs and this can really brighten up the phone usage experience. After watching a couple of demo videos, showing of the ROM functions I had to have it. The next step was to figure out how to install the ROM on the phone and here the trickier part began.

As I quickly found out, there is no lack of tutorials on how to root and how to install custom ROMs on Xperia SP out there. I also quickly realized that many of these tutorials were inconsistent, had information missing regarding important steps in the process, and were throwing around with terms such as lockedbootloader, unlocked bootloader, kernel, recovery and fastboot without properly explaining the meaning of the terms and the actions first. Even as relatively experienced user of custom ROMs (from my HTC Desire times) I got confused while finding the head and tail in some of these tutorials. This is the primary reason for this post: to gather all of the background information, the needed links and steps for the custom ROM installation process in one place.

Disclaimer

First a disclaimer! Rooting a phone and installing a custom ROM on it will void your warranty and, if done incorrectly, can result in a malfunctioning or a bricked device. If you choose to follow the procedures described in this post, you are doing so at your own risk!

The Background

First of all, in order to understand what we are dealing with, lets get familiar with the way an Android based phone functions, the underlying structure and some of the terminology used in conjunction with installation of custom ROMs. First some terminology:

Rooting is gainging root access privileges on the device. The term comes from the fact that the Android operating system is based on a Linux kernel. On Linux systems the user called root is the default superuser, who has full access to all files on the system. A root user has the ability to modify the system files. In case of Android having root access opens up for additional functionality and additional tweaking possibilites.

ROM / firmware are terms often used to describe the entire operating system with all of the potential tweaks, preinstalled programs, settings etc. The ROM acronym stands for Read Only Memory and is a legacy term from the old days, when embedded software (often referred to as firmware) used to be stored in small-size read only memory chips, which were impossible to reprogram, or required additional hardware to do so.

Flashing is a term used for writing a file into the memory of the device. The term itself originates from the fact that the memory chips inside electronic devices are based on flash memory technology.

Fastboot is a diagnostic/engineering protocol which is available on some android phones. When the phone is booted into fastboot mode and connected to a computer over USB, it is possible to use the protocol for various tasks, such as, unlocking the bootloader and flashing the different partitions.

Dalvik is the process virtual machine in the android operating system which is used to execute applications written for Android.

Bricked device is a term used to describe a state of a device, when it is unable to boot, for instance, when the bootloader has been deleted. It is possible to restore some devices from this state by either restoring the bootloader using fastboot mode, or by booting from an SD card. However, it is impossible to restore some devices, without manually reprogramming the memory chip inside.

The internal memory of the phone is, similarly to a normal computer harddisk, divided up into several partitions, all of which have a specific function. Some partitions are mandatory while other partitions are manufacturer specific. A typical memory layout of an Android device is illustrated in the figure below.

/boot - contains the bootloader, which isis the first piece of software to run when the phone powers up. The bootloader is responsible for early hardware initialization, loading and booting the kernel and initial RAM file system. The bootloader is also responsible for system maintenance, including loading and flashing new kernel and system images and providing the possibility to boot the phone into recovery mode. Phones usually come with a locked bootloader, which means that the bootloader will not allow you to flash any images not officially signed by the manufacturer. This partition also contains the boot kernel and ramdisk - the kernel (Linux kernel, in case of Android) is a fundamental part of a computer operating system, which translates requests from software into low-level hardware specific instructions to the hardware components the device is made of, such as the display, the touch screen, acclerometer, LED, buttons etc. The kernel is device specific and creates a so-called abstraction layer between the software and the hardware, which allows the software to access common hardware functions, unaware of the specifics of the actual underlying hardware. The ramdisk contains a small file system which is copied into RAM and presented to the kernel as a harddisk. This allows the kernel to run programs there before the actual root filesystem is mounted.

/recovery - contains the recovery kernel and ramdisk. The recovery is a piece of software, which, as the name suggests, provides some tools for recovery of a malfunctioning phone. Recoveries, which are provided by the phone manufacturers, typically allow for simple operations such as doing a phone factory reset, wiping all user data or installing a ROM update from a .zip file on the SD card, but not much else. When dealing with custom ROM installation, the built-in recovery is typically replaced with a custom recovery, such as the Clockworkmod (CWM) or TWRP, which provides additional functionality, such as full phone backup, ability to wipe selected partitions, flash multiple zip files, reset battery statistics and many more.

/system - is the partition which actually contains the operating system, the boot animation, the built-in apps etc. When the phone is operational this partition is mounted as read-only.

/data - is the partition which contains all user data, such as downloaded apps, app data, contacts, messages, photoes, etc.

/cache - is a partition which is used by the operating system to store temporary files, during operation.

/misc - can contain miscellaneous system information in form of on/off switches. These settings could include CID (Carrier or Region ID), USB configuration and other hardware settings.

The Update Process

Now we have got the terminology in place, lets begin. First of all, if you have not done anything like this before, your phone will most likely have a locked bootloader. This means that it has to be unlocked, before anything custom can be installed in the phone Read Only Memory. Luckily for us, Sony allows users to unlock the bootloader, by accepting the loss of warranty.

Back up all of the important information on your phone, such as pictures, contacts, etc. Remember, when you are finished with this process, all of your personal data, apps and app settings will be lost. Personally I don't have any important data stored on my phone (all my contacts, etc. are cloud synced), except for the camera pictures, which are stored on the SD card and will not be affected by the process.

Unlock the bootloader by following the Sony instructions here. As a part of this process you will be required to install the Android SDK (Software Development Toolkit), as it contains fastboot, which is the tool used to communicate the unlock code to the phone. In order to use fastboot, open a terminal and navigate to the folder where the fastboot executable is located. This will be in the platform-tools folder inside the Android SDK installation folder, in my case (I use Linux), the path is ~/bin/android-sdk-linux/platform-tools.

Download the CyanogenMod ROM for the Xperia SP, the latest version can be found here. The ROM will be contained in a zip file. If you want Google functionality on your phone (Gmail, Calendar, Google Now, etc.) download Gapps from here.

Extract the bootloader from the ROM. Open the zip file and extract the boot.img into some folder on your computer. Although called boot.img, this image also contains the CWM recovery.

Copy the downloaded ROM zip file (and the Gapps file) to the root of the phone SD card, preferably using a card reader.

Flash the bootloader over fastboot. Plug one end of the USB cable into the computer and boot the phone into fastboot mode by first switching it off, and then plugging in the other end of the USB cable, while holding the "Volume up" button. The notification light on the phone should turn blue, to indicate that the phone is in fastboot mode. Use the terminal to navigate to the fastboot location on your computer and type: fastboot flash boot /path/to/boot.img substituting the path with the correct path for the folder whereto you extracted boot.img. Press enter and ensure that the program exits correctly, the process should take a few seconds.

Reboot the device into recovery. This can be done by either issuing the fastboot reboot command (which for some reason didn't work for me) or by pulling out the USB cable and pressing the power button for turning on the phone. The recovery is chosen by pressing the volume buttons back and forth while the phone boots.

Once the device is in CWM, use the volume buttons to navigate up and down and the power button to select. Select "wipe data/factory reset" and then "wipe cache partition". Choose "install zip", then "choose zip from /storage/sdcard1" navigate down to the downloaded zip file and select it. Confirm the next dialog, and wait for the process to finish.

Install the Gapps zip file in a similar way. When the installation finishes, select "reboot system".

The phone boots up into CM11, enjoy your fast and upgraded device!

General remarks

During the described process some complications can arise. In Linux fastboot might not be able to detect your phone at once, try fastboot devices to show a list of connected devices, in case no devices are found it is most likely a permissions problem, try the command with sudo. I have not tried to use or install fastboot in windows, if you are a Windows user, I would recommend you to check the fastboot installation guide here. First time I booted into Cyanogenmod the phone had problems with turning on WiFi, the slider in settings was simply grayed out. In order to fix this I had to reboot into recovery, wipe the cache, data and dalvik cache, which can be found in the "advanced" menu in CWM and then repeat the installation steps 8 and 9.

Conclusion

Once the process of flashing a custom ROM is broken down, it is actually quite simple. I am very happy with the result, which feels like getting an entirely new phone. Kitkat is very snappy, and CM11 provides a lot of customization and theming options. In Settings->About phone->CyanogenMod updates it is possible to configure the phone perform a weekly check for CyanogenMod updates. A new "nightly" version, containing any updates, commited by the developers during the day, is automatically generated and made available every night.