Whether it's Ice Cream Sandwich, Jelly Bean or iOS - find out how we test every software update, and get it on to devices as soon as possible.

It doesn’t matter whether you’re salivating for Ice Cream Sandwich, or it’s iOS that sets your pulse racing – we know how much it matters to our customers to be running the latest version of software available on their handset and to get it as soon as possible.

It can be a complicated process though, as Vodafone’s Chris May explains.

“The first step in the process is for manufacturers to decide which devices will receive or need an update. In the case of a major firmware upgrade – such as Android devices moving from Gingerbread to Ice Cream Sandwich – it’s largely dependent on the experience that the combination of the new software and the hardware capabilities of the device can offer customers. That call is made by the manufacturer in the first instance”

“It’s the first time we’ve seen Ice Cream Sandwich on a Huawei”

With the decision made to updgrade the firmware of a device – as Huawei plans to do soon with the G300, moving from Gingerbread to Ice Cream Sandwich – the manufacturer can start to develop their device’s version once they receive the software from Google, in the case of Android.

It’s at this point that the Vodafone-specific elements of the software build are implemented.

“We work through our global team to ensure that the features and settings we want to provide our customers with are incorporated into the build of the update. In some instances we add branding and services to a device that we believe will benefit our customers – be that music, apps or even remote diagnostics capabilities. In other instances, we need to pre-configure services such as voicemail or network settings. This means the customer gets the best possible experience when they fire up their newly updated device for the first time.”

Testing times

Once Vodafone engineers get their hands on an upgrade they start a rigorous series of tests, with the aim of being first to deliver the highest quality product to customers – but there can be several iterations of the software before it’s made available for customers to download.

“Firmware testing can typically take anything from one day to one week, depending on our previous experience with an individual manufacturer, and the complexity of the upgrade itself,” May says, “Security releases and bug fixes are usually quickest to test, but platform upgrades with new features take longer”

“We conduct smart testing,” IT Technical Manager Kevin Ralph explains. “If it’s a manufacturer we know well, we might conduct more limited tests – focusing on how new features perform. Assuming that goes smoothly, this process can result in the software making it back to the manufacturer pretty quickly.”

The team also studies the manufacturer release notes that outline any tweaks to the device.

“We test data, the operation of the apps, battery life, music services and so on – but if it’s a device we’ve seen before we wouldn’t go back and test the antenna again, unless there’s been a specific issue identified with it.”

In the case of the Huawei G300 Ice Cream Sandwich upgrade, however, engineers are conducting the full range of tests, including network bearer tests:

“We’re doing nearly all the testing we’d do for the launch of a new device. It’s the first time we’ve seen Ice Cream Sandwich on a Huawei, and the first time we’ve seen them do this kind of upgrade – so it’s important we test everything and make sure the user will get the best experience.”

Ralph added that the Huawei has a relatively straightforward manufacturer overlay on Ice Cream Sandwich, which makes it less complicated than more customised devices.

In addition to testing the platform on a live network, engineers also look at the proposed delivery mechanism – and whether it can be supplied FOTA (Firmware over the air).

“A platform upgrade is a big file, so we suggest to customers that they upgrade via WIFI. In fact, where we can we like to ensure there’s an alert included for users to that effect during the update process itself” Ralph said.

“Typically software and firmware now goes through a two stage testing cycle. After conducting a first round of tests we report back to suppliers with our recommendations for any improvements – and then we would expect to see that upgrade again before we could be confident that it meets the highest standards for our customers.”

Although the Vodafone team is responsible for testing all of the software updates released to our customers, a given software’s release date is determined by a number of other factors.

“We discuss an outline of delivery times with the manufacturers at an early stage to ensure that customers don’t get too many upgrades in too short a period – but once we have tested and approved the upgrade it goes back to the manufacturer to review before being sent for approval by Google, in the case of Android,”

“In terms of testing, it’s our job to ensure that we turn it round as quickly as possible, while still ensuring the best customer experience of the final software,” May says. “We work very closely with our manufacturing partners throughout this process as you’d expect.

“Once our testing’s completed we’re keen for the update to get through the remaining steps in the process smoothly and quickly so it’s in our customers’ hands sooner rather than later. In many cases, our customers are among the first to receive software updates, and we’re proud that’s the case”