Balky carriers and slow OEMs step aside: Google is defragging Android

Versions don't matter, because Google now controls the platform behind the platform.

Android 4.3 was released to Nexus devices a little over a month ago, but, as is usual with Android updates, it's taking much longer to roll out the general public. Right now, a little over six percent of Android users have the latest version. And if you pay attention to the various Android forums out there, you may have noticed something: no one cares.

4.3's headline features are a new camera UI, restricted user profiles, and support for new versions of Bluetooth and OpenGL ES. Other than the camera, these are all extremely dull, low-level enhancements. It's not that Google is out of ideas, or the Android team is slowing down. Google has purposefully made every effort to make Android OS updates as boring as possible.

Why make boring updates? Because getting Samsung and the other OEMs to actually update their devices to the latest version of Android is extremely difficult. By the time the OEMs get the new version, port their skins over, ship a build to carriers, and the carriers finally push out the OTA update, many months pass. If the device isn't popular enough, this process doesn't happen at all. Updating a phone is a massive project involving several companies, none of which seem to be very committed to the process or in much of a hurry to get it done.

Since it's really hard to push out an Android update, Google's solution is to sidestep the process completely. The company stopped putting all the good stuff in Android updates. It's not that good stuff isn't coming out at all, the exciting features are just not being included as part of a big Android release.

This year's Google I/O was a show of force for this new delivery concept. No new Android version was at the show, yet Google announced Google Hangouts, Google Play Games, cloud saving of game and app data, a complete redesign of Google Play Music and Google Maps, a new version of the Google Maps API, and new location and activity recognition APIs. Post I/O, we've seen seemingly OS-level features added like the Android Device Manager, a remote wipe and device tracking system, without needing to touch the base OS.

It's such a simple idea: Android updates roll out too slowly, so start releasing all the cool stuff separately. The hard part is making it actually work. But the first reason this is now possible is a little app that has finally come of age: "Google Play Services."

Calling Play Services an "app" doesn't really tell the whole story. For starters, it has an insane amount of permissions. It's basically a system-level process, and if the above list isn't enough for whatever it needs to do next, it can actually give itself more permissions without the user's consent. Play Services constantly runs in the background of every Android phone, and nearly every Google app relies on it to function. It's updatable, but it doesn't update through the Play Store like every other app. It has its own silent, automatic update mechanism that the user has no control over. In fact, most of the time the user never even knows an update has happened. The reason for the complete and absolute power this app has is simple: Google Play Services is Google's new platform.

Enlarge/ What happens when you try living without Google Play Services.

Andrew Cunningham looked at this shortly after Google I/O, but now things are truly crystallizing. Google's strategy is clear. Play Services has system-level powers, but it's updatable. It's part of the Google apps package, so it's not open source. OEMs are not allowed to modify it, making it completely under Google's control. Play Services basically acts as a shim between the normal apps and the installed Android OS. Right now Play Services handles the Google Maps API, Google Account syncing, remote wipe, push messages, the Play Games back end, and many other duties. If you ever question the power of Google Play Services, try disabling it. Nearly every Google App on your device will break.

Play Services supports over the entire Android install base.

The reason for all the permissions and sneaky updates is best illustrated in that chart above. While the latest version of Android is on six percent of devices, Play Services rolls out to everyone in a week or two and works all the way back to Android 2.2. That means any phone that is three years old or newer has the latest version of Google Play Services. According to Google's current Android statistics, that's 98.7 percent of active devices. So at Google I/O, when Google announced their slew of new APIs, nearly every Android device was immediately compatible in a week. Play Services is a direct line from Google to the core of your phone, and, really, no one outside of Google is quite sure of just how powerful it can get.

Google Play Services takes care of lower-level APIs and background services, and the other part of Google's fragmentation takedown plan involves the Play Store. Google has been on a multi-year mission to decouple just about every non-system app from the OS for easy updating on the Play Store. Take a quick look at Google's Play Store account and you'll see a huge list of apps, many of which ship by default in Android. Gmail, Maps, Search, Chrome, Calendar, the keyboard, YouTube, and even the Play Store itself are all separately updatable.

The above list is a good representation of the current update situation in Android. Nearly everything that can be moved out of the main OS has been. The only features left that would require an OS update are things like hardware support, Application Frameworks APIs, and Apps that require a certain level of security or access (like the lock screen, Phone, and Settings apps).

This is how you beat software fragmentation. When you can update just about anything without having to push out a new Android version, you have fewer and fewer reasons to bother calling up Samsung and begging them to work on a new update. When the new version of Android brings nothing other than low-level future-proofing, users stop caring about the update.

This gets even more interesting when you consider the implications for future versions of Android. What will the next version of Android have? Well, what is left for it to have? Android is now on more of a steady, continual improvement track than an all-at-once opening of the floodgates like we last saw with Android 4.1. It seems like Google has been slowly moving down this path for some time; the last three releases have all kept the name "Jelly Bean." Huge, monolithic Android OS updates are probably over—"extinct" may be a more appropriate term.

Not having to package everything into a major OS update means Google can get features out to more users much faster and more frequently than before. Android feature releases can now work just like Google's Web app updates: silent, continual improvement that happens in the background. Your device is constantly getting better without your having to do anything or wait for a third party, and developers can take advantage of new APIs without having to wait for the install base to catch up. This should all lead to a more unified, less fragmented, healthier Android ecosystem.

Since this framework is closed source and depends on the Google Play store, it also cuts off anyone who tries to fork the underlying Android layer. That means you, Amazon. It also limits what Samsung can do with their big Android market share. In some ways that's good, in other ways it is bad for users.

lovely, so even less control. not sure what phone i will need to use in 3 years time when all you find are "smart" phones (android/IOS/M$/let's skip BB - most likely dead by then as phone os) without giving godlike control to my device to one or the other 3the party.

Well, guess what? For parents, the lack of the ability to lock android devices down so kids can't buy infinite apps, music, etc was very annoying. Or lock them so settings can not be changed, apps deleted, and the rest of what the restricted user profiles allow finally catches up Android to what iOS has had for a while. Years, even.

I would really only consider handing a 4.3 Android device to a kid unsupervised full time.

What happens when this thing starts introducing major platform changes that conflict with patches from carriers and handset manufacturers?

There are a lot of things that are practical to decouple from the underlying platform and update independently, but once you start doing that with pieces that affect the API surface and core platform services you are inevitably going to create the same exact problems that make it difficult to push updates today.

Also seems like a special always-on system process that can magically grant itself new privileges is a security accident waiting to happen.

I'm not sure that's Google has beaten fragmentation from its most visible and annoying manifestation. Judging by the table you've posted, key UI elements, and application framework / API are still firmly bound to the particular Android version a user is running.

Which means that as a developer, beyond the general API part of Google Play Services, you still have to worry about fragmentation, ie whether older versions of Android do have a particular UI component, or supports a particular API or framework element. (and as they still account for above 30% of installed base, is something developer would tend not to overlook)

If I compare with iOS, Apple ships dozen of new API / UI elements in every version of iOS, and they are massively usable for developpers in a matter of weeks, as update cycles tend to be very short (even more so since OS update can be done OTA)

While Google owned OTA updates are clearly the way to go (as leaving it up to OEM / users clearly doesn't work), I don't believe that fragmentation in its annoying part can be properly defeated unless Google controls the API/UI part...which would kind of ruin the layer ecosystem OEM have built, and I guess reduce the attractiveness of Android vs a system perceived as "closed" such as iOS.

I don't know, that point of view seems to apply mainly to end users. Sure it's nice to have a all the google stuff update without OS updates, but for developers the important APIs are the Android APIs and not so much the little additions the Play Games API etc provides.So I don't really think this "tactic" does a lot for app developers, it's more Google's way of making sure their own apps and services stay up-to-date.

There are a few problems with this. One, you can't move all the APIs away from the core OS. Two, if you do move them, then the OS without Google becomes almost unusable. And most importantly, when users stop caring about updates, then carriers and manufacturers care even less about them, and that has grave consequences. Just because all the features are no longer part of the platform, doesn't mean that low-level security is also patchable by Google. This means when critical security vulnerabilities are found, it will still be up to the manufacturers and carriers to post updates, and since they won't be using the latest versions of Android, that means patching the issues with their own updates, which could take a long time to produce.

Also, the boring stuff tends to be what brings the platform forward, things like restricted profiles and OpenGL ES 3, both of which are somewhat small, but very powerful features. I'm glad Google is making an effort to keep as many features available for as many devices as possible, but they really need to put a fire under manufacturers and carriers to keep their devices up to the latest updates. One thing that would help is if manufacturers and carriers stopped modifying the core OS for their devices (aside from mandatory kernel and driver changes). They can move skins and all that to packages that get installed on top with no issue, just like how most of Google's apps install on top just fine. If they'd stop messing around with core changes just for the heck of it, they could update with a lot less issues.

Good on Google for making new things work even on old versions of Android, but they really need to get the manufacturers and carriers to update to the latest version of Android for all their devices in a timely manner. Because security and base features are still very important.

More importantly: There's nothing in Google Play Services that even requires Android.

It abstracts all the important stuff and frees Google to develop a matching set of services, using the same API, on any other platform. Specifically, ChromeOS, but there's no reason Google couldn't build Play Services for Windows 8 or Linux or OS X.

This is why it's not called Android System Services or something similar. Long term, Google is a cloud and services company, not an OS company. Android is secondary.

Given how aggressive that Google is becoming about demanding location info from users and either nagging the user or turning off services which don't directly rely on locations, I find a Google app that can do anything it wants at any time disturbing.

Since this framework is closed source and depends on the Google Play store, it also cuts off anyone who tries to fork the underlying Android layer. That means you, Amazon. It also limits what Samsung can do with their big Android market share. In some ways that's good, in other ways it is bad for users.

Capture 80% of the market with an open source project and lock it in with a closed source app!

People complain about fragmentation and feature exclusion. Complain again when Google introduces a work around.

Classic case of damned both ways.

A bad solution to a bad problem isn't a good thing. The solution is incomplete (app developers' problem is largely ignored, which means more work for app developers, worse app support for users, and less apps in general). Bug fixes (including security updates!) and performance improvements to the base OS will also continue to not make it to the end users. It's also bad, in that more and more of Android is being made closed source (it being open source was one of the reasons Android was touted as a good thing).

At this point, I'm not sure whether Google is exerting more control over Android to try to get updates out, or using updates as an excuse to exert more control over Android.

As a former Android user (now on iOS for almost 2 years), this isn't anywhere near enough to remove "updates" from my complaints-against-Android list.

The article is missing another important part of what Google's been doing to deal with fragmentation: the Android Support library. Google's been building on it for years, adding controls, adding new versions, etc.

The controls in the support library work with specific API levels: anything in the v4 support library will support all the way back to API level 4, Donut. A v7 support library is fairly new and supports all the way back to the latest patched version of Eclair. Any control in those libraries will work on virtually any device made in the last 3 years without issue.

The number of controls in this library has been growing for years. It originally started out mostly to ease the transition to using Fragments with Honeycomb, but it's since started to provide huge compatibility layers. Things that used to need an entire third party library now work as long as you use the support library controls instead of directly using the high-number API version of them. For example, a recent update added an AppCompat package to the v7 support library which allows use of the ActionBar all the way back to Eclair, for the most part like if the platform had supported it when it shipped nearly 4 years ago. Prior to this, you had to use third party libraries, ActionBarSherlock being the most popular one, that were fairly different compared to the support library API.

Honestly, I think the support library is the bigger success story. Google Play Services is fantastic but it's mostly there to support Google Accounts, the proprietary Google APIs, and the play store. If you don't use those, and most apps I don't think actually do, it's less important to you that it's there and runs on everything. The support library though helps out most app developers in cutting through android version differences. Oh, and it's completely open source, as it's typically statically compiled into the app.

Since this framework is closed source and depends on the Google Play store, it also cuts off anyone who tries to fork the underlying Android layer. That means you, Amazon. It also limits what Samsung can do with their big Android market share. In some ways that's good, in other ways it is bad for users.

Capture 80% of the market with an open source project and lock it in with a closed source app!

80% of phone buyers don't decide on which phone they get based on if the source code is open or closed. To some it matters, and it was useful for google getting manufacturers support in the beginning but it really doesnt have much of an impact on the market as a whole these days.

People complain about fragmentation and feature exclusion. Complain again when Google introduces a work around.

Classic case of damned both ways.

A bad solution to a bad problem isn't a good thing. The solution is incomplete (app developers' problem is largely ignored, which means more work for app developers, worse app support for users, and less apps in general). Bug fixes (including security updates!) and performance improvements to the base OS will also continue to not make it to the end users. It's also bad, in that more and more of Android is being made closed source (it being open source was one of the reasons Android was touted as a good thing).

Well, if you think "closed source" is inherently bad, then iOS is clearly worse...

This is a clever solution that solves one of the major issues under the "fragmentation" label. Specifically, it solves the problem of new features not being accessible to users because their OEM/carrier is slow at moving the device to a new release of Android.

However, some parts of "fragmentation" are inherent to giving the user more choices. Developers need to deal with a wide range of screen sizes, screen resolutions, CPU capabilities, and GPU capabilities, and yes, that makes things harder for developers. But it's clearly part of the reason why (80% of) users prefer Android to iOS.

"Have the core OS do only the bare minimum of what it needs to be able to do" sounds like a pretty darn good idea to me.

Sure, building a closed-source "all the stuff you actually WANT to do" program on top of that while you bill your "open-source" OS is a bit of a shady business practice, but from a software engineering standpoint, that kind of separation of responsibilities is good.

Is there going to be a way to opt out of this or is Google going to completely take over my phone against my will?

You can use an application to block permissions from Google Play Service / Framework, like Xprivacy (root required). One problem is that this may cause some Google applications to malfunction, but you can experiment.

People complain about fragmentation and feature exclusion. Complain again when Google introduces a work around.

Classic case of damned both ways.

Exactly. I see it as a 'good' thing myself, having been seriously stung by the 'build-then-abandon' strategy. My LG LS670 (Optimus S) is sufficient to run at least 4.0.4 (and newer), but the last Sprint update was 2.2.3. That is one step past what it was released with.

After that, I jailbroke the phone, then dropped in CM9, got double the battery life, and MUCH smoother running system and virtually no crash issues. Now - why doesn't Sprint want to do this? Or many of the other carriers? To keep selling you new phones.

This way, you can still keep the low-level functionality stable, but everything else is driven by the self-updating core service which becomes the new 'os'. Thus, security fixes, app updates, core service updates are all handled by the company with the vested interest in keeping the os updated, not the one who'll push users to new devices by refusing to update and leaving customers out in the wind. This overall is a win for the consumers.

Good on Google for making new things work even on old versions of Android, but they really need to get the manufacturers and carriers to update to the latest version of Android for all their devices in a timely manner. Because security and base features are still very important.

This might actually be the end-game. The OEMs should only be responsible for the low-level hardware support and drivers. Now that they don't need to concern themselves with the functionality provided by the Play Store/Play Services middleware, there should be no excuse anymore why they can't release updates within a timely manner.

Big tangent, but I would actually like to see Microsoft adopt a similar model with Windows Phone 8. A lot of the out of the box functionality is great, but since it's tied to system updates they don't get updated nearly enough.

Yeah, really, I view this as a good thing. Maybe I'm insufficiently paranoid, but it's a good thing IMHO that devices won't be trapped in limbo, lingering in years-old OS versions, for no reason other than the carriers and manufacturers want to sell you new phones.

My current phone, an (original) Galaxy Note, is on 4.3 right now via CarbonROM. If I were using the stock Samsung rom, however, I'd just have updated to 4.1.2 in the summer. That's pretty dramatically behind. It shipped with Gingerbread, and was over a year late upgrading to ICS.

Thankfully, I was able to bypass that bullshit with custom roms, but I shouldn't have to. I'd far, far rather Google go this route to ensure the OS and key features Just Work, while keeping my device open to whatever I want to install.

The writing for this was on the wall at I/O. Google allowing new software features to be used out of Play Services is a genius idea. It was great for them to demonstrate new features that peoples phones JUST got, automatically.

This mostly solves "fragmentation" when it comes to big features. Google can implement as much as it can in Play Services (such as Play Games) while continuing with lower-level hardware abilities in the OS itself (such as Bluetooth 4.0 support in 4.3).

No, this cannot solve all security risks that pop up. That's typically too big of a low-level problem. But Google also can't really be blamed for manufacturers opting not to patch security holes.

I don't know, that point of view seems to apply mainly to end users. Sure it's nice to have a all the google stuff update without OS updates, but for developers the important APIs are the Android APIs and not so much the little additions the Play Games API etc provides.So I don't really think this "tactic" does a lot for app developers, it's more Google's way of making sure their own apps and services stay up-to-date.

As a developer, this definitely makes my job easier. Combined with the support lib you really can ignore is version 95% of the time now, whereas two years ago no way.

As an open source and open platform advocate this has me worried about the future of the platform though

While this is nice for certain things its, it is terrible for security. Most security issues have to do with system services that are available for the other applications to access or exploit. It seems that they are providing some services through the Google Play "super service" this really just a hack of how they should be able to manage their OS. They really need to create a version of android where they can define a set of services that Google has power and a set the OEMs can customize. eg

They need to make sure the coupling between the layers is loose enough so that carrier and OEM monkeying doesn't effect their ability to deliver timely and robust updates. Currently it is not only fragmented on the application level, but it also seems to be fragmented at the individual OS component level. It is almost like google is fighting for the parts of the OS that it can grab control of.

With android being open sourced what it did was always in well "the open" Now with google going closed source (with a small open source kernel) no-one knows what they are doing or more importantly what they are being ordered to do.

People complain about fragmentation and feature exclusion. Complain again when Google introduces a work around.

Classic case of damned both ways.

A bad solution to a bad problem isn't a good thing. The solution is incomplete (app developers' problem is largely ignored, which means more work for app developers, worse app support for users, and less apps in general). Bug fixes (including security updates!) and performance improvements to the base OS will also continue to not make it to the end users. It's also bad, in that more and more of Android is being made closed source (it being open source was one of the reasons Android was touted as a good thing).

Well, if you think "closed source" is inherently bad, then iOS is clearly worse...

This is a clever solution that solves one of the major issues under the "fragmentation" label. Specifically, it solves the problem of new features not being accessible to users because their OEM/carrier is slow at moving the device to a new release of Android.

However, some parts of "fragmentation" are inherent to giving the user more choices. Developers need to deal with a wide range of screen sizes, screen resolutions, CPU capabilities, and GPU capabilities, and yes, that makes things harder for developers. But it's clearly part of the reason why (80% of) users prefer Android to iOS.

80% of the people who want an open source phone don't know what "open source" means.

Is there going to be a way to opt out of this or is Google going to completely take over my phone against my will?

Sure: You can opt out with an iPhone or a Windows Phone. Or even a Blackberry. Just remember when you pick one of those choices, though, that you're willfully allowing Apple/Microsoft/BBM to completely take over your phone

My phone (Virgin Mobile Epic 4G Touch, aka Galaxy S2) only received the 4.1.2 upgrade from ICS at the end of March.Samsung/Sprint still has yet to release a patch for the master key exploits found in June despite the fact that Google released the fix a couple of weeks later.

Talk on some of the boards I hang out on is that Sprint will never release a patched 4.1.2 fix or an upgrade to 4.2.2. or 4.3 for the phone.

My workaround was to install the Xposed framework and the Master Key Dual Fix app.I really shouldn't have to depend on XDA-Developers for major security fixes such as this.

Another fix would have been to install Cyanogenmod, but it has its own issues (data speeds and graphics corruption) on my phone.

Ron Amadeo / Ron is the Reviews Editor at Ars Technica, where he specializes in Android OS and Google products. He is always on the hunt for a new gadget and loves to rip things apart to see how they work.