From Android Auto to OnLive: how video is becoming the secret glue of connected devices

Here’s something you may not know about Android Auto, Google’s attempt to enter the automotive space: Cars compatible with Android Auto won’t actually run any Android apps, but instead simply play back a video stream rendered by your phone – kind of like OnLive, but for your dashboard.

An Android connected Hyundai at Google I/O (photo: Signe Brewster)

This makes Auto the latest example for a growing trend to virtualize user interfaces. Video games on your game console, the channel guide on your cable box and even the productivity apps on your mobile device are increasingly not based on locally executed code, but compressed video streams that are rendered on another device within the same network, or even in the cloud.

In this move towards virtualization, video streams have become the secret glue that connects screens and devices without the need for complicated, platform-specific implementations. And the upside of rendering video instead of code could be that devices have to be replaced less often while getting more frequent updates.

The technology behind Android Auto

Google officially introduced Android Auto at its Google I/O developer conference in San Francisco last week, announcing partnerships with two dozen car manufacturers ranging from Ford to Honda to VW. Much like Apple’s CarPlay platform, Android Auto won’t take over your dashboard. Instead, it will be just one item on your car’s dashboard menu, and it will only be usable if you have a compatible Android mobile phone.

Photo by Janko Roettgers/Gigaom

Once users connect their phone to the USB port of an Android Auto-equipped car, they will be able to access a number of Android apps through the car’s dashboard. However, these apps — all of which will have driving-optimized user interfaces — won’t actually be installed on the car’s dashboard. Instead, they will render their output onto a virtual display of your phone, which is then compressed and streamed as a video stream to your car’s dashboard. Here’s Google software engineer Fabian Tamp explaining this process in detail:

There are numerous advantages for Google to go down this route. Car manufacturers don’t have to run full instances of Android on their dashboards, but can instead simply render a video stream on screen, and interact with Google’s Android Auto APIs to send user input and sensor data back to the phone, which makes integration much easier.

And without directly running Android and apps, cars also will have to do a lot fewer software updates, not to mention the hardware: A three- or four-year old phone typically isn’t capable of running the latest apps or operating system, but people only replace their cars every ten or so years. By relying on a video stream as a way to virtualize apps and user interfaces, Android Auto will in theory be able to serve cars coming out this year for another ten years.

Video-streamed cloud virtualization is everywhere

Android Audio is not the first platform to use video as a means for virtualization. One of the best-known examples is OnLive, which renders video games in the cloud, allowing it to stream complex action shooters to legacy desktop computers, mobile devices and even TV streaming devices like the Vizio CoStar.

OnLive is best known for its cloud gaming, but the company is also using the same technology to stream Windows desktops to mobile devices.

OnLive has also been looking to bring the same technology to other areas. The company is also offering a cloud desktop service to consumers, which makes it possible to run a Windows desktop and Microsoft Office apps on iPads and Android tablets, and has started to pitch its technology to enterprises that want to bring their apps to a wide variety of devices.

Rendered by the most powerful processor in your house: your phone

What’s notable about Google’s Android Audio implementation is that the video stream is not rendered in the cloud, but on your mobile device, which is quickly becoming one of the most powerful and most frequently-upgraded devices in people’s lives.

And Google isn’t just looking to take this approach to Auto: The company also announced at Google I/O that it is bringing Android device mirroring to Chromecast. Mirroring your phone on your Chromecast-equipped TV will provide an easy way to share personal media or the occasional video stream on the TV, but it will also effectively outsource processing of more resource-intensive applications to the handset.

One area that Chromecast has been lacking in particular is gaming — the device simply isn’t equipped to render full-blown video games. However, modern phones are more than capable of doing so, and by adding mirroring to Android, users will be able to play their favorite games on the TV screen. Apple has been doing similar things with AirPlay for some time, but the $35 Chromecast stick is a perfect example of why transmitting video streams is a great way to connect powerful applications with cheap and lean devices.

In a way, all of this — from Android Auto to OnLive to Chromecast mirroring — may foreshadow how devices and screens communicate with each other in a future where everything in and around your house is connected. Instead of trying to figure out common standards across devices and operating systems, companies may just decide to fall back on a format that every screen understands: video.