Android N gets a low-latency VR mode

We don’t know if Google will launch its own virtual reality headset at its I/O developer conference this year, but Android N, the next version of the company’s mobile operating system, will feature a special mode for VR apps.

This new mode gives VR apps exclusive access to the device’s processor cores when they are in the foreground. Combined with an improved sensor pipeline and the work Google did on bringing support for the Vulkan graphics API to Android, the company claims the VR mode can bring latency on the Nexus 6P down to about 20 milliseconds, which is pretty much the gold standard for mobile VR at the moment (i.e. about what Gear VR operates at).

“When we started Android, we didn’t foresee many use cases,” Android VP of Engineering David Burke told me, adding that, with VR, latency is one of the main problems. The “motion-to-photon ratio,” as Burke calls it, has long been an issue with VR headsets, specifically those of the mobile variety. When the latency between your head movement and the image on the screen changing is too high, you simply don’t feel like you are present in the scene (and you are more likely to get sick).

Though there’s always room for improvement, latency below 20 milliseconds has long been seen as the sweet spot for pleasant usage. Oculus CTO John Carmack has said that delays in this range are “generally imperceptible.” Indeed, even expensive tethered headsets like those from Oculus, HTC and Sony are still roughly in this ballpark though they have much more intensive graphics needs.

So with this new VR Mode, apps now get full access to all of the power of the phone’s CPU and GPU to render images as fast as possible. The team also changed how its graphics buffering works by using a single buffer and having the app chase the scan line on the screen.

In addition, the VR Mode also uses a process similar to Oculus’s Time Warping where the system first creates an image based on your head movement and then, just before it displays it, checks your current head position and very quickly transforms it to match your current head position.

If your phone doesn’t have a low persistence display, though, all of this work is likely for naught because you end up with lots of motion blur as pixels can’t change their colors fast enough.

While the focus here is clearly on performance, the team also built other functionality into this mode. You will still see notifications, for example, as well as alarms and incoming calls — all of which will display on cards that are now rendered in stereographic 3D.

Google’s name for its VR platform is Daydream. This includes the VR mode, as well as hardware certification and support for headsets and controllers. Google also built a special version of the Play store to run inside this VR experience and a number of app developers like the New York Times, Netflix and others are already working on Daydream apps and Google is bringing its own Play app, StreetView, YouTube and Google Photos to this platform.

The Daydream platform will launch in the fall, but the software

Google will certify phones to be Android VR ready and unsurprisingly, the Nexus 6P, the company’s own current flagship phone, is the first to receive this label. Google noted that Samsung, HTC, LG, Huawei, ZTE and others are also releasing VR-ready phones or having older phones certified.