Windows Phone Mobile VR (Gyro Head Tracking in Unity3D)

Mobile VR

Recently I have been tracking the surge in interest in virtual reality and things certainly seem to have moved on in the last 20 years! Having experienced a number of Oculus Rift DK2 demos and various other mobile VR demos I have become excited about the rapid tech advances and the possibilities for the near-term future. Leaving future speculation aside, as a techie my first thoughts were to find some demos to run on my windows phone. I couldn’t find any so started to look into ways to create them myself using an inexpensive headset of some kind.

If you haven’t seen any mobile VR solutions here is an example of the lowest budget version I could find which illustrates the main components required:

The phone slides in behind the lenses at the back.

The basic idea behind how this works is that each eye receives a 2d image of a 3d perspective view which are rendered offset from each other. The brain assembles these projections from the retina giving the illusion of depth.

The lenses allow a much wider field of vision but they also distort the image. These factors result in the need to render your 3D scene twice, side-by-side each from a slightly different viewpoints and also to render it in such a way as to ‘undo’ the lens distortion and keep any straight lines looking straight.

I ordered this functioning cardboard headset at a cost of £2; unfortunately, I can’t fit my Lumia 930 inside it but it illustrates the parts involved.

Since this won’t accommodate my phone for subsequent posts I will be using a prototype of the ViSR headset see their Kickstarter here https://www.kickstarter.com/projects/1122924030/visr-virtual-reality/description. This is a much better built version made of laminated corrugate which feels really solid and is expected to be sold at a £15 price point. If you’re into mobile VR go to the Kickstarter and pledge – there are even pre-release developer edition headsets available. The ViSR developers are working on a Unity package which will be available for this headset which will enable devs to get up and running quickly.

Gyroscope

Another aspect of mobile VR is that when you have your headset on and you rotate your head the scene view should orient appropriately to give you a sense that you are in the scene and looking around. This is what I will focus on in this post to get the ball rolling and I’m going to try to keep it as simple as possible. Most mid-high end smart phones come equipped with a gyroscope and that is what I’m going to use to achieve the head tracking. We can get readings from the gyroscope in a Unity script which will allow us to orient the view appropriately.

publicclassHeadTrack : MonoBehaviour

{

privatebool gyroBool;

privateGyroscope gyro;

privateQuaternion rotFix;

privateVector3 initial =newVector3(90, 180, 0);

// Use this for initialization

void Start()

{

Screen.orientation =ScreenOrientation.LandscapeLeft;

Screen.sleepTimeout =SleepTimeout.NeverSleep;

gyroBool =SystemInfo.supportsGyroscope;

Debug.Log("gyro bool = "+ gyroBool.ToString());

if (gyroBool)

{

gyro =Input.gyro;

gyro.enabled =true;

rotFix =newQuaternion(0, 0, 0.7071f, 0.7071f);

}

else

{

Debug.Log("No Gyro Support");

}

}

// Update is called once per frame

void Update()

{

if (gyroBool)

{

var camRot = gyro.attitude * rotFix;

transform.eulerAngles = initial;

transform.localRotation *= camRot;

}

}

}

Note that the coordinate systems need to be aligned between Unity and the data coming from the phone. So simply taking this script and attaching it to the Main Camera in a Unity scene, building it for Windows Phone 8.1

Running the resulting app will show using orientation in an app in a pretty simple, but effective way.

First Person Camera

It would be nice to use this to drive a first person camera so that we can use a different input method to allow the camera to move forward and backwards as at the moment the camera is stuck in one spot. Unity has a package to help here:

Use File > Assets > Import Package and choose Character Controller and import everything into your project. Now, under Standard Assets > Character Controllers you will see two character controllers; one for third person and one for first person.

You can drag the First Person Controller into your scene and delete the existing Main Camera as there is one included with the Character Controller. You will also need to position your character controller above the terrain floor otherwise it will just keep falling; it’s a rigid body so reacts accordingly.

Out of the box you can use mouse to orbit and keyboard to navigate the camera forwards, backwards and to jump. We’re going to replace the input with our head tracking and add some buttons to ‘walk’ forwards and backwards. So now, as before drag the HeadTrack.cs script onto the new Camera. This gets us back to the functionality we had before but we can now wire up some of the other actions. I’ll just stick to forwards and backwards for now.

I then provided the direction movement vectors to the FPSInputController.cs script using some static variables which I marshalled to the unity render thread from the xaml environment where I altered those values in response to button presses. (Note that this was done for convenience).

Thx for the post! When your app will be published? I tired of waiting for the VR game with head tracking on Windows Phone (there’s no anything yet). Can you send me .xap file, please, my e-mail is arthurovsky@hotmail.com

How about to create the app which consider the Remote Desktop streaming with Headtracking feature? It would be great to have the app like Tinus Gyro on Android, which allows to stream PC games to mobile with enabled headtracking.