Monday, February 22, 2016

VR for Beginners: Cardboard, Unity and iOS - Part 2

Update 23-May: Note that the Cardboard SDK has now been replaced with the GoogleVR SDK. In most cases you can simply replace Cardboard with GoogleVR or GVR in the text below. There may be some steps below that are no longer needed, (e.g. some of the XCode steps are already done for you, like disabling Bitcode and including security.framework.)
I posted on this subject before, but back then Google didn't officially support iOS with their Cardboard SDK. That has changed, so things are a little easier now. Yesterday I decided to try it out and figured I'd share what I learned.

What I'm going to cover:

I'll point to a simple tutorial to build a quick 3D game. We'll be extending this a little too.

From there we'll convert that game to support VR mode using Cardboard.

We'll add the VR camera.

We'll add some interaction to the scene using gaze and clicks.

We'll add some UI, including HUD and 3D buttons.

We'll add support to switch between regular and VR modes.

Finally, we'll build through XCode to get it running on an iPhone.

You could do these steps earlier to try things out on your phone at each stage, but I just used the built-in Unity player in the IDE most of the time since it takes a while to build.

Copy the Roll-a-Ball project folder and name it Roll-a-Ball VR. Open the project in Unity.
Import the Cardboard SDK: Go to Assets > Import Package >
Custom Package. Select the CardboardSDKForUnity unitypackage where you
downloaded it and click Open. Since we are using Unity 5 we can uncheck any items marked Legacy. Notice the Cardboard and Plugins folders are added to your project.
Delete the Main Camera item in your project hierarchy, (note that this means our CameraController script that we wrote as part of the Unity tutorial is no longer used.) Replace your camera by dragging Cardboard > Prefabs > CardboardMain from your project view to the scene hierarchy. Set the position of CardboardMain to (x,y,z)=(0,10,-10) and rotation x=45 to get it to the same spot as our old camera. In your hierarchy select CardboardMain > Head > Main Camera, then set Clear Flags = Skybox in the inspector.
Press Play, and you should see the game view showing a stereo rendering of the Roll-a-Ball board. While in play mode, press Alt and move the mouse around to pan
horizontally and vertically to simulate moving your head around. Use Ctrl to
simulate tilting your head from side to side. Your keyboard arrow keys or gamepad directional buttons will still work to move the ball around too.

Roll-a-Ball in VR!

Adding Interaction

We're going to add the ability to interact with the game world by just looking at it. In the hierarchy select the Event System and add a Script component Cardboard > GazeInputModule. Make sure to move this component above any other input components in the inspector, (you can use the cog button next to the component's name.) In our case, we need to move Gaze Input Module up once, above Standalone Input Module. Also in the inspector, check VR Mode Only.
Next select CardboardMain > Head > Main Camera in the hierarchy and add a component Event > PhysicsRayCaster, (note, make sure not to pick a Physics 2D Ray Caster.) This will project rays from the camera into the world so we can detect collisions with the object you are looking at and react.
Now we want something to happen in response to our gaze. We'll do something simple to start with to prove things are working, switching the colour of the ground. In the hierarchy select Ground and add an Event > Event Trigger component. Within that component click the button to Add New Event Type, OnPointerEnter, and set MeshRender.material to Pick Up. Do the same for OnPointerExit to have the material change back to Background.

Setting up the Event Trigger

Click play and try looking around. If you have part of the ground in the middle of your view you should notice it change colour. Look away and it should change back. You may notice some flicker. Lets see if we can find out why.
To make it easier to see what we are pointing at we are going to add a reticule to our view. In the project we'll drag Cardboard > Prefabs > UI > CardboardReticule to under CardboardMain > Head > Main Camera in the hierarchy. If you click play again you should notice a white point in the middle of the screen which changes size as it approaches objects. You should also now see that the ground doesn't change colour when we are pointing at some object in the way, (e.g. the player ball or pick ups.)

Look at the ground to change the colour

Now let's remove that Event Trigger component so we can add something more useful in a moment.

UI in VR

Where did our UI go from the original tutorial? You may notice it is still displayed at the top left of the screen in the Game view, but we wouldn't see that when wearing our Cardboard device. So we need to make it part of the 3D world. Unity have a doc explaining more about this. Select Canvas in the hierarchy and see the Canvas component in the inspector. Set the Render Mode to World Space and set Event Camera to Main Camera. Now in the Scene view if you just zoom out you should see a large rectangle, which is our UI canvas. We need to move it to a more reasonable/useful position. In the inspector set the position to (x,y,z)=(0,2,0), size to (128,128), rotation to x=90, scale to (x,y,z)=(0.1,0.1,1). Select Canvas > Count Text in the hierarchy and change the Anchor Preset from top-left to top-center, (hold Shift and Ctrl when clicking.) Change the Paragraph Alignment to Center. Note I've removed the Win Text from the tutorial and just use the Count Text to show the win message, which you can see in a script a little later in this post.
Click play and you should see the player's score hovering above the game board.

That text is diegetic UI, (in-world UI,) there at the top

For fun, put the Canvas object under CardboardMain > Head in the hierarchy. If you play now you should see that the UI follows the player's gaze, like a HUD. While this is a badly positioned example, it should give you the idea if you wanted to try this again in future. For now, move the Canvas back to the root of the scene hierarchy.
We're going to add a button and some script in order to reset the game board which pops up once the player has collected all of the pick-ups. So select the Canvas and from the main menu click GameObject > UI > Button. Position it somewhere appropriate within the canvas, (I went for a bottom-center anchor preset.) Name this button Play Again Button and set a Highlighted colour. Now click to disable it, (top-left checkbox of the inspector.)
On the Canvas object we'll add a New Script component in C#. Call it GameplayMenu. Very simply, we just need one function here:

// Reset the game and start again
public void PlayAgain () {
UnityEngine.SceneManagement.SceneManager.LoadScene ("MiniGame");
}

Save the file and go back to Unity. Select the Play Again Button, set the OnClick event to use the Canvas object and choose the GameplayMenu.PlayAgain() action. If you have an issue finding that method, try clicking elsewhere in the editor and in the main menu click Assets > Open C# Project and try again.
Finally we'll update the PlayerController win code to show the button:

// Reference to a UI element so the player can reset the gamepublic Button playAgainButton;

Be sure to go back to the inspector for the Player object and set the Play Again Button setting to point to the Play Again Button object.
Try playing the app again, collect the pick-ups and see the Play Again Button appear. Try looking at it to see the colour change, then tap it to reset the level.
Note, when I did this I noticed the world would go dark when testing within the Unity editor. To work around this you can click Window > Lighting on the main menu, then in the Lighting dialog box uncheck Other Settings > Auto, then click Build.

Bonus Interaction

We're going to make it so that you can control the game using only the Cardboard device. You'll be able to gaze at a point on the ground, click the button on your Cardboard device and the ball will move towards it. In the PlayerController script:

// Position that could be passed in from Gaze, Touch, Mouse to move the player towardsprivate Vector3 moveTowardsPosition;
// Only move after a player clicks/taps, not each frameprivate bool shouldMove;

// Happens just before Physics calculations
void FixedUpdate() {
// If we got input from clicks, taps or cardboard, move towards it. Otherwise, see if there is keyboard/gamepad input.if (shouldMove) {
// Get a vector from the player position to the point passed in
Vector3 movement = moveTowardsPosition - gameObject.transform.position;

// Multiply the input vector by the object's speed, then apply that result as a force to the object to move it
rb.AddForce (movement * speed);

// Make a vector based on that input
Vector3 movement = new Vector3 (moveHorizontal, 0, moveVertical);
// Multiply the input vector by the object's speed, then apply that result as a force to the object to move it
rb.AddForce (movement * speed);}
}

Select the Ground object and add a New Script component called GroundTriggers. Add this code, which will forward the intersect point of the viewer's ray with the ground plane each time the player clicks their Cardboard device's button.:

Be sure to set the Player object in the Ground inspector, then try playing again. Look around, point and the ground and press your Cardboard button. To simulate the button press in the Unity editor you can use a mouse click. Note you may want to turn off Raycast Target as an option on the Count Text object so that your rays still reach the ground behind that floating text.
Awesome, we have a complete VR game for Google Cardboard!

Toggle VR mode on/off

But not everyone has a Cardboard device, so what if they still want to play our great game without us having to release a separate version from the original tutorial? Well why not have an option to toggle VR on and off?
Add another GameObject > UI > Canvas to the root of our hierarchy. Name it Canvas - Settings so we can differentiate from our other Canvas object more easily, (perhaps name that Canvas - Gameboard.) Set position (x,y,z)=(0,0,16), size (128,128), rotation (x,y,z)=(0,0,0) and scale (x,y,z)=(0.1,0.1,1). Add a New Script component called CardboardModeManager and add this code:

using UnityEngine;
using System.Collections;
using UnityEngine.EventSystems;
using System.Collections.Generic;
using UnityEngine.UI;

public class CardboardModeManager : MonoBehaviour {

// We need a reference to camera which
// is how we get to the cardboard components.
public GameObject cardboardMain;

// This is to enable/disable the reticule
private CardboardReticle cardboardReticle;

public void Start()
{
// Save a flag in the local player preferences to initialize VR mode
// This way when the app is restarted, it is in the mode that was last used.
int doVR = PlayerPrefs.GetInt("VREnabled");
Cardboard.SDK.VRModeEnabled = doVR == 1;
CardboardHead head = cardboardMain.GetComponentInChildren();
head.enabled = Cardboard.SDK.VRModeEnabled;
cardboardReticle = cardboardMain.GetComponentInChildren();
cardboardReticle.gameObject.SetActive(Cardboard.SDK.VRModeEnabled);
Cardboard.SDK.TapIsTrigger = true;
}

// The Cardboard SDK can show a back button, (escaspe key
// on keyboard,) which we will use to get out of VR view
void Update () {
if (Cardboard.SDK.BackButtonPressed) {
ChangeCardboardMode();
}
}
}

Make sure you set the CardboardMain value in the new Canvas object's inspector.
Finally, under the Canvas - Settings object add a Button named Toggle VR. Set a highlight colour, set the label text and set the OnClick event to point to the Canvas - Setting object and use CardboardModeManager.ChangeCardboardMode() as the event.
That's it! Try playing again and switching VR mode on/off.

Donate with PayPal

About Me

A Technical Project Manager for EA Mobile at Electronic Arts, currently working out of the Redwood Shores, CA. Credits include titles in the Need for Speed, Medal of Honor and Battlefield franchises among others.www.HiAsh.comVR Portfolio