How ARCore enables you to create brand new types of user interaction (Part II)

In Part I of our ARCore series, we shared some creative ideas on how you can use capabilities like light estimation to unlock new forms of user interaction and gameplay, part II shares some more practical use cases for additional ARCore features like Instant preview and Motion tracking.

Can you picture a Jenga game in AR? (If you’re not familiar with the game, the purpose is to build a progressively taller tower by removing blocks from the structure and then placing them on top of the tower.) In part II of our ARCore blog post series, we share some more practical ideas for leveraging ARCore capabilities like motion tracking to build a Jenga handheld AR game or the AR Session Pause function to keep users engaged when your ARCore app loses focus.

Motion tracking and proximity

With features like motion tracking, you can do anything from triggering animations based on the device’s position and orientation, to using the device as a physical bumper based on its proximity to a digital object. An interesting use case of proximity is Jenga. Using the device’s position relative to the digital tower, you can move around to find the right block you want to manipulate and use simple and familiar gestures to remove the block and then place it back on top of the tower.

A super fun ARCore experience: In this example, you are manipulating the digital object directly using familiar and simple gestures, such as holding the block while pressing it on screen and moving your device to find where to release the block by letting go of the screen

User interactions based on proximity and motion tracking will continue to play a major role in handheld AR user interaction. If you have any more ideas on how to leverage these ARCore features, please share them on our Connect handheld AR channel!

Pausing AR session

Enabling users to pause their AR session is a must for keeping users engaged. A user might want a pause in the action or simply stop using augmented reality in the app. In order to allow them to pause, you could make it possible to stop the entire AR session and start a new one when the user is ready to return to AR mode. In this way, however you risk losing user engagement as for now this would require the user to find planes or place objects in the world once again.

Another option is to use ARCore’s functionality for short term pauses. This can maintain planes and placed objects where they are while stopping the app from searching for new planes and trying to track motion.

Because pausing an AR session is supported natively in ARCore, as a developer all you have to do is disable the ARCoreSession component in Unity.

C#

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

usingSystem.Collections;

usingSystem.Collections.Generic;

usingUnityEngine;

usingGoogleARCore;

publicclassPauseAR:MonoBehaviour

{

publicARCoreSession SessionController;

privateboolm_sessionPaused=false;

publicvoidTooglePauseMode()

{

if(SessionController==null)

{

return;

}

SessionController.enabled=m_sessionPaused;

m_sessionPaused=!m_sessionPaused;

}

}

Pausing your AR app doesn’t only apply to allowing users to take a random break. Some other notable use cases are a multiplayer game that uses a single device briefly pausing a session between turns, or simply allowing the user to take a phone call without losing where they left off.

The action is paused and a full screen UI appears, after unpausing the AR functionality the same planes and flower come back.

Something to keep in mind is that this is still just a temporary pause, if the user moves drastically and the same plane or feature points can’t be found again, objects might not be kept between pauses.

Instant Preview for AR: test as you build

New in ARCore v1.1.0 for Unity is Instant preview. Available only in Unity, this tool allows you to skip the build process and test any changes to your AR app on ARCore-compatible devices as you build your project in Unity. This shortens the process from minutes to milliseconds enabling you to iterate in near real-time, check the state of objects, debug errors and view the position and scale of objects directly from the Unity Editor.

To get started with Instant preview, setup ARCore 1.1 for Unity and install the HelloAR scene on a compatible device. By default, data from the device such as the camera’s position and orientation, plane finding, feature points, light estimation and touch-input will be streamed into the Unity Editor and displayed in the game window. By making all of this information available directly in Unity, you can test and iterate on new features much quicker.

While the majority of AR is experienced through the lens of a mobile device in real-world conditions, you are normally developing on a laptop or desktop with no connection to that world. Instant Preview makes it possible to test in the real world — perhaps one of the hardest and most important things to do when developing a mobile-AR app. It enables you to test real-world aspects, lie like the contrast between shaders and real physical materials or how different lighting conditions may affect your AR experience, directly in the Unity Editor.

Imagine testing and iterating on object materials with a view into the real world from the Unity Editor.

Instant preview is already on by default in ARCore v1.1.0 for Unity, so get started right away! For more information and details, check out Google’s documentation.

ARCore resources and how to share your ideas

Share your ideas with the community and use ARCore 1.1.0 for Unity to create high+quality AR apps for more than 100 million Android devices on Google Play! Here’s how.