Hands-on Augmented Reality for Game Developers

This training course teaches you how to build a fully functioning AR game with several features that interact with the player’s device and environment. Several of these skills will easily carry over to building your own games or other AR applications, making you an expert at developing AR games and applications.

Augmented and Mixed Reality has emerged as the next wave of user interface augmentation and interaction to hit computing. It has outpaced Virtual Reality in its ability to be more accepted across domains and to gain widespread commercial use. Of course, without the game Pokémon Go, most people would not even know what AR is. Go introduced the world to AR in multiple forms and this course will revisit building your own AR, GO-like application.

This course takes the student on a journey to build a full Pokémon Go style application in a very short time, using some powerful tools like Unity, and assets freely available in the asset store. We start by showing how to use and set up a mapping interface on your mobile device that allows you to track your movement using the GPS sensors. We will use a 3D mapping interface called WRLD.

Next, we will show how to consume various localized services for displaying points of interest on the map around the player. We then learn to use these techniques to place targets or goals. After being able to locate and place objects on the map, we now allow the player to interact with the objects in AR mode using the player's environment and the device's camera. We show the student how to set up an AR cross-platform tool that supports both ARCore and ARKit. In Go, the object is to catch the Pokémon using balls, we will also show how to build a similar interface in order to catch the creatures.

Finally, we cover a couple of advanced techniques in AR like creating portals and adding AI to the monsters. We use the last couple of hours of the course to explore fun enhancements that can be quickly added to the game. We first show how to add portals to allow the player to move into specialized game locations. Then, we finish up by quickly introducing the addition of the Unity ML-Agents to control our creatures when in interactive mode.

What you'll learn-and how you can apply it

Install and set up a 3D map interface and how to customize it

Learn how to correctly consume and display GPS/GIS coordinates on a map

Animate 3D characters

Consume web services with Unity and render them correctly on a map

Track and render objects on a map in real-time

Learn the basics of AR platforms and how to use a cross-platform tool

Use AR platform to identify surfaces

Use touch to interact with physical objects in the environment

Basic particle effects

Build AR portals for viewing player inventory or other game scenes

Use the ML-Agents SDK to power your creatures with AI

This training course is for you because...

You are a game developer or an experienced developer new to GIS or GPS development or an enthusiast and wish to learn how to develop your own AR game.

Prerequisites

Basic understanding of ARKit or ARCore

It is critical you be able to build and deploy a game or app developed with Unity to your mobile device (Android or iOS) in order to work through the labs in this course. Other than that, bring your imagination and think about what games or apps you can build with your new AR skills.

About your instructor

Micheal Lanham is a proven software and tech innovator with 20 years of experience. During that time, he has developed a broad range of software applications. Including games, graphic, web, desktop, engineering, artificial intelligence, GIS and machine learning applications for a variety of industries as an R&D developer. He currently resides in Calgary, Alberta, Canada with his family. At the turn of the millennium Micheal began working with neural networks and evolutionary algorithms in game development. He was later introduced to Unity and has been an avid developer, consultant, manager and author of multiple Unity games, graphic projects and books since.

Micheal has previously written Augmented Reality Game Development, Game Audio Development with Unity 5.x and Learn ARCore: Fundamentals of Google ARCore, Learn ML-Agents: Fundamentals of Unity Machine Learning all published by Packt Publishing.

Schedule

The timeframes are only estimates and may vary according to how the class is progressing

DAY 1

Section 1: Mapping your location in WRLD (1 hour)

Lecture: Learn to install and setup WRLD to work with your devices GPS and map your location in real-time. You will learn some GIS/GPS basics and how you can use that knowledge to consume your devices GPS sensor. Then we understand the correct way of mapping GIS coordinates on a map.

Lab: Install WRLD and integrate with GPS to display location on a map.

Break: 10 mins

Section 2: Mapping the Player (1 hour)

Lecture: Learn how to load, display and animate a low-poly 3D character that will function as the player’s character. We learn how to mirror the movement of the device in the real-world and mirror the actions in the game. Note: It is recommended, you use the 3D character recommended for the course. Advanced users may be able to use their own character.

Lab: Learn to display a game character that is controlled by the device's orientation and location.

Break: 10 min

Section 3: Seeing the WRLD (1 hour)

Lecture: Learn how to consume various location services that can be used to display localized information to the player/user. This allows the player to further interact with the game environment.

Lab: Consume web services from a few different providers and display the objects/information on the map.

Break: 10 min

Section 4: Finding the Monster/Creature (1 hour)

Lecture: Learn to load, display, render and animate the monster/creature on the map. While this version of the game only resides on the players device, we show how various information services can be used to generate monsters/creatures from events. If given time we will also look at how to customize the map for your specific game theme.

Lab: Add monsters/creatures to the game and have them appear and animate on the map.

DAY 2

Section 5: AR Platforms (1 hour)

Lecture: Learn the fundamentals of an AR platform and what services it provides to your app. We look at how to integrate a cross-platform AR platform and use that to power either ARCore or ARKit. Then learn one of the fundamentals of an AR platform is identifying surfaces and rendering 3D objects fixed to those services.

Lab: Use a cross-platform AR toolkit to identify surfaces and render a 3D object on a fixed surface point.

Break: 10 min

Section 6: Catching the Creature (1 hour)

Lecture: Learn how to interact with the 3D creature in AR. Use the devices camera and player’s environment in order for them to throw balls (or other objects) in order to catch the creature/monster. This lecture covers how a player can physically (touch) interact with the environment and even allow the player to bounce objects off surfaces. We also cover some basic physics and particle effects for enhancing the game.

Lab: Build a scene that allows the player to toss an object at a creature in order to catch it or net it.

Break: 10 mins

Section 7: An Inventory Portal (1 hour)

Lecture: Learn how to construct an AR portal that allows the player to enter their own trophy/collection room. We learn how to construct an AR portal and render another game scene behind.

Lab: Build an AR portal that links to another in-game scene.

Break: 10 mins

Section 8: Adding Monster AI (1 hour)

Lecture: Learn how to use the ML-Agents SDK to build a simple AI for the game monster. This lecture shows how to quickly set up and train a reinforcement learning agent that can control the creature/monster. This is intended to be an introduction to the Agents SDK and a demonstration for creating smarter more interesting creatures for your game. Note: This is an advanced lecture that introduces advanced concepts in machine and deep reinforcement learning but still taught at an introductory level.

Demo: The demo will show how to train an agent creature for use in your own game. Code and examples will be provided for further learning by attendees.