Learn how John Keefe and his AR studio, Draw & Code, developed a wonderful new kind of toy, get access to a comprehensive Vuforia in Unity development workflow and discover their tips for AR design, which they learned along the way.

Augmented reality is here to stay. There will come a day in which the real world will become our leading user interface. But, until then, there are many developers who, through pragmatic product engineering and clever design, have been unlocking the potential of what is possible beyond the screen today.

One such developer is John Keefe. His small studio in the UK, Draw & Code, spent years of AR development in Unity and Vuforia to create SwapBots, an AR-enabled toy that has gained the attention of the world’s leading toy companies.

SwapBots Co-founder & Director, John Keefe

The SwapBots team

SwapBots is a toy that combines physical and digital play. Kids can arrange and rearrange physical blocks, which when viewed through the lens of a handheld device, bring characters to life. Evoking elements of RPGs and tabletop games, users can create characters each with their own skills and abilities, or play in story mode and call on different characters by combining the blocks in different configurations to resolve puzzles or play pass-game challenges.

Some SwapBots characters and example game-play mechanics.

With SwapBots, Draw & Code has demonstrated an interesting way to create characters and make physical objects part of an AR experience. To make this possible, they became experts in the use of multiple AR markers and in embedding markers in the actual game mechanics using Unity and Vuforia. Below, John Keefe shares five lessons his team has learned in the process and offers a full AR Markers best practices guide valuable for anyone interested in breaking into AR gaming.

“Unity and Vuforia are like a flywheel. It’s so easy to get started and pretty quick to get something demoable. You play with the tools, build on your previous experiences and push the boundaries further and further as you develop.”

Five lessons learned in AR design

1. Consider external factors.

Mobile AR implies that users have the freedom to go anywhere. Yet developers need to get attuned to the impact that external factors, such as poor lighting, can have on the experience.

2. Test your assumptions beyond the desktop screen.

Putting SwapBots into the hands of kids or taking it to shows such as GDC to demo has helped Draw & Code put the user front and center. If you can’t get out there, just keep testing and prototyping, using tools like Unity and Vuforia to their full potential and always keep your user in mind. AR should feel effortless and natural to use.

SwapBots concept drawings from Draw & Code

3. Make the physical object relevant to the AR experience.

In the case of SwapBots, the idea was to create a toy where the physical object doesn’t become redundant after scanning the marker. The user continuously manipulates and interacts with the artifact. Draw & Code accomplished this by creating a game mechanic where users need to constantly swap the toys’ components in order to solve puzzles or win battles. This is a unique quality of AR: it brings physical gameplay into the digital world.

4. Consider the design of the physical object and how it fits in.

Character design was also a major consideration for Draw & Code. The target images printed on the toys work as AR markers but, at the same time, they need to work well as characters and while maintaining an appealing aesthetic. This required designing markers that resembled a persona and kept their essence when interchanging them with markers from other characters.

5. Test in the real world.

Remember that while the majority of AR is experienced through the lens of a mobile device in real-world conditions, you are developing on a laptop or desktop that has no relevance of that world; therefore, testing and prototyping is essential.

According to John, Unity and Vuforia allowed him and his team to experiment and test early on until they found a feasible design that would work on all devices. By iterating and prototyping, they were able to identify what type of game mechanics made the most sense, the ideal shape and material for the physical objects, and how users would interact with the different characters they had created.

The Draw & Code team at work.

Marker design also required testing in the real world. Through iterations, Draw & Code found that while the markers often worked well on screen, the camera would have difficulty tracking them when transferred to a 3D object. The marker would lose accuracy or it would not generate enough contrasting points to be recognized.

The only way to overcome this was by testing the marker on top of the 3D object itself. Draw & Code tested multiple 3D-printing techniques, such as printing a full core object that included the image target. They concluded that the best solution is to print a marker on a sticker and then overlay it on top of the 3D-printed physical object. These iterations helped inform the final SwapBots toys that are to be built by Creata, one of the world’s leading toy manufacturers.

A SwapBot progression from early 3D printed blocks onto the professionally manufactured version.

Unity 2017.3 for Vuforia 7 workflow

Designing AR applications for devices that users already own is a significant challenge for developers. Often conflated by sci-fi-level expectations, creators come up with incredible ideas only to find that they are limited by the computational and sensor capabilities of the target devices and all the external environmental conditions.

Draw & Code is amongst the pioneers of AR development that have been tackling this challenge for years. Based on their experience, they’ve created a full AR Markers best practices guide and highly recommend anyone interested in breaking into AR gaming to prototype AR development with Vuforia’s free sample assets provided in the Unity Asset Store. Vuforia’s sample assets provide you with a complete Unity scene that shows you how to create AR experiences for objects and environments.

Vuforia provides cross-platform support for Android, iOS, and UWP devices, through a single API, allowing developers to write their apps once and run them using the best available core technology.

With the Unity and Vuforia workflow, Unity developers can create Vuforia-enabled apps using a simple authoring workflow and event-driven scripting directly in the Unity Editor. The Vuforia integration includes performance optimizations, tight ongoing synchronization of features and fixes, and a native Unity workflow that enables developers to focus on creating the best AR experiences.