Natural User Interface in Mobile AR: SOUR PATCH KIDS – In Sour Vision

Bully! recently launched a new Augmented Reality game for Mondelēz International promoting the “SOUR THEN SWEET” SOUR PATCH KIDS candy and their X-box game WORLD GONE SOUR.

Bully! recently launched a new Augmented Reality game for Mondelēz International promoting the “SOUR THEN SWEET” SOUR PATCH KIDS candy and their X-box game WORLD GONE SOUR.

The game, found on the App Store (here) uses AR image markers (found here and here) to spawn a Bully! created set featuring Dolly Doll, a main boss character from the X-box game. Dolly, controlled by the mischievous Yellow SOUR PATCH KID character, throws anything at hand – toy blocks, doll heads, firetrucks – at the user. The user must dodge the projectiles and, with the help of the Green SOUR PATCH KID character, throw tag-alongs (small SOUR PATCH KIDS) back at Dolly to win.

Beyond being among the first AR advergames, a key differentiator of the game is that the user must actually duck to avoid being hit by virtual projectiles – not just move back and forth with on-screen buttons but physically move his or her body to avoid being hit by the Dolly’s barrage.

This technique, utilizing an operation that comes ‘naturally’ to people as an input to computer interaction, is referred to as a natural user interface (NUI.). This kind of interaction is being actively explored by groups such as but in its infancy with mobile device experiences.

“One of the things we wanted to explore with the SOUR PATCH project was how NUI inputs could be used to add a new dimension to mobile game play – where the user has to move in real space to interact with virtual content – or in the case of SOUR PATCH game, duck the virtual blocks Dolly Doll throws.” said Carlson Bull, Creative Director and founder of Bully! Entertainment.
“As mobile technology progresses we’ll be able to utilize this kind of real-world interaction to create some very interesting engagements.” Says Bob Berkebile, VP of Technology and Innovation. “We can envision a new genre of experiences that seamlessly blends natural inputs and virtual content to dramatically enhance game play, learning and brand activations.”

“The SOUR PATCH game is a step toward this future,” said Bull. “When AR and gesture recognition systems make their way into eye ware, a world of possibilities will open up.”