The iPad’s next killer app: Osmo connects iOS kids’ games with the real world

“Everything we are doing here is magic.” I’m not usually someone who embraces Silicon Valley’s tendency to paint every little product iteration as magical and revolutionary. But when Tangible Play co-founder and CEO Pramod Sharma uttered those words while demonstrating his company’s Osmo gaming device at our office, I couldn’t help but agree, and I couldn’t really stop smiling either.

The gist: What Osmo is all about

Osmo is best described as an iPad accessory that brings real-world gameplay to the tablet. The device itself consists of a simple white iPad stand that props up the tablet in portrait mode, and a small red clip that covers up the iPad’s camera. Once in place, an internal mirror redirects the camera view point to an area on the table in front of the iPad that’s roughly the size of an open magazine. It’s that area that Osmo uses for gameplay.

At launch, Osmo comes with two packaged games with extra accessories. One of them is Tangram, the traditional Chinese puzzle game consisting of seven objects that can be combined to create an endless number of shapes. Just like the original, Osmo uses differently-colored wood shapes for its Tangram game. But unlike the original, this one also comes with an iPad app.

Osmo’s camera mirror allows the iPad to recognize objects placed in front of it, extending game play into the real world.

That app tasks players with solving ever-more-complicated Tangram puzzles by showing them the shapes on the screen. Players then pick up the physical shapes, and arrange them in front of the iPad, which gently provides feedback, and offers up some clues for harder levels.

The second game, simply called Words, is essentially Osmo’s version of Pictionary. It consists of two sets of letter cards in two different colors, allowing you to play with or against another player as well as in single-player mode, and yet another iPad app. The Words iPad app shows you a picture and tells you how many letters the word you are looking for consists of. To play, you simply start to put letter cards in front of the iPad, and the app automatically starts to count the right and discount the wrong answers.

For a first look at Osmo, check out this promotional video:

Words is a simple game, but it shows some of the technology behind Osmo at work. During our demo, I first tried to align all of my letters neatly in front of the iPad, only to be interrupted by Sharma. That’s not how kids play, he told me. Instead, he just dumped a handful of letters on the table, and Osmo quickly recognized all of them, regardless of alignment and orientation. It’s a pretty addictive game, and very impessive technolgy as well.

But the real magic of Osmo revealed itself when Sharma showed me a third game, one that comes without any additional objects. Newton simply works with any white sheet of paper and a pen, and of course yet another iPad app.

Osmo’s Newton game is controlled by drawings, and even real world objects.

The goal of this Osmo game is to redirect balls that are falling on the iPad screen to hit other objects on the screen. Except you don’t do the redirecting on the screen, but on your piece of paper. Draw a line, and the iPad app automatically imports it into the game. Draw another one, and you might have a funnel for those falling balls. Now add a physical object — a key, a pen, a toy, your own fingers, or any object, really — and Osmo automatically registers its outlines and adds them to the game as well. It’s augmented reality meets scribbling, and it’s a whole lot of fun.

The idea: using proven games to connect the iPad to the real world

Sharma told me that one of the inspirations for Osmo was his five year-old daughter, who is “constantly glued” to the iPad. Sharma wanted to help her embrace real-world play, and also give her a chance to be more social and play with others — something the iPad doesn’t exactly lend itself to. At the same time, he recognized that some traditional toys and games have started to lose appeal to a generation that’s growing up with the iPad. “Lego has lost the battle,” he argued, adding that it’s increasingly being replaced by Minecraft.

Tangram has been around for hundreds of years. Osmo is trying to reinvent the game for the mobile age.

That’s why Tangible Play decided to marry the two. Sharma’s team purposefully concentrated on existing and proven games for its launch: “They are good for kids, we know that.” Of course, you could also play Tangram without the iPad. But Osmo is trying to make games like it more interesting, he said, adding: “It’s not if you can play it. It’s how long you can play it.”

Tangible Play has been testing Osmo in a number of schools over the last couple of months, and Sharma said that watching kids play with it and getting feedback from teachers helped the company shape its vision. “It has given meaning to what we are doing,” he added.

The challenge: kids don’t play under ideal conditions

The other inspiration for Osmo was Sharma’s previous job. Before launching the company, he worked for seven years at Google(s GOOG), where some of his duties included working on Google’s book scanning project. Back then, he started thinking about what else you could scan, and how you bring real objects into the virtual space.

Turns out, making this work in the real world isn’t actually all that easy. Osmo has to work under variable lighting conditions, on different kinds of tables and with objects that align in countless ways. Conventional image processing doesn’t work for this unless you force kids to always align their letter cards (the way I did the first time I tried it), require play on white backgrounds and impose all other kinds of unrealistic constraints.

Kids don’t perfectly align letters, and everyone’s living room table looks different — so Osmo has do do some heavy lifting on the computer vision front.

Cornell University Professor Serge Belongie, who is a computer vision expert and who helped Tangible Play tackle some of these issues, described Osmo’s approach this way:

“We make use of statistical machine learning methods together with advanced image feature extraction techniques to accommodate real world variations in the patterns we need to detect, ranging from letters on cards to shapes of Tangram pieces. In this respect, the approaches we use are data-driven, taking into account real world noise, rather than based on ideal templates of the patterns of interest.”

The future: where Osmo goes from here

Two things struck me about Osmo when Sharma showed me the product. First, it’s incredibly well-designed, from the stand itself to the packaging to the delightfully minimal games that don’t come with a whole lot of directions, but instead encourage kids to explore and figure out how to play on their own.

Osmo is incredibly well-designed, and uses magnets to make sure the mirror doesn’t get lost in a household with kids.

Sharma credits his co-founder Jérôme Scholler, who previously worked as an engineer for LucasArts, with all of these design decisions, and explains that the goal was to build a kids’ game with the look and feel of an Apple product. The company succeeded with this, and I can see Osmo popping up in Apple stores once it’s commercially available later this year.

Osmo also has great potential for future games or even experiences that go beyond traditional gaming. Sharma told me that his team has “at least 50” ideas for additional games that could make use of the Osmo hardware. After playing with it for a few minutes, I couldn’t help but think about other apps as well. For example, what if Osmo allowed kids to arrange music by manipulating real-world objects? Or what if it transformed kids’ watercolor drawings into digital animations, without the constraints of touch screens that make us jump through hoops to recreate things that really work better on paper?

Sharma said that he doesn’t quite know yet how Tangible Play is going to make money. Maybe it will be hardware sales, maybe paid apps, maybe a mix of both. For now, Osmo is available for pre-order through a self-hosted crowdfinancing campaign, which will give early adopters a chance to buy a device for $50, as opposed to the $100 Tangible Play plans to sell it for when it retails later this year. The company has a very modest campaign goal of $50,000, which it will surely (and justifiedly) surpass in a matter of days, if not hours.

I know I’m putting in my order, and I can’t wait until I get one shipped in a few weeks.

i-waste your money on this sorta crap. Still nothing with lasting power. The iPad is having an identity crisis right now. It was meant for entertainment and information, not a bad gaming platform. But it’s not very good for productivity. We have 3 iPad’s and 2 Windows Surface Pro 2’s. We mainly use the Surfaces now, but for those few apps we can’t find a windows program for, we use the iPads. It is sort of a shame to see such expensive tablets sit there not being used. Gave away 2 iPad 3’s to family members.

Tough time we are in, No one platform is quite right, except for the MAC Desktop and the Windows Desktop. All tablets are trying to be more than what they are.

There are already children specific tablets that can do this using a positionable camera. Not to mention a Microsoft Surface Table project that doesn’t require keeping track for so many stray pieces that can get lost or chewed up by the dog! ….it’s a FAIL in my Book!!!

But it won’t be shipped ‘in a few weeks’. It is still many months away according to their site. 50 bucks for a clip on mirror and a plastic stand? Seriously? Which company with any credibility whatsoever makes people buy something months before it’s even available? As for the ‘discounted price’: you can’t advertise something at a discount until it has been sold at the original price. Anyone can sell something ‘at a discount’ on a price they have no chance of ever selling something. Sadly, some people will be suckered into this scam in the same was they did with that other scam, the electronic Tag. Wave goodbye to your money people!

That’s right. Android. Fail.
Don’t worry, when they can afford to take on a fragmented, low quality, inconsistent hardware platformâ€”wait, they’ll probably be working on the next iPad game and investing their energy efficiently.

Very clever. Another similar approach is available from Tiggly, which offers apps for younger children in which the kids have to place plastic shapes onto their matching images on the screen of the iPad: http://tiggly.com