3-D Maps, Camera Phones Put ‘Reality’ in Augmented Reality

Earthmine says its three-dimensional reality augmentation software can help locationally-aware directories and search engines label points of interest with more precision than Google Street View or other solutions can.

Augmented reality software promises to let us see the world as the Terminator saw it: overlaid with digital information. Given that most of us aren’t cyborg assassins from the future, we’re more likely to use the technology to see restaurant ratings as we walk past that new taqueria, status updates floating over friends’ heads or a trail of digital breadcrumbs leading us back to our hotel rooms.

Most of us will use smartphones instead of a head-mounted display because cellphone cameras help these services reconcile digital and physical reality without making you look like a cyborg assassin.

But as the video below shows, the digital tags in today’s early augmented-reality mobile apps float unmoored, as if not quite tethered to reality. This “floating tag” problem is more than just a quirk or annoyance because of how our brains process information. In order to convince us they’re real, augmented-reality tags must stick to the world they’re describing.

Google Goggles sidesteps the problem by displaying tags in two dimensions at the bottom of the screen rather than trying to overlay them.

The solution is on the way, but it’s not from Google. Earthmine, which takes 3-D pictures of cities that assign every pixel a specific point in real space, launched three software-development kits in October to help developers affix augmented reality tags more firmly to reality. The company captures more data per street than Google Street View, which it claims is its only competitor.

By matching Earthmine’s 3-D city data with what a phone’s camera sees, developers can solve the floating-tag problem using Earthmine’s iPhone, HTTP and Flash software development kits. Company CEO Anthony Fassero says they “localize [a developer’s] camera position to less than a foot, and its orientation just fractions of a degree, versus 20 to 30 feet with a GPS, and 20 degrees with the [iPhone’s] compass.”

“Right now, [augmented reality] is kind of cheesy,” he said. “You pull up your augmented-reality app and things are kind of floating around, and it doesn’t look like Terminator, where you’re looking around and all the information’s popping up. The thing Earthmine does for augmented reality is we enable it to look like it does in the movies.”

“For it to line up and ‘stick’ in the real world, they need to use our data,” Fassero said. “There’s no other way to get that accurate localization.”

A number of local search companies apparently are taking him up on that, but seem to be doing it in stealth mode. Munich-based Metaio, an early Earthmine client already has an augmented reality iPhone app on the market, (Junaio video), but it, too, suffers from the floating tag problem.

“It was a big decision for us to put it out there, because the technology part of Metaio said, ‘Hey, this is not ready yet in terms of accuracy, it’s just GPS, man, that’s jumping around like crazy, and we can’t do it,'” said Metaio CTO Peter Meier. “But then of course, on the other side, we have to be out there.”

The iPhone’s GPS is accurate to 30 feet or so most of the time, but that’s not close enough.

“Imagine there are two restaurants next to each other, and one gets a 5-star rating and the other gets a 1-star rating as people tag it,” said Meier. “[Using GPS], the 1-star restaurant [can] get the 5-star rating.”

Metaio’s app eventually will incorporate GPS, Earthmine’s 3-D maps and Metaio’s optical tracking technology in what could be the first augmented-reality app to solve the floating-tag problem, assuming another company doesn’t beat it to the punch. Metaio plans to debut its first working demo for Earthmine’s hometown of Berkeley, California, in the coming months.

Earthmine’s small size meant it could provide specific information to Metaio, such as where the sun was when a picture was taken. Metaio wants to know because it helps mesh photos taken at different times into a cohesive view.

“If the shadows are different, it makes a big difference for our computer vision algorithms…. We would never get that [data] from a big company like Google because we’re probably too small.” He also backed up Fassero’s claim that Earthmine’s more-detailed 3-D maps work better than Google Street View for mixing augmented reality and real-world information.

As much as a journalist might like to know what augmented reality’s killer app is, Meier says one doesn’t exist.

“What is the killer application? The same question would be ‘What is the killer application for a touchscreen?'” he said. “We’re not revolutionizing everything, we’re not creating really new applications, we just make what’s possible right now easier to use, easier to understand, and better to grasp.”

That said, he does see location-based, augmented-reality mobile games as a promising area. He expects game developers to use Metaio’s and Earthmine’s technology to create realistic, nongimmicky games that take place in real-world cities starting in 2010. An ad-supported model for AR gaming is easy to envision: Local businesses could pay to have the game send players past their storefronts.

“When there’s a dragon flying in front of a building, and you take one step and the dragon comes one step closer — exactly one step closer — and it’s really flying right next to that building, and it’s not jittering around, it’s a completely different experience — close to special effects in real time,” Meier said.