Can’t find the words? Google with a photo instead

Smartphone users no longer need to think up an appropriate phrase when searching the web on the go – at least, not if they are using Google’s Android operating system.

“You take a picture of an item and use that picture as the query,” explains Vic Gundotra, one of Google’s vice presidents for engineering.

The service, called Goggles, attempts to identify the object – for example, it might recognise a landmark like the Golden Gate bridge in San Francisco – and returns a page of conventional search results, as if a user had typed the name of the landmark. Goggles can also identify artworks and extract the text from a photo, for example of a business card. It might find places to buy a product you’ve seen or identify local attractions.

When a user takes a photograph, Goggles sends the image to a Google server where it is analysed by algorithms looking for “signatures” of objects within the image, says Gundotra. The signature is then compared against a database of more than a billion images.

“The best matches are ranked and sent back down to your device, all in a fraction of a second,” says Gundotra.

Point, shoot, buy

Goggles is not the first visual search engine, nor is it the first available as a mobile app. Since 2006 Bandai Networks, the largest cellphone applications provider in Japan, has enabled users to buy products such as CDs by taking photos of them. The system uses software developed by Evolution Robotics in Pasadena, California.

The usefulness of searching visually was always going to come to Google’s attention, says Paolo Pirjanian, Evolution Robotics’ president and CEO.

“It makes it possible to do a lot more than you can describing something by words alone,” he says. “It is an obvious next step for search.”

A mouse pointer for the world

Google’s product is tailored to work best on certain types of object, such as landmark buildings, says Gundotra, but the goal is to identify any object in any image. “It will be as simple and easy as pointing at an object. You will be able to treat it like a mouse pointer for the real world.”

However, Google has set itself some limits, to fend off privacy fears.

“While we have the underlying computer science technology to do facial recognition, we have decided to delay that,” Gundotra says, citing privacy concerns. Google acquired face-recognition technology developed by Neven Vision in 2006.