Robots Search Google Goggles to Pick New Things Up

Some days Google seems like it’s more of a science fiction factory than a search engine, developing products like driverless cars, and augmented reality glasses. An academic project at Berkeley adds another element to the mix – Robots. Robots that can help pick up commonplace objects around your home, and put them in their proper places.

A paper submitted to the IEEE International Conference on Robotics and Automation, to be held in Karlsruhe, Germany on May, 2013, describes the role that Googles visual search queries plays in helping robots understand the objects that they might try to pick up, before they do. In Cloud-Based Robot Grasping with the Google Object Recognition Engine, we’re told about cloud-based robots that can view objects, and send queries about them to version of Google Goggles on the cloud to learn more about those objects and the best way to grasp them.

Google Goggle’s is Google’s visual search app, which enables you to take photographs and send them to Google to potentially perform facial recognition searches, OCR searches for text in images, product and bar code recognition, recognizing landmarks and places and named entities, and more. I spent a few hours at my Mom and Dad’s house a couple of weekends ago taking pictures of almost every photo and painting they had on their walls, and seeing if Google Goggles recognized any of them.

Another feature that the visual search engine is capable of is recognizing objects, and the Berkeley team, with the assistance of James Kuffner of Google, appears to have achieved a goal that had eluded them in the past with the use of Google Goggles. From the paper’s introduction:

One as-yet unachieved goal of robotics and automation is an inexpensive robot that can reliably declutter floors, tables, and desks by identifying objects, grasping them, and moving them to appropriate destinations such as shelves, cabinets, closets, or trashcans.

Errors in object recognition can be costly: an old chocolate bar could be mistaken for a cellphone and moved to the charging station, or vice versa — a cellphone could be placed in the trashcan. The set of objects that may be encountered in an unstructured environment such as a home or office is essentially unbounded and dynamically grows as our increasingly capitalist global economy designs new products to satisfy demand from consumers (and shareholders).

The project used a customized version of Google Googles to perform object recognition, training the system on 15 commonplace objects that tend to be around a home, such as a bar of mustard, an air freshener, and a bar of soap.

When I was writing about the future of search being spoken and visual queries not long ago, I hadn’t anticipated that some of those searchers might not even be human.

Related

Reader Interactions

Comments

Amazing. I can’t help but wonder what would happen if you had one of these AND a Roomba? Would they fight each other? Who patrols the floorspace? Would a dollar on the floor be trash or an object to pick up and place on the counter?

It is hard not to take Google as an adversary when your site’s fortunes are changing drastically every other week but then you read about the kind of innovations Google is spearheading and you have to laud their efforts in creating a new future.

The paper definitely had me thinking of whether or not we might see cloud connected roombas sometime in the not too distant future, too. Will we need Isaac Asimov’s three rules of robotics sooner than we think?

Hi Bill
Thank you, fascinating, but now I have a vision of Marvin the Paranoid Android (Hitchhikers Guide to the Galaxy) wandering about the house tidying… all the time muttering ‘brain the size of a planet’ and telling me ‘you could have bought this cheaper at [insert adwords driven sponsor].

This seems like an effort for android to try and place themselves a step a head of everyone else. Glasses are an interesting concept. Do we think that society is ready for such a thing? Perhaps this is a gadget that should slow down until is is perfected. While is certainly true that mobile devices are becoming popular, this might be jumping too far ahead. Thoughts?

I’m attending this event at Google Campus in London this week:https://plus.google.com/u/0/+CampusLondon/posts/UqnBrRVhyAm The gist of it: “Amit Singhal, Google’s VP of Search, recently said that he believed that ‘the destiny of search is to become the Star Trek computer, a perfect assistant by my side whenever I need it”

Too far ahead or not I don’t think they will stop anytime soon and some of these ideas may flop but others will succeed.

Looks like it has the potential to be a pretty interesting event. Between self driving cars, Google Glass, Google Now and Google Field trip, this robots paper, a couple of Google whitepapers on telepresence robots, Google really is venturing into worlds of science fiction. Would love to hear Amit Singhal’s take on that.

One of your earlier commenters mentioned the “matrix”. I concur. That is exactly what I thought about when I read this article. It’s interesting and migraine inducing at the same time. I am glad I discovered this site. It is not as SEO geeky as I expected. I actually understood everything you wrote…although I now have the words “Google Googles” stuck in my head. Thanks a lot.

You know it is stuff like this that just makes me wonder what Nicoli Tesla would do in this era. I love hearing about new inventions especially ones that let us peer into the not to distant future. To bad its google and I only say that because I really wish more companies would have far reaching unique ideas as well. Remember 3 years till hoverboards.

Trackbacks

[…] lead to a “Star Trek “ type computer, as are a few lesser known (projects and papers) such as: Robots Search Google Goggles to Pick New Things UpAmit spoke about his dream as child at the foothills of the Himalayas, about building such a […]

[…] might be used by robots to learn about objects that they might have to interact with, in the post Robots Search Google Goggles to Pick New Things Up. Google’s new patents involving robots also describe how cloud computing might play a role in […]