Please Tell This Robot What a Turtle Looks Like

Pretend you're a robot. (I do this all the time, it's great!). Okay, are you pretending? Awesome! Now, take these colored blocks and BUILD ME A TURTLE, ROBOT!

If you're panicking right now, that's understandable. A general purpose robot would probably have no idea what a turtle was, much less how to build one out of blocks. There are ways that you could teach the robot about turtles and blocks, but you, being a human, are hopelessly flawed and would only be able to teach it your conception of what a turtle should look like and how to use blocks to make one. What the robot really needs is to be able to examine a bunch of different examples of a bunch of different turtles, and then use machine learning to choose the best, most reliable, and most efficient one to build. And rather than have you try and do that all on your own, researchers at the University of Washington are paying strangers to do it as part of a crowdsourced effort.

Amazon's Mechanical Turk service provides access to thousands of real humans who are willing to do very small amounts of work for very small amounts of money. It functions best when you have a simple task that you need to be done many, many times, and that a human can do very quickly but that would be difficult for a computer/robot. For example, one task (currently available, if you're interested) is to look at a series of images and determine whether the given keyword accurately describes each image. Very easy for a human, very difficult for a computer.

For researchers, a mechanical turk system provides a cheap and easy way to get huge, simple datasets from real humans. For example, if you use the service to ask people to make a turtle with some available blocks, you'll get results that look something like this:

My personal favorite is third row, sixth from left.

So now that you've got all of these hand-built turtle models, you can apply machine learning techniques to the lot, and get a robot to figure out which turtle to build, "based on difficulty to construct, similarity to the original, and the online community’s ratings of the models."

What's useful here is the ability to leverage a large crowd of humans with a minimal amount of cost. Eventually, robots may get to the point where they can use the mechanical turk service (or something like it) to autonomously request assistance from humans when presented with an unfamiliar task.

This idea of robots asking for help is a powerful one, because it makes general purpose robots a much more realistic proposition: making a robot that can perform something like 75 percent of tasks may be difficult, but bumping that 75 up to 90 percent could easily be as difficult as the entire first 75 percent was. And don't even think about 100 percent: we're talking about robots here. But with the direct or indirect assistance of humans, robots that can ask for help when they're stuck on something will be significantly more capable than robots that can't.

Future work will focus on exploring the application of the approach to more challenging tasks such as 3D object assembly and tool use, self-learning of heuristic measures of task-difﬁculty, and endowing the robot with the ability to use decision theoretic methods to decide when to crowdsource and when to ask the physical user for more examples.

You can read the entire paper here; it was presented last month at ICRA 2014.