Why Teaching a Robot to Fetch a Cup of Coffee Matters

In robotics, as in life, it often takes small steps to reach a big goal.

We try to post about robotics research that’s engaging and fun, and it’s easy to look at some of what we cover and wonder why it’s at all relevant. [Editor's Note: We dedicate this post to a recent commenter named David.] Being fun and being relevant are not things that are mutually exclusive, and we put a lot of energy into finding things that have a bit of both. At the 2013 IEEE International Conference on Robotics and Automation (ICRA) this week, we spotted a good example of this: a presentation by Stanford researchers about a PR2 robot autonomously fetching a cup of coffee from a coffee shop. It’s a cool video to watch, and we’ll lay out for you why getting a PR2 to do this is advancing the field of robotics as a whole.

Okay, it’s not like you’re going to get a PR2 to fetch you coffee any time soon, and in that sense, this research isn’t what you’d call practical. What you have to keep in mind when you see fun demos like this is that the demo itself is just a way to demonstrate the functionality and practicality of the underlying research. In this case, there are four challenges that have been solved:

Navigating between multiple 2D maps. Lots of robots rely on 2D maps to get around, and generally, researchers avoid having their mobile robots navigate multiple floors (because they’re bad at it). In order to get the multiple floor thing to work, the Stanford researchers developed a “gateway” system that links one 2D map to another 2D map through a portal (like a door or an elevator), and all robots that run ROS can now use this method to enhance their navigation capabilities.

Robust operation of an elevator. Elevators can be tricky for robots, especially when they are sharing it with humans. In this experiment, the entire run was autonomous, so the PR2 had to find the elevator, find the call buttons, push the right button, detect when the elevator arrived, make sure that the elevator was heading in the right direction, make sure the elevator didn’t have any humans in it (for safety reasons), enter the elevator, find the control buttons, push the right control button, detect when the elevator has arrived at a floor, make sure that it’s the correct floor (it checks the floor numbers on the elevator doors for this), and then exit. Whew! That’s a huge number of steps that are heavily dependent on robust vision and manipulation.

Opening heavy, transparent, spring-loaded doors. Imagine how hard it is for a robot to detect an object like a glass door, which has been specifically designed to be as invisible as possible. There are some complicated ways around the problem, but in this case, the researchers solved it by simply reaching out to physically touch the door. The Americans with Disabilities Act mandates that door handles (shiny silver and also hard for robots to see) are at a set height, so once the robot found the surface of the door at that height, it could just move horizontally until it found the handle. Getting through the (very heavy and spring loaded) door really should qualify as a separate challenge, and it was solved by relying on the base of the robot for power to pull the door open, and then keep it open as the robot moves through. This approach can be applied to any robot with a strong base and an arm.

Passing objects between robots and humans. The coffee shop employees weren’t roboticists: they had to be able to interact naturally with the robot without any training at all. Humans usually don’t have a problem taking stuff from robots, and they’re happy to yank objects out of a robot’s gripper. But humans hand things to robots much differently: think about handing someone a glass of water. You don’t shove it into their outstretched hand, but rather you hold it in front of them and wait for them to grab it. The robot would really rather you shoved it into its gripper so that it could detect when to activate a grasp, but the researchers solved this problem by waiting for the camera on the PR2’s forearm to be covered up by the object, and initiated the grasp based on that.

And succeeding at these tasks is just part of the story. For each one of the challenges, a key component of autonomy was to be able to detect failure, and then try again. Since it’s the failure cases that reveal what the real challenges to autonomy are, it’s important to find a task (like coffee retrieval) that you can use to test the robot out in the real world (or as real a world as a college campus qualifies as). Reading about where the robot had issues is very interesting. From the paper:

The system avoided a variety of obstacles and failure conditions during the testing. A common case was the presence of people and carts in the environment. The robot successfully avoided these obstacles by either driving around them or by waiting for them to clear. A second issue occurred when humans called the elevator to another ﬂoor before the robot could successfully press the control button. The robot avoided this problem by detecting and rejecting incorrect ﬂoors during the exit phase of the elevator navigation process. A third common issue was failure to push the buttons inside or outside the elevator. Recovery was achieved because the robot re-ran these processes when the elevator did not act as expected within 30 seconds.

In addition to the failures of the robot described above, the robot avoided other cases caused by humans and environmental issues. One failure mode was that the coffee shop employee did not fully push the beverage cup into the robot’s hand. This cause the robot’s hand to slip off the beverage cup as the robot attempted to pull the cup back from the human. However, the robot detected that no object was present in its gripper and asked the human to try again. Another challenge arose from a case where a human pushed a door open and then released it while the robot was lining up to navigate the door. The robot avoided this problem by waiting for sensor readings to stabilize, given its quasi-static approach to navigating the world.

It’s one thing to try to plan for all of these cases in advance, but it’s quite another to robustly and successfully execute them in practice. This is why it’s necessary to find a task to perform, and you want that task to be something complex and useful (and, why not, fun to work on!). So really, the end goal of this research, where the PR2 brings someone a cup of coffee, is incidental: it’s the fact that it was able to complete all of these challenges (five times in a row, in fact) that shows that progress has been made, and that problems really have been solved, outside of a laboratory.

Hopefully, this gives you a sense of why research that seems like it’s just for fun can actually make very useful contributions to robotics as a whole. This is not true in every case, of course, but just keep in mind that there are lots of things that need to be figured out in robotics, and there’s no reason why roboticists shouldn’t find exciting ways to solve them.

"Replacing the Ofﬁce Intern: An Autonomous Coffee Run with a Mobile Manipulator," by Anthony Pratkanis, Adam Eric Leeper, and Kenneth Salisbury from Stanford University, was presented this week at ICRA 2013 in Germany.