IROS 2013: Robots That Pay Attention To You

Having a robot obey your commands is hard. Having a group of robots obey your commands is really, really hard. The good news is roboticists are making good progress in solving this problem. Research presented at IROS early this month shows how we may be getting to the point where teams of robots can pay attention to what we’re saying, where we’re looking, and what we’re doing, leading to a much more natural way to control them.

Think for a second about how you’d direct a subset of a group of people to do something. You’d probably look at each person in turn, saying something like “you, you, and you” while perhaps pointing to emphasize what you wanted them to do. It’s an intuitive thing for humans, because we convey attention through our gaze, and researchers from Simon Fraser University (which, incidentally, has an awesome bagpipe band) are teaching robots to pay attention to what we pay attention to.

Here’s a video illustrating the concept:

The AR Drones are watching the human with their cameras, and cooperatively using face detection to decide which one of them the human is looking at while listening for specific audio commands. Based on where the human’s attention is focused when verbal commands are issued, the robots can then decide which one of them the human is directing. It works if you want to select multiple robots (one at a time), or single robots, whether or not they’re already in the group.

The photos below show the camera views for the three robots. The robot that the user is staring at has the highest "face score" (left image).

The user gets feedback from the robots by watching their LEDs, which change color when the robot thinks it’s being paid attention to.

This same basic principle is just as effective with gestures:

The point of all this is to develop an intuitive command system that even an untrained user can quickly pick up on. To that end, future work will allow robots to be grouped into named teams, so that it’ll be possible to easily direct specific robots to join or abandon different groups. And gesture control will be expanded to allow for directional control. Put it all together, and you can imagine a scenario where you might be able to say to a group of six quadrotors: “you and you are Blue Team, go fly over there. You three are Red Team, patrol this area. Oh, and you, you’d better go join Blue Team.” It sounds so easy to do, and thanks to some very hard work by roboticists, we're getting there.

“ ‘You Two! Take Off!’: Creating, Modifying, and Commanding Groups of Robots Using Face Engagement and Indirect Speech in Voice Commands,” by Shokoofeh Pourmehr, Valiallah (Mani) Monajjemi, Richard Vaughan, and Greg Mori, and "HRI in the Sky: Creating and Commanding Teams of UAVs with a Vision-mediated Gestural Interface," by Valiallah (Mani) Monajjemi, Jens Wawerla, Richard Vaughan, and Greg Mori, all from Simon Fraser University in Burnaby, B.C., Canada., were presented at IROS 2013 in Tokyo, Japan.