A robotic arm is sublime at stacking boxes on a production line, but workplace robots struggle with social niceties. Now researchers are teaching robots the right way to act in social situations in the hope of making it easier for humans and robots to work together.

Human social interactions are full of subtle cues that are tricky for robots to interpret, says Song-Chun Zhu at the University of California, Los Angeles. So Zhu and his team set out to teach Baxter – an industrial robot designed to work alongside humans – to respond to social cues in a more natural way.

The team trained Baxter on videos of humans shaking hands, waving, helping each other up, passing over a cup and high-fiving. The robot’s learning algorithm generated rough skeletons of each person’s movement and used those outlines to infer human intentions and mimic their responses in social situations.

Advertisement

After watching 20 videos of each type of interaction, Baxter was told to take the place of one of the humans in each of the five situations. The robot, which has two arms and a moving base, attempted to mirror the movements of the humans it had learned from. It used a motion sensor to detect the location and body position of its partner, and a pressure sensor on its right hand to let it know when it was touching them.

Nice cuppa

Baxter used visual cues about the position of its partner to work out the most appropriate response in each situation. If it detected that its partner was reaching out with a cup in their hand, Baxter responded with an open hand to take the cup. A different group of volunteers then rated how successful and human-like each of these interactions were compared with simulations with an untrained robot. For each of the five types of interaction, the volunteers rated the trained robots as much more successful and human-like than the untrained ones.

The more Baxter watches humans interact, the better it becomes at responding appropriately in similar situations, says Zhu. He hopes that his learning model, which is being presented at this year’s International Conference on Robotics and Automation in Singapore, will improve robot-human collaboration on production lines or on tasks like building furniture.

Heni Ben Amor at Arizona State University says that having robots learn from watching humans interact won’t just make them better collaborators, it will help humans feel more at ease around robots too. “There’s a high demand for robots that are socially aware,” he says.

If robots can learn the basics of social interaction, like knowing how fast to move their arms or how close to stand to a person without making them feel uncomfortable, then humans are much more likely to accept robots in their home or workplace, he says.