This copy is for your personal non-commercial use only. To order presentation-ready copies of Toronto Star content for distribution to colleagues, clients or customers, or inquire about permissions/licensing, please go to: www.TorontoStarReprints.com

The moral dilemma of programming self-driving cars

When researchers ask whether the vehicles should save some passengers or many pedestrians in the event of a crash, a contradiction emerges. Respondents chose the greater good — but they wouldn’t buy a self-sacrificing car themselves.

An autonomous version of Acura's RLX Sport Hybrid SH-AWD navigates around a dummy at carmaker Honda's testing grounds at the GoMentum Station autonomous vehicle test facility in Concord, Calif., on June 1, 2016. (NOAH BERGER / REUTERS)

This May 13, 2015, file photo shows Google's self-driving car during a demonstration at the Google campus in Mountain View, Calif. The U.S. auto industry's home state of Michigan is preparing for the advent of self-driving cars by pushing legislation to allow for public sales and operation — a significant expansion beyond an existing law that sanctions autonomous vehicles for testing only. (Tony Avelar / The Associated Press file photo)

Self-driving cars could save a million lives a year by eliminating the 90 per cent of car crashes caused by human error. But as autonomous vehicles proliferate on real-world roads, they will still inevitably face life-or-death dilemmas.

Should an autonomous vehicle’s algorithms be programmed to swerve and sacrifice its passengers if it means saving the lives of many pedestrians? Or should the car protect its occupants at all costs?

In a series of studies described in the journal Science, a trio of U.S. and French researchers tried to gauge the public’s response to this moral quandary — and discovered a typically human contradiction.

In six carefully designed online surveys, respondents voiced a strong moral preference for machines that would choose the greater good, sacrificing one or two passengers to save five or 10 pedestrians. But the respondents would not want to buy such a car themselves.

“You can kind of call this the tragedy of the algorithmic commons,” said Iyad Rahwan, a study co-author and a professor at MIT’s Media Lab.

Article Continued Below

“Even if you started off as one of the noble people who are willing to buy a self-sacrificing car, once you realize most people are buying self-protective ones, then you are really going to reconsider why you are putting yourself at risk to shoulder the burdens of the collective when no one else will.”

Even if these passenger-versus-pedestrian scenarios are rare — in fact, even if they never occur — these are the types of discussions the public needs to have, other researchers emphasized.

“We’re at the edge of an age where we’re programming behaviours into machines,” said Wendy Ju, executive director of interaction design at Stanford University’s Center for Design Research.

“One of the quandaries in this, and a moral dilemma in itself, is whether machines should necessarily be programmed the way everyone wants them to be. Because sometimes the crowds aren’t the most ethical decision-makers.”

The surveys found a consistent preference for self-sacrificing autonomous vehicles in general. For example, a car programmed to swerve and avoid hitting 10 or even two pedestrians while sacrificing its solo passenger received high moral approval. Even when participants were asked to imagine themselves and a family member as the car’s occupants, the approval of its morality dipped but remained above neutral.

Yet in these studies, the participants responded with a low likelihood of actually purchasing an autonomous vehicle that would sacrifice themselves and their family members for the greater good. They still thought the cars were programmed to do the right thing — they just didn’t want one for themselves. They also didn’t want the government to regulate autonomous vehicles to be self-sacrificing.

“That’s a big challenge to the wide-scale adoption of autonomous vehicles, especially when there’s already a basic fear about entrusting a computer program to zip us around at 60 miles an hour or more,” said Azim Shariff, a study co-author and director of the Culture and Morality Lab at the University of California Irvine.

Ju questioned whether these attitudes would truly hinder the adoption of autonomous vehicles or skew market forces: car manufacturers would not likely advertise how the vehicles would behave in these very rare scenarios. An accompanying perspective in the journal points out, however, that the greater ethical dilemma may be whether manufacturers even make the weighting of their algorithms transparent.

Moreover, many engineers believe autonomous vehicles will function more like a shared transit service. The public may accept a different risk profile for transportation that seems more like infrastructure than an individually-owned commodity.

Either way, we have to ask the questions posed in the Science study, says Bertram Malle, co-director of the Humanity Centered Robotics Initiative at Brown University.

“These machines have to make decisions, and if we don’t think about it in advance, the machines will still do something,” Malle says. Humans generally dislike randomness, he points out. It is unlikely society will be comfortable putting these vehicles on the roads without any idea of how they will react in such scenarios, even if those scenarios turn out to be hypothetical.

Malle points out that autonomous vehicles are just the tip of the robot-human interaction iceberg. While the international community has extensively discussed the ethics of autonomous weapons, artificial intelligence in health care and robots that care for the sick and elderly are proliferating — yet there has been little discussion about the moral decision-making of robots in homes and hospitals. Interestingly, his own research has found that people have stronger moral expectations for robots than for humans, especially if the robots appear more machinelike than humanoid.

“One of the major things I really endorse, and that we’ve written in our papers, is that we need to start a public discourse,” Malle said.

The Toronto Star and thestar.com, each property of Toronto Star Newspapers Limited, One Yonge Street, 4th Floor, Toronto, ON, M5E 1E6. You can unsubscribe at any time. Please contact us or see our privacy policy for more information.

More from the Toronto Star & Partners

LOADING

Copyright owned or licensed by Toronto Star Newspapers Limited. All rights reserved. Republication or distribution of this content is expressly prohibited without the prior written consent of Toronto Star Newspapers Limited and/or its licensors. To order copies of Toronto Star articles, please go to: www.TorontoStarReprints.com