Semi-autonomous driving systems like Tesla's
Autopilot and
Cadillac's Super Cruise are designed to reduce the stress
of driving by automating tasks like steering, accelerating, and
braking under certain circumstances, but the driver needs to be
ready to take over at any moment.

Those systems rely on unrealistic expectations about
how well humans are able to pay attention to the road and
quickly react if the semi-autonomous system needs them to take
control, Mary Cummings, a professor at Duke who studies the
interaction between humans and autonomous driving systems, told
Business Insider.

"We can't sustain attention, especially in boring
environments like highway driving," she said. "Expecting the
human to be able to just step in when we know they haven't been
paying attention is a huge problem."

Semi-autonomous driving systems like Tesla's Autopilot and
Cadillac's Super Cruise are designed to reduce the stress of
driving by automating tasks like steering, accelerating, and
braking under certain circumstances, but the driver needs to be
ready to take over at any moment. That's a problem, Mary
Cummings, a professor at Duke who studies the interaction between
humans and autonomous driving systems, told Business Insider.

Ad

"Any reliance on the human to intervene in any kind of
safety-critical capacity is a red herring. They're never going to
be able to do it reliably enough to make these systems, I think,
safe for the road," she said.

Systems like Autopilot, Super Cruise, Nissan's ProPilot Assist,
and Volvo's Pilot Assist rely on unrealistic expectations about
how well humans are able to pay attention to the road and quickly
react if the semi-autonomous system needs them to take control,
Cummings said.

Ad

"We can't sustain attention, especially in boring environments
like highway driving," she said. "Expecting the human to be able
to just step in when we know they haven't been paying attention
is a huge problem."

A Tesla representative directed Business Insider toward a
page on its
website that describes its approach to vehicle safety and
includes statistics about accident rates for Tesla vehicles with
and without Autopilot engaged.

"We believe the unique combination of passive safety, active
safety, and automated driver assistance is crucial for keeping
not just Tesla drivers and passengers safe, but all drivers on
the road," the page says.

During the fourth quarter of 2018, Tesla said there was one
accident per 2.91 million miles driven by Tesla vehicles with
Autopilot engaged, compared to one accident per 1.58 million
miles driven by Tesla vehicles without Autopilot engaged and one
car crash every 436,000 miles driven for all vehicles in the US.
But Tesla's data does not account for factors, like the
environments in which Autopilot is most often used, that could
also account for some of the differences in accident rates.

Nissan and Volvo did not respond to a request for comment.

There are semi-autonomous features, like those that alert the
driver if the car is drifting out of its lane or some advanced
cruise control systems - which can brake or accelerate based on
surrounding traffic - that can be useful, particularly at low
speeds, Cummings said. The danger comes when vehicles take
control of driving tasks at higher speeds.

"I'm a big fan of slow-speed autonomy," she said. "It's when we
see cars at increasing speeds under unstructured environments
with a lot more uncertainty that the car control systems really
fall apart."