Driverless Cars for the Road Ahead

Dan Neil says computer-controlled cars will be a reality. The question isn't "if" but "how" can technology allow humans to take a back seat to the computer

ENLARGE

Illustration by Matt Dartford for The Wall Street Journal

By

Dan Neil

Updated Sept. 27, 2013 12:03 p.m. ET

GREETINGS FROM the back seat of the world's smartest car.

Mercedes-Benz's autonomous research vehicle—cleverly disguised as a large black sedan—has been taking this reporter on a tour of hallowed ground. In August 1888, Bertha Benz, the wife of Karl Benz, actually stole her husband's prototype motorwagen, then the only automobile in the world, and drove it from Mannheim, Germany, to her mother's house in Pforzheim, some 60 miles away. It's a whole big thing with the company...

Look! Over there! A tree! See, you have a lot of downtime in this car, because it drives itself. To be clear, it drives itself under the wary eye of at least a couple of engineers at all times, who are watching the car's autonomous-driving operation and carefully recording errors so that they can dial in software fixes back at the lab.

A fully autonomous vehicle is at least more than a decade away. That's one way to look at it. The other is, Holy Cow! Robot cars in a decade? Where do I sign up? Dan takes a ride in Mercedes-Benz's autonomous research vehicle, and looks at the robo-cars of today and tomorrow. Photo: Getty Images

Eberhard Kaus, sitting in the driver's seat, is also there to grab the wheel in case the car becomes baffled by one of the route's many intersections, roundabouts, parked cars or random pedestrians crossing the street in Bruchsal's Imperial district.

At the frontier of making cars drive themselves, one big technical lift is the writing of pattern-matching algorithms so the machine brain understands what the machine vision sees. As the speed of traffic increases, these algorithms must become increasingly predictive, anticipating, as humans do, the probable intention of drivers in other vehicles.

To evolve, this research needs the car to experience randomness so that it will react appropriately in even the unlikeliest situations. "We have a washing-machine test," said Dr. Ralf Herrtwich, a leader in the autonomous-research program. "Something suddenly falls off a truck ahead of the car. Can the system react faster and better than a human?"

As of this unseasonably warm day in September, the test car has 6,500 kilometers on the odometer. It will have a million before it retires. A fully autonomous vehicle, something you'd put your mom in, is "maybe more than a decade" away, said Dr. Herrtwich, with a touch of regret.

ENLARGE

Mercedes-Benz's autonomous research vehicle
Mercedes-Benz USA

That's one way to look at it. The other is, Holy Cow! Robot cars in a decade? Where do I sign up?

Giving automobiles auto-piloting features—up to and including completely hands-free, eyes-closed operation with trusting souls aboard—is the Space Race of global auto makers, and you are the monkey in the capsule. Last month Nissan and Renault chief Carlos Ghosn promised that Nissan would bring affordable autonomous cars to the public by 2020. Mercedes-Benz already markets some of its driver-assist technologies as "semiautonomous": automatic lane keeping (positioning the car between the lines during brief periods of hands-off operation); and Stop & Go Pilot, an optics-and-radar-based cruise control that can see traffic ahead and adjust speed in heavy traffic.

"To be honest the auto industry hasn't had any wows in a while," said Rachel Nguyen, director of global upstream planning for Nissan Motor Company. "This is something we can shout about."

The first tremors of self-driving hit the market in 2007, when Lexus offered an auto-parking feature that automatically parallel parks the car, a skill few human drivers possess. The technology, which piggybacks on the ultrasonic sensors and low-cost cameras already on board to help the driver maneuver in tight quarters, has moved into mass-market cars such as the Toyota Prius, Ford Escape and VW Tiguan.

As other technologies have emerged—optically-based automatic lane-keeping assist and radar-based all-speed cruise control—cars have acquired fairly robust senses that allow them to briefly and tactfully fill in for the driver, in congested traffic or during long dull spells on the highway, or in the case of inattention.

The falling cost of computer horsepower is key. Moore's Law can be applied to the rapid growth in the speed and performance of devices based on semiconductors, and it is playing out in the cost and sensitivity of the sensor technology, said Larry Burns, former GM research chief, professor at University of Michigan and a Google adviser. "These systems' computational demands are enormous," said Mr. Burns.

Not if, but how

What is being considered isn't whether, but how to give cars more automation in limited situations, including driverless operation, meaning nobody in the driver's seat.

Brace yourself. In a few years, your car will be able to drop you off at the door of a shopping center or airport terminal, go park itself and return when summoned with a smartphone app. Audi demonstrated such a system at this year's Consumer Electronics Show.

Related

At your next dinner party, ask for a show of hands of the people who'd want that.

Everybody?

Anybody want a car that doesn't crash? At this month's Frankfurt auto show, mega-auto supplier Continental announced a partnership with IBMIBM0.64% to help bring autonomous vehicles to market, with "zero accidents" as a possible result. Volvo has promised to injury-proof its cars by 2020. GM and Carnegie Mellon aim to develop autonomous technology to eliminate car accidents.

That would be nice. In 2011, Americans were involved in more than five million auto accidents, with more than 32,367 killed. The cost to the U.S. economy in mayhem, lost productivity and lives totals $300 billion per year, according to the AAA.

Autopilot pioneers

Google's fleet of auto-piloting Priuses and Lexuses is perhaps the best-known autonomous-research program, and most recognizable, thanks to the cars' roof-mounted laser emitters on the roofs. As a harvester and sorter of titanic amounts of information, Google is in a unique position to develop, and profit from, the data-enriched 3-D virtual maps upon which autonomous cars will depend. The Google project, was led by Sebastian Thrun, former director of Stanford's Artificial Intelligence Lab, has logged hundreds of thousands of test miles without incident.

More on Cars

For the Bertha Benz autonomous route—a lovely tour through vale and village—Mercedes-Benz joined with Nokia, whose mapping service digitally reconnoitered the path, actively pinging the world to form a reliable data map.

At the end of the autonomy rainbow is a very different relationship with the automobile than we are accustomed to. Because if we can order vehicles hither and yon, and pay for them only as needed, why own them? The costs of transportation to young people are driving them away from automobiles in historic numbers. Now imagine a car-sharing service like Uber dispatching a car, or truck, or SUV, driverlessly. (Google Ventures invested $250 million in Uber in August.) It appears in your driveway when you want it, and disappears when you don't. Mr. Burns noted that private vehicles spend 90% of their time parked and unoccupied.

Autonomy "changes everything," said Mr. Burns. "Giving automobiles the ability to drive themselves is the biggest thing to happen to the automobile since the automobile."

How will it work? No one is quite sure.

The idea of self-piloting cars has been around a long time, but the vision belongs to Norman Bel Geddes, the American utopianist who designed General Motors' Futurama exhibit for the '39 World's Fair in New York. In the future city (as per Bel Geddes' sprawling set piece of miniatures), cars would move in tight squadrons safely under the control of a central traffic authority. They would be in constant radio contact with each other and the road. They would be electronically crash-proofed in a way that rules out random collisions. Speeds could rise, proximities close and road carrying capacities increase.

‘Can 'Skynet' serve the greater good for our automobiles?’

A beautiful order would be imposed on interurban traffic, and the cascading efficiencies would include a smaller infrastructural footprint, lower per-mile energy costs and higher system productivity (this vision being from the 1930s, the era of scientific management).

As of 2013, almost everything technically necessary to enact a real-world version of the Bel Geddes dream has been invented. But the details are shaping up differently. For this first generation of autonomy, for example, cars will rely on their own wits—their own cameras, sensors and map-keeping—rather than cede control to some master computer.

But, for now

In the near-term, autonomous cars will likely operate like "human beings on steroids," said David Strickland, an administrator of the National Highway Traffic Safety Administration. Independent of the infrastructure or traffic information, these vehicles will rely on powerful sensors (including long-range radar and thermal imaging) and onboard brains to assist the driver and, if appropriate, take the wheel. Among the first functions likely to be highly automated is highway driving, when driver workloads are low and the safety risk is fatigue. And distraction. NHTSA reports that in 2011, 17% of injury crashes (387,000) and 10% of fatal crashes were caused by driver distraction (3,331 killed), such as eating, grooming or chatting on the phone.

ENLARGE

A camera installed on the Mercedes-Benz autonomous vehicle.
Mercedes-Benz USA

The Mercedes autonomous vehicle sees the world primarily through a wide-baseline stereoscopic camera installed ahead of the rearview mirror at the top of the windshield. This camera can provide range, closing rates and direction of other traffic. The car also uses radar sensors front and rear to paint a picture of the world at long range (up to 650 feet) and midrange. Additionally, two radar sensors are installed in the front fenders to look out for cross-traffic threats. If the car approaches a green light and detects another car running the light from another direction, the car should react, avoiding, if it can, the collision and otherwise bracing the car and occupants for impact.

The car uses two other video cameras. One forward wide-angle unit helps interpret lights. "The color of traffic lights has been a challenge," said Dr. Herrtwich. "Which traffic light is your traffic light? Also, with a wide-angle camera, the light registers only as a few pixels. The computer has to recognize that pattern in real time, 100% of the time."

One challenge for these systems is the weather sensitivity of onboard computation. Stuffed into the trunk of the Mercedes is what looks like a mini server farm, a rack of three desktop computers and assorted power electronics to crunch the numbers. These chipsets and motherboards have to tolerate bitter cold and blazing heat. "We'd love to have the processing power of a PC," said Dr. Herrtwich.

The focus is on two types of autonomy. One is based on vehicle-to-infrastructure (V2I) communications, rather like what Bel Geddes had in mind. Cars orient themselves, navigate, talk to each other (V2V) and avoid collisions as part of a dedicated electronic network.

"The problem with V2I is penetration," said Dr. Herrtwich. For such systems to be even minimally effective, a large percentage of vehicles would have to subscribe, and they would rely on an infrastructure not yet engineered, built nor paid for. Beyond that, many of the promises of V2I—the fuel-saving platooning of vehicles on major highways—can be accomplished using the mobile broadband network.

"We're bullish on both technologies," Mr. Strickland said. "With V2I, cars we'll be able to see around the corner and nine blocks away. Vehicles will be able to predict scenarios and avoid them. They'll be able to deal with the chaos of traffic much faster than humans."

The architecture for an inter-automotive network is up in the air. NHTSA has reserved 75MHz in the 5.9 GHz band of the communication spectrum for auto makers who are doing research in V2X (the X stands for either infrastructure or vehicle), known as Intelligent Transportation Systems. This technology would involve equipping cars with various kinds of black box-like devices, to communicate with other cars and the infrastructure as well as to record and share telemetry from the vehicle.

In an age of connectivity based on mobile broadband, the more disruptive notion is one in which the rolling stock is connected on the "Mobility Internet," as mentioned in Columbia University's 2010 Earth Institute report. This would do "for the movement of people and goods what the Internet has done for the movement of information by coordinating large amounts of real-time spatial and temporal connectivity and infrastructure data."

That's right, John Connor. It's Skynet.

Regardless of safety, convenience, energy and transportation advantages of autonomous cars, are you really ready to turn over control to somebody's science project?

Seven years ago, Nissan did a study describing to consumers over-the-horizon features of autonomous cars. "Then it was...'Oh, you mean the car is going to take control? I'm not comfortable with that," said Nissan's Ms. Nguyen. "Now it's almost a complete 180 in attitude. And I think that's because they are doing other stuff in their car,They have a personal alliance with their smartphone, and the car is the last place where the Internet is not part of their lives."

Traffic-safety regulators in Europe and the U.S. are struggling to catch the tail of this comet. Several states, including Nevada and California, have adopted rules governing research and testing of autonomous vehicles on public roads. The policy debate to come is how far to trust our new robot chauffeurs.

The promise of autonomy is this: It doesn't have to be perfect, at first, as long as it is better than the faulty, flimsy wetware currently occupying the driver's seat. "Human error is behind almost 100% of car crashes," noted Dr. Herrtwich.

Robo-Cars, Today

ENLARGE

Ford Focus

Ford Focus Active Park Assist feature

The Ford Focus is the least expensive car available with a parallel-parking assistance feature. The driver switches on the system while the car is moving at low speed. The car scans for a place to park (the driver is responsible for determining legal parking). Once the ultrasonic sensors have located a space, the driver brings the vehicle to a stop, relinquishes control of the wheel and watches as the car swings into place (the driver controls the brakes and throttle). Other systems enable nose-first and tail-first parking.

Infiniti's flagship sedan is available with an optics-based lane-keeping system that will intervene with steering if it senses the car drifting out of lane. Also, the Q50 has what it calls Predictive Forward Collision Warning. This function, piggybacked on the car's adaptive cruise-control radar system, is able to judge the relative speeds of the cars ahead, and it can alert the driver of imminent danger caused by a potential rear-end collision and intervene to lessen the impact.

ENLARGE

Volvo

Volvo City Safety

Ever egalitarian, the Swedish company Volvo has made pedestrian safety an important part of its message. The company's Pedestrian Detection system sweeps the area ahead of the car with radar. If a pedestrian wanders into the path of the vehicle, the system will automatically brake, avoiding, or at least minimizing, pedestrian injury.

ENLARGE

Audi

Audi Piloted Parking

At this year's Consumer Electronics Show, Audi demonstrated a late-prototype system that would allow drivers to exit their cars and send the cars off to park themselves. Drivers could then summon their cars remotely with a phone app, to return when and where needed.

This intrigues me. Humans lack the ability to concentrate for any real length of time, and repetitive tasks ease us into mindlessness. Computers can hold speed and distance with far more accuracy than a human, never gets bored or angry or drunk and can recognize patterns. The patterns are actually there in the highway code.

My one question is this - Would the passengers be liable for any accidents? It seems like a mess. I have a good driving record and enjoy pretty cheap insurance rates ($26/month from Insurance Panda.. woohoo!). I also enjoy taking my car out for a spin and enjoying the 'freedom' of being able to drive anywhere. Will the driverless car allow all this? If not, I'll have to pass.

IMO.. Until they can ensure that there are no humans taking control of the wheel, insurance will be needed... at least uninsured motorist. Who knows? Maybe insurance as we know it will go away, replaced by any number of models that would more accurately represent the new risk distribution.

Law enforcement will love this. If your car is computer-controlled, your movements can be tracked in minute detail. And a really advanced system would be programmable to the extent that you'd be allowed to go some places but the car would not take you to others - or anywhere at all.

I've experienced Nissan's autonomous car and am certain this technology will be ready much sooner than 10 years. The bar has been set low by the "wetware" Mr. Neil describes. Humans can be excellent drivers, it's just that many of them choose not to be.

I'm a hypermiler who drives an EV (Nissan LEAF) and it's clear to me that the cars of the future will be electric and programmed to drive like I do. Safety and efficiency are the two most important aspects of my driving style. Most other drivers are incredibly inefficient and dangerous. Speeding and tailgaiting are the two most likely reasons for accidents. Autonomous cars will do neither. As more of these cars occupy our roads, it will have a calming effect on traffic, and commuting times will shorten as a result. Today, speeders rushing to "save time" end up creating large traffic jams. If they'd slow down a bit, the traffic would dissipate allowing for a steady speed that is higher than the average of a combination of speeding and stopping. Much less energy will be used as a secondary result.

Lastly, Neil is correct that an Uber type service will be preferable for most folks. Economics will drive this since purchasing, housing, maintaining, washing, parking and fueling a car costs lots of money. Using an app to call a car to take you somewhere is going to be much cheaper, and you can get work or play done while in route.

I'll keep my electric motorcycle for the fun stuff, but gladly sell my EV when the time comes.

Some people will believe anything. GM makes cars that do not break. MSFT has software with not glitches. The Government is here to help you. The check is in the mail. I promise I'll only put the........

I believe the key to regulations on these systems will be something similar to the NTSB investigation of plane crashes, along with a commensurate set of laws on liability which limit the liability to what one human would have caused - not the entire industry.. Each failure of an autonomous car system will then be analyzed and programmed not to be repeated. Over a relatively short (though still measured in months / years) period of time - the systems will become highly reliable - orders of magnitude better than human drivers. The public will accept that like an airplane accident, the few and between are worth the convenience.

Check out GM's EN-V 2.0. It will be launched in Tianjin, China next year. It is an autonomous vehicle. I drove it (the previous version) in Flint, Michigan. They are ahead of Mercedes. I have written a case on it and will be happy to share with anyone interested in it.

If it happens and is available for the average person it would be great . My disability doesnt allow me to drive. I have C.P and that could be over come, but if i drive for about 30 mins i fall asleep even I'm rested.so something like this would be welcome.Also think people that drink would benefit as would the rest of society.

Owing money for non- driving court ordered misdemeaner should not stop your driving rights to travel, while the roads are filled with dangerous drivers. I had not had an accident in 32 years, only a few parking tickets and my licence is suspended nationwide on that damned computer system. I was falsely accused of some "theft " charge in a other state. been fighting it by mail...m what a hassle. !!!!!!!!!!

boy, is the millennial gen. brainwashed by technology, technology has risks too, automation is not infallible! far from it. we will just be victimized by big brother even more than now, why not turn over all aspects of our lives and decision making, thinking to the government!!! I know we are a stupid society, but do you really think government is smarter?

The radar cruise control on my 05 Lexus has been terrific. I set the following distance and it maintains the gap. The other function of the radar that involves applying full braking and seat belt tightening in anticipation of a collision is perhaps less advanced. It can do some alarming things when there is a sudden movement in front of the car. Has it prevented and accident yet.... maybe there was a close call where it contributed but I'd like to think my razor sharp reflexes were right on top of that anyway.

Reminds me of the old joke about the first pilotless air flight... Shortly after take off it goes something like this:"We'd like to welcome you to the first pilotless flight. Every precaution has been taken to make this the safest flight ever. The computer is now in full control. Nothing can possibly go wrong, go wrong, go wrong, go wrong...".

It's astounding that neither the developers of these technologies – nor journalists – have learned anything from our experiences in automating flight. There are far fewer problems associated with creating “pilotless” aircraft yet this dream is still but a dream.

Interesting article. I'm not sure I buy into the car-sharing argument, though. There will probably a a market for it in places like Manhattan where there really is no place to park a car, but, by and large, I think most people will want to own their own car. A car may sit unused more than 90% of the time, but there are times of peak demand. During rush hours, or after sporting events, or on holiday weekends there will be high demand for cars and I can envision people waiting a long time before they would be able to get a shared vehicle.

And if a car is driven 12 to 16 hours a day instead of 1 or 2 hours a day it will wear out much faster. I think this will tend to lessen the economic advantages of car sharing relative to individual ownership.

"Giving automobiles auto-piloting features—up to and including completely hands-free, eyes-closed operation with trusting souls aboard—is the Space Race of global auto makers, and you are the monkey in the capsule. "

That's the problem, isn't it? YOU are the monkey. What happened to the first monkeys shot into space?

Wikipedia says:

"The first ever monkey astronaut was Albert, a rhesus monkey, who on June 11, 1948 rode to over 63 km (39 mi) on a V2 rocket. Albert died of suffocation during the flight.

Albert was followed by Albert II who survived the V2 flight but died on impact on June 14, 1949 after a parachute failure."

On and on. Eventually, yes - we got it right and now private companies are sending shuttles into space on a regular basis. But that took around 40 years and it was rocket science.

I have no doubt we will have full functioning autonomously driving vehicles with a 99.99% success rate. But until we iron out all the bugs, I don't want a car to ever have to make a decision whether or not it veer off a cliff - killing you and your entire family - because the algorithm prevents it from hurting another vehicle.

Let's see now, ... driverless cars will be programmed to avoid accidents, right? So all people in older cars need to do is to swerve close to them and the driverless car will veer off into the ditch. Sounds like a fun game for bored teens and gang-bangers. And what happens when those "driverless" features fail (and they will, from time to time)? People will be so accustomed to sitting back unaware, that there will be many deaths due to these failures. Who is responsible? Google? Ford, GM, and Chrysler? Or the federal government who funded this B.S. and pushed it into production? Oh, no matter, under Obama(don't)care, we will all be dying off early to save money for those who choose to not work and get free stuff on our dollars.

you can reconfigure the software to mae car hit the deer. Put some spikes on the front of your car along with reinforcement and shock asorbers, and you have a driverless deer hunting device . a couple of drives thru a wooded rural area at high speed, should provide you with all the meat you need for a year. yeah, i am a deer hater nd proud of it.

It is NOT an autonomous vehicle. It has potential just like all the others.

See GM's future in China includes EN-V 2.0; new renderings show light evolution on green.autoblog.com:

"....The new EN-V could potentially be available in 2020, but pilot studies will take place in China before then.......EN-V stands for Electric Networked-Vehicle, so it's no surprise that the EN-V 2.0 concept includes mobile Internet access and telematics along with an EV powertrain. Kevin Wale, president and managing director of the GM China Group, said in a statement that the EN-V 2.0's 'combination of sensing technology, wireless communication and GPS-based navigation establishes a technology foundation, pieces of which could potentially lead the way to the creation of future advanced vehicle systems.' More work needs to be done, as the vehicle has shown somewhat flawed autonomous driving capabilities thus far."

I have been hearing that but it is puzzling. If these cars are now driving autonomously, what problems remain to be solved? Why should commercial implementation be a decade or more away? I am guessing that there are limitations (weather, route, required periodic human intervention) that are not telling us about.

What does an autonomous car have to do with turning decision making over to the government? If this technology pans out, and I believe it will, you will be able to tell your car where you want to go anytime you want to go, without the government getting its mitts on things, as they did with the airlines and the railroads.

I guess this will eventually eliminate the need for cab drivers, so now you'll be able to discuss politics with a computer. Coming to a dysfunctional future near you: the Obamacab. Guaranteed to put you to sleep within 30 seconds.

It probably won't totally replace ownership, but I can see huge benefits. People claim they need an SUV for towing but only tow twice a year (if that). commute in your sports/economy car and borrow the big vehicle only when you really need it.

And to follow-up on that...Harry Reid has done precisely the same thing in the Senate. So much for being ready, willing and able to NEGOTIATE with the peoples' representatives! BTW, word has it that Mr. Reid is first on the waiting list for spanking new Mercedes driverless car with a manual transmission!! (between his legs, that is).

We shouldn't have to negotiate on a law that was passed over three years ago. The negotiation occurred before the bill was passed and there were plenty of compromises - no public option, states allowed to run their own systems, etc. If you want to negotiate, how about we start here: Top income tax bracket rolled back to pre-Reagan level and all income (dividends, capital gains and inheritance) treated equally. Still want to negotiate?

This copy is for your personal, non-commercial use only. Distribution and use of this material are governed by our Subscriber Agreement and by copyright law. For non-personal use or to order multiple copies, please contact Dow Jones Reprints at 1-800-843-0008 or visit www.djreprints.com.