Agreed, Larry. The idea that a human in control, behind the wheel, is inherently "safer" than having an algorithm do the driving is really odd. Not saying that truly autonomous cars are realistically possible right this minute, but surely, in principle, such automation can easily beat human intervention, when it comes to consistency, reliability, and consequent safety.

Perhaps the problem is that people think they are more in control of their own driving safety than they really are. Sure, I too think I'm the most expert driver on the road. My problem is all those other unpredictable half-wits around me.

Even in the simplest case, i.e. trains running on determinstic tracks, how many times have we seen recently in the news that the operator dozed off? This is safe?

There are risks in self-driving cars, but as another post in this area says humans are bad at assessing risk. They are also not that great at driving cars. A self-driving car fatality would make headlines, but the numerous daily fatalities from drivers on cell phones, falling asleep at the wheel, or otherwise just not paying attention rarely make it above the fold (obscure reference for those who still remember newspapers). This case illustrates a net lowering of the safety in the activity of driving a car, where there is real evidence that self-driving cars would raise the safety level.

"There are two ways of constructing a software design.One way is to make it so simple that there are obviously no deficiencies.And the other way is to make it so complicated that there are no obvious deficiencies."-- C. A. R. Hoare, 1980 Turing award lecture

While you may be able to skew the predictions and rationalize not doing a good job, you can't fool the field results. It's unfortunate that this had to take lawsuits to get in front of this, but given the analysis by the plaintifs experts, it's abundantly clear that they not only screwed up, but didn't even have the first clue on how to do real-time, let alone saftey-critical software.

Moreover, really Toyota has three HUGE problems they have to fix. In addition to the SW problem, they also lacked a proper design validation program that should have alerted them to the design flaw early, and they weren't tracking their field failures and getting to root cause in a day. Very uncharacteristic for a Japanese company, especially Toyota.

Finally, humans are notoriously bad at assessing risk. Even NASA, the world leaders in safety-critical systems blew it with Challenger and Columbia. At least they try (and got in trouble when they cut corners trying.) With Toyota's ignorance of how to even DO real-time safety-critical software, is it any surprise that they totally whiffed the risk ANALYSIS?

To me the takeaway as a consumer is not to avoid the benefits of new auto technology, but to make sure there is a hardwired physically-interrupting-the-power OFF switch (which Toyota also missed).

NHTSA doesn't have the budget to hire a crack team of software experts, and congress isn't about to give them more teeth to enforce standards. OTOH the EU could take the lead and demand that all automotive software confirm to ISO 26262 and that it be certified by an independent testing body such as Germany's TUV SUD. That would be a big step forward from having the public be the beta testers and the court the enforcers.

As anyone who has done a MTBF calculation to MIL-HDBK-217 knows, you can make the numbers say what you want. Our current culture of "sqeeze every last dollar out" of a business means that safety reviews can easily be skewed by a "severity" or "probability" rating in a FMECA analysis. I have witnessed a similar conclusion within my own company when such a safety analysis (although not as severe as a car crash) was discussed and discounted as not enough to warrant a redesign. Tools will have to be worked on to verify designs better.

Yes, the self driven cars are too far from reality and safety is a major challenge. But the companies definitely want to invest in this to be in race and to make the business in the future. The real challenge lies in the hands of governments to make sure the automobiles are really safe and they are tested perfectly.