Why 'Toyota's Killer Firmware' May Not Have Been a Killer After All

As he delved deeper into the evidence, David Cummings says he did not find a convincing argument that Toyota's software was to blame for the accident.

As some readers may remember, in 2013 an Oklahoma jury found that Toyota's embedded software was to blame for unintended acceleration that resulted in a fatal accident. The embedded software expert who convinced the jury that software was to blame made his trial testimony and slides available to the public and invited us to "judge for ourselves." A flurry of articles in technical publications followed, most of which (to the best of my recollection) described the expert's testimony in favorable terms, accepting the conclusion that software caused the accident. Many of these articles had attention-grabbing titles such as Toyota's killer firmware and Toyota Case: Single Bit Flip That Killed.

Having developed embedded software for many years, and having written an opinion piece in the Los Angeles Times years before the trial about the possibility that Toyota's software was to blame for reported incidents of unintended acceleration, I read the expert's testimony with great interest and excitement. I was expecting to find a convincing argument based on the evidence that the software was indeed to blame for the accident in question.

As I delved deeper and deeper into the testimony, however, my excitement turned to disappointment. It became clear that there was no credible theory based on the evidence. I felt it was important to set the record straight, and so I submitted an article with my technical analysis to the IEEE Technology and Society Magazine. That article was peer reviewed and then accepted for publication in their most recent issue (December, 2016). (You can also find a pre-publication version of the article on my company's website, as well as two video interviews here and here providing additional context.)

In this column, I summarize some of my findings (please refer to the IEEE article for a more complete discussion that includes all the technical details).

As discussed in the IEEE article, the plaintiffs convinced the jury that Toyota's embedded software was responsible for the accident by employing the following approach:

First, they bombarded the non-technical jury with criticisms of the quality of Toyota's software from two different software experts. The first expert did not see any of Toyota's source code, but nonetheless his entire testimony, which is also publicly available (see Part 1 and Part 2), was directed toward criticizing the quality of Toyota's software. As anyone with extensive experience developing real-world software knows, software quality assessments can be highly subjective.

The second of the two experts, who did examine Toyota's source code, also criticized the quality of Toyota's software. Then he told the jury that "to a reasonable degree of engineering certainty, it was more likely than not" that the death of a task running on the engine control processor (referred to at trial as "Task X") was responsible for the accident, despite the fact that the evidence presented at trial did not support that conclusion.

Car & Driver found that at 70mph, the braking distance was increased slightly in full throttle versus closed throttle situations. It was only at highly elevated speeds (100mph), involving a lot of kenetic energy to be dissapated, that some of the brake systems tested struggled or could not stop the vehicle before overheating. Most "UA" events I have heard claims of are in low speed situations (i.e. parking) where brakes should be able to dominate the throttle no problem. http://www.caranddriver.com/features/how-to-deal-with-unintended-acceleration

OK, with the engine stopped and vacuum reservoir emptied by repeated brake applications after the engine stop, it's more difficult to stop the car. Congratulations, you found a theoretical corner case.

Now show me a single real-life case where this happened -- not a badly-researched press story, not a driver saying "I couldn't stop, my brakes were burned out", a verified post-crash forensic analysis showing that this really happened.

Have you ever wondered why all these cases only seem to occur in America?

"In other words, if you press the brake pedal -- not the accelerator -- hard, the car will stop. Every time. No exceptions, even with a stuck throttle. Unless you pressed the wrong pedal..."

This is simply not true. NHTSA has data showing that it takes >100 pounds of force with brake vacuum depletion, which is more than a little old lady is likely to muster in a crisis. For that matter the Saylor crash showed brakes burned out, which is inconsistent with your statement.

Consumer Reports has a video showing that an adult male has trouble stopping a car without pumping, and is unable to stop it after pumping brakes (which is a normal human reaction, especially for older drivers who trained before ABS). Their scenario is stuck pedal or floor mat entrapment, but there is no reason to believe that is any different than a software defect that commands wide open throttle.

Bookout pulled on the parking brake. The parking brake activates only the rear tires. To me, this is not evidence of a bug, but just indicates that the car was dragging the rear tires while she simultaneously had her foot on the gas and hand on the parking brake. This does not seem like a great mystery that can only be solved with cosmic rays and bit-flips. Bookout was also geriatric, which is not proof of anything, but old age is highly correlated with pedal misapplication, according to the NHSTA.

Regarding logic, the burden of proof is on the accuser. You are not guilty of robbing a bank because you cannot prove you didn't rob the bank. You are guilty because someone proved you did rob the bank.

In a case like this, it comes down to statistical probability. it's not a good sign when the judge and attorneys are asking "what are statistics" and the reply is "something with numbers". Having an expert witness who declares code "unsafe" for having 10,000 global variables shoots his credibility for me. Sloppy perhaps, not necessarily unsafe. Again the burden is on him to prove it is unsafe.

The push-button start-stop switch requires the driver to depress the button for 3 seconds continuously to turn the engine off when it is in gear. ...

That is the dumbest interface ever... It's releated to the reason that I didn't buy a Toyota pickup. This whole concept that you can start and run the car without even finding the key to put it in the ignition was a bit creepy (apparently the dealer has universal keys that work for every car on the lot)

Every motorcycle that I've owned has a big red "engine off" switch that cuts the power to the engine. Of course they also have a clutch which works just as well to cut power to the wheel.

The suggestion that you can't stop a Toyota -- or almost any car -- with the accelerator pedal mashed to the floor is bullsh*t. A normal car like this can't pull more than about 0.3g under acceleration, but can decelerate at about 0.9g -- and there's enough braking power to lock the wheels, so some to spare.

Even without allowing for this excess it can decelerate at at least 0.6g with a wide-open throttle, and maybe as much as 0.9g if the brakes have enough spare power (which most have). In other words the worst that can happen if the throttle sticks open is extended stopping distance, the best is no increase at all.

No modern car will suffer from brake fade with a single stop even from maximum speed, where the energy dissipated is far higher than any of these so-called "runaway" accidents. If it did, it would be banned from sale as being unsafe.

All this has been proved using real cars in real tests, not simulations -- the worst case is that stopping distance increase slightly, none ever showed inability to stop or brake failure/fading.

In other words, if you press the brake pedal -- not the accelerator -- hard, the car will stop. Every time. No exceptions, even with a stuck throttle. Unless you pressed the wrong pedal...

My point exactly. Even as the Toyota firmware turned out to be a complete BBM (Big Ball of Mud), the author of the article assures us that it certainly was "Not a Killer". He bases this verdict on his second-hand analysis of a redacted deposition of an expert witnesses.

To implicate the firmware, the author accepts nothing short of reproducing the killing of the passengers at will.

The problem with this standard of proof is that highly intermittent problems that show up only once per millions of hours of operation are by definition difficult to reproduce in the lab, or in tests of a fleet of just a few dozen cars.

This is easy: there are software safety standards that answer precisely that question. These days it is ISO 26262 for passenger vehicles.

Back in the late 1990s and through the 2000s it was MISRA Software Guidelines, which Toyota could have followed, but did not. Note that these are much more comprehensive than just MISRA C. It is an entire software safety process of many hundreds of pages in length that outlines a SIL approach and acceptable practices for safety.

If you could follow the standard but you don't, and your vehicle kills people, that tells me that your software was bad enough to be a problem.