Toyota Case: Inside Camry’s Electronic Control Module

MADISON, Wis. -- As we continue to explore the Oklahoma court judgment against Toyota for unintended acceleration, EE Times readers have raised many astute engineering questions, ranging from the probabilities of bit-flip occurrence and safety standards applied to software and hardware to the safety system architecture built into Toyota cars.

Meanwhile, in a number of Toyota cases (including the Oklahoma one), one nagging question recurs among consumers: If there are software bugs in the system, why have millions of Toyota owners like ourselves never experienced unintended acceleration?

It turns out that Jean Bookout, the plaintiff in Bookout v. Toyota case, had driven her Toyota for several years and put 9,000 miles on the odometer without a problem -- until the crash that injured her and killed a passenger.

We would like to take you through how Michael Barr, CTO and co-founder of the Barr Group and an expert witness who testified in the case, concluded that a random hardware flaw -- combined with a software bug that's latent and lurking -- "can get through or knock down the fail-safes that are in place" under certain driving conditions on certain days.

Excerpt from the court transcript
EE Times is publishing a portion of the court transcript relevant to the Toyota Camry's electronic control module ECM. The following Q&A was carried out between Barr and Benjamin E. Baker Jr., an attorney representing the plaintiffs. This excerpt begins with Barr on the witness stand describing the ECM, which consists of two CPUs: a V850 supplied by NEC (which later became Renesas) and an ESP-B2 supplied by Denso acting as a second CPU (sometimes referred as a monitor CPU).

A. So this is a photograph of the ECM. And this ECM, or engine control modules, has two big chips on it. Has a bunch of other chips, capacitors, circuit tracers that you can see, and other things. This biggest one, the square one, is the main CPU. It is a type of a CPU or a model of CPU called a V850. That is kind of the equivalent of calling it a Pentium. V850 is the model number of that processor. Comes from a company, a supplier of Toyota that used to be called NEC. It has since changed its name.

Then there is a second rectangular chip here, and that chip is what has been referred to by various witnesses as the monitor CPU, the ESP-B2, and sometimes the sub-CPU.

Importantly, each of those is a processor with its own software. Then, of course, all together they comprise an embedded system.

Q. So the software that we're going to talk about is stored within components on this board?

A. Almost always when I'm talking about the software, I'm talking about the software on this main CPU, which performs the throttle control, the combustion, monitors the accelerator, and all those things, cruise control. But there is also software, and I will specifically call out when I'm talking about this monitor CPU and its software.

Q. This is from a 2008 Camry?

A. This particular photo is from 2008 Camry.

Q. Is the 2005 generally very similar to this?

A. The chips would be moved around a little bit, but in terms of the electronics of what is there, there is a V850 processor, there is an ESP-B2. From a substantial similarity point of view, they are very similar.

NASA physicist Henning Leidecker has now voiced his concerns about the increased risk of unintended acceleration in '02-'06 Camrys due to the ongoing growth of "tin whiskers," comparing the risk to a game of Russian roulette. Toyota redesigned the pedal sensors in '07 and '08 for the express purpose of eliminating the tin whisker risk, and Dr. Leidecker questions why Toyota would have done this if tin whiskers aren't a problem. Instead of Toyota, the Department of Justice, and NHTSA trying to keep the electronics issue quiet, a recall should be issued to update the sensors in the models Dr. Leidecker is concerned about.

Sorry, Steam Kid, but my description of what the ECU is doing was meant to show that it's taking into account all manner of variables and acting on them constantly. That's what drivers do when they drive a car, but they often make huge mistakes, are slow at reacting, or freeze at the wrong time.

No question that this will take some new communications schemes, between cars and between the roadway and cars. And sensors on cars too.

Driving is just not as phenominal of a chore as some seem to think.

"I recently had a car that during the night would decide to open the windows, initially part way then fully even when it had been raining! and I was told that there was no problem, but got a re-call a year later for a software upgrade."

There you go. A software upgrade fixed the problem. Instead, if you get hit by some moron who was too busy texting, a software upgrade won't prevent that from happening again.

1. Autonomous driving has to rely on V2V as well as V2I communications. The intentions of the dump truck, as well as its mechanical health, are part of V2V. Much more reliable than something a human driver "thinks" he notices.

2. Construction zones and lane merging are part of V2I. Way more reliable than what some "old lady" in front of you may or may not decide to do.

3. Optical sensors on cars, another necessary ingredient, can detect odd motions in the vehicle in front a lot better than some idiot who is busy texting on his cell phone.

4. Obviously, drunk drivers are part of the reason you want self driving cars. Initially, in the driver assist-only mode, where a mix of human and autonomously driven vehicles share the same roads, the solution won't be as good. Mostly, in this intermediate phase, you'll have V2V comms warning that the other guy is doing unpredictable things.

5. Vision algotirthms can *easily* tell apart a floating plastic bag from a brick. And if a floating platic bag gets stuck on your windshield, it wouldn't matter.

And so forth. You simply haven't appreciated what sensors and what comms are needed for autonomous driving. For most repetitive work, and driving is a prime example, properly programmed computers can do a far better job than the majority of humans. Even playing chess, for instance. For purely creative work, perhaps that's different.

You write: "...it should be taught more widely how to use gear shift to neutral in emergency".

Yes agreed, but this has to be in the context of where shifting to neutral would be appropriate. Those preparing training material and instructions to be put in owners manuals would have to qualify the kinds of emergencies in which shifting to neutral would be appropriate . For example, hypothetically:

"in the unlikely event of a sudden uncommanded acceleration, put the vehicle into neutral, then brake etc."

This in turn would mean that automobile manufacturers would have to admit in public the possibility that some UAs are NOT caused by driver error.

In other words, they would have to admit what they, with NHTSA's tacit support, have consistently denied for three decades: namely, that intermittent electronic and software malfunctions in electronic throttles may be causing sudden accelerations.

Brake shift interlock is an electro-mechanical interlock that prevents the driver from moving the gear stick from PARK unless they have their foot on the brake. It comes in various different flavours, see Shift Interlock system.

When it comes to getting the car into neutral from DRIVE or REVERSE with the vehicle moving at speed this is an entirely different matter, not connected in any way with the brake shift interlock. You can certainly put the gear stick into NEUTRAL, but whether this actually puts the transmission into neutral will depend on the transmission control strategy implemented by the manufacturer - remember that the automatic transmission is also now largely under electronic control. As to what the situation is in Toyota cars, I do not know. Perhaps someone else can answer this question.

@Bert22306 @grg9999. Sorry Bert, you seem to think I was commenting on the ECU complexity whereas I was replying on your comment that driving is easy. The post by grg9999 is similar to what I was thinking, in that anticipation can not be computed. Humans are prepared to accept a less optimum solution to avoid a potential poor or disastrous situation.

When automobiles were first built the driver was also the engine ECU with various levers to move to keep the engine running as well as driving, then mechanical systems took that over. As enviromental concerns arose then ECU's have come to the fore, but even then they are far from perfect, I recently had a car that during the night would decide to open the windows, initially part way then fully even when it had been raining! and I was told that there was no problem, but got a re-call a year later for a software upgrade.

I think many, including myself, still are not quite sure, the whole story regarding which gear shifts require brake action. As mentioned earlier, I fault inadequate driver instruction, as well as education by the car manufacturers themselves.

Otherwise, it should be taught more widely how to use gear shift to neutral in emergency. Because a current common teaching is to avoid switching gears without brake action. Drive to neutral is sometimes even a discouraged gear shift, there is so much confusion about this. A dangerous shame.

Reading between the lines Toyota found that in this instance they could not blame the incident on the absence of a guardian angel, or on the presence of loose all-weather flying floormats (which allegedly become active in the absence of a guardian angel), or sticky pedals, or loose beer cans, or backseat drivers. They could not blame it on the age of the driver, nor on their youth, nor on the width of their feet, They could not blame it on driver pedal error because there were too many witnesses of the driver braking and the vehicle flying through the air. They could not blame it on a malfunctioning electronic throttle control system because that would damage the public perception of Toyota's perfection in all things electronic.

Replacing the car and presumably getting the driver to sign a non-disclosure agreement would cost less for Toyota than having driver and passengers telling friends and acquaintances that the car "ran away and tried to kill the whole ruddy lot of us." thereby rubbishing the pedal error hypothesis.

In commercial aircraft the pilots are given about 120 hours per month of non-flying time some of which they use for practicing fault conditions in a simulator - I wonder if driving simulators are needed due to the increasing complexity of cars and the addition of steer by wire and brake by wire, and throttle by wire -- The issue is that these change the way the car can react under some fault conditions and result in issues.