@Junko, Thank you for that response. I am surprised that they did not employ ECC given the spectacularly noisy electrical environment that is present in a typical automobile. Ignition noise itself has always been a problem in cars, but even current diesel engines, with their Direct Injection systems, are electrically noisy beasts.

Is anyone using hardware controlled access to device space or memory? This is fairly common in cellular handsets, both for security and runtime stablility. It also makes errors readily observable since out of bounds accesses drive immediate hardware faults instead of leaky data errors that may or may not be observed in testing.

@Wobbly, according to the expert witness, "Toyota claimed the 2005 Camry's main CPU had error detecting and correcting (EDAC) RAM. It didn't."
As you accurately pointed out, the expert witness also agrees that EDAC, or at least parity RAM, is relatively easy and low-cost insurance for safety-critical systems.

You immediately addresses the topic of threads in safety critical products, but you did not address the two points that I raised, and those were ECC on memory and hardware memory region protection on client devices.

In thirty years of delivering core network equipment in telecom, including sixteen years within a Network Systems associated division of Bell Labs, I have had to deal with high reliability requirements. Not safety critical systems such as aerospace or medical, but still equipment that was intended to run unattended in locked vaults buried under ground in very remote locations, and perform its own diagnostics and fault reporting and mitigation. So I am not completety out of touch on those issues.

Even in single threaded systems with well defined task definitions, you can gain stability and safety through having well defined hardware access control limiting tasks to only those devices and memory regions that are associated with that particular task.

Having shipped systems that were expected to operate non-stop with five-nines of uptime in deployment, I have had the opportunity to observe things, such as the fact that any significant amount of RAM is going to show correctable single bit errors through a year of continuous operation, so bit flips do happen. Around mid 2005, we had appoximately six hundred router blades in a single distributed network, each blade had 1GB of DDR2 RAM, and over a year of collecting fault data, each blade expeienced about six or seven correctable ECC events. We were able to swap out two or three blades before they failed by having ECC event thresholds that flagged the cards for replacement.

Now admittedly, these where blades with 1GB of memory, running 24x7. But if you count the total installed RAM in all the cars on the highway, times the total run hours, there have to be distributed single bit error events occuring.

That's what I was referring to, Junko. That testimony was misleading. The "brake override" he was referring to was only the feature where applying the brakes simultaneously cuts the throttle. The implication was that the brakes didn't work at all, which isn't the case. And the throttle override feature does work, except in cases where task X dies while the driver is braking. So it's not as bad as I thought.

Specifically, this quote here:

"Q Where is the function for that brake override? Where is the task located, as you understand it?

"A Yes. So the brake override that is supposed to save the day when there is an unintended acceleration is in task X, of course, because it is the kitchen sink."

Don't you get the impression from this that the brake override won't work when task X dies? And is it made clear that the brakes do work, even if the throttle isn't cut in worst-case scenarios? The brakes would STILL "save the day," if the driver can overcome his or her moment of astonishment.

My approach would have been to make the whole situation clearer from the start, especially in view of the fact that the attorney doing the questioning did not seem well versed in these matters.

@Antoney Anderson, you have been absolutely critical in our Toyota discussions on this EE Times Forum. Thank you so much for chiming in often, offering pointed guidances and bringing clarity to the issues.

Now, if somehow that task X death had affected ABS in such a way that the brakes didn't work, the situation would have been a whole lot more dire. In the early reports, this "small" detail was never brought out.

@JIMAshby, what the root cause is for a single bit flip is apparently hard to find.

As the expert witness Michael Barr noted, among dozens of tasks, there are16 million different ways those tasks can die. The experts group was able to demonstrate at least one way for the software to cause unintended acceleration, but there are so many other ways that could have happened.

You may not conisder it as a conclusive evidence. But in a trial like this, it raised enough reasonsable doubt to convince a jury to deliver a verdict against Toyota.

You are absolutely right about this, hence, the expert witness was talking that corruptions could happen "on certain road conditions on certain days." That makes it imperative to have a built-in selft test of the hardwrae by software, as you point out.

Sure, it's better to have the engine throttled back in these emergemcies. But it's also true that the brakes can overpower the engine, even at full throttle.

I too was somewhat relieved to discover that the brakes did work throughout these instances of task X death (on the third article or so of the series - it was definitely not clear before that). The power from the engine is just not that huge of a concern, if you plant your foot firmly on the brakes, because, as the Audi tests showed in the mid 1980s, the stopping distances do not change by much, power on or power off. That means, there is not a big difference in the amount of energy the brakes need to dissipate as heat. It's more important to catch the problem before the car really speeds up.

It is true that the vacuum assist will go away if the engine is at full throttle, but that would only occur if task X died while brakes were being applied. Otherwise, if task X died while the brakes were NOT being applied, the throttle would shut down, and you'd have vacuum assist. And here's the really interesting part, even in the worst-case scenario (task X death while brakes are being applied), if the driver HAD pumped the brakes, as the Toyota is programmed, she would NOT have lost power assist! Because apparently, you need to release the brakes for a couple of tenths of a second and then reapply, in order for the throttle to be shut down, even in this worst case.

Now, if somehow that task X death had affected ABS in such a way that the brakes didn't work, the situation would have been a whole lot more dire. In the early reports, this "small" detail was never brought out.

From a functional safety point of view, the most effective way of stopping a runaway vehicle is first to remove the source of energy causing the acceleration and then to apply the brakes. It is inappropriate in my view to treat the driver exercising the brakes as the fail-safe for an engine that is out of control.This presumably is why Toyota now fit brake override software.

here are some of the factors that make it inadvisable to rely on the brakes as a fail-safe:

Brakes only have a limited capacity for absorbing heat. If the temperature of the brake cylinders rise too far the hydraulic fluid will boil and cause vapour locks which greatly reduce braking efficiency. The temperature at which the hydraulic fluid boils is dependent on the moisture content of the hydraulic fluid and drops as this rises. Hydraulic fluid readily absorbs moisture- hence the importance of changing it on a regular basis.

With a racing engine, there is no vacuum produced and hence if you pump the brakes you will rapidly lose vacuum brake assist

with a racing engine there may well be sufficient slip in the torque converter to give somewhere between a 2 and 2.5 times torque multiplication factor, which means that you have to press twice to two and a half times as hard to get the necessary braking force at the wheels.

I for one think that the three part series based on the trial transcript has provided an extremely useful and helpful insight into the evidence presented by Dr Barr to the jury. The the resultant discussion has been wide ranging, constructive and fruitful. I certainly have learnt a great deal. Many thanks Junko!