Posted
by
samzenpus
on Monday April 08, 2013 @12:18PM
from the surgery-safety dept.

coondoggie writes "When it comes to having robotic surgeons slicing around inside your brain, heart or other important body organ, surgeons and patients need to know that a software or hardware glitch isn't going to ruin their day. That's why a new technique developed by researchers at Carnegie Mellon University and the Johns Hopkins University Applied Physics Laboratory that promises to reliably detect software bugs and verify the software safety of surgical robots could be a significant development."

I am not convinced there is a reliable system short of market forces for rooting out
* People who would take advantage of the system (whatever it is)
* People who would charge unreasonable rates (of course, insurance seriously hampers this)
* Underachievers

You need to completely disassemble something so every surface is visible in an autoclave. If you don't do that, you don't kill all the bacteria. Which means, you failed at your job -- with potentially lethal results. If you ever put something in that had a battery compartment to begin with, that tells me before you even close the door that you should be ejected out of the fourth floor window in a firey blob as a lesson to other technicians to not mess around with safety procedures.

Reliably finding all "bugs" is not possible without an intimate knowledge of exactly what the software is trying to accomplish. You can find certain kinds of errors algorithmically (what amount to "compile time" errors), and even prevent many kinds of run-time errors.

But there is simply no way to prevent the program from doing something unintentional (like cutting the wrong thing) without prior detailed knowledge of the actual intent.

But there is simply no way to prevent the program from doing something unintentional (like cutting the wrong thing) without prior detailed knowledge of the actual intent.

Back in the '80's David Parnas argued that software verification was fundamentally different from hardware verification precisely because software has very nearly infinite bandwidth, which is precisely the opposite of the argument being made in the article.

That is, for hardware, a small change in some parameter will in almost all practical cases result in behaviour that is not wildly different from the behaviour with the original value. This means that we can test it under a finite set of conditions and smoothly extrapolate to the rest.

With software, a single bit flipped can result in behaviour that is arbitrarily different from the original behaviour. As such, nothing short of exhaustive testing over all possible accessible states can prove software correct.

Algorithmic testing of the kind described here will catch some bugs, but provably correct algorithms can still run into practical implementation details that will result in arbitrarily large deviations from ideal behaviour. For example: the inertial navigation module on the Ariane V could have had provably correct code (for all I know it actually did) and the rocket still would have destroyed itself.

Most problems with embedded systems are of the kind, "The hardware did something unexpected and the software responded by doing something inappropriate." Anticipating the things the (incorrectly assembled, failure-prone, unexpectedly capable) hardware might do and figuring out what the appropriate response is constitutes the greater part of building robust embedded code, and this kind of verificationist approach, while useful as a starting point, won't address those issues at all.

I think referring to it as "useful as a starting point" very much under values this approach. None of that other stuff you referenced matters, at all, if the algorithm is guaranteed to screw up under certain circumstances. The example they gave in the article was of a provably screwed-up algorithm.

However, I'm really commenting along the lines of "tone" or "emphasis". It is true that once you have a known-good algorithm, there's still other stuff that can go wrong that you need to pay attention to in how

I'm pretty sure that the number of possible states in a moderately sized software system can be regarded as unfathomably too large, which isn't "infinite" but as close to "nearly infinite" as you can get... What's possibly saving these guys is that they are probably validating that the robotic "scalpel" is following all possible inputs and positions of the input device in the usual tree cosy and well behaved dimensions... OTH I didn't bother to read the paper, so what do I know...

Back in the '80's David Parnas argued that software verification was fundamentally different from hardware verification precisely because software has very nearly infinite bandwidth, which is precisely the opposite of the argument being made in the article.

That is, for hardware, a small change in some parameter will in almost all practical cases result in behaviour that is not wildly different from the behaviour with the original value. This means that we can test it under a finite set of conditions and smoothly extrapolate to the rest.

With software, a single bit flipped can result in behaviour that is arbitrarily different from the original behaviour. As such, nothing short of exhaustive testing over all possible accessible states can prove software correct.

More accurately, this is a distinction between stateless and stateful systems. Stateless machines always perform the same operations. They makes them much easier to test than stateful systems whose behavior depends on memory of prior events.

Software is always stateful although it can approach stateless behavior. Analog electronics and mechanical systems are generally stateless. Digital systems can be built either way but registers and memories are pretty fundamental so almost all modern digital chips

"Software is always stateful although it can approach stateless behavior. Analog electronics and mechanical systems are generally stateless. Digital systems can be built either way but registers and memories are pretty fundamental so almost all modern digital chips are stateful even if they don't embed cpu's and software."

This is all true, but the very nature of the kinds of applications being discussed demand that they be stateful.

The halting problem says that you can't determine if a program halts or not in the general case. You definitely can check that for a chosen restricted set of programs (usually relatively small ones, but the core control system of a robot or the avionics control system for an airplane are small enough). See model checking [wikipedia.org] for how this is done in general. The article actually talks about a different technique which is to develop of a methodology for formally proving that class of programs correct, which i

That's not insightful, it's clueless about the subject.The halting problem is very rarely a problem in program verification. It's possible to write a program for which halting is undecidable. That's a bug in the program.

The Microsoft Static Driver Verifier, in practice, runs into problems it can't decide about 5% of the time. The goal is to determine that the driver is memory-safe to be in the kernel and calls all the driver APIs with valid parameters. If it does that, the driver can't crash the rest of t

In the general case it is true that you can not tell if an program will halt. However most programs do halt and most programmers understand why. Theorem provers present a language to their users that does not admit the general case in which halting is guaranteed which is part of a more important guarantee that all programs can be reduced to a normal form.

Which means that the halting problem is solved for a program written in a language like coq.

.The reason why I wrote my prior comment was that I read the reason the GNU-Linux people do not want to adopt BSD's strlcpy function is that the strlcpy encourages sloppy coding. The rationale given for that assertion is that strlcpy checks for the destination buffer being too small and zero-terminates regardless (unlike strncpy, which returns a non-zero-terminated buffer if the destination buffer is too small).

The proof verifier won't verify sloppy software without a proof of correctness. It's much harder to write a formal proof of correctness for sloppy code, so anything that can pass the verification stage will most likely be well written, simple, and very human readable since a human has to read and understand it sufficiently to write the formal proof of correctness for it. Writing the formal proofs will also probably be the largest part of the workload in the software development process so it makes no sens

The whole article is written with the notion of a total replacement debugging/designing solution for all software.

It is really dealing with mechanical and human motion behaviours like inertial overshoot, path avoidances and realistic feedback to the operator rather than things like buffer overruns, atomic accesses and race conditions.

The problem is to construct a control system such that the robot scalpel (controlled by a person by a force-feedback device) will not exit some pre-defined boundaries. E.g. boundaries that prevent the scalpel cutting a nerve. The preivous system was flawed as it didn't take into account delays in the reponse of the scalpel. By creating a more accurate model, and using methods that are able to prove things about the dynamic system, they prove that usin