Never Say Never, Part Two

Whether we have a “never” return type or not, we need to be able to determine when the end point of a method is unreachable for error reporting in methods that have non-void return type. The compiler is pretty clever about working that out; it can handle situations like

The compiler knows that N either throws or it doesn’t, and that if it doesn’t, then the try never exits, and if it does, then either the construction throws, or the construction succeeds and the catch throws. No matter what, the end point of M is never reached.

However, the compiler is not infinitely clever. It is easy to fool it:

Here x is always going to be a small, non-negative integer, and a small, non-negative integer is always greater than or equal to half of it. You know that, I know that, but the compiler doesn’t know that. The compiler complains that this method has a reachable end point, even though clearly the end point will never be reached. We could put that logic into the compiler if we really wanted to; we could be more clever.

How much more clever could we be? Is there any limit to our cleverness? Could we in theory produce a compiler that perfectly determined whether the end point of a method was reachable?

Turns out, no. A proof of that fact is a bit tricky, but we’re tough, we can do it.

First off, let’s restrict the problem to a particular pattern. Consider programs of this form:

The question is “given values for ‘some data’ and ‘some code’, does DoIt have a reachable end point?” If we can’t solve the problem for programs with this very restrictive format then we can’t solve it in general, so let’s explore whether we can solve problems in this limited format.

Let’s suppose that we have a class in our compiler library with a public method that answers that question. We assume that “code” is a legal body of a C# program and “data” is the legal contents of a string literal.

public class Compiler{ public static bool HasReachableEndPoint(string code, string data) { // [Omitted: work out the result and return it] }}

What is “result”? Suppose it is true. Then that means that DoIt has a reachable end point. Which means that when we run DoIt, the call in the conditional statement returns false. But “data” and “weird” are the same string, so why is “result” true if the result of the same call is going to be false? That’s logically inconsistent, so result cannot be true. Clearly I cannot drink from the cup closest to me.

Suppose “result” is false. That means that DoIt does not have a reachable end point. Which means that Compiler.HasReachableEndPoint(data, data) always either returns true, or throws an exception, or runs forever. But again, “data” and “weird” are the same string, so why would it return true, throw, or run forever here, when by assumption it returns false normally when given that input? Clearly result is not “false” either, since that leads to a contradiction. Clearly I cannot drink from the cup closest to you.

Since result can logically be neither true nor false, HasReachableEndPoint must either throw or go into an infinite loop when given these inputs. But if it does that then there are some inputs for our reachability tester which cause the compiler to fail or run forever, which seems bad. The compiler needs to be able to compile all legal programs and error on all illegal programs; ideally we don’t want to crash, run forever, or give a wrong answer for any possible input.

What we’ve shown here is, first, that you should never go up against a Sicilian when death is on the line, and second, that an endpoint reachability tester logically must either (1) sometimes give the wrong answer, or (2) sometimes fail to give an answer. The reachable endpoint problem is in general not reliably solvable by compilers no matter how clever the compiler writers are.

Now, you might reasonably push back on this proof, noting that the proof relies upon an absolutely crazy situation, namely, a program that contains part of its own code as data that itself calls the reachability analyzer in what Douglas Hofstadter calls a “strange loop”. Even if you don’t like the proof though, we have other evidence that a “perfect” reachable end point detector that always returns a correct result in finite time is probably impossible. Consider for example this totally trivial little method, with just five loops and some calculations on an arbitrary-precision integer type:

Want to know if Fermat’s Last Theorem is true? Just run your magical perfect C# compiler on a class containing that method and see if it compiles. If it fails with a “reachable end point in non-void-returning method” error then the goto must have been reachable, which means that there is a non-trivial solution to the equation, and Fermat’s Last Theorem is false. If compilation succeeds (with a warning that there’s an unreachable label!) then the goto was unreachable and the theorem is true. To answer the question you don’t even need to run the program, you just have to run the compiler! The fact that this program is insanely inefficient in the amount of recalculation it performs is irrelevant, since we’re not going to even run it.

Such a compiler would enable us to answer any open question in finite mathematics simply by writing a program that terminates if the hypothesis is true and runs forever if it is not. It seems inconceivable that we could even in theory construct a compiler that had the ability to answer all unsolved problems in the entire history of finite mathematics.

This famous problem is called the “Halting Problem” because it is usually posed for computational systems that do not have “throw an exception” as an option. The only alternatives are “halt with a result”, or “go into an infinite loop”.

The Halting Problem is of course solvable by heuristics provided that you can tolerate false positives. The C# compiler will cheerfully tell you if the end point of a method is absolutely guaranteed to be unreachable, but as we’ve seen, you can fool it into telling you that some unreachable end points are reachable. As long as it does not have false negatives (that is, sometimes tells you that the reachable end point of a method is unreachable) everything is fine.

This doesn't seem right. Surely the contents of DoIt should be valid code, but it's attempting to "return 123" from a function declared to return bool. If you change "return 123" to a throw your argument seems to make more sense.

"But "data" and "weird" are the same string, so why is "result" true if the result of the same call is going to be false?"

"But again, "data" and "weird" are the same string, so why would it return true, throw, or run forever here, when by assumption it returns false normally when given that input?"

I don't understand why the result is expected to be false in these cases?

The proof is one by cases leading to contradiction. Assume that "result" is true; OK, then logically, that implies that it must have been false. Therefore the assumption is invalid; result could not have been true. Now assume that "result" is false. OK, then logically, that implies that either there was no result, or, if there was a result, then it was true. Therefore the second assumption is also invalid. Result could not have been false. Since it could not have been either true or false, what could it have been? The only conclusion is that the whole situation is impossible; there is no perfect reachability detector. Does that make sense? – Eric

"Such a compiler would enable us to answer any open question in finite mathematics simply by writing a program that terminates if the hypothesis is true and runs forever if it is not."

Usually it's the other way around–the program terminates if the hypothesis is *false* (because it found a counterexample and exited, like your FLT method) or diverges if the hypothesis is true. You can always invert the sense of the hypothesis, but being precise here is important when you're talking about [co-][semi-]decidability.

"Such a compiler would enable us to answer any open question in finite [= discrete, I guess] mathematics" – not quite. Precisely, it would enable you to solve any question classified in Computability Theory as "solvable with an oracle to the halting problem" –

"Finite mathematics" is another name for what is usually called "discrete mathematics"; it was called "finite" when I was taught it, and old habits die hard. And yes, you are absolutely right about the set of solvable problems, though of course the definition of that set is somewhat circular when stating it informally. (The set of problems that could be solved by such a device is… the set of problems that could be solved by such a device!) I decided to not go into that level of detail in this brief sketch. What I'm getting at here is that a whole host of unsolved (or until-recently-unsolved) problems believed to be extremely difficult would be solved by such a magical device; Goldbach's Conjecture, Fermat's Last Theorem, and so on. That's evidence that such a device is at the very least far beyond our present capabilities, and probably impossible. — Eric

There is a vast literature about questions that fall into this class and others that don't. See the Wikipedia article on the Arithmetical Hierarchy, or better yet, a good textbook like Floyd & Beigel's "The Language of Machines".

Clearly, this class of questions is tantalizing enough to justify the "inconceivability" of an algorithm to solve it; but given that any PC can now perform tasks that 50 years ago were "inconceivable", I trust the proof by contradiction better.

>> But if it does that then there are some inputs for our reachability tester which cause the compiler to fail or run forever, which seems bad. The compiler needs to be able to compile all legal programs, even crazy ones.

I think there is a logical inconsistency here. The legality of the program depends on whether it has unreachable code in it, so we cannot claim that program is "crazy but legal" until we have solved the reachability problem. Clearly, in the most general case, the answer may be not just "yes" and "no" (as it is now), but also "undecidable". In which case we can just say that only programs for which the answer is "yes" are legal.

Though I take your point, there are some subtleties here.

First off, as a general statement we would like the compiler to always terminate normally, either with success or with failure, but the compiler should not go into an infinite loop or crash, ever.

Second, the point of the reachability detector is to take programs which are syntactically legal and determine if they are semantically legal. It's acceptable for the reachability analyzer to consider only syntactically legal programs, but it is not acceptable to say that the reachability analyzer is allowed to produce bad results or no results if the program is "illegal". It is the illegality of the program that the analyzer is attempting to determine; like the compiler as a whole, it should terminate with success or failure for it to be reliable.

Third, adding another category called "undecidable" doesn't help. By a similar argument you cannot produce a logically consistent, always-terminating method "CategorizeReachability" which categorizes method bodies into "reachable end point", "unreachable end point" and "undecidable" either. The existence of such a method would then allow for the creation of yet another "strange loop" input string which refered to "CategorizeReachability" in such a way that it could be shown to either be logically inconsistent or not terminating normally itself.

I recently saw a question on StackOverflow that was something like "Sometimes the user initiates an operation that never completes; how do I terminate the operation" and one of the answers was essentially "Rather than attempting to terminate the operation, just put it in a condition like this:

if (/* operation would complete for this input */) {

// perform operation

}"

Apparently the person who wrote that answer failed to understand that the "operation would complete" function is decidedly non-trivial. Although in this case, it wasn't quite the traditional halting problem, but still it was an answer that you could only determine by actually trying the operation.

Pavel, I'm not sure I see the contradiction. "Crazy" is a subjective term, and "legal" isn't actually strictly defined in this example, right? Just because a compiler can't determine if code is dead code doesn't make the program non-legal.

Fermat's Last Theorem talks about positive integers – a set having infinite number of numbers -, however on computers I think one can have types with finite value range. Would it really help to answer Fermat's Last Theorem question to know that this code never reaches the label ? Since it would mean only that the theorem is true only for numbers from 0 to Integer.MaxValue ( assuming that You represent number in a format that Integer.Maxvalue+1 = 0).

I think we can check if the function reaches the label or not in finite (but quite long) time since we just have to check all possible combinations knowing that the value range of the variables is finite.

Correct, I did not mention Alan Turing, who first proved that the Halting Problem was not solvable by any conventional means of implementing algorithms. Neither did I mention David Hilbert, who first proposed a version of the Halting Problem (though in his time it was called the Entscheidungsproblem which means "Decision Problem"; I also did not mention Martin Davis, who actually coined the term "Halting Problem"). Neither did I mention Kurt Gödel, whose earlier work on undecidability of formal proof systems is clearly related. Neither did I mention Georg Cantor, whose technique of "diagonal argument" is used in the proof. I did however make passing mention of Douglas Hofstadter, whose book Gödel Escher Bach has a delightful discussion of the Halting Problem, amongst many other things.

I'm not writing a history of the theory of decidability here, I'm writing a little blog post about the interesting things you run into when writing a C# compiler. See Wikipedia if you want a summary of the history of the problem. – Eric