Not enough programs!

Now we come to the final knockout blow for computation.

How many programs are there?

Any program is simply a sequence of bits so the set of programs is enumerable.

All I have to do is write a for loop that counts from 1 to 2^N and I have all the programs that can be stored in N bits. Godel came up with this idea and numbering programs in this way is usually called Godel numbering.

Even if you let this process go to infinity the number of programs is enumerable - but the number of irrational numbers isn't enumerable which means that there are more irrational numbers than there are programs.

So what?

What a moment think about the task of computing each possible number.

That is imagine each number has a program that computes it and displays it say. You can do it for the integers and you can do it for the rationals but you can't do it for the irrationals.

Numbers like the square root of 2, pi and e clearly do have programs that compute them so not all irrational numbers are non-computable - but the vast majority are non-computable.

I'll say that again - most of the irrational numbers do not have programs that compute them.

What does this mean?

Well if you think about it a program is a shortish regular sort of construct and if it generates an irrational number then some how the information in that number must be about the same as the information in the program that generates it. That is computable numbers are regular in some complex sense but a non-computable number is so irregular that you can't compress its structure into a program.

This leads on to the study of algorithmic information theory which is another interesting area of computer science full of strange ideas and even strange conclusions.

Math too

If you are now thinking that programs to compute numbers aren't that important let's take one final rephrasing of the idea.

Consider a mathematical formula. It is composed of a n enumerable set of symbols and as such the set of all formulae is itself enumerable. Which means that there are more irrational numbers than there are mathematical formulae.

In other words, if you want to be dramatic, most of the numbers in mathematics are out of the reach of mathematics.

There are some subtle points that I have to mention to avoid the problem of you inventing "solutions" to the problem.

Surely you can write an equation like x=a where a is an irrational number and so every irrational number has its equation in this trivial sense.

The problem here is that you cannot write this equation down without writing an infinite sequence of digits - i.e. the numeric specification for a. Mathematical formulae can't include explicit rational numbers unless they are assigned a symbol like pi or e and these irrationals, the ones we give a symbol to are exactly the ones that have formulae that define them.

That is the computable irrational numbers are the ones we often give names to. The same argument holds for programs.

Also notice that if you allow general irrational numbers within a program or a formula then the set of programs or formulae is no longer enumerable. What this means is that if you extend what you mean by computing to include irrational numbers within programs then the irrationals become computable. So far this idea hasn't prove particularly useful because practical computation is based on enumerable.

The standard theory

Now that we have arrived at these major conclusions it might be worth giving some of the more standard math.

Mathematicians say that two sets are equal in size if you can pair up items in one set with the items in the other one-to-one with none left over - they have the same cardinality and this idea works for infinite sets as well as finite ones.

All finite enumerable sets have cardinality equal to the number of elements they contain {a,b,c} has cardinality 3.

All infinite enumerable sets can be put into one-to-one correspondence and so have the same cardinality which is traditionally called Aleph Zero or enumerable infinity.

Aleph zero the size of the first or countable infinity

The set of real numbers i.e. including the irrationals isn't enumerable and is a "bigger" set than an enumerable set - it has cardinality Aleph One and is the infinity of the continuum.

There are a whole set of increasingly large infinities with cardinality Aleph Two, Three and so on..

And, yes, the set of infinities is thought to be an enumerable infinity - but I don't think anyone has proved the enumerable part as yet.

If you want to know more look up transfinite numbers.

This is more or less the end of the story but it really is just the beginning. Read some computer science to find out more.

The stack is a very simple idea. It is a data structure that has only two simple operations and yet not only is it powerful, it is at the heart of modern computing, both theory and practice. Let's fin [ ... ]