Main menu

Vindicating Turing and von Neumann?

Submitted by egdaylight on Fri, 01/17/2020 - 17:09

Dated:

17 January 2020

Thomas Haigh & Mark Priestley have written a `historical reflections' column entitled “Von Neumann Thought Turing's Universal Machine was 'Simple and Neat.' But That Didn't Tell Him How to Design a Computer” (CACM, January 2020). The negation of the previous sentence is clearly false: That Did Tell Him How to Design a Computer. Hence, if a binary resolution, a `yes' or `no' answer, is all that is required in the history of computing, then everything has been said. Reality is more complex though. Not every form of knowledge, such as von Neumann's background knowledge in engineering, is expressible in writing. In the sequel I shall pretend — in contrast to my on-line lecture on Turing's legacy and my reading of Andrew Hodges's 1983 biography of Turing [5] — that John von Neumann's role in computing history was as pivotal as Haigh & Priestley's recent writings seem to suggest.

A Bird's-Eye View

Haigh & Priestley appropriately distinguish between a mathematical (Turing) model of computation and a computer architecture. In my words: the former mathematically describes physical computations performed by actual machinery, the latter (physically manifested) amounts to an energy consuming machine. The computer design documentation, in turn, prescribes work to be accomplished by engineers. This descriptive/prescriptive distinction can be re-cast as one between science and engineering, which I have extensively written about before and lectured about here. To recapitulate from previous blog posts and Edward A. Lee's work [16], I conceptually distinguish between (1) the descriptive role and (2) the prescriptive role of a Turing machine:

A scientist who mathematically models a real computer with a Turing machine.

An engineer who implements a practical realization of a Turing machine with a real computer.

Three historical points can now be conveyed to the casual reader:

I have argued in my 2015 article [3] that Saul Gorn in the 1950s, along with other soon-to-be-called “computer scientists,” appropriated the `universal Turing machine' concept in the first sense above, not in the second.

Joint work with Liesbeth De Mol and Maarten Bullynck [1] shows that various machine building projects in the postwar years did exist in (at least partial) conformity with the second sense above.

It is anachronistic to frantically re-cast the entire history of the modern computer in the second sense above.

All aforementioned historians agree on the contents of the first and third point. To the best of my knowledge there is no disagreement with Haigh & Priestley on the second point, with one of them serving as a peer reviewer of our article [1].

Zooming in on the third point, recent publications of Haigh & Priestley convincingly show that addressable memory and the like do not originate from Turing's 1936 paper; see e.g., [12] and the references therein. With that said, I would always spare some room for unlikely, but possible, exceptions, e.g., Willem van der Poel in the 1950s (discussed below) and Turing himself in the 1940s.

Methodological Assumption

Haigh & Priestley write that, as a computer designer, von Neumann “made no reference to any feature of the Turing machine” and they fail to find “any feature” of von Neumann's “architecture” that he “might have derived only from Turing's paper.” Likewise, Haigh & Priestley dwell on the following statement: “Von Neumann freely acknowledged the contribution of Turing's 1936 paper to his work on automata theory but made no such connections in his discussion of computer design.” These observations bring me to Haigh & Priestley's methodological assumption, which goes like this, in my words:

If Turing sufficiently influenced von Neumann in computer design, then that influence should clearly reveal itself in von Neumann's prescriptive (implementation) work, e.g., in his “First Draft of a Report on the EDVAC”. It should not solely show in von Neumann's descriptive work on automata theory.

I can express sympathy with this doctrine in connection with my own research concerning the Dutch computer builder Willem van der Poel [9, Part I]. In my words:

In the 1952 design of the arithmetical unit of his new ZERO computer, van der Poel referred to Turing's 1936 universal-machine concept, expressing his desire for the simplest computer that was still universal in Turing's theoretical sense [10, p.376]

This extract, which comes from my 2013 blog post for the CACM, presumably points to the kind of archival evidence Haigh & Priestley sought but failed to find in the case of John von Neumann. There's a catch though. Even without this particular archival finding, I would still have come to the following conclusion:

Van der Poel thought Turing's Universal Machine was simple and neat. It did not play a significant role nor a negligible role in his design of various computers.

I can convey van der Poel's reception of the `universal Turing machine' concept with regard to computer design by pointing to his PhD dissertation [2] but not by referring to his extensive collection of computer architecture documents. The only exception I have is his commentary on his 1952 design document, published in the form of an article [10]. These observations are, in retrospect, not so surprising. After all, an engineer examining a computer will not, upon viewing the material units, inquire `Where is the Turing machine or where is its infinitely long tape?'. In other words, lack of specific computer design comments does not lead to a binary resolution (a 'yes' or 'no' answer) concerning Turing's legacy in computer building.

More on Willem van der Poel

Not a single model is considered perfect by a professional engineer [6, Ch.1]. Van der Poel was no exception in this regard. He stressed that the universal Turing machine was not an ideal model of a modern computer:

[Van der Poel] did not want to abstract away the finiteness of real machines. In his words:

“In “On computable numbers”, TURING [7] does not exercise restraint as regards the size of the machine or the extent of the store. Therefore the class of the computable numbers is infinite. From a practical point of view it is, however, better to restrict oneself to finite machines. Then the latter can no longer be called universal in the sense of TURING, because they cannot generate all “computable numbers”. [2, p.102]

In this fragment, which comes from my “Turing Tales” book [6, p.56], van der Poel alludes to a gap between the mathematical infinite and the practical finite. That same gap also comes to the fore in von Neumann's writings. On the one hand, we see von Neumann connecting Turing's mathematical world of machines to real apparatus:

“Suppose this machine is provided with a tape with a finite number of symbols, the question is whether there is a permanent apparatus which will solve the problem by the Turing method.” — John von Neumann quoted in [Haigh & Priestley, p.29, my emphasis]

On the other hand, von Neumann also emphasized the discrepancy between the infinite abstraction (due to Turing) and the finite limitations of man-made artifacts:

“Here the analogy to a high-speed computing machine breaks down, for one cannot wait for the machine to go all eternity for his answer.” — John von Neumann quoted in [Haigh & Priestley, p.29]

These words from von Neumann are subsequently interpreted by Haigh & Priestley in the following, uninformative manner:

“Computer builders, von Neumann realized, must be concerned more with practical than theoretical limits to computability.” [Haigh & Priestley, p.29]

This observation is a no-brainer because every mathematical model has limitations in engineering.

The previous fragments from von Neumann's writings, which are very similar to those of van der Poel, do not at all suggest to me that the universal Turing machine had no bearing on von Neumann in the context of computer design. In fact, John von Neumann almost said the opposite himself:

“We shall consider two systems of logic which could be used in building a computing machine. The first, developed by Turing, is essentially a logic machine.” — John von Neumann quoted in [Haigh & Priestley, p.28]

Therefore, the historically relevant question is actually this: How and to what degree did Turing's 1936 work shape von Neumann's thinking in engineering?

Claude Shannon

The following extract from my previous work (with Liesbeth De Mol and Maarten Bullynck) may also point to the kind of evidence Haigh & Priestley have sought in the case of John von Neumann:

During the same early fifties, Edward F. Moore and Claude E. Shannon at Bell Labs embarked on a study of simplifying universal Turing machines, with an eye on a possible application in the design of complex calculators. During this time, Shannon obtained his famous result that two symbols suffice for a universal machine [...] In 1952 Moore presented his `simplified universal Turing machine' at the ACM meeting in Toronto [...] Moore described a 15-state two-symbol three-tape universal Turing machine. [1, Section 4]

This extract suggests that Moore and Shannon connected simplified universal Turing machines to physical computing machinery. Now, compare the previous extract with Haigh & Priestley's own observation that von Neumann had concocted a precursor to a game of “identifying the smallest number of states and symbols a Universal Turing Machine could operate with.” It thus seems that von Neumann did seize connections between theory and practice in the arena of computer design, connections that only he, Turing, and few others were able to see in the 1940s. Why? Because von Neumann's game is akin to `logical minimalism' and, hence, relevant in computer building itself. The notion of 'logical minimalism' is introduced in our article in the IEEE Annals of the History of Computing [1]. Towards the end, we write:

[T]he Turing machine [...] fits in a tradition of logical minimalism: the search for a minimum of operations, of axioms, of length of propositions etc [...] pioneered by people like Curry, von Neumann and Turing [...]

A bit further on, we comment on a select group of engineers, including van der Poel and Fraenkel:

Though hardly any detail of Turing's original construction of a universal machine made its way into an actual computer, the very idea that 1) such a thing as a universal machine exists, and 2) it does not need more than 4 or 5 instructions, was very important to this select group of engineers.

Closing Remarks

The excellent archival findings of Haigh & Priestley (e.g., their Figure 1) lead me to the following working hypothesis:

[Correction on 18 January 2020:] Von Neumann's approach to computer architecture was shaped by his background in Turing's writings.

A supporter of the Haigh & Priestley paper has conveyed to me that one can't simply use the word “influence” as a sufficient ground for whatever one wants to assert, especially not in the face of strong evidence to the contrary. I concur, as long as we all agree that somebody's background (e.g., von Neumann in logic) conditions somebody (e.g., von Neumann in engineering). In my personal case: not only did my engineering background condition me in computability theory (and as a historian), I am also not able to express this conditioning process in writing, cf. [13].

In sum, Haigh & Priestley appropriately describe von Neumann as somebody who mastered both the practical and theoretical. But they have yet to appreciate the specific way in which both domains were interrelated for him. The history of modern logic, and the role von Neumann, himself, played in that history as a key actor, is completely missing. Could von Neumann's reception of Turing's work have been any greater? Future research will tell.

I do continue to concur with Haigh & Priestley that there is no reason to believe that von Neumann's acquaintance with — or, perhaps, thorough comprehension of — Turing's 1936 paper was a necessary condition for him, let alone the rest of the world, to enter the modern age of so-called 'stored-program' computers [3,4]. This position is best captured by the historian Leo Corry [8, p.51]:

[The ideas from Turing 1936] provided at most additional input (arguably not the decisive one) that was incorporated to a broader, complex array of (mostly engineering and only partly mathematical) considerations [...]

References

[1] L. De Mol, M. Bullynck, E.G. Daylight. Less is More in the Fifties: Encounters between Logical Minimalism and Computer Design during the 1950s. IEEE Annals of the History of Computing, Vol. 40, No. 1, pp. 19-45, January-March, 2018.

Appendix

Every logician will read that statement — contra Haigh & Priestley — as follows: von Neumann's approach to computer architecture was deeply shaped by his background in modern logic and, hence, also in computability theory and, hence, also in terms of Turing's 1936 work.

“Even Martin Davis, who in 1958 [by 1952] was one of the first to note the potential of the universal machine as a model for what could be calculated by real computers, nevertheless advised readers that the page he devoted to the topic was “a digression and may be omitted without disturbing continuity.”” [Haigh & Priestley, p.29]

Perhaps Davis was pressured by his publisher to advise his readership in the way he did. For the record, here's what I wrote about the same event in my 2012 book “The Dawn of Software Engineeirng”:

In fact, one of the reviewers of his book derided the connection he was proposing with actual computing machines. [15, p.20]

Martin Davis thinks the anonymous reviewer was the loigcian J. Barkley Rosser as he conveyed to me in a private discussion in Ghent, Belgium on 8 November 2011. [15, endnote 35, p.204]