Introduction to Turing Machines

Posted 09 January 2014 - 05:00 PM

Introduction to Turing Machines

Introduction
In modern computability theory, there exist numerous models of computation, such as Lambda Calculus, Von Neumann Computers, and Turing Machines. Each of these, while vastly different on the surface, are equivalent models of computation. That is, anything each of these models can (or cannot) compute can (or cannot) also be computed by any of these other models. The Turing Machine is considered the standard model of computation. This tutorial will explore Turing Machines and their uses in determining the limits and power of computers. Some familiarity with Formal Languages is assumed.

Turing Machines- Introduction
So what exactly is a Turing Machine? Conceptually, it is a finite state machine with memory. That is, a Turing Machine acts like a finite state machine, but it also has a single tape of infinite capacity and a tape head. The Turing Machine can write to the tape on the tape cells, one character per cell, and move the tape head in either direction arbitrarily. There are also variations on the standard deterministic Turing Machine, such as the Non-Deterministic Turing Machine and the Multitape Turing Machine.

Now that the concept of a Turing Machine has been introduced, let's define it more precisely. A standard deterministic Turing Machine is a 6-tuple containing the following: (Γ, β, Q, δ, s, h).

Γ: This is the tape alphabet. That is, Γ contains all the non-blank symbols that can appear on the tape.

β: This is the blank symbol that can appear on the tape head.

Q: This is the finite set of states. Note that the halting state is not in Q.

s: This is the initial state, which is in Q.

h: This is the accepting halting state, which is not in Q. Note that a Turing Machine may halt in states other than h, but those are all reject states.

δ: This is the state transition function. It takes the current state and an input on the tape, and returns a state from Q or the halting state, as well as either a letter to write or a direction to move the tape (if L or R is the second value in the 2-tuple returned). So δ is defined as δ: Q x (Γ U {β}) -> (Q U {h}) x (Γ U {β} U {L, R}). Note that it is always clear to which state the standard Turing Machine transitions. That is, on a given input from the current state, there cannot be two states to which are indicated for transition.

Before exploring variations on the standard Turing Machine, it is first important to understand how the standard Turing Machine behaves. Conceptually, a standard Turing Machine starts with an input string, which is written to an initially blank tape starting at the far left cell. It then executes starting at the initial state s, transitioning to other states as defined by the function δ, based on the current state and input from the given tape cell. If evaluating this string in such a manner results in the Turing Machine reaching its accepting halting state h, then the Turing Machine is said to accept the input string. A Turing Machine may also halt on a non-accepting state, or it may continue to execute and never halt. A language is accepted by a Turing Machine if all the strings in the language are accepted by the Turing Machine. Let M be a Turing Machine, then L(M) denotes the language of strings that M accepts.

Non-Deterministic and Multitape Turing Machines
This section will briefly introduce the Non-Deterministic and Multitape Turing Machines. The Non-Deterministic Turing Machine differs from the standard Turing Machine in an important way: the transition between states may not always be clear. For example, with a non-deterministic Turing Machine, given an input 'A', there may be instructions to move the tape head left, and there may be instructions to print 'x.' Each of these instructions directs the Turing Machine to transition to distinct states. Hence, the name of "Non-Deterministic" Turing Machine.

Σ: This is the choice alphabet. Since the Turing Machine is non-deterministic, the choice input describes the option to take at each juncture, through use of a choice string c in Σ*.

Γ: This is the tape alphabet. That is, Γ contains all the non-blank symbols that can appear on the tape.

β: This is the blank symbol that can appear on the tape head.

Q: This is the finite set of states. Note that the halting state is not in Q.

s: This is the initial state, which is in Q.

h: This is the accepting halting state, which is not in Q. Note that a Turing Machine may halt in states other than h, but those are all reject states.

δ: This is the state transition function. It takes the current state, a character from the choice string, and an input on the tape, and returns a state from Q or the halting state; as well as either a letter to write, a direction to move the tape (if L or R is the second value in the 2-tuple returned), or the no valid state option (⊥) . So δ is defined as δ: Q x Σ x (Γ U {β}) -> (Q U {h}) x (Γ U {β} U {L, R} U {⊥}). Since the Turing Machine is non-deterministic, a choice string is necessary to determine the transition.

The definition of string acceptance on a non-deterministic Turing Machine varies slightly compared to that of the standard deterministic Turing Machine. A string is said to be accepted on a non-deterministic Turing Machine such that there exists a choice string c in Σ* such that an input string w in Γ* reaches h, the accepting halt state, when started on the initial state s with only the input string w written on the tape. Finite State Machines are commonly described with graphs, and a graph theory intuition is quite useful here. Think of the choice input string as a graph path on the representation of a finite state machine. If there is a path on the graph representation of the finite state machine from s to h on input string w, then w is said to be accepted by the Turing Machine. The choice string dictates this path. L(M), the language accepted by a non-deterministic Turing Machine M, is still the set of such strings w that are accepted by M.

The multitape Turing Machine is the other important adaptation of the standard Turing Machine. As the name suggests, the major change is that there are k-tapes, each with their own tape head. Non-deterministic Turing Machines can also be described with multiple tapes.

As it turns out, the standard Turing Machine and deterministic multi-tape Turing Machines are equivalent in terms of their computational power. That is, anything the standard Turing Machine can (or cannot) computer can (or cannot) also be computed by a multitape deterministic Turing Machine. Note that the time and space bounds are not identical between the two Turing Machines.

Before proving this, let's walk through it conceptually. The standard deterministic Turing Machine only has one tape, so it is easy to see a multitape Turing Machine can compute anything a standard Turing Machine can compute. Simply, the multitape Turing Machine just uses a single tape.

Showing that a standard Turing Machine can compute anything that a multitape Turing Machine can compute is a bit more complicated. The general idea is to take the strings on each tape of the multitape Turing Machine and to place them on a single tape of the standard Turing Machine, using a delimiter to separate them. That is, let s1, s2, ..., sn be the the strings on tapes 1, 2, ..., n of the multitape Turing Machine. Let # be the delimiter symbol for the standard Turing Machine. Then the tape of the standard Turing Machine is arranged as follows: s1#s2#...#sn#. Marked symbols with the dots above them on the single tape Turing Machine are used to indicate the positions of the tape heads on the multitape Turing Machine.

Proof: The standard deterministic Turing Machine is an equivalent model of computation to the multitape Turing Machine. (See the spoiler for the proof)

Spoiler

Let M be a single tape Turing Machine, and let M' be a multitape Turing Machine. It will be shown that M = M', by showing that M subset M', and M' subset M.

In order to show that M subset M', it is sufficient to show that anything M can be simulated by M'. By definition, M only has a single tape. Thus, M' only requires a single tape, ignoring the remainder of its k-1 tapes, to simulate M. It follows that M subset M'.

In order to show that M' subset M, it is sufficient to show that M' can be simulated by M. The proof is by construction. Since M' has k-tapes, it is necessary to represent the strings on each of the k-tapes on a single tape. It is also necessary to represent on M where the kth tape head appears on M'. This is done by using a marker character for each c in Γ'. Let c^ represent the marker character. It is also necessary that M delimits each string to distinguish between tapes. Thus, |Γ| = 2|Γ'| + 1. M then simulates M' in the following way:

M scans the tape from the first delimiter to the last delimiter to determine which symbols are marked as under the tape heads on M'.

M then evaluates the transition function of M', then makes a second pass along the tape to update the symbols on the tape.

If at any point, the tape head of M falls on a delimiter, M' would have reached the end of that specific tape. So M shifts the string, cell by cell, starting at the current delimiter inclusive. A blank symbol is then overwritten on the delimiter.

Thus, M simulates M'. Therefore, M subset M'.

Since it has been shown that M' subset M and M subset M', it follows that M = M', so multitape Turing Machines and standard deterministic Turing Machines are equivalent models of computation.

Note: If a multitape Turing Machine takes T steps, it uses at most T+1 cells. Thus, the standard Turing Machine (T+1) passes times T steps, for T2 + T time, which is O(T2) time complexity compared with O(T) for the multitape Turing Machine.

The next important point is that any language accepted by a non-deterministic Turing Machine can be accepted by a standard deterministic Turing Machine. The proof for this is by simulation. Before introducing the proof, let's conceptualize this. Earlier in this section, a graph theory intuition was introduced for understanding the definition of what it means for a string to be accepted by a non-deterministic Turing Machine. That definition of string acceptance dealt with the existence of a choice string such that the non-deterministic Turing Machine would reach the halting state h from the starting state s. The graph theory analog was that there existed a path for the input string from s to h.

So the way a deterministic Turing Machine simulates a non-deterministic Turing Machine is through, essentially, a breadth-first search. More formally, what actually happens is that a deterministic multitape Turing Machine is used to simulate a non-deterministic Turing Machine. It does this by generating choice strings in lexicographic order and simulating the non-deterministic Turing Machine on each choice string until the string is accepted or all the possibilities are exhausted.

Given the non-deterministic Turing Machine has a finite number of transitions, there are a finite number of choice input strings to generate. Thus, a multitape deterministic Turing Machine will always be able to determine if an input string is accepted by the non-deterministic Turing Machine. It was already proven that a multitape Turing Machine can be simulated by a standard deterministic Turing Machine, so it follows that any language accepted by a non-deterministic Turing Machine can also be accepted by a deterministic Turing Machine.

Proof: Any language accepted by a non-deterministic Turing Machine can be accepted by a standard deterministic Turing Machine.

Spoiler

This theorem will be proven directly. Let M be a non-deterministic Turing Machine and let L(M) be the language accepted by M. Let w in L(M). Let M' be a multitape deterministic Turing Machine to simulate M, where M' has at least 3 tapes. Let the first tape have w written to it. The second tape will be used to generate choice strings in lexicographic order. Finally, the third tape will be used to simulate M.

M' will start by clearing the third tape, if it is not already blank. Then w will be written to the third tape. M' will then generate the first choice string in lexicographic order on the second tape, and then M' will simulate M on the choice string and w. If the simulation of M fails to reach the accepting halt state, then the next choice input will be generated, the third tape will be cleared and have w rewritten, and M' will begin again by simulating M. Thus, M' deterministically accepts w if and only if there exists a choice input such that M accepts w. QED.

Turing Machine Encodings and Universal Turing Machine
This section will deal with the topics of Turing Machine Encodings and Universal Turing Machines. For the sake of concept, this section will only focus on standard deterministic Turing Machines. The alphabet of the Turing Machines Γ = {0, 1} for this section.

The idea behind Turing Machine encodings is that the behavior and structure of Turing Machines can be captured in string format. As computer scientists, this concept is not foreign. Anyone who has written a computer program has captured the behavior of a Turing Machine in string format.

Conceptually, a Turing Machine has |Q| states, with 3|Q| possible transitions. That is, on a given state, the encoding of the Turing Machine will have to describe how a state transitions on inputs 0, 1, and β. Let's now look at formalizing the encoding of Turing Machines. The alphabet describing the encoding of a Turing Machine is Λ = {<, >, [, ], #, 0, 1, β, R, L}.

Let qi, qj be states in Q, and let c in (Γ U {β}), and let z in (Γ U {β, L, R}). Consider the transition function evaluated on δ(qi, c) = (qj, z). That is, on input c, the Turing Machine transitions from state qi to qj and performs the action described by z. This transition is described by the string <z#1j>, where 1j represents the character 1 repeated j times. In other words, the number of 1's represents the index of the state to which to transition. If the transition function does not return a state, there can be no 1's returned. Thus, the Turing Machine encoding is a series of triples of the format: [<z#q> <z'#q'> <z"#q">]|Q|, one ordered triple per state in Q. The encoding of a Turing Machine is denoted p(M).

Why bother encoding Turing Machines? There is no point in encoding anything, unless there is a decoder. In fact, the decoder is a Turing Machine itself- the Universal Turing Machine (UTM), a Turing Machine which is capable of simulating arbitrary Turing Machines (including itself) on arbitrary inputs. The UTM is constructed with an alphabet Λ' = Λ U Λ^. That is, for each character c in Λ, c and c^ are in Λ'. Thus, the UTM is a single tape Turing Machine capable of simulating multitape Turing Machines.

The UTM is given p(M) and w in Γ*. Starting with a blank input tape, β is written to the far-left cell of the tape, followed by p(M) and then w. The string w is marked by replacing it with w^, and the current state of the simulation of M is marked by replacing the left-most [ on p(M) with [^. The UTM then reads the next character c of input from w, marking it with c^, then the UTM marks the desired <z#q> in the triple by replacing the left < with <^. If a write is required, c^ is replaced with d^, a new symbol. Before transitioning to the next state of M, the UTM replaces all marked characters with their unmarked counterparts.

The Turing Machine encoding language and UTM have a powerful implication: Turing Machines are countable and enumerable. That is, it is possible to generate strings in lexicographical order over Λ*. If the ith string in lexicographical order over Λ* is not a valid Turing Machine encoding, then it references M0, a Turing Machine that simply halts on all inputs. Otherwise, the ith string corresponds to some Turing Machine. Thus, Turing Machines can be algorithmically generated. Turing Machines are also equivalent to algorithms (which will be explored more in the following tutorial), so algorithms can be algorithmically generated by the brute force and ignorance method. That puts a lot of computer science in check, given all the work we do to come up with new and clever ways to solve problems.

Conclusion
This tutorial served as an introduction to Turing Machine and computability theory. The next tutorial in the sequence will cover uses for Turing Machines, including language acceptance. It will also begin to explore how Turing Machines can be used to understand and define the power and limits of computing.