I try to learn Computation Complexity by Sipser's textbook "Introduction to the Theory of Computation".

The problem is I have a lack in understanding how Turing Machine is working. Example from the textbook.

Let's analyze the algorithm we gave for the language $A=\{0^k1^k | k \geq 0\}$. We repeat the algorithm here for convenience.

$M_1$="On input string w:

Scan across the tape and reject if a 0 is found to the right of a

Repeat if both 0s and 1s remain on the tape.

Scan across the tape, crossing off a single 0 and a single 1.

If 0s still remain after all the 1s have been crossed off, or if 1s still remain after all the 0s have been crossed off, reject. Otherwise, if neither 0s nor 1s remain on the tape, accept."

The question is why do we need to repeat before each crossing, why we cannot just cross over all 0s and all 1s one by one in $O(n)$ and continue to the step 4. According to the textbook each scan uses $O(n)$ steps and each scans occur $n/2$ times. So if each scan occurs $n/2$, on every step is should cross over 2 symbols: one 1 and one 0, which is can be done in linear time, so where $O(n)$ comes from? In result, the following algorithm runs in $O(n^2)$.

An improvement that comes later suggest to crossing over two 1s and 0s on every step that improves the running time to $O(nlogn)$.

1 Answer
1

In his first example, Sipser gives a particular decision algorithm for the language $A$. He then analyzes it and finds that it runs in $O(n^2)$ steps, where $n$ is the length of the input string. This is because his algorithm will take $O(n)$ steps for each full scan and since each scan will cross off 2 symbols, we'll be done after $n/2$ scans, giving us a running time of $(n/2)O(n)=O(n^2)$ steps, showing that $A\in\text{ TIME }(n^2)$. In other words, his particular TM demonstrates that $A$ can be decided in time no worse than a fixed multiple of $n^2$.

Here's an example, with input $0011$:

Step 1: scan all four cells. Input is the right form, so continue.

Loop test: Scan all four cells, to determine that some zeros and ones remain.

First iteration through the loop: scan all four cells, crossing off a 1 and a 0, giving us the tape $0X1X$.

Loop test: Scan all four cells, to determine that some zeros and ones remain.

Second iteration through the loop: scan all four cells, crossing off a 1 and a 0, giving us the tape $XXXX$.

Loop test: Scan all four cells, to determine that some zeros and ones remain. Nope, so accept.

We did two ($=n/2$) loop iterations, each involving two scans of the entire tape ($n$). In general, we'll take $O(n^2)$ steps

Your idea, that we cross off all the zeros and ones in one step, is a good one, but in order to do that we'd need to keep a count of the number of zeros and ones we crossed off, otherwise we've lost any information about how many zeros/ones there are as soon as we complete the first pass. Keeping this count is indeed possible, if we use a two-tape model, and this is exactly what Sipser does in his third algorithm, a couple of pages later, where he demonstrates that $A\in\text{ TIME }(n)$.

As an aside, it's easy to see that we can't do any better than $O(n)$, since it'll take that many steps just to look at each cell in the input. Hope this helps.