Using a specially designed cellular
automaton capable of emulating completely the function of a human brain, we
explore ways in which counterfactual sensitivity, i.e. the ability of a
computational system to respond to any valid inputs, relates to discrete state
machine consciousness. In this computational framework, the distinction between
a computation and the recording of a computation can be blurred arbitrarily,
yet the physical implementation of the computation itself is unchanged. From
this, we conclude that a purely computational account of consciousness is
unsatisfactory.

The question of whether consciousness could be
realized in a purely computational system has been widely discussed, but any
kind of consensus on the question is still remote. We assume that consciousness
is produced by the brain, but is it by virtue of purely computational
processes? If so, an emulation of those computational processes should result
in consciousness, since an emulation of a computation is just another,
equivalent, computation. But if consciousness is not the result of
purely computational processes, we must seek explanations elsewhere, starting
with the physical components of a brain considered as physical entities, rather
than abstract representations of computational states.

In this paper we meet “Woody,” a discrete
computational system which implements an emulation of a human brain, and
consider a number of experiments that could be done with such a system,
experiments that call into question one of the requirements that has been
postulated for any conscious computation. The reader can test his or her
intuitions regarding the effects of these experiments on the putative
subjective state of Woody, our volunteer experimental subject. Recognizing that
there is already a large literature on the subject of computational
functionalism, the modest hope is that these thought experiments will provide a
more intuitive framework for discussing some of these ideas.

Before going further, it will be useful to review
related work by Tim Maudlin, Hilary Putnam, and Mark Bishop, as their ideas,
and attempts to refute those ideas, are germane to this discussion, and we will
be referring to them throughout. We will start with Maudlin [1]. This brief
overview can hardly do justice to Maudlin’s paper, but we will review some key
points needed for our purposes. Maudlin sets up a scenario with a mechanical
computer he calls “Klara” which, when fed some appropriate input tape τ, will
compute a set of states and outputs which a computationalist argues would
instantiate a conscious computation. Once the computation has run, a new
computer, which he names “Olympia”, is constructed, which processes the same
input τ, but in all places where a conditional state transition would occur,
the unused branches are effectively replaced with “hard coded” state
transitions that ignore any inputs that differ from τ. The computationalist
will insist that in spite of transitioning through the same set of states as
Klara, what Olympia is doing is too trivial to even label a computation, much
less a conscious computation. The missing element is what we will call counterfactual
sensitivity, which we can define as the ability of a computational system
to correctly respond to arbitrary inputs. A computational system has full
counterfactual sensitivity only when all components of that system are capable
of operating as designed when presented with any valid input.

To remedy Olympia’s missing counterfactual sensitivity,
Maudlin asks us to imagine placing an instance of the fully capable Klara next to
each conditional state of Olympia, arranged so that if any inputs not identical
to τ were to arrive, a Klara would take over the computation. This restores
counterfactual sensitivity, and therefore presumably consciousness, even though
the Klaras never actually do anything, since we continue to use only τ as input.
Maudlin goes further and has us imagine that some of the Klaras have rusty
components so that, were the Klara to actually run, it might not work. He also
considers a scenario where the Klaras are physically blocked so that they
couldn’t run. But since the Klaras are never actually invoked when using input
τ, Maudlin argues that whatever their condition, they can make no difference to
the physical implementation of the computation. From this he concludes that
computationalism implies that consciousness arises from (supervenes on)
something more than the physical activity taking place in a computation, since
only the original “trivial” Olympia is actually doing anything. Maudlin
proposes that rejecting computationalism is more tenable than rejecting
physicalism.

Mark Bishop [2][3]
takes a different tack, but arrives at a similar conclusion. Again, our review
is brief, and the original works should be studied for more detailed
explanations and justifications. Extending ideas first put forward by Putnam [4],
Bishop notes that any discrete state machine M with fixed, known inputs
can have its conditional state transitions replaced with direct transitions to
the state that is selected based on that particular fixed input. This process
prunes the combinatorial explosion of possible state transitions down to a
single set of simple state transitions (which we can call U for unrolled)
that corresponds to the trace of the execution of M with a given set of
inputs. Now we can take any system that generates discrete, non-cyclic state transitions
(e.g. a simple counter, C) and map the “unrolled” states of M to
the states of C by a simple correspondence. The interpretation of any
state is arbitrary, so we choose to associate via a mapping function each state
Ctwith state Utfor all states t in
U. At any time, then, we can both find the current state of U, and
predict the next states of U, by mapping the current and next states of C
to U. In short, any time C is running, we may consider U
to be running.

Now suppose that we have a computation that a
computationalist claims is conscious, say the experience of a robot seeing a
red square. Bishop’s construction allows us to translate the computation plus
its inputs into a sequence of states U that correspond to the actual
state transitions that occurred during the conscious computation. Again, this
translated sequence can then be considered to run whenever the state machine C
is running, meaning that the operation of a simple counter can be interpreted
as the execution of a program resulting in consciousness. Worse still, Putnam
puts forward an argument claiming to show that any open physical system, say a
rock, can be considered as passing through a series of discrete, non-cyclic
states that we could also associate with the states of U. If that is
true, computationalism must imply a sort of panpsychism, since the
interpretation of these states is arbitrary, and must include states that
correspond to every sort of conscious experience imaginable. Putnam and Bishop understandably
reject this possibility. However, once again, counterfactual sensitivity
figures large in this argument, because constructing U is only possible
when the inputs are fixed. Critics of this line of reasoning can once again
point to the requirement of counterfactual sensitivity for anything we might
call a real computation, and since the discrete state machine U does not
satisfy this requirement, Bishop’s and Putnam’s arguments lose their force. [5]
Bishop has responded to this, concluding as Maudlin does that the lack of
counterfactuals can only affect the putative consciousness of the computation
if we are prepared to abandon physicalism.

Clearly, the notion of counterfactual sensitivity is
crucial to the question of whether a computation is, or could be, conscious.
But as Maudlin and Bishop maintain, exactly how the presence or lack of counterfactual
sensitivity might affect consciousness has not been satisfactorily explained.
The rest of this paper considers thought experiments which explore some of
these ideas further, in a framework that may be more intuitive for some readers.

One important point to make at the outset: for the
following thought experiments it is not necessary to draw fine distinctions
between the different types of consciousness that a human brain can experience.
Because the whole brain is emulated, everything from creature consciousness, to
dreaming, to higher order beliefs can be considered. And if one takes the
position that consciousness is an illusion, one may freely substitute “the
illusion of consciousness”, or “the illusion of subjective experience”,
wherever desired in what follows. Regardless of whether it is an illusion, the
phenomenon of subjective experience, i.e., whatever it is that vanishes during
anesthesia and slow-wave sleep, is the phenomenon in question here.

For the purposes of this paper, let us assume that
the problems of scanning the necessary structures of a human brain and modeling
the relevant behaviors of those structures have both been solved: we have a
computational system which emulates the behavior of an individual human brain
sufficiently well that the system tells us that it has normal human phenomenal
experience [6]. The level of emulation required (whether neural circuit,
neuron, macromolecule, atom, or even string) does not matter for the purposes
of this thought experiment; we only require that the emulation must accurately
reproduce the behavior over time of a physical brain (in its normal embodied
state). Obviously, the level of detail required for the emulation greatly
affects the quantity of computational resources needed, but we will assume that
we have enough to run the emulation.

For concreteness, let’s say that we have scanned the
brain of our friend, Woody, who has volunteered to undergo the procedure. Once
the scanning is complete, we run the emulation for a while, feeding it inputs
that correspond to environmental events such as sights and sounds. We interpret
the outputs as signals to the muscles of the tongue, arms, etc., which in turn
can be interpreted as speech, gestures, and the like. The complete input stream
should of course include everything needed to keep a human brain functioning
normally, such as proprioception, and additionally, the outputs must be acted
on by the world (virtual in our case), and fed back into the emulation. And
there might well be a need to pause the emulation from time to time while
staging and preparing the input stream. But these and a myriad other details
need not occupy us here.

Once we have satisfied ourselves (and presumably
both the human Woody and the emulated Woody) that everything is working as
expected, the human Woody exits the scene, so from this point on, the name
“Woody” refers to the emulated Woody. To begin our experimentation, we take a
checkpoint of the entire state of the computational system, then we directly ask
Woody, “Are you conscious? And do you smell anything?” At the same time, we
have arranged for inputs corresponding to the smell of roses to arrive at
Woody’s emulated olfactory center. The system runs for a while, then Woody
answers, “Yes, I’m conscious, and I smell a lovely rose.”

As previously stated, it is the premise of these
thought experiments that this whole brain emulation does result in normal
conscious behavior, so there can be no question of whether the existence of
inputs corresponding to smells would result in behavior appropriate to the
perception of sound, say, instead of odor. The issue of embodiment does not
arise, given that the scenario postulates a complete emulation of an entire
virtual body and environment. The emulated environment provides all necessary
inputs and feedback for normal operation of a human brain in a human body.

At this point it is necessary to mention one detail
about the implementation: whenever the computation requires a random number, it
retrieves it from a table of random numbers which were previously generated by
an appropriate source, say a quantum device. This table of random numbers is
part of the initial configuration. That means we can reset the state of the
computer to the checkpoint, feed it the same inputs, and observe that the
system behaves identically whenever the emulation runs, since the computation
is perfectly deterministic. We can run this particular computation as many
times as we like, and in each instance, the result will be a pattern corresponding
to “Yes, I’m conscious, and I smell a lovely rose.” Nothing has changed from
the first run, so we must conclude that if the computation had a subjective
experience the first time, it would have the identical experience in subsequent
runs. This is an important distinction between physical brains and emulated
brains, since it is not possible to set up a physical brain that goes through a
given set of states more than once (if indeed the notion of a sequence of states
even makes sense for physical brains).

For the next set of experiments we choose a new
computational system, a two-dimensional array of unit cells which together implement
Conway’s Game of Life [7]. Such an array is an example of a cellular automaton
(CA), and it has been shown that a sufficiently large array of these Game of
Life cells can implement any computation that can be performed by a
general-purpose computer [8].

All the unit cells of this system are identical,
made up of three parts (Figure 1):

• input photodetectors, which respond to the
presence of photons arriving from the eight immediately neighboring unit cells,

• a logic unit, which receives the information from
the input detectors and sets the next output state based on the cell’s current
output state plus the neighboring cells’ on/off states, following Conway’s
simple rules[1],
and

• a light source which turns on and off, based on
the output of the logic unit.

Figure 1. Unit cell components.

One further component is needed for the array: a
system-wide clock signal that controls when the inputs are to be sampled. We
arrange things so that the inputs are sampled in the middle of the cycle,
leaving adequate time for the next state to be computed and for the light to
reliably indicate each cell’s state before the next clock cycle.

The array needed would doubtless be enormous, but in
principle, the original Woody emulation could be set up on such a cellular
automaton. With identical inputs, staged to arrive exactly as they had in the
original setup, and with appropriate interpretation of outputs, the CA
implementation would be computationally equivalent. If we have done the
conversion properly, after the computation has run, we should see outputs
corresponding to “Yes, I’m conscious, and I smell a lovely rose.” We expect
this because the computation is deterministic, and computational equivalence
guarantees that if a computation results in a given set of outputs on one computational
system, it will result in the same set of outputs on any other equivalent
system. Computationalism goes further and asserts that since the first
computation creates a feeling of subjective awareness, the identical feeling
should arise any time that computation is performed, regardless of the
computational substrate.

In what follows, we will call this most recently
described system the “free-running” version of Woody, for reasons that will
become obvious. We can now use the Woody CA to test our intuitions regarding
counterfactual sensitivity. As explained above, counterfactual sensitivity is a
reasonable precondition for anything which might be called a computation; it
allows us to distinguish between, on the one hand, the mere replaying of a
recording of states, versus a computational process which actually generates
states, on the other. Many scholars (e.g. Chalmers [9], Hardcastle [10],
Chrisley [11]) have advanced counterfactual sensitivity as a requirement for
any conscious computation. David Chalmers, for example, states in [9], “Cognition
requires at least the possibility of functioning in more than one way”.

To explore this idea, let us now imagine that as
usual we run the “Are you conscious, and do you smell anything?” computation on
the Woody CA, but now we

1. film or otherwise record the pattern of lights of
the computation,

2. reset the array back to the original checkpoint,

3. run the computation again,

4. project the results of the prior, identical
computation back on top of the Woody CA as it runs.

In this case of projecting the results of a prior
computation onto the array, given that the projection will register exactly
with the computation, nothing changes in terms of the computation and local
information flows, and naturally, at the end, the expected output shows up. But
would anything happen to the feeling of subjective awareness, assuming there
had been one in the unprojected case?

If one insists on full counterfactual sensitivity
for full consciousness, one might suspect that the moment the projection of the
previous computation turns on, consciousness would vanish, because suddenly the
counterfactual sensitivity of the system would vanish. Why? Because some of the
photons from the projector that hit a unit cell’s output light will scatter
into neighboring cells’ photodetectors. When projecting light onto unit cell A,
unit cell A’s neighbors will detect photons from both sources of light: photons
internally generated by unit cell A, and scattered photons from
the projection onto unit cell A. So in terms of the definition of
counterfactual sensitivity given above, no unit cell that was projected on
would be able to signal “off” if the projection were signaling “on”. That
component would not be capable of operating as designed, thus violating our
rule. Further, if any different inputs were introduced while the projector was
on, the computation would almost certainly fail, due to the brittle nature of
cellular automata: any lights that were on in the projection, but not on in the
new computation, would disrupt the pattern, and doubtless in short order the
entire array would cease to represent the computation of anything meaningful.

Counterfactual sensitivity is definitely removed by
projecting the results of the previous computation, but the conclusion that
projecting onto the array obliterates consciousness when the input stream is not
changing seems rather mysterious, since physically the computation proceeds as
usual. This scenario is distinguished from both Maudlin’s trivial Olympia and
Bishop’s unrolled discrete state machine U, in that at the level of the
basic computational elements (the unit cells), everything is physically present
and operating normally: the photons (or lack thereof) that show up on unit cell
A’s photodetectors continue to set the next output state according to
the logic unit’s rules. It happens that with these inputs the output will always
agree with the projection, but nothing physically different is going on in
terms of the local information flows and logical processing.

Concluding that consciousness is lost when the
projection is turned on can be made even more dubious by supposing that we turn
on the projection and leave it on, but now we add a simple input checker which
will monitor the input stream, comparing it to the original inputs. If any
different inputs arrive, the input checker will immediately turn off the
projector so that it will not disrupt the new computation. Now the system as a
whole does have full counterfactual sensitivity, so presumably consciousness
returns, but we are once again left with the question of how this could affect
the consciousness of the system, since the actual computation that results in
conscious behavior relies only on the information flows and other causal
interactions of the computational elements, which are unchanged by the addition
of the input checker. This is analogous to Maudlin’s installing of Klaras to
restore counterfactual sensitivity to Olympia, except that here the presence of
the input checker does not have anything to do with the computation going on in
the array. It merely switches off the projection if different inputs are
detected.

At this point, someone inclined toward a
computational account of consciousness might be tempted to give up on the
requirement for counterfactual sensitivity, and argue that it is only the
causal structure that counts for consciousness. But this cannot be done lightly,
because as discussed above, without the requirement for counterfactual
sensitivity, one is led to results that are uncomfortable for computationalism,
such as those pointed out by Maudlin and Bishop.

One way to preserve the requirement for
counterfactual sensitivity would be to show some meaningful physical difference
between the projected system and the free-running system, with the hope that
this could explain why only the free-running computational system actually
experiences anything (e.g. Chrisley [11]). In our case, someone could point out
that the system with projection is physically not the same as the system
without projection. That is true, but so would be two different
unprojected arrays implementing the same computation, but with the clock
running at half speed on one array, or with the lights twice as bright on one
of them. A hallmark of computationalism is that the substrate does not matter,
so just being different is not enough. The differences must actually affect the
physical processes that underlie the computation such that the second system is
not logically equivalent to the first. In our scenario, at the level of the
unit cells, all components are operating normally, in both the projected and
non-projected cases.[2]

Furthermore, to claim (as a consistent
computationalist must) that the addition of an input checker rescues
consciousness is in some ways even more problematic, since it is causally distant
from the low-level processes. For example, suppose that instead of installing
an automatic input checker, we, the experimenters, leave the projection on, but
because of a side-effect of the design of the machine we are forced to turn the
projector off before we can reach the switch used to change the inputs. Now the
system has full counterfactual sensitivity, but only by accident. Or suppose we
just promise to turn off the projector before any new inputs are given. Now we
are part of the system, and Woody is presumably conscious to the degree that we
keep our promises.

With no explanation why the projected Woody should
fail to be conscious in spite of having no counterfactual sensitivity, it seems
a committed computationalist has no choice but to accept that projected Woody
is conscious, and try to live with the consequences, or if that is too
unpalatable, perhaps to seek other explanations for conscious experience.

Regardless of whether one is convinced by arguments
like Maudlin’s, Putnam’s and Bishop’s, the idea of a conscious system without
counterfactual sensitivity may well strike one as extremely counterintuitive,
if not absurd. The notion feels similar in plausibility to ascribing
consciousness to images of actors projected on a movie screen. Let’s consider
some objections that could be raised.

Objection 1:The projected Woody system is not conscious because it lacks counterfactual
sensitivity.

It is the burden of anyone offering this objection
to point out a mechanism for the projected system to detect that counterfactual
sensitivity is missing. It cannot be at the level of the logic units, since at
that level, inputs, outputs, and computations are fully equivalent to the
free-running Woody. The laws of physics assure us that all photons of a given
wavelength are identical, so what could be the other source of information
which causes subjective awareness to arise or not? If one wants to claim that
it is an emergent property of the lower levels, how could any emergent property
be different, given that all lower-level behaviors are identical? Again, the
question here is not whether there are physical differences between the
free-running and projected systems: clearly they are different. But if
consciousness arises from a computation, and if the computational elements are
operating in an identical manner with and without projection, the defender of
the requirement for counterfactual sensitivity must explain clearly how these
physical differences affect subjective awareness.

Objection 2:The argument is too powerful: if we substitute the notion “computing 6×7 =
42” for “digital emulation of a brain yields consciousness” in the foregoing
thought experiments, we should presumably be led to doubt whether a
computational account can be given of multiplication.

This objection fails immediately because it begs the
question: the substitution only works if we already accept that the feeling of
consciousness arises from (or just is) the right sort of computation. It is
granted at the outset in these thought experiments that the digital emulation
of a brain will result in “the right answer” (conscious-like behavior), but the
question is whether subjective awareness is also produced. A small parable
might help make this clear: suppose there are people who claim, for what are to
them good reasons, that whenever 6 and 7 are multiplied, the answer 42 results,
of course, but that in addition, a wonderful phenomenon arises called
Life-the-Universe-and-Everything-ness, or LUE-ness. Our thought experiments
might trouble these believers, because their claim is that LUE-ness happens
only when that computation is actually carried out, so by blurring the
distinction between a computation and the recording of a computation, we call
into question the LUE-ness that results, even though we always get the right
answer. Other people accept the existence of LUE-ness, but are skeptical that
multiplying 6 and 7 produces it, preferring instead an account involving
specific arrangements of physical entities rather than abstract representations.
Still others deny that LUE-ness exists, in spite of finding themselves in the
middle of it.

Objection 3:
None of these systems is conscious because they all use a previously
generated table of random numbers.

As long as the numbers are generated from a genuine
random source, it seems unreasonable to insist that one set of random numbers
is better for consciousness than another. Again, the burden is on the person
bringing this objection to demonstrate how a computation could feel different because
of the “freshness” of the random numbers used in the computation.

Objection 4:Since the end state is always the same, it doesn’t matter whether Woody was
conscious during the intervening states, regardless of any projection of the
previous computation.

It doesn’t matter much in the scenario where we are
merely asking Woody if he is conscious, and if he smells anything. It might
matter immensely if the experiment were to see what would happen if all of
Woody’s pain sensors were made to fire at the maximum rate for an extended
subjective time. If this is just lights turning off and on with a certain
pattern, fine. If not, a real moral question arises.

A great many variations are possible in this Woody
“Game of Life” cellular automaton framework, only a tiny fraction of which we
have explored here. For example, we can consider scenarios where

We project only onto one or a few cells, not
the entire array.

We project onto unit cells with functioning
input detectors and logic units, but which have had their lights
disconnected. Nevertheless, because of the projected photons, such cells
continue to signal correctly.

We completely power off certain unit cells, but
project onto those powered-off cells, so that they continue to signal correctly
to neighboring cells, in spite of being functionally inert.

We vary the proportion of light provided by the
unit cells versus the projector.

We change the source of the photons: instead of
giving each cell an internal light to signal the “on” state, we design the
cell so that it has a shutter on top which opens to signal “on” by
allowing in photons which must be provided by an external source of light.
Now we can remove counterfactual sensitivity by projecting light only onto
those cells that were “on” in the recorded computation.

We power off all unit cells that were never
“on” during a given computation.

These are just a few possibilities, some of which
allow us to explore variations in both the counterfactual sensitivity and the
causal structure of these computations. All of them raise difficult questions
about the prospects for computationalism by blurring in various ways the
distinction between a computation and the recording of a computation.

So what are we to make of the conscious awareness of
the Woody array when it is not in a free-running state? The most parsimonious
explanation is that since projecting onto the array can have no effect whatever
on the physics of the computation, it would have no effect on any subjective
awareness produced by the computation. But since the projection definitely
removes the counterfactual sensitivity of the system, if we insist that
counterfactual sensitivity is a requirement for subjective awareness, we must
conclude that there is some non-physical explanation for the lack of awareness
when the Woody CA is not free-running. We can (reluctantly) abandon
counterfactual sensitivity as a requirement for computationalism, but then we find
ourselves in the awkward position of granting consciousness to too many entities,
including Maudlin’s trivial Olympia, and Bishop’s and Putnam’s trivial state
machines. So we reject computationalism, thereby avoiding these problems, since
if the original, unprojected Woody does not have conscious awareness, we would
not expect the projected version to have awareness either.

But rejecting computationalism is not without its
own issues. Assuming we are someday able to construct something like Woody,
what do we make of it? We will have built a system that can answer any question
about the consciousness which the brain it is emulating would be experiencing,
were it a brain, and not an emulation of a brain. And among the outputs of the
system are states that correspond to behaviors that are indistinguishable from
those of human beings. Evolution has not prepared us for interactions with an
entity which exhibits conscious behavior but which is not in fact conscious; we
would find it tempting, perhaps overwhelmingly so, to grant full rights and
responsibilities to an entity which can pass every imaginable test of
sentience. But if Woody says, “Come on in! The water’s fine!” and suggests that
you submit to a destructive brain scan in order to upload to a computational
substrate, you would be wise to do so only if the nature of consciousness is clearly
understood, and if you have included as part of your upload package any
hardware (or wetware) necessary for conscious experience.

Acknowledgment

Though I cannot say I am persuaded by his
conclusions, (e.g. that all reality results from the operation of a universal
computer whose mere Platonic existence implies a sort of modal realism),
Marchal [12] discusses a thought experiment with projection onto a two
dimensional computer which was an important inspiration for some of the ideas
in this paper.

[1]
Next output state is “on” if
exactly three neighboring cells are on, unchanged if exactly two neighboring
cells are on, otherwise “off”.

[2]
To remove any
question of whether there are “extra” photons in the projected case, we can
arrange things so that when projecting onto a unit cell, the total light is the
same as the unprojected case, with either source individually brighter than the
detection threshold. So when not projecting, each unit cell could output, say,
4× the photons needed for detection by the neighbors, but when the projector is
on, each unit cell could output only 2× the threshold, with the projector also outputting
2× the threshold, so that the total brightness is the same for both the projected
and unprojected cases. It is also important to remember that the construction
of the CA involves the use of a clock signal that indicates when each unit cell
is to sample its inputs, and this signal changes only after both photon sources
have fully mingled. This means that there can be no notion of which photons
“really” indicated the state, since the laws of physics guarantee that all
photons of the same wavelength are identical.