Epistemological Debate

This is a summarized version of a long debate I had on the
epistemology of conscious experience, or the debate between Direct Perception
v.s. Representationalism on the
PSYCHE-D mailing list in April - June,
2005. I have selected choice threads that led to interesting exchanges,
with links to the original messages.

This debate is interesting not only as a discussion of the nature of
consciousness, but also as a prime example of a paradigmatic
debate. In fact, this issue is perhaps the ultimate paradigm
debate, because the alternative paradigms present such radically
different inside-out-inverted views of the world relative to each
other, no wonder the opposite camps could never reach agreement even
on the meaning of terminology!

Regarding the most interesting discussion on projection geometry
related to visual conscious experience between (primarily)
Alex Green and Brian Flanagan, I highly recommend the
work of Steven Lehar. See
http://cns-alumni.bu.edu/~slehar/Lehar.html

He has written several books and a target article in
Behavioral and Brain Sciences on this and related topics, and
his work is not only insightful but (as you can see on his website)
extremely well illustrated with artistic color cartoons.

In fact there is no need for an internal observer of the scene, since
the internal representation is simply a data structure like any other
data in a computer, except that this data is expressed in spatial form
(Earle 1998, Singh & Hoffman 1998). For if a picture in the head
required a homunculus to view it, then the same argument would hold
for any other form of information in the brain, which would also
require a homunculus to read or interpret that information. In fact
any information encoded in the brain needs only to be available to
other internal processes rather than to a miniature copy of the whole
brain. The fact that the brain does go to the trouble of constructing
a full spatial analog of the external environment merely suggests that
it has ways to make use of this spatial data.

Sizemore:
I do not think that the absurdities of representationalism
go away so easily. It is becoming widely known that saying we see a
representation immediately raises the issue of how seriously to take
it. If it is a metaphor, then it is useless (unless we can show that
it is likely to somehow be literal), but if it is taken literally,
then it raises the issue of infinite regress.

So, the ploy is simply to say that the representation is information
and information is "used" by some other part of the nervous system. By
substituting some other word for "see" (here "use" is substituted) the
glaring absurdity is obscured. But the notion of "using information"
needs to be critically examined, and when it is, it can be seen to
raise the same issues as "garden-variety" representationalism. In what
sense do nervous systems "use information"? Does it mean any more than
events cause receptors to "fire" which cause other neurons to "fire,"
and so on? If not, then we are simply saying that somehow physiology
mediates the behavioral functions that we wind up calling "seeing,"
and "hearing," etc. and this only raises issues of infinite regress
and homunculi if representationalism is assumed. If it means more
than that, then we are probably back where we started. That is, in the
conventional view, we say that we see the world, and that we do so by
creating an inner copy that is seen. But we may easily say that
animals (human and non-human)"use information" in the environment
(colors, sounds, etc.) in order to behave in the ways that we observe
and that they do this by creating an inner copy of the information
which is what is "used." If "using" information in the environment
requires further "usage" than why doesn't "using" the internal
information?

The key issue is not the word "see" or "use", to describe how the
brain makes use of internally represented information, it is the
question of whether the internal representation is processed by other
internal processes in the brain, or whether it requires a copy of the
*whole brain* in order to "see" that represented data. Only the latter
formulation leads to the infinite regress.

If Sizemore claims that a picture in your brain requires a miniature copy
of your whole brain to "see" that picture, then surely this objection would
apply to *any* information represented in the brain, including verbal,
linguistic, and cognitive knowledge, all of which would require a miniature
copy of the whole brain to interpret or process that cognitive
information. Why is it that the homunculus objection is only raised
against pictorial data? What is so special about image data that requires a
homunculus to "see", when other data do not?

Now I acknowledge a profound philosophical issue here, that applies to
*any* explanation of mental function, that is, after we are done
explaining the mechanism of perception or cognition, there is the
question of how come we get to *experience* that information
processing. The mechanistic explanation of neural signals and sensory
processing says nothing of the experience of perception and why we
have it. Even if we have actual pictures in our brain, how is it that
we *experience* those pictures? Why doesn't a computer experience the
image data that it processes? That is indeed a profound issue, but
again, it is one that applies to cognitive and verbal information
processing just as much as to pictorial processing.

But whatever the explanation might be for the
"Ultimate question" of consciousness, the indisputable fact remains
that we *DO* in fact experience the operation of our brain during
mental processing.

Lehar >
The key issue is not the word "see" or "use", to describe how the
brain makes use of internally represented information, it is the
question of whether the internal representation is processed by other
internal processes in the brain, or whether it requires a copy of the
*whole brain* in order to "see" that represented data. Only the latter
formulation leads to the infinite regress.
< Lehar

GS:
I disagree. The key issue is whether or not what
has to be explained is simply placed inside the head
(as a category error). We ask "What is 'seeing'"? and
representationalism answers that "it is the 'seeing'
of a representation." It doesn't matter if the answer
is "it is one part of the brain 'seeing' the
representation located in another part." Or: "it is
one part of the brain 'making use of' a representation
in another." Or: "it is one part of the brain
"processing" the representation in another part."

Lehar >
If Sizemore claims that a picture in your brain requires a miniature
copy of your whole brain to "see" that picture, then surely this
objection would apply to *any* information represented in the brain,
including verbal, linguistic, and cognitive knowledge, all of which
would require a miniature copy of the whole brain to interpret or
process that cognitive information.
< Lehar

Glen Sizemore >
I do not only raise the issue with respect to "pictoral data." I am
critical of all metaphorical uses of 'representatation.' I put no
stock in 'information' being 'represented' in the brain.
< Glen Sizemore

Lehar >>
The key issue is not the word "see" or "use", to describe how the brain
makes use of internally represented information...
<< Lehar

Sizemore >
I disagree. The key issue is whether or not what
has to be explained is simply placed inside the head...
< Sizemore

I acknowledge that placing a representation in the head does not solve the
whole problem of how we see. It *is* however a *prerequisite* for being
able to see something that that something must be represented in your
brain. Is it not?

Sizemore >
I do not only raise the issue with respect to "pictoral data." I am
critical of all metaphorical uses of "representatation." I put no stock
in "information" being "represented" in the brain.
< Sizemore

Yow! That's pretty radical man! I'm not sure I can debate you if we don't
even share this much in common!

Sizemore >
I claim that explanatory fictions like "representations," and
earlier versions like "beliefs" and "knowledge," shed
no light on what is going on in the nervous system
... What does it mean for "information" to be "processed?" ...
< Sizemore

Wow! Thats pretty bizarre!

Do you agree with the robot metaphor, that people are like a robot
that receives sensory input, stores it in internal representations,
and processes that information in order to compute an appropriate
behavioral response? Do we at least agree on that metaphorical image?

If not, then I don't think we can have a meaningful exchange beyond just
agreeing to disagree.

Ok, NOW I understand where you guys are coming from! Its the Gibsonian /
O'Regan, "organism interacting with the environment" idea.

Well, in the first place, Representationalism has never been shown to be
false. Au contraire, mon ami, the *principle* of representationalism has
been actually demonstrated in robots that use video cameras for sensory
input, computations in a computer brain, and behavior by way of servo
actuators. Now admittedly current robots are pretty primitive
and "robotic", and very different from animal perception. But if anyone has
any ambiguity about the meaning of terms
like "information", "representation", and "processing", just look at a
robot and the concepts become perfecty clear.

Robots offer an *existence proof* that the *concept* of representationalism
is *feasable* at least in principle.

So lets have no more talk about not knowing the meaning of information or
representation or processing. Those terms are perfectly clear, and
obviously workable in principle.

Furthermore, in the absence of *compelling* evidence to the contrary, a
representationalist assumption is the most *reasonable* understanding of
perception, given the eye that appears to work like a video camera, the
optic nerve that sends data to the brain, given the complex wiring
suggestive of computation in the brain, and motor neurons from the brain
back out to the muscles as if to produce behavior. The representationalist
thesis comes directly from inspection of the wiring of animal bodies, that
looks for all the world *AS IF* it were a representationalist system! It
may not be, but until the strong contradictory evidence comes in, that *IS*
the most *REASONABLE* initial assumption!

...o0o...

Now, the Gibsonian / O'Regan / direct percepton concept, on the other hand,
has *NEVER* been demonstrated in ANY kind of artificial system, and there
is very good reason to believe that the whole concept is totally incoherent
and impossible *IN PRINCIPLE*!

How would you even build a robot that works by direct perception??? Would
you equip it with video cameras for eyes? If so, what do you do with the
data generated by those cameras? You can't send it to the brain, because
that would be computation and representation again, so we can save
ourselves the expense of video cable and computer brain. But what would
drive the servo-actuators? Where does that signal come from? And how does
the direct-perception robot project its experience out of its body into the
world so as to produce behavior *as if* it were actually *seeing* that
environment *without* representations and computations???

HOW WOULD IT BE DONE????

The concept is so vague as to be totally meaningless! Gibson himself
refused to discuss what gets sent from the eye to the brain, or what kind
of computations might occur in the brain. In fact Gibson even denied that
the retina records anything like an image! But he NEVER OFFERED ANY
ALTERNATIVE EXPLANATION for how behavior works besides a few vague
mumblings about being tuned to invariants in the environment. Gibson spoke
as if the computation of perception occurs OUT IN THE WORLD, rather than in
the brain. But there is no computational or representational *machinery*
out in the world, the only machinery is found in the brain, right where the
sensory nerves terminate, exactly AS IF the sensory organs were sending
data to the brain for processing!

As for O'Regans concept of using the external environment as a
representation of itself, that idea is DEMONSTRABLY FALSE, because probing
the world with visual saccades, especially in the monocular case, is
**nothing like** accessing a memory, internal or external, because every
saccade presents only a two-dimensional pattern of light. The three-
dimensional spatial information of the external world is **by no means**
immediately available from glimpses of the world, but requires **the most
sophisticated** and **as-yet undiscovered** algorithm to decipher that
spatial information from the retinal input.

Secondly, O'Regan's concept of direct perception is totally inconsistent
with the *phenomenal experience* of vision, where we do in fact experience
the world as a spatial structure, and we perceive individual saccades to be
located at the location in the global framework of space that we perceive
that saccade to be located.

Thirdly, the absurdity of O'Regan's concept is highlighted by the condition
of *visual agnosia* (specifically, *apperceptive* agnosia) which is a
visual integration failure. An agnosic patient can see individual features,
but cannot distinguish a picture of a bicycle from a picture of
disassembled *parts* of a bicycle. They can see a wheel here, and a wheel
again, but they cannot tell whether the second wheel is not just a second
glance at the first wheel, or if it is a separate wheel, they cannot see
the spatial relation between the two wheels. In fact, the experience of
visual agnosia is *exactly as if* vision worked as O'Regan proposes, with
individual glances picking out features in the world in the absence of a
global framework or spatial representation to store or record those
features.

The condition of apperceptive agnosia is the absence of a visual function
whose existence O'Regan effectively denies!

So don't be complaining about the vagueness of terms like information,
representation, and processing. Those terms are **perfectly clear**, and
are demonstrable in **actual physical robots** that receive sensory input
and compute behavioral responses.

Instead, the real concern is the vagueness of terms like "active
interaction with the environment" and "responsive to invariants in the
environment" IF NOT by way of sensory input and internal representations.
What does that even ***MEAN***?

Until the *principle* of direct perception is demonstrated in a simple
robotic model, the concept is totally incoherent and ill-defined!

I have been lurking in the recent discussion but I want to say that,
modulo a few excess asterisks, Steve Lehar's message on
anti-representationalism seems to me to be exactly right. Theorists
who present themselves as denying the existence of representations
almost always, on closer inspection, turn out to be denying that
certain kinds of representations exist (Brooks) or to be denying that
representations are as plentiful or play as big a role as the
tradition would have it (O'Regan, Noe, Clark), or whatever. Some just
refuse to talk about central issues at all (Gibson). Mounting a
flat-out denial that there is anything in the brain that stands for,
indexes, refers to, even pictures items other than itself is a pretty
tough task. (Actually, Lehar might be a bit hard on O'Regan on this
score, though some of his rhetoric invites Lehar's kind of response.)

I think the argument against representation should be made based on
physics. Namely, what is the nature of the environment-organism
interaction allowing the brain to make a copy of it, how exactly can a
copy of this environment be made? Is it a tape-like recording? A CD?
And despite its enormous capacity, can the brain really afford to hold
a copy of the entire universe together with whatever extra brain power
is needed to analyze it? Judging by nature's preference for parsimony,
the answer is no.

Andrew Brook >
Steve Lehar's message on anti-representationalism seems to me to be
exactly right. Theorists who present themselves as denying the
existence of representations almost always, on closer inspection, turn
out to be denying that certain kinds of representations exist
< Andrew Brook

Yes, I am more accustomed to debating people who contest the notion of
*spatial* representations in the brain, given that neurophysiology has
not (yet) found "pictures" in the brain. It threw me for a loop to
find people like Glen Sizemore who deny representationalism
altogether! Even the most ardent supporters of Gibson's theories
generally take care to disclaim his most radical views (Bruce & Green
1987 p. 190, 203-204, Pessoa et al. 1998, O'Regan 1992 p. 473)
although they present no viable alternative explanation to account for
our experience of the world beyond the sensory surface.

Many of Gibson's observations on environmental affordances and invariants
were very valuable and insightful, even from a representationalist
viewpoint. In fact, it was Gibson's refusal to discuss physiology and
computation that released him from the burden of having to consider issues
of "neural plausibility", and that is why he dared to make such bold and
generally valid observations on the nature of perception.

But Gibson's profound epistemological error backed him into a corner which
is ultimately indefensible, which made him get defensive and dogmatic in
his later years, as often happens to those who commit themselves to
defending the indefensible.

Carreno >
I think the argument against representation should be made based on
physics. ... can the brain really afford to hold
a copy of the entire universe together with whatever extra brain power
is needed to analyze it? Judging by nature's preference for parsimony,
the answer is no.
< Carreno

I am viscerally sympathetic with this argument. Given what we know about
neurophysiology, it seems totally implausible that the brain could
construct a model of the world as rich and complex as our experience of it,
and maintain it in real time as we move about in the world. Carreno is
right: that does ideed stretch credulity to its elastic limit. But before
deploying Occam's razor we must first balance the scales, and take a full
accounting of the alternatives under consideration. For the alternative is
that we experience the world directly, as if bypassing the causal chain of
sensory processing. As incredible as representationalism might seem, the
alternative is even more incredible.

But before
deploying Occam's razor we must first balance the scales, and take a full
accounting of the alternatives under consideration. For the alternative is
that we experience the world directly, as if bypassing the causal chain of
sensory processing. As incredible as representationalism might seem, the
alternative is even more incredible.

< Lehar

Fallacy of the excluded middle. There are other options. One is the
one I sketched in the message from which Steve quotes: that our
representations give us access to the world itself, not just to the
end point of the representing process. Our brain has the capacity to
work its way down the causal chain to experience the kickoff
point. How we can do this is a wonderful mystery but that we do it
seems, to me at least, pretty much beyond question. When I open my
eyes, I see the world around me. This is to be in epistemic contact --
to see, know, experience, be conscious of -- the world, the part of it
in my immediate vicinity anyway, not any representation or construct
of mine.

Brook >
How we can do this is a wonderful mystery but that we do it
seems, to me at least, pretty much beyond question.
< Brook

Wonderful mystery indeed! Downright *miraculous*, wouldn't you say?

I mean, how would you demonstrate this in a simple robot model?

Experience or awareness of an object means having posession of
information about that object, color, shape, location, etc. Exactly
*as if* one had a miniature colored model of the object right there
inside your brain. Except we *DON'T* ?

So how would this work in a robot model? The video camera picks up a
2-D image which is sent to the computer brain, that extracts a few
features and performs a few computations, and suddenly, miraculously,
a three- dimensional colored data structure appears--not in the
computer brain in some kind of holographic 3-D imaging mechanism at
the end of the causal chain, but right back out there in the world!
With NO high-tech holographic imaging machinery involved!!! The image
just appears out there, disconnected computationally from any of the
hardware of the robot.

And when the robot closes its lens covers, the model out there
DISAPPEARS! As if it were causally connected to the computational
hardware downstream of the video signal, except it *ISN'T!*

I say again: Until the *principle* of direct percepton can be
demonstrated in an artificial sensory system, the whole idea is
completely implausible and ill-defined. I don't mean anything fancy,
just a simple demo like the representationalist robot. *HOW* does the
robot "work its way down the causal chain to experience the kickoff
point" ???

I'm not sure what's going on here. We should be on the same side. Here
is why I think that some form of direct realism is virtually
unrejectable. When I open my eyes, I see a chair just as directly as
when I kick it, I kick a chair. In both cases, it is the chair that I
am in contact with. In neither case am I in (interesting, relevant)
contact with any intermediary. And -- here is why the view is
essentially unrejectable -- if you say, in either case, 'the contact
is not direct', then I will invite you to tell me what you mean by
'direct'? What could be more direct than seeing or kicking an object
in plain view in front of me? If this is not direct, what would be?

Simple robot model? I sketched how at the end of my message. In the
same way that vision systems since Marr's have been able to extract
three dimensional objects from two dimensional arrays, our vision
system not only extracts threre dimensional objects but allows us
'reverse infer' down the causal chain to be directly aware of
them. What more do you want?

Brook >
I'm not sure what's going on here. We should be on the same side.
< Brook

Yeah, thats what *I* thought when you chimed in to reject the extreme
Gibsonian (Sidemorian) version of direct perception! I guess that this
issue is not a clean binary choice, but there are a number of
intermediate positions between direct perception and
representationalism.

Brook >
Here is why I think that some form of direct realism is virtually
unrejectable. When I open my eyes, I see a chair just as directly as when I
kick it, I kick a chair. In both cases, it is the chair that I am in
contact with. In neither case am I in ... contact with any intermediary.
What could be more direct than seeing or kicking an object in plain view in
front of me? If this is not direct, what would be?
< Brook

Ok, stand in front of a chair, and before you kick it, touch your
finger to one eyeball (through the eye lid) and push it gently to one
side, until you see a double image. Now KICK! Now you see TWO chairs,
and TWO feet kicking them! Which one is the "real" chair and the
"real" foot? And what is the actual objective location of that chair?
If this is not IN-direct, what would be?

Brook >
1. Simple robot model? I sketched how at the end of my message. In the same
way that vision systems since Marr's have been able to extract three
dimensional objects from two dimensional arrays, our vision system not only
extracts three dimensional objects but allows us 'reverse infer' down the
causal chain to be directly aware of them. What more do you want?
< Brook

Au contraire! Marr's vision model is entirely representational. From the
moment the image registers on the (synthetic) retina, all computational
processing of that image occurs inside the computer, or inside a brain.
Nothing gets projected out into the world again! There is *NO* "reverse
inference" going on here, but a feed-forward progression of *forward*
inference, from input stimulus to internal mental model. The computer never
has access to *any* information that is not explicitly represented in the
machine. That is an indirect representational algorithm!

But Gibson's profound epistemological error backed him into a corner which
is ultimately indefensible, which made him get defensive and dogmatic in
his later years, as often happens to those who commit themselves to
defending the indefensible.

Yet I could nowhere find a clear statement of what Gibson's "profound
epistemological error" is presumed to be.

For sure, Gibson avoided questions of mechanism. But that hardly seems
to be an epistemological error.

I am quite sure that Gibson understood the physics and biology, and
recognized that stimulation of retinal cells and the transmission of
signals on the optic nerve were part of the causal processes involved
in that direct perception.

Rickert >
But I am quite sure that Gibson understood the physics and biology, and
recognized that stimulation of retinal cells and the transmission of
signals on the optic nerve were part of the causal processes involved in
that direct perception.
< Rickert

Actually, Gibson made his views on the role of the retina perfectly clear,
and they were very much at odds with the consensus view on it.

"The very idea of a retinal pattern sensation that can be impressed on the
neural tissue of the brain is a misconception, for the neural pattern never
even existed in the retinal mosaic. There can be no anatomical engram in
the brain if there was no anatomical image in the retina. The retina jerks
about. It has a rapid tremor. It even has a gap in it (the blind spot). It
is a scintillation, not an image. ... The whole idea stems from the
persistent myth that there has to be something in the brain that is
visible, and from Johannes Müller's assumption that the nerves telegraph
messages to the brain."

The reason why Gibson denied that the retina records an image and
transmits it to the brain, is that to even allow this much
representationalism in the visual process is to acknowledge that the
principle behind representationalism is perfectly feasable, and that
the first stage of visual processing is apparently representational.

Steve Lehar said (among much else):
Ok, stand in front of a chair, and before you kick it, touch your finger
to one eyeball (through the eye lid) and push it gently to one side,
until you see a double image. Now KICK! Now you see TWO chairs, and TWO
feet kicking them! Which one is the "real" chair and the "real" foot?
And what is the actual objective location of that chair? If this is not
IN-direct, what would be?

I respond:
Well, it is still the chair that we see -- and we have extremely
reliable means for sorting out when the perceptual medium is working
well and when it is distorting things. All I want to insist on is that
when we see, we see the world (most of the time). We see *via* various
media, but we not aware of these media (except when things go wrong or
we otherwise pay attention to them), we are aware of the things in the
world that kick them into action. That's all.

Try another way:
of course there is intermediate *machinery" but that
does not mean that the resulting *perception* or *consciousness* is
indirect, i.e., by inference from something in the head.

Steve:
Unless, that is, you mistake our experience for a direct view of the
world itself, bypassing the chain of sensory processing, which in
impossible in principle!

Me:
Why should a 'direct view of the world' have to bypass sensory
processing? It proceeds via, though, sensory processing. What makes it
direct is that it makes us aware of the world, not the processing. How
it does this is, at this point in history, mostly wide-open.

Steve:
It is extremely difficult to banish the last vestiges of naive realism
from our philosophy.

Me:
I don't know what naive realism is but I certainly do not want to banish
realism because it is true! (IMHO, of course.)

Ok, we've been round and round the direct perception v.s.
representationalism debate enough times to see that nobody is about to
change their minds, no matter HOW eloquent or persuasive the arguments
are on either side. Why is this so?

This is a sure sign of a *paradigm* debate!

As Kuhn explained, (Kuhn 1970 The Structure of Scientific Revolutions)
there is a profound difference between theories and paradigms, and how
to debate them. The problem is that what we are debating is not a
question of theory, like the question of whether the representation in
the brain is digital or analog, or whether time-to-collision
information is available from the optic array, which would be resolved
by the normal rules of logic and evidence. But in this case what we
are debating are the *foundational assumptions* with which we come to
the debate in the first place. Debates between paradigms tend to go
round and round in futile circles, because the participants are
debating from different foundational assumptions. We take our
foundational assumptions as a *given*, as obviously self-evident
*fact*, and from that perspective the opposing paradigm appears
patently absurd, it makes us wonder how intelligent, educated people
could possibly defend such an absurd and indefensible view.

But if alternative paradigms are to be fairly evaluated, it is
necessary to temporarily and provisionally suspend one's own
paradigmatic assumptions, (a feat that many find impossible to do) and
accept the assumptions of the alternative paradigm **as if they could
actually be true**. Only then can the competing paradigms be fairly
compared, not on the basis of the perceived incredibility of their
initial assumptions, but on the overall coherence and self-consistency
of the world view that they implicate in total.

In the case of our debate, we hear some state that it is "obvious"
that we experience things directly, while others state that it is
"obvious" that perception is indirect. If we begin with either of
those assumptions, we are sure never to reach agreement. In the case
of the historical debate between an earth-centered or sun-centered
cosmos, the earth-centered people argued that the idea of the whole
earth with all its mountains and forests and oceans spinning and
flying through space is so absurd and incredible on the face of it,
that it does not matter what evidence you might cite, they would never
be convinced!

There is a parallel with the current discussion, because as in that
ancient debate, there is a certain asymmetry in the two views: one
alternative is the "naive" view, in the sense that that is the view
that we adopt by default, even before giving the question any serious
thought, while the other view appears initially to be patently absurd.

In paradigmatic debates, the argument that one view "seems incredible" is
no valid argument. Many of the greatest discoveries of science seemed
initially to be so incredible that it took decades or even centuries before
they were generally accepted. But accepted they were, eventually. And the
reason why they were accepted was not because they had become any less
incredible. Facts such as the immensity of the universe, and its
cataclysmic genesis from a singularity in space and time, as well as the
smallness of the atom, or the bizarre properties of quantum phenomena, are
just as incredible today as they were when they were first discovered. And
yet all of these incredible theories have taken their place in the realm of
accepted scientific knowledge, not because they have become any less
incredible since they were first proposed, but because the evidence for
them has been irrefutable. In science, irrefutable evidence triumphs over
incredibility, and this is exactly what gives science the power to discover
unexpected or incredible truth.

There is an asymmetry in this debate: all representationalists were once
naive realists, whereas most direct perceptionists have never been
representationalists. I can tell you that I find representationalism **just
as incredible** as any of you do. It is absolutely incredible that my
physical skull should be larger than the dome of the sky. And it is
incredible that the brain can construct and maintain a real-time volumetric
moving image of the world with the rich detail and fidelity of the world I
see around me. Given current knowledge of neurophysiology, that appears to
be absolutely incredible!

But before we deploy Occam's Razor, we must first balance the scales and
take a full accounting of the alternatives under consideration. For the
alternative is that we can somehow become aware of objects and surfaces in
the external world *without* the mediation of sensory processing and
internal representations. That, in my view, is not just incredible, it is
incoherent, as demonstrated by the fact that nobody has ever, or could ever
possibly build a robot that can demonstrate the *principle* behind direct
perception in a simple model.

I am therefore suspicious of direct perceptionists who focus exclusively on
the incredible aspects of representationalism, without also acknowledging
the incredible aspects of direct perception. I can accept someone who
argues that both views appear incredible, but that they consider this view
to be somewhat less incredible than that one. That is a valid and
reasonable position. But anyone who does not see the profound problems in
direct perception (as I see the profound problems in representationalism)
is suspect of being a paradigmatic partisan, that they accept one view as
plainly obvious, thus requiring no further proof, while the other appears
patently absurd no matter what the evidence. One suspects of such people
that they never really understood the alternative position enough to give
it any serious consideration.

To those people I implore that they entertain the *possibility* that they
may perhaps be mistaken.

"There are more things in heaven and earth, Horatio, than are dreamt of in
your philosophy."

Ok, we've been round and round the direct perception v.s.
representationalism debate enough times to see that nobody is about to
change their minds, no matter HOW eloquent or persuasive the arguments are
on either side. Why is this so?

Steven perhaps sees this as a problem. I don't. While I tilt toward
the direct perception side, I am not all all concerned that Steven and
Alex are strongly committed to representationalism. We don't need to
put all of our eggs in the one basket. Science is best served when a
problem is studied from several different perspectives. And may the
best perspective (whichever that is) win.

There is an asymmetry in this debate: all representationalists were once
naive realists, whereas most direct perceptionists have never been
representationalists.

I don't think I agree with Steven's assessment.

No doubt naive realism is pretty much the received view. But direct
perception is not the same thing as naive realism. I suspect that if
people with an ordinary science education were asked to choose
between Gibson's account of vision and Marr's account of vision, most
would prefer Marr's account. So the default view would be closer to
that of the representationalists than to that of the direct
perceptionists. There are subtleties to Gibson's theory that make it
a little difficult to appreciate.

But before we deploy Occam's Razor, we must first balance the scales and
take a full accounting of the alternatives under consideration. For the
alternative is that we can somehow become aware of objects and surfaces in
the external world *without* the mediation of sensory processing and
internal representations.

However, that is not the alternative. It seems that Steven can only
see one side of the paradigm shift.

The alternative, or one alternative, is that we can become aware of
objects and surfaces *with* the mediation of sensory processing, but
*without* the mediation of internal representations.

Incidently, I don't doubt that some of our interactions with the
world are mediated by internal representations.

Steven perhaps sees this as a problem. I don't. While I tilt toward
the direct perception side, I am not all all concerned that Steven
and Alex are strongly committed to representationalism. We don't
need to put all of our eggs in the one basket.

< Rickert

Are you serious? It does not bother you in the least that people believe
two mutually contradictory theories of perception? Surely the goal of
science is to discover which of those two views is right, and which is
wrong. I am passionately interested in that question!

slehar >>

But before we deploy Occam's Razor, we must first balance the scales and
take a full accounting of the alternatives under consideration. For the
alternative is that we can somehow become aware of objects and surfaces in
the external world *without* the mediation of sensory processing and
internal representations.

<< slehar

Rickert >
However, that is not the alternative. It seems that Steven can only see
one side of the paradigm shift.
The alternative, or one alternative, is that we can become aware of
objects and surfaces *with* the mediation of sensory processing, but
*without* the mediation of internal representations.
< Rickert

And exactly *HOW* would this be implemented in a simple robotic model? How
can a robot "become aware" (aquire immediate parallel access) to
environmental information *without* the mediation of internal
representations?

We hear these *words* loud and clear, but the *concept* behind the words
remains as clear as mud. Again, until you can demonstrate the *principle*
behind direct perception in a simple model, this concept is so vague as to
be virtually meaningless.

...oOo...

Given my comments at the launching of this "Theories v.s. Paradigms"
thread, isn't it perfectly clear what is happening here?

Isn't it perfectly clear that Rickert himself does not have a clear idea of
what he means by direct perception, or at least not clear enough to tell us
how to build the robot to demonstrate the principle. And yet he is
strangely blind to this gaping hole in his concept of perception. He
appears to be strangely blind to the profound problems inherent in the
concept of direct perception, he does not even acknowledge that any kind of
problem exists.

Indeed, Rickert exhibits all the signs of a paradigmatic partisan. He is
arguing from the initial assumption that perception is direct, but he is
unable to question that initial assumption itself. To him that assumption
appears so manifestly obvious that it requires no explanation.

I believe that the whole notion of direct perception is an elaborate
rationalization to try to make some kind of logical sense out of the
profoundly paradoxical observation that experience is outside of our head,
as if in the world itself, and yet vision is clearly representational, from
the retina on in to the brain.

But at a very deep level there is one small part of Rickert's mind that is
aware of this paradox, although his conscious mind is in denial over that
recognition. So, like Gibson, his response is to get more dogmatic and
emphatic about his certainty that his view is right, even though he cannot
explain it to us in any kind of detail, all he can do is to emphasize again
and again that you can perceive the world directly without representations.

Until Rickert can find it in himself to acknowledge the *problem* that we
are discussing, that is, the difficulty of actually implementing direct
perception as he conceives it, further debate is simply useless, because we
are arguing from different foundational assumptions. Rickert BEGINS with
the assumption that perception is direct, so he cannot contemplate the
possibility that it might not be.

"We can somehow become aware of objects and surfaces in the external world
*without* the mediation of sensory processing"

But that is not my view at all!!!! As I have said over and over and
.... OVER! (The caps may get me censored.) Nor, given that it is an
utterly implausible view, is it the view of most other direct
realists. A rough approximation of my view would be:

"we can somehow become directly, i.e., noninferentially, aware of
objects and surfaces in the external world *through* the medium of
sensory processing"

Furthermore, I don't think the difference between us is a paradigm
shift difference, I think it is a difference induced by an implacable
belief, totally untouchable by evidence or argument, that *if* there
is a medium of perception, *then* the resulting perceptions cannot be
direct. To which I respond:

You can define the word *direct* this way if you want, but if you
define it as it is usually defined, that we are aware of objects
around us, not just end-products of sensory processing in our brains,
then the inference needs argument and evidence to support it. And we
have seen not a single bit that the direct realists in the crowd have
not been able to deal with handily.

Lehar >>
"We can somehow become aware of objects and surfaces in the external world
*without* the mediation of sensory processing"
<< Lehar

Brook >
But that is not my view at all!!!! As I have said over and over and
.... OVER! (The caps may get me censored.)
< Brook

Well in that case you are not the target of that criticism.

Brook >
we are aware of objects around us, not just end-products of sensory
processing in our brains
< Brook

And exactly *HOW* would that occur??? If you cannot explain how this
concept might be implemented in an artificial robot, then the idea is
so vague as to be *meaningless*! Simply stating "over and over and
.... OVER!" that perception is direct is just not going to cut it!

Lehar:
There is an asymmetry in this debate: all
representationalists were once naive realists, whereas
most direct perceptionists have never been
representationalists.

Sizemore:
Obviously, you are asserting that "representationalism" is a view
battling for acceptance, but nothing could be further from the
truth. The fact is that it is one of the first academic positions
concerning human behavior to be taken seriously. It wasn't really
until the 20th century with behaviorism (and I include later
Wittgenstein here) that any serious challenge was mounted. Mental
"theories" of representation simply morphed seamlessly into
neurobiological "theories".

I agree with Glen as against Steve on which view comes first. Though I
agree with Steve that some kind of inchoate realism would seem true to
most people who have never studied cognition, I was thoroughly
indoctrinated in indirect representationalism in school as though it was
the obvious and even the only conceivable point of view (Steve?). It
then took me years to battle my way out of that bewitchment and back to
a view that allowed me to accept what seems to me obviously true -- that
I see, touch, taste, feel, and smell things other than myself,
especially including other people. One good name for my view (he said
being deliberately provocative) would be: direct representationalism.

Which comes first? When we are children we begin with the most naive of
naive realism--that the world of experience is the world itself, viewed
directly where it lies.

Then we learn about the eye with retina and optic nerve that project to the
brain, an obviously representational system. At this point our mental image
of the problem goes into a bistable state. We understand the causal chain
of vision, but at the same time we observe that the experience, from the
end of the causal chain, jumps back out of your head again and appears back
out in the world, "like a stretched rope when it snaps", as Bertrand
Russell expressed it. This is the state in which direct realists become
permanently stuck.

Andrew Brook>
One good name for my view ... would be: direct representationalism.
< Andrew Brook

A very apt name for a befuddled epistemology! Experience is both at the end
of the causal chain, and it is also back out at its beginning. Vision is
obviously representational, at least as far as the retina (except for
extremists like Gibson and Sizemore) and yet at the same time perception is
direct, as if bypassing the retina and viewing the world directly again.
Paradoxically, both appear to be true simultaneously. "Direct
representationalism" indeed!

It is only by really taking representationalism seriously that the
epistemological paradox is resolved, although that comes at the cost of an
almost *incredible* neurophysiological hypothesis, that the brain is
capable of constructing a three-dimensional volumetric real-time moving
model of the external world with as much rich spatial detail as you see in
the world around you.

Even if you continue to find that hypothesis too incredible to swallow,
will you not at least admit the profound paradox inherent in the direct
perception view? As I said in the "Theories v.s. Paradigms" thread, if you
cannot even see that there is a paradox at all, then further debate is
really a waste of time.

We've beaten this direct perception v.s. representationalism debate nearly
to death, without much progress. And yet one view is right, and the other
is wrong. One day the conclusive evidence will come in that will finally
settle the issue once and for all. Maybe even in our own lifetimes! That
raises the question:

What would it take to convince you? What kind of evidence can you possibly
imagine that would settle the issue finally and conclusively? Or are we all
so dogmatic that we will go to our graves in obstinate denial no matter
what the evidence?

Just for the fun of it, let me hypotheticalize two alternative future
scenarios that once and for all *PROVE* the correctness of
representationalism, and of direct perception, respectively. Would the
paradigmatic partizans among us be convinced by these? I suspect the
responses might be illuminating.

H1: REPRESENTATIONALISM IS PROVEN!

It is discovered in the future that the brain is one giant resonator, just
humming with one global resonance through all its tissues, right down to
the spinal cord. This sets up volumetric spatial standing waves in the
various cortical areas, in patterns like three-dimensional Chladni figures.

Although the different cortical areas are connected, they are also
independent resonators, each one sustaining its own spatial standing wave,
as in a separate Chladni plate. And yet they are also coupled, so that the
pattern in one cortical area is coupled to the patterns in all the other
areas. If you modulate the resonance in one area, it has an influence on
the pattern in all the other areas simultaneously. This immediate parallel
coupling between resonating areas turns out to be the solution to
the "binding problem". If you *were* the standing wave in your brain (which
in fact you *are*), there would be no way for you to tell that your
resonance is distributed over different resonators, because the pattern of
your resonance would be unified.

Careful neurophysiological recordings reveal that the experience of
volumetric substance and void, solid matter surrounded by empty space, is
encoded by the *phase* of the vibration: positive phase within perceived
objects, and negative phase in the surrounding void. Experienced colors
correlate with a cylic phase representation (as in the NTSC color
television standard, for those who are familiar) which finally explains why
phenomenal color defines a circular space (the color circle) even though
the spectrum is linear. When a subject views a three-dimensional scene,
neurophysiologists can sample that volumetric experience in various parts
of the visual cortex, and actually "read" what the subject is experiencing
in any portion of his visual space by the resonance in that part of his
cortex.

When a person turns their head, or moves about in the world, the spatial
image in each of the cortical areas rotates and translates in synchrony,
like the images in an array of television sets in a shop display that are
all tuned to the same channel.

If the mapping of experience in the brain were decoded so that *every
aspect* of experience could be deciphered from the outside with appropriate
probes, which revealed a complete world of experience all encoded inside
your brain, and if neurophysiologists could transmit signals into the
visual cortex and predictably cause the subject to experience a red square,
or a blue circle, or whatever, by using the right Fourier code, would THAT
be sufficient to finally convince the doubting direct perceptionists out
there?

I suspect not.

Ok, then lets switch to hypotheticalization #2

H2: DIRECT PERCEPTION IS PROVEN!

Uh, here I have a little more difficulty imagining the "experimentum
crucis" (Oops! My paradigmatic partizanship is showing!) But lets give it
a try regardless.

In the future it is discovered that there are in fact NO representations in
the brain! Even the image on the retina is not really an image as such,
transmitted from the eye to the brain, but instead, the eye is a tool for
active exploration of the environment that detects environmental
invariances OUT IN THE WORLD where the objects of perception reside.

This bizarre notion is finally proven beyond a shadow of doubt with the
invention of the "experience meter", a device tuned to some quantum-
mechanical cat-in-the-box state of existence in the world, so that it can
detect when a material object is being experienced by someone. If you point
the experience meter at an object in front of a person, the meter lights up
when that person's eyes are open, and blinks out when their eyes are closed.

The meter has a screen that shows the information content of the subject's
experience--monochrome for people who are color blind, full color for
people with normal vision. When a subject with visual agnosia views an
object, the experience meter displays a chaotic jumble of ever shifting
visual features instead of a coherent image. There is a peculiar void, or
loss of signal from the rear faces of objects that are not exposed to the
subject's direct view, as well as for background objects that are occluded
by foreground objects. This finally proves that experience is not some
mysterious non-existent entity, but a real physically measurable quantity
that exists out in the physical world, not in a person's brain.

The experience meter also records *affordances*. When a subject views a
chair, there is a measurable "sit-onable" affordance detected that appears
superimposed on the chair, and that affordance has the mysterious property
that it has a causal influence on the subject's motor system, so as to tend
(when circumstances are right) to make the subject actually move toward the
chair and attempt to sit on it. Furniture manufacturers use the experience
meter to objectively measure the "sit-onable" appeal of various furniture
designs. Neurophysiologists map out in exquisite detail the relation
between the measured affordances, and the spatial influence that they exert
on the various muscles of the body, by some kind of mysterious action-at-a-
distance from the affordance out in the world, to the muscles inside the
body.

Would THAT be sufficient to finally convince the doubting
representationalists out there?

Steve, absolutely the right approach, ingeniously carried out. Trouble
is, it does not address the issue between, for example, you and me. I
can't imagine anything that would 'prove' direct perception *as you
describe it*, anything compatible with what we now know about the brain,
anyway. But within representationalism, there are two houses.

There is the house of those who think 'If a representational medium is
present, the results can only be indirect perception/consciousness.'

And there is the house of those who think, 'The right kind of
representational medium lets us go right through it all the way to the
world itself, so that the result is direct perception/consciousness.'

For me, this debate between direct and indirect representationalists,
which is within representationalism, is the interesting one.

Steve, absolutely the right approach, ingeniously carried out. Trouble
is, it does not address the issue between, for example, you and me. I
can't imagine anything that would 'prove' direct perception *as you
describe it*, ... But within representationalism, there are two houses.

[1]:
There is the house of those who think 'If a representational medium is
present, the results can only be indirect perception/consciousness.'

[2]:
And there is the house of those who think, 'The right kind of
representational medium lets us go right through it all the way to the
world itself, so that the result is direct perception/consciousness.'

< Andrew Brook

Well then can you imagine the "experimentum crucis" that would select
between THOSE two alternatives?

Lehar has conflated conceptual and empirical issues. The only slight
difference is that he considers future experiments, whereas most
cognitivists claim that representationalism has already been
proven. Once again, the notion that psychological phenomena require
current representations and stored and retrieved representations (as
well as a host of related notions like expectation, unconscious
inference, unconscious rule-following, etc. etc.) for their
explanation is an assumption, as is the view that they do not.

As long as these views remain assumptions, and it is not clear that
they can ever be anything else, philosophical debate and conceptual
analysis is the only avenue via which they can be compared. Did Lehar
not, in fact, argue this himself when he invoked Kuhn a couple of days
ago?

No one has this stuff worked out by any conceivable stretch of the
imagination, but, quite bluntly, the representationalists think that
they do and this faith is, I assert, largely a product of the
ubiquitous, literal, usage of real representations such as photographs
and paintings etc. It is bewitchment by metaphor.

But even paradigmatic issues can be determined by experiment, at least in
principle. The earth-centered cosmos has now been conclusively rejected by
the "experiment" of flying around the back side of the moon without
colliding with the crystal sphere that supposedly supports it in space.
Although this experiment was technologically not feasable in Ptolomy's
time, Ptolomy and Copernicus could have predicted the different outcomes
of this "experimentum crucis" based on the two theories.

Any "theory" that is *NOT* testable by any experiment, even in principle,
is not a scientific theory at all, but a pure *belief*. For example the
existence of immaterial souls, if they are by definition undetectable by
physical means.

If you cannot describe an experiement that could prove direct perception at
least *in principle* with some kind of future technology, then direct
perception is a *belief* not a theory, since it predicts nothing different
than the alternative representationalist hypothesis.

But the truth is that the principle of representationalism *is*
demonstrable in a simple robotic system, while the principle of direct
perception remains as mysterious as the immaterial soul! After all these
rounds of debate, we still have no idea how such a system could possibly be
built in a real physical system.

...If the mapping of experience in the brain were decoded so that *every
aspect* of experience could be deciphered from the outside with appropriate
probes, which revealed a complete world of experience all encoded inside
your brain ... would THAT be sufficient to finally convince the doubting
direct perceptionists out there?

I suspect not.

< Lehar

You are right. That would not convince me.

Lehar >

H2: DIRECT PERCEPTION IS PROVEN!

[some of the details snipped]

This bizarre notion is finally proven beyond a shadow of doubt with the
invention of the "experience meter", a device tuned to some quantum-
mechanical cat-in-the-box state of existence in the world, so that it can
detect when a material object is being experienced by someone. If you point
the experience meter at an object in front of a person, the meter lights up
when that person's eyes are open, and blinks out when their eyes are closed.

< Lehar

This would convince me that I should consult
the Amazing Randi, so
that he can uncover and debunk the trickery involved in the
demonstration.

The trouble with Steven's two examples is that they are both
quite implausible. They look like sleight of hand parlor tricks.

A scientist needs better evidence than that.

To convince me, I need a detailed account of the relevant processes.
This should, preferably, be at the level of information processing.
It should plausibly account for all of the unresolved questions. And
it should be supported by empirical evidence.

-NWR

[Not published on PSYCHE-D. Censored?]
From: Steve Lehar
Date: 4/22/2005
Subject: What would it take to convince you?

Neil Rickert >
You are right. That would not convince me.
< Neil Rickert

Well then is there *ANY* possible future experiment that would prove it to
your satisfaction? If not, then you are a hopeless paradigmatic partisan.

Rickert >
The trouble with Steven's two examples is that they are both
quite implausible. They look like sleight of hand parlor tricks.
A scientist needs better evidence than that.
< Rickert

What *kind* of evidence are you talking about? Describe the experiment!

Rickert >
To convince me, I need a detailed account of the relevant processes.
This should, preferably, be at the level of information processing.
It should plausibly account for all of the unresolved questions. And
it should be supported by empirical evidence.
< Rickert

Notice that the direct perceptionists are always long on high sounding
verbiage and vague concepts, but very short on specific mechanisms and
mechanical details.

They cannot describe a simple robotic system that would demonstrate the
*principle* of direct perception in an actual physical system, and they
cannot describe a future experiment that would prove it one way or another.

All this just confirms my original suspicion that the concept of direct
perception is no *theory* at all, but merely a series of vague
rationalizations used to justify their naive realist intuitions. It is
impossible to pin them down on *any* kind of specifics.

Lehar:
But even paradigmatic issues can be determined by experiment,
at least in principle. The earth-centered cosmos has now been
conclusively rejected by the "experiment" of flying around the back
side of the moon without colliding with the crystal sphere

Sizemore:
This only shows that "paradigmatic issues" are not
necessarily "conceptual issues." Besides, you yourself emphasized the
incommensurability of "opposing paradigms."

Lehar: Any "theory" that is *NOT* testable by any experiment, even
in principle, is not a scientific theory at all, but a pure
*belief*.

Sizemore: But, as I have pointed out, the issues in question are
NOT theoretical issues. They are conceptual issues. They concern the
ASSUMPTIONS that underlie theory. And the empirical side of things is
frequently distinct from both theory and concepts

Lehar: If you cannot describe an experiement that could prove
direct perception at least *in principle* with some kind of future
technology, then direct perception is a *belief* not a theory, since
it predicts nothing different than the alternative representationalist
hypothesis.

Sizemore: Yes. Whether or not there are representations that are
stored and retrieved etc. etc. are [not] beliefs, they are
assumptions.

Lehar:
But the truth is that the principle of representationalism *is*
demonstrable in a simple robotic system, while the principle of direct
perception remains as mysterious as the immaterial soul! After all
these rounds of debate, we still have no idea how such a system could
possibly be built in a real physical system.

Sizemore:
I don't think that representationalism is "demonstrable
in a simple robotic system". ... showing orderly relations between
"the world" and a "pattern of neural activity" is not "proof" of
representation. Representation, like "computation," is defined by its
functional relation to the behavior of an animal, human or
otherwise. To invoke "representation" is to invoke behavioral
characteristics of whole animals, hence the charge of homunculism.

An alternative view is that perception is behavior, and that
behavioral function is mediated by physiology. Unless all "mediation"
is "representation" (and I argue it is not) this simple statement
constitutes the beginning of a scientific approach. You are correct
that "we still have no idea how such a system could possibly be built
in a real physical system," and we have representationalism to thank
for the wild-goose chase that constitutes much of "cognitive
neuroscience." We understand, in some complete sense, behavior at
about the level of habituation of the gill-withdrawal reflex in
Aplysia, and maybe some classical conditioning of the system (and
despite the language that Kandel uses, there is little to be called
"representation" here, unless everything orderly is
representation). Little wonder that "we still have no idea how such a
system could possibly be built in a real physical system," if the
system in question is the behavior we call seeing and hearing etc.,
and the other behavior of which it is a part. We know some of the
behavioral observations that need to be explained, at least to the
extent that some portions of psychology actually demonstrate
behavioral regularities in individual subjects, but we do not yet know
how physiology mediates such behavior. But to depend on the obviously
category-error-ridden conceptual muddle that constitutes
representationalism is no solution.

The question whether psychological phenomena require internal
representations is not just a conceptual question, as Sizemore
suggests, it is also an empirical question that will one day be
confirmed one way or the other experimentally, as soon as we figure
out the code of the brain. Because the theory of direct perception
states that there is no need for internal representations, or in the
softer version defended by Andrew Brook, the internal representations
do not need to encode ALL of the information of experience, because
SOME of that information can be perceived directly *through* the
representation (whatever that means). In any case, if it is discovered
that the brain actually *does* explicitly encode *ALL* of the
information in our experience, that will pretty much prove direct
perception to be false.

Of course there will still be dogmatic defenders of direct perception even
after that discovery, such as Glen Sizemore who tells us that he would not
be convinced even if representations *are* found in the brain, because we
would still not understand how we *see* those representations. (But then
neither does Direct Perception tell us how we can *see* the world *without*
representations, so the mystery of experience remains regardless.)

But you can never convince *everyone*. After all, there are still people
who believe in God and intelligent design, as opposed to evolution, despite
the overwhelming evidence to the contrary.

The *only* kind of paradigm that remains purely conceptual, in Sizemore's
usage, are theories that make no testable predictions whatsoever. For
example a version of direct perception that posits that there *are*
complete and explicit representations in the brain that encode *all* the
information of experience, but that perception is still direct, and does
not actually *use* those representations. Or the theory that God did design
the world, but that he used evolution has his mechanism of creation. Those
kinds of theories are indeed purely conceptual, not at all empirical,
because the experimental evidence is identical for both alternatives. But
such theories are not theories at all, they are *beliefs*, and thus fall
outside the realm of science.

Is there *any* information in our experience that is *not* explicitly
represented in the brain? Like the information about the world that is
experienced *through* the representation instead of *in* it? If so, then
your concept of direct perception *through* representations could be tested
in principle by seeing whether that information is in fact encoded in the
brain or not.

If on the other hand you posit that *all* of the information in our
experience *is* explicitly encoded in the brain, but we still view the
world "directly" *through* that representation, then we are in perfect
agreement, that in veridical (non-illusory) perception we are viewing the
world "directly" in that sense. But in that case your theory becomes
indistinguishable from represntationalism. (Viewing "directly" *through* a
representation strikes me as a contradiction in terms.)

Maybe this will help situate my position. I agree with Glen Sizemore that what
is between us is mainly conceptual, not empirical. Another way to put this is
that we are not disagreeing about the facts of human information processing and
behaviour, we are disagreeing about how these facts should be interpreted, what
they entail for theory.

It is hard to tell whether your "direct representationist" position really
is conceptual, or whether it has empirical implications. It is very
slippery the way you describe it. But in any case, if it *is* purely
conceptual, and makes no predictions, then it is entirely vacuous. It is
like saying that "the representation is so good, it really feels as if I am
seeing the world directly". Well we all agree with that! But if your theory
does not make any predictions, then it is no theory at all! It is a belief,
and a rather vague and ill-definable one at that.

Glen Sizemore calls them "assumptions" rather than "beliefs". But the issue
is what are they assumptions about? If they are *untestable* assumptions,
such as the existence of immaterial souls, then they are not scientific
assumptions at all. If they *are* scientific assumptions they *must* make
some kind of predictions, even if they are only testable in principle.

In fact there is a very profound empirical issue that is wrapped up in this
debate, and that is the question of how one would construct a robot that
operates by the same principle as our own brain. Either it needs to be
equipped with a representation of the world, or it needs to be designed to
extract that information from the world directly. And the debate also has
profound implications for neuroscience, that is, should we even be *looking
for* representations in the brain, and should we expect those
representations to encode *all* of our experience, or only part of it?

It is exactly the profound implications of this debate for artificial
intelligence, philosophy, and neuroscience, that make the epistemological
question interesting in the first place!

if your theory/interpretation/whatever and mine are both
compatible with all the facts, they are *equally* untestable, at least
at the bar of the facts. As I said yesterday, I suspect that that is the
case.
Andrew

*IF* our theories are indeed both identical with respect to
predictions and future discovery of facts, then they are identical,
because you would claim, as I do, that every aspect of experience must
necessarily be explicitly encoded in the brain, and thus your theory
is indistinguishable from representationalism. Welcome to the most
reasonable explanation!

But then your rhetoric about seeing things directly, *through* the
representation amounts to no more than that the representation is so
damned convincing that it really *seems* as if we are viewing the
world directly, although you acknowledge that you are seeing it
"through" the representation, which is tantamount to saying that you
are seeing the representation, while having the vivid impression that
you see the world itself, as when watching through a television
monitor.

If on the other hand you continue to insist that seeing *through* the
representation is something different than observing the state of your
brain, while believing that you are seeing the world directly, then
you will have to explain what that actually *means* in real
down-to-earth terms with sufficient specificity that we would know how
to build a robot that sees directly by that same principle, or that we
could describe the experiment that would distinguish between the
two--for example that there is information encoded in the
representationalist view that is not encoded in the direct
representatioanlist view. We still do not know what you actually mean
by this term.

Doesn't it bother you to defend a view so adamantly and with such
conviction, while being unable to specify exactly what you mean by it? It
sure has me confused!

But then so do all the other attempts at rationalizing direct perception!

2. I have put a lot of effort into specifying my position in detail,
much more than you have put into specifying yours. That you either do
not understand what I am saying or do not believe that I could mean is
another matter.

I am sorry, the intent was not to intimidate, nor to belittle your
contribution. You have done a masterful job of articulating a very slippery
issue, and you have my deepest respect for your willingness to explain your
thinking very honestly and in excruciating detail. I am really very
interested to hear what you have to say.

My point was that I am truly baffled at how you perceive your own position.
You freely admit that you cannot describe a robot that operates by direct
perception, and that you cannot conceive of an experiment that would
differentiate between our two views. And now in the most recent exchange
you raise the level of ambiguity of your concept of direct perception one
more level by saying that you **don't know** whether your view suggests
that the representation in the brain encodes *all* of the information in
your experience or not. Given all that, can you understand my puzzlement
about exactly what it is you are proposing?

Contrast that with the powerful conviction and certainty with which you
argue your position, and it makes me wonder just what it is that you are so
powerfully certain *of*? As far as I can tell, you acknowledge *everything*
about representationalism, except for the one single statement that you
know for a fact that your experience is *direct*, *not* an experience of a
representation. Can you not see a contradiction here? The concept
of "direct representationalism" is a contradiction in terms, because
representationalism is by definition indirect, by way of a representation.
In all our long debate, I have never once heard you acknowledge this as any
kind of paradox. It need not be fatal; you might still argue that you
consider that paradox more tolerable than the incredible notion that the
whole world is represented in your head. That would be a reasonable and
understandable position. But do you not even see what we
representationalists see as a problem with your view?

And how can you maintain your view with such supreme conviction when you
cannot describe an experiment that would differentiate our views? Isn't it
in the very nature of scientific theories that they make predictions?
Shouldn't this at least dampen the magnitude of your conviction, or at
least your expectation that others be persuaded to join your faith? As I
explained earlier, I can see the profound problems in the
representationalist view, but in my view they are outweighed by the deeper
paradoxes of the direct perception view. But do you even see where we see a
problem in your position? Why do we not hear you at least acknowledge that
much?

Your arguments just confirm my initial suspicion that the theory of direct
perception is logically indefensible, but that people hold it with great
conviction simply because it *SEEMS* so obviously to be true, and the whole
theory of direct perception is just an elaborate rationalization of that
initial assumption. Can you disabuse me of that suspicion?

Brooks >
2. I have put a lot of effort into specifying my position in detail,
much more than you have put into specifying yours.
< Brooks

I have invited you to read Chapter 1 of my book (available on-line) but I
have not hear your commentary on it. I would be interested to hear your
reaction to it. What do you make of the "introspective retrogression"? I
would really like to hear what you have to say.

Lehar:
Because the theory of direct perception states that
there is no need for internal representations, or in
the softer version defended by Andrew Brook, the
internal representations do not need to encode ALL of
the information of experience, because SOME of that
information can be perceived directly *through* the
representation (whatever that means).

Sizemore:
Here is where Lehar ignores a point I have made several times. We
cannot identify "representation" with some observation of a
correspondence between two domains. If we did, then all
correspondences would be a matter of representation. That would make
all of science representation. ... Whole persons look at
representations and they "use" their eyes and their brain. How does a
"part of the brain" do this, and how do we justify "our" belief that
this could possibly make sense? ... Lehar may say that we don't see
the representation ... but the problem is that that is what we do with
literal visual representations, we see them. If the things we say
about literal representations do not apply to "brain representations,"
what does that say about "our" cavalier use of language?

Lehar:
The *only* kind of paradigm that remains purely conceptual, in
Sidemore's usage, are theories that make no testable predictions
whatsoever. ... But such theories are not theories at all, they are
*beliefs*, and thus fall outside the realm of science.

Sizemore >
As long as these views remain assumptions, and it is not clear that
they can ever be anything else, philosophical debate and conceptual
analysis is the only avenue via which they can be compared. Did Lehar
not, in fact, argue this himself when he invoked Kuhn a couple of days
ago?
< Sizemore

Sizemore >
Here is where Lehar ignores a point I have made several times. We cannot
identify "representation" with some observation of a correspondence between
two domains. If we did, then all correspondences would be a matter of
representation. That would make all of science representation.
< Sizemore

Whether or not something is a representation depends on whether somebody or
something uses it as a representation. A picture in a newspaper is only
representational when viewed as a depiction of something else, not when the
newspaper is used to line your bird cage. Voltages in a computer memory, or
patterns of activation in a retina, are not representations unless or until
some process further downstream interprets them as such, at which point
they become representations only to that process.

Sizemore >
Whole persons look at representations and they "use" their eyes and their
brain. How does a "part of the brain" do this, and how do we justify "our"
belief that this could possibly make sense?
< Sizemore

The retina simply responds to light, it does not experience any kind of
representation, just its own state. Lower brain functions interpret the
signal from the retina as a pattern of light, like the meaningless patterns
in an abstract hallucination. The next higher brain functions interpret
those patterns of light as meaningless objects in an illuminated scene,
like the view of an abstract sculpture. The next higher levels interpret
the pattern of objects as a meaningful scene, for example a view of a face.
And the highest levels of brain function interpret the face as someone you
know, and call up the appropriate response such as nodding or greeting the
experienced person. Only the last stage involves the *whole* visual brain,
although not necessarily the auditory, olfactory, or limbic functions,
which may or may not engage in any particular experience. The lower level
functions do not involve the whole brain, and in fact there are an array of
visual deficits caused by failures of various regions of the brain (e.g.
visual agnosia) that clearly demonstrate that the whole brain does not
always have to be involved in every kind of experience.

And likewise in synthetic vision. An image on a photodiode array is just a
pattern of voltages, it does not represent anything until the video camera
is plugged into a computer that interprets that signal as a pattern of
light, or an image. Further algorithms work on that data on the assumption
that it is a pattern of light, to detect presumed features in the scene,
and then further algorithms work on that feature data to extract presumed
objects from that presumed scene. Those are all representations, whether or
not there is any correspondence between two domains. If the lens cover is
on, then there is no pattern of light, although the rest of the algorithm
mistakenly interprets the signal as an image of light nonetheless.

Sizemore >
Lehar may say that we don't see the representation ... but the problem is
that that is what we do with literal visual representations, we see them.
If the things we say about literal representations do not apply to "brain
representations," what does that say about "our" cavalier use of language?
< Sizemore

The word "see" implies a viewer, and that does not apply inside the brain,
because the viewer is the whole brain. Instead, we experience the states of
our own brain, and when the whole brain directs its attention to an
external object (by way of its internal representations) then we call that
process "seeing". It is a fallacy to insist that experience necessarily
involves eyes and a separate viewer. But we've been over this ground once
before already.

Sizemore >
Some questions are empirical questions and some are conceptual. It may be
that conceptual questions may become empirical, but the fact of the matter
is that the notion that "representations" are necessary to explain
perception is simply an assumption.
< Sizemore

True enough, but it *is* a *testable* assumption, at least in principle,
and a good way to test it is by demonstration with a simple model system.
Representationalism is easy enough to demonstrate. Direct perception is
more difficult to demonstrate, because nobody has ever articulated the
concept with enough specificity to either build a model, or to make
predictions.

Ok, lets try the *ontological* approach to the Direct Perception
v.s. Representationalist debate. What is the ontology of experience?

First let us agree what we mean by experience. For example visual
experience is the colored three-dimensional volumetric world you see
around you when you open your eyes. This is distinct from the
objective external world in the fact that when you close your eyes,
the visual world disappears, or rather, it is transformed into a foggy
brownish space of indefinite extent, while the real world continues to
exist unaffected by the blinking of your eyes. Whether you are a
direct or indirect perception advocate, we can agree on the definition
of visual experience, even though representationalists believe it to
be located inside the brain, while direct perceptionists locate
experience out in the world beyond the retina.

In either case, visual experience takes the form of modulations of
color qualia across a volumetric space. For example a checkerboard
pattern is experienced as an alternating modulation of black and white
squares. What is the *ontology* of those alternating qualia? What is
it that flips from black to white and back again? We know that it is
an experience, and that experience is spatially extended in a
pictorial fashion, but is there anything in the external physical
world that corresponds to that experienced alternation? What is its
substance? Does it even have one?

The representationalist answer is that qualia are different states of
the physical brain, and thus they are located inside the brain. In
other words the brain must posess some continuous spatial medium
across which extends a pattern of alternating states. Whether these
states correleate with voltages, spiking frequencies, or some standing
wave representation, remains an open question at this point. But
experience has physical presence in the physical universe known to
science.

But what is the direct perceptionist's answer? What is the "stuff"
that changes color across space that you experience? And where is it
located? And is it in principle detectable by scientific means at that
location?

If my experience of a chess board is out there where the chessboard
exists, is the black and white pattern I experience the alternation of
the pigment in the paint on its surface, perceived directly? Or is it
the reflectivity of the surface experienced directly? Or is it the
intensity of reflected light experienced directly? Or is it a pattern
of activation in my retina? What is its ontology?

I think that direct perceptionists are uncertain about the ontology of
experience, they perceive it in a bistable manner, as being both an
external objective, and an internal subjective entity, and their
answers, when probed, flip back and forth between these two as if a
pattern in your brain could somehow be also outside of your head. The
whole concept of direct percption is founded on a profound
epistemological error, that we can in principle be conscious of things
which are not explicitly represented in our brain. Direct perception
states the *problem* of experience, it does not offer a *solution* to
it that can either guide the construction of a model of the concept,
or even an experiment to test the concept. The theory of direct
perception is every bit as mysterious as the property of consciousness
that it is supposed to explain.

The representationalist position is more coherent because it posits a
single ontology, that the modulations of the qualia of visual
experience are modulations of the physical state of your brain across
some spatial representational medium. And it makes the testable
prediction that that medium and its modulations will one day be
discovered and decoded in the brain. My vote is for a standing wave
representation using a Fourier code to produce moving volumetric
holographic images in the brain, and that those images correspond
directly to our experience.

Glen Sizemore will complain that there remains the problem of
experience, and why it is we have it when our brain is in certain
states. But if you accept that mind is a physical process taking place
in the physical mechanism of the brain, and you acknowledge that the
brain is conscious, then that already is an admission that a physical
process taking place in a physical system can under certain conditions
be conscious.

Besides, the mystery of experience, or why consciousness exists in the
brain, is by no means unique to representationalism. Direct perception
cannot resolve that one either. Representationalism at least offers an
account of the functional aspects of experience that can be expressed
in actual models and make testable predictions. Direct perception does
not even offer that much.

It's an ingenious argument. It seems that you could use that method
to prove that we don't eat food, we eat representations of food.
It makes you wonder how we get our nutrients.

You seems to be assuming the kind of Cartesian theater that Dennett
criticised. I don't agree with that. But even assuming a Cartesian
theater, you are misusing "experience". For example, my experience
in watching a movie includes my emotions and thought. It isn't just
what was played on the screen. Contrary to your assertion, I don't
locate experience out in the world. I don't consider it a thing. It
seems to me that treating experience as a thing is a category mistake.

The argument about blinking the eyes is interesting, because I think
that actually argues for direct perception. If there is some sort of
volumetric representation, then you would think your visual
experience would persist during a blink, perhaps slowly fading out.
May I suggest that the "foggy brownish space of indefinite extent" is
closer to what is represented.

SL >
But what is the direct perceptionist's answer? What is the "stuff"
that changes color across space that you experience?
< SL

There is no such stuff. You have confused the issue by your misuse
of "experience".

...

I seriously doubt that there is enough DNA in the human genome to
encode the hardware specifications that would be required for the
proposed system.

In another message ("Conceptual vs. Empirical Issues"), Steven wrote:

SL >
For you direct perception offers the "simplicity" of being intuitively
believable, whereas for me representationalism offers the "simplicity" of
being causally explanatory.
< SL

There is nothing explanatory about representationalism. Most
representationalists admit that they are unable to explain conscious
experience. The argument about an infinite regression of homunculuses
keeps coming up precisely because representationalism explains
nothing.

Whether a system uses direct perception, or is based on
representations, in an implementation issue, not an explanatory
issue. So lets stop the arguing, and wait for until there is enough
empirical evidence to answer questions about implementation details
in homo sapiens.

Rickert >
It's an ingenious argument. It seems that you could use that method
to prove that we don't eat food, we eat representations of food.
It makes you wonder how we get our nutrients.
< Rickert

No, just that we experience internal representations of the
external "nouminal" food that nourishes us.

Rickert >
you are misusing "experience". ... I don't consider it a thing. It
seems to me that treating experience as a thing is a category mistake.
Rickert

It seems there are a lot of these "category mistakes" in direct perception.
I see a spatial structure that is my experience, it is distinct from the
world itself, and yet we are not permitted to think of that spatial
structure as anything we can talk about. It seems that a lot of direct
perception involves prohibitions against certain concepts, as in
behaviorism, that fobade discussion of conscious experience. Curiously,
all of these forbidden concepts are the very things that reveal the
incoherency of direct perception.

Rickert >
The argument about blinking the eyes is interesting, because I think
that actually argues for direct perception.
< Rickert

I see. Because closing the eyelids makes the real world out there cease to
exist momentarily. Hmmmm...

Rickert >
If there is some sort of volumetric representation, then you would think
your visual experience would persist during a blink, perhaps slowly fading
out.
< Rickert

No, you would expect it to blink out immediately when the input data stream
is blocked, like the image on a photodiode array when you put the lens
cover on.

Rickert >
I seriously doubt that there is enough DNA in the human genome to
encode the hardware specifications that would be required for the
proposed system.
< Rickert

There I believe you put your finger on what I believe is the principal
reason why representationalism is generally not given serious consideration.

Rickert >
There is nothing explanatory about representationalism. Most
representationalists admit that they are unable to explain conscious
experience. The argument about an infinite regression of
homunculuses keeps coming up precisely because representationalism
explains nothing.
< Rickert

Well it does explain the *functional* aspect of vision, that is, how the
information of the world gets in to the computational hardware of the
brain. Direct perception does not even explain that much. And direct
perception does not explain experience either, it merely prohibits
discussion of it.

Rickert >
So lets stop the arguing, and wait for until there is enough
empirical evidence to answer questions about implementation details
in homo sapiens.
< Rickert

I think that is wise. We are not making any further progress in
understanding each other. With this last ontological argument I have
expended my last big arrow from my quiver. I will provide a summary
overview of all the arguments we have covered in these various threads
under the Subject:

We have had a full and informative exchange on the question of
Direct Perception v.s. Representationalism, and it seems we have
pretty much exhausted that topic by now, as we are now just repeating
the same arguments over and over again. So I will bow out of the
discussion in a major way at this point, although I will answer any
residual questions that people might have. I have no more major
arguments to make on the subject. Most sincere thanks to all who
have contributed, and to all those lurkers out there who have found
this debate worth following. If nothing else, it is in my view a most
fascinating topic, and resolving this issue once and for all would be
of the greatest significance for philosophy, psychology, and
neuroscience. I hope we have advanced that cause if only by a small
notch.

I will take this opportunity to summarize the whole debate as I see it
from the representationalist perspective. The origin of direct
perception is naive realism. It is almost impossible to shake the very
vivid impression that what we see in experience is the world
itself. But this concept is profoundly at odds with the
neurophysiology of perception, with sense organs that transmit
information to the brain, an obviously representational system. One of
these views must be right, and the other is wrong, unless as Andrew
Brook claims, the truth lies somewhere in between.

It turns out however that the direct perception view is incoherent,
whether in its pure or partial form, because it involves the organism
having knowledge of things which are not explicitly represented in its
brain. Proponents of direct perception, from Gibson onwards, must
sense some kind of a problem at least subconsciously, as seen in their
supreme confidence that their view is right, even though they cannot
articulate their position with sufficient specificity as to explain
how an artificial robotic system could possibly be built that operates
by direct perception. Even more surprising, they cannot seem to
formulate any possible future experiment that would resolve the issue
definitively one way or the other. And strangest of all, they appear
to have a peculiar blindness to this gaping hole in their view of
perception, as if they simply cannot understand the objections of the
representationalists. It would be one thing if they acknowledged that
there is some kind of paradox involved, but that they consider that
paradox more tolerable than the incredible representationalist
view. But direct perceptionists cannot seem to even bring themselves
to acknowledge that there is any kind of problem at all.

The more extreme form of direct perception espoused by Gibson is like
behaviorism: it is full of prohibited concepts and so-called "category
errors" that forbid one to acknowledge experience as a "thing", or to
recognize our experience as pictorial, or that it has spatial extent,
when those facts are plain for all to see as soon as they just open
their eyes. "Seeing is behavior", Glen Sizemore tells us, and there is
supposedly no meaning to words like "information" and "representation"
and "processing", concepts which have become part of our everyday
lexicon since the arrival of computer technology in our lives. One
gets the sense that direct perception is a religion rather than a
scientific hypothesis, as seen in Gibson's stark refusal to even
discuss sensory processing at all.

Andrew Brook's more moderate concept of "direct representationalism"
sounds at first more reasonable, because he allows for representations
in the brain, but paradoxically, he insists that we view the world
"directly" *through* the representations in our brain, a concept that
strikes me as a contradiction in terms. If our brain uses
representations, then of course our experience consists of those
representations, and we cannot view the world directly except by way
of them. When questioned more closely, Brook wriggles and squirms
until his explanation becomes almost identical to a
representationalist thesis, with the sole exception that he continues
to insist that our experience of the world is direct, although he
cannot even explain exactly what he means by that term.

Until the *principle* of direct perception can be demonstrated in a
simple robot model, the concept is so vague and incoherent as to be
essentially meaningless.

Conscious experience has an information content, and information
cannot exist without a physical medium or carrier to carry that
information. That medium can only be the brain, where the sensory
nerves terminate, and from whence the motor nerves originate.

A theory that makes no predictions about possible future discoveries
in the brain, is no scientific theory at all, but is more like an
article of faith.

Direct perception cannot explain the *ontology* of the vivid spatial
structure of visual experience, those volumetric objects bounded by
continuous colored surfaces that disappear when we close our
eyes. These are obviously a product of the brain, and yet they appear
out in the world. They can only be states of our own brain, and thus be
located in our brain.

Like Behaviorism, direct perception only survives by simply
prohibiting discussion of concepts like the information,
representation, and processing in the brain, and by prohibiting
discussion of the manifest properties of the vivid spatial structure
of experience.

The chief argument raised against representationalism is the question
of experience, and why we have it when our brain processes sensory
information. This is indeed a deep philosophical quandary. However we
know for a fact that experience does exist, and that the brain is the
organ of conscious experience. The only reasonable location for
experience is inside the brain as a representation.

We do not need "internal eyes" to "see" the representations in our
brain, we simply experience the structure of certain patterns of
energy in our brain directly. While this concept may seem deeply
troubling to some, it is nowhere near as troubling as the concept of
awareness of the world out beyond the brain, where there is no
computational or representational hardware to do the experiencing. Direct
perception does not offer any better explanation for the question of
experience, except by prohibiting discussion of
experience as a "category error".

O'Regans concept of probing the external environment as if it were an
internal memory is demonstrably false, because the three-dimensional
spatial information of the external world is by no means immediately
available from glimpses of the world, but requires the most
sophisticated and as-yet undiscovered algorithm to decipher that
spatial information from the retinal input.

The absurdity of O'Regan's concept is highlighted by the condition of
visual agnosia, a visual integration failure, because the condition of
apperceptive agnosia is the failure of a visual function whose
existence O'Regan effectively denies.

The question of direct perception v.s. representationalism is not a
conceptual issue that is beyond the reach of science, it is a very
significant empirical issue with profound implications for the nature
of perceptual representation in the brain, and for our attempts to
replicate the principle of perception in artificial robot models.

The phenomena of dreams and hallucinations clearly demonstrate that
the brain is capable of constructing vivid spatial experiences in the
absence of an external world available for direct inspection. Perception
is a guided hallucination, constrained by sensory input.

In the face of this overwhelming array of indisputable evidence, how
can intelligent, educated people continue to insist that our
experience of the world is direct? The answer is the very vivid
impression that our experience simply *appears* direct. Some people
just cannot bring themselves to accept the view towards which all of
the evidence inevitably points. This issue is the ultimate example of
a paradigm debate, because seeing things from the representationalist
perspective requires that one inverts one's entire epistemology to
recognize that the world which appears outside is actually inside
one's head. It is admittedly a difficult concept to swallow.

I have prepared an exerpted summary of this whole debate on-line, which
can be found at:

Steve Lehar has presented a strong defense of representation in his
summary. Unfortunately, as he himself has acknowledged previously, it
is unlikely to cause too many minds to change. Many people have
suggested that maybe there is a way to reconcile the opposing views of
direct vs indirect perception, and this seems desirable since the
absence of empirical evidence that would clearly determine the correct
position suggests that there is a possibility that both approaches
could be wrong, or both could be right. I submit that both could be
right.

...

In sum, yes there is a picture, but it is not a copy of the world. It
is, for all intents and purposes, the world.

Carreno >
there is a possibility that both approaches could be wrong, or both
could be right. I submit that both could be right.
< Carreno

The only form of direct perception that might be right is one that is
experimentally indistinguishable from representationalism, and thus makes
no testable predictions to distinguish between the two. That form of direct
perception however is indistinguishable from representationalism, because
it posits that every aspect of experience is necessarily replicated in the
brain.

The more extreme version of direct perception that prohibits
representations of any sort, cannot be right if representationalism is also
right.

Carreno >
In sum, yes there is a picture, but it is not a copy of the world. It
is, for all intents and purposes, the world.
< Carreno

But there are two aspects of the "picture", one that disappears when you
close your eyes, and the other that continues to exist unchanged. That
clearly places one world on the outside beyond your eyelids, and the other
one inside on this side of your eyelids. The failure to distinguish these
fundamentally different worlds is exactly the theory of direct perception.

Carreno >
if we stipulate that the world is indeed already there, then the advantage
of making a copy of it over taking it at face value is difficult to
elucidate.
< Carreno

It is very easy to elucidate. If "taking it at face value" means making a
sensory image of it and sending it to the brain for processing, then that
is representationalism, and "direct perception" would be wrong. If "taking
it at face value" means behavior and experience *as if* we had a copy of
the world in our brain, but we actually *don't*, then the causal loop
between perception and behavior is critically broken, and perceptual motor
function remains forever a deep dark mystery.

Rickert >
Steven indeed gave the representationalist perspective. It was
perhaps more a polemic than a summary.
< Rickert

Obviously it was a view from the representationalist perspective; feel free
to compose a summary from the "direct perception" perspective. And there
was an overall message to that polemic, which is that direct perception may
seem plausible enough when debated issue by issue, but it falls apart when
viewed in the aggregate, when one takes a "big picture" view as revealed in
the summary. Advocates of direct perception cannot even agree on what their
theory states, whether there are any representations in the brain or not,
or if there are, whether they encode all of experience or only some of it.
They cannot make testable predictions of future experiments that would
resolve the matter one way or the other, nor can they build a functioning
robot to demonstrate the *principle* behind the concept.

The motivation behind direct percepton on the other hand is perfectly
clear, it is motivated by the very vivid naive impression that what we
experience is the world itself, rather than an internal representation. In
that sense it is very much like the "animism" of the turn of the last
century whose advocates insisted that life is "something more" than just
chemical reactions, although they were unable to define what the added
ingredient might be, or how it would be detected in principle, or how it
could be implemented in a simple model.

Lehar >>
Until the *principle* of direct perception can be demonstrated in a
simple robot model...
<< Lehar

Rickert >
We should put this in perspective. There is no robot model that
demonstrates representational perception. There is no robot model
that credibly demonstrates any kind of perception.
< Rickert

Any robot model with a camera, computer, and servos, demonstrates the
*principle* behind representationalism, and thus clarifies concepts such
as "information", "representation", and "processing" in terms that are
perfectly clear to anyone who has used a computer. There is *no* such
simple demonstration of the *principle* behind direct perception, nor is
there even consensus among advocates what that term actually means, or what
it says with any specificity about the perceptual process, or how it would
be implemented in a robot model. It is quite extraordinary how dogmatically
and with such supreme confidence the advocates defend what is a pretty
vague concept.

Direct perception is entirely consistent with the neurophysiology
perception. It is inconsistent merely with a certain ideology of what
follows from these facts. Steve, saying what you do here shows as
serious an unwillingness to come to grips with what we are actually
saying as your earlier statement that we believe that representation
is outside the head. We believe that representation can be of or about
thing outside the head. But your statement is about ... the nature of
the vehicle, mine is about .... what it is about, what
it makes us conscious of. If you refuse to even try to grasp this
distinction, I hypothesis that you are letting ideology substitute for
evidence and argument.

Lehar >
It turns out however that the direct perception view is incoherent,
whether in its pure or partial form, **because it involves the organism
having knowledge of things which are not explicitly represented in its
brain.**
< Lehar

[The clause in asterisks] No, it does not ... and I have said why over
and over and over. What does it take to get you at least to
acknowledge that we believe what we say we believe? Go back over my
messages. I have addressed this point at least half a dozen
times. Again, I suspect you are letting unshakeable ideology go proxy
for looking at the arguments.

Brook>
Steve, saying what you do here shows as serious an unwillingness to
come to grips with what we are actually saying as your earlier
statement that we believe that representation is outside the head. If
you refuse to even try to grasp this distinction, I hypothesis that
you are letting ideology substitute for evidence and argument.
< Brook

You have yet to tell us whether the representations in the brain
necessarily encode *all* of experience or just *some* of it. This may
seem like an irrelevant detail to you, but for me that issue is
*everything*. Because if you acknowledge that some experience is not
explicitly represented, then you actually *ARE* claiming that part of
your experience is out in the world, not in your head, and I challenge
you to demonstrate *that* in a robot model. So are you herewith now
denying that *any* aspect of experience can exist without being
explicitly represented in your brain?

Lehar >>
**because it involves the organism having knowledge of things which are not
explicitly represented in its brain.**
<< Lehar

Brook >
[The clause in asterisks] No, it does not ... and I have said why over and
over and over. What does it take to get you at least to acknowledge that we
believe what we say we believe?
< Brook

Is that really what you believe? That all of experience *IS* necessarily
represented in the brain? But in that case your "theory" is
indistinguishable from straight representationalism. You can't have it both
ways, and then complain when I point out the weakness in either one of the
two versions which you refuse to choose between.

I say again, how can you feel such supreme confidence that your view is
right, when you cannot even explain to us what your view is? Could it be
that it is you who are letting ideology substitute for evidence and
argument?
Steve

Steve once again says that we have not explained our position. So let
me once again say what I have done. I have:

1. ... defined the position. Direct perception is being aware of
objects in the world, nothing short of them, and not by inference from
anything else of which I am aware.

2. ... given examples. If Steve is reading this, the state he is in a
a perfect example of direct awareness. He is aware of the words on the
screen directly, not by inference from anything else of which he is
aware. And he is aware of the words on the screen, not any
intermediary. He won't accept this as an example but has given no
reason for this refusal.

3. ... said why this position works, indeed is really the only game in
town -- no notion of anything more direct can even be articulated.

4. ... said that all this is perfectly compatible with there being a
complex information-process connecting brain to objects, with there
being representations, and so on. What matters here is that it is
crucial to distinguish between the representational vehicle, some
state of the brain I expect, and what that vehicle represents, what it
makes us conscious *of*, its object.

What more do you want?

If experiments cannot distinguish direct from indirect
representationalism, that most emphatically does *not* mean that the
indirect view is supported. It means that on this issue both views are
unscientific -- not selectable by evidence -- and the issue has to be
settled other ways. If A cannot be distinguished from B by evidence, B
cannot be distinguished from A either. 'Distinguish' is a symmetrical
relationship.

It should be obvious that robot models, as with Bach-y-Rita's
prosthetic devices, demonstrate direct and indirect
representationalism equally well -- at best. More likely, they tip the
scales in the direction of direct representationalism. Steve's reading
of the transition in B-y-R patients from experiencing the tickles to
seeing objects in the world is this:

Final comment: Steve is entitled to say,

"The motivation behind direct percepton on the other hand is perfectly
clear, it is motivated by the very vivid naive impression that what we
experience is the world itself, rather than an internal
representation. In that sense it is very much like the "animism" of
the turn of the last century ...",

only if he accepts the same judgment on his own view. My own view is
that civilized debate should avoid speculations about people's
motives. It is clear that Steve refuses to consider any position other
than his own but I wouldn't dream of speculating about why that is so.

Let me repeat. I have said all these things before. Nothing in this
message is new. I am not supremely confident that our position is
right but I sure would like it if Steve and others convinced that we
are wrong would try to understand the position, rather than
attributing to us over and over views that we don't hold, don't need
to hold, and would be silly to hold. Our position is as articulated in
this message. If you don't like it, say what you find to be wrong
about it.

Brook >
What more do you want?
...
Our position is as articulated in this message. If you don't like it, say
what you find to be wrong about it.
< Brook

This is the more that I want:

1: Predictions of possible experiments that would differentiate the
two positions. If there are no such experiments, then it is no theory
at all, and the default is representationalism, because...

2: Define what "direct representationalism" means in terms of an
actual mechanism--how would a direct representation differ from an
indirect one? We all understand indirect representationalism, that is
how robots work. But how would you build the direct
representationalist robot? How would it differ?

So far the only "definition" you have given us is "being aware of
objects in the world, nothing short of them, and not by inference from
anything else of which I am aware.". We understand the *words*, but
what does that actually *mean*? What difference would it make to our
*experience* if representationalism *was* indirect? What difference
would it make for neurophysiology? Your definition is perfectly
consistent with pure representationalism, that also gives a vivid
impression of perceiving directly.

Brook >
What matters here is that it is crucial to distinguish between
the representational vehicle, some state of the brain I expect, and what
that vehicle represents, what it makes us conscious *of*, its object.
< Brook

But that is just as true in indirect representationalism. The voltage
in a photodiode that represents the brightness of light, or the
electrical state of a retinal cone that represents a color component
of a point of light. The voltage gives information *of* the
brightness of light. But that information is nevertheless necessarily
inside the computer, or at least downstream of the photodiode where
the transduction occurs. What is the difference between saying 1: the
voltage gives indirect information about the brightness of light, and
2: that the voltage gives *direct* information *of* the brightness of
light measured *through* the representation. Is there a difference
that makes a difference? Or are we just playing with words? The onus
is on YOU to demonstrate that direct representationalism is different
in any way from indirect representationalism.

Brook >
If experiments cannot distinguish direct from indirect representationalism,
that most emphatically does *not* mean that the indirect view is supported.
It means that on this issue both views are unscientific -- not selectable
by evidence
< Brook

Wrong. Representationalism is the "obvious" concept that is thoroughly
understood, to the point that we have built representationalist
systems whose principles of operation are clear to all. Not only has
direct perception never been demonstrated, it has never even been
clearly defined sufficient to make predictions about possible future
experiments. If it makes no such predictions, then the default returns
to the concept that we *do* thoroughly understand, and that *does*
make specific predictions.

Brook >
It should be obvious that robot models, as with Bach-y-Rita's prosthetic
devices, demonstrate direct and indirect representationalism equally well
< Brook

Wrong. The Bach-y-Rita device has a human being in the loop to do the
perceptual integration, so the fact that they report having a spatial
experience outside of themselves, as also with any ordinary visual
experience, can mean either that experience is escaping their heads
and projecting into the world around them, *or* that a spatial percept
is being constructed in their internal representation of surrounding
space.

When I'm talking about a robot model I don't mean anything more complex
than, for example, a single photodiode and a circuit that interprets its
voltage as a brightness of light. That there is already a representational
system. How would it differ if it were built as a "direct
representationalist" system? Does it not already perceive brightness
*through* voltage? If there would be no difference, then the words "direct
representationalist" are meaningless.

Brook >
My own view is that civilized debate should avoid speculations about
people's motives. It is clear that Steve refuses to consider any
position other than his own but I wouldn't dream of speculating about
why that is so.
< Brook

I am sorry, I was aware that that might sound a little offensive. But the
point I was making is that the two sides of this debate are not
symmetrical, one is the "naive" view that we understand from earliest days
of childhood, while the other is admittedly incredible, or seemingly so,
which is why it remains so controversial. I can understand your position,
having been there myself, whereas I don't think you can even fully
conceptualize my position, it sounds so self-evidently absurd to you, that
it sounds like a theory that "black is white" or "up is down". And that
asymmetry in the debate is a significant factor, even if it remains (of
course) invisible to you. For example:

Brook >
... said why this [direct representationalism] work, indeed is
really the only game in town -- no notion of anything more direct can even
be articulated.
< Brook

and

Brook >
If Steve is reading this, the state he is in a a perfect example of direct
awareness. He is aware of the words on the screen directly, not by
inference from anything else of which he is aware.
< Brook

That does not distinguish the two alternatives, indirect perception also
predicts an experience *as if* it were direct. How does Andrew think
experience would appear if representationalism were indirect? It would
appear just like that.

In summary, we need

1: predictions of possible experiments, and

2: a definition of what the concept means that differentiates it from
indirect perception in a way that would have implications for biology or
robotics.

Lehar:
Even if we found representations for *every last* experience in the
brain, that would not disprove direct perception. But then again, what
would? If there is no imaginable experiment that would distinguish
between direct and indirect perception, then they are indistinguishable.

Me:
As I have said repeatedly, if this is so, then neither theory is
empirical on this point. If evidence does not distinguish A from B, then
it does not distinguish B from A either, and Steve has no evidence
whatsoever that perception is indirect rather than direct. 'Distinguish'
is a symmetrical relationship. Since I think that indirect
representationalism is at bottom not a theory at all, just an utterly
confused misinterpretation of the scientific evidence, that result is
fine with me.

SL>Obviously it was a view from the representationalist perspective;
SL>feel free to compose a summary from the "direct perception"
SL>perspective.

I don't see any need for a one-sided summary. I'm involved because
of scientific interest, not because I want to push a particular
ideology. I welcome a diversity of methodologies.

SL> Advocates of direct perception cannot even agree on what their
SL> theory states, whether there are any representations in the brain
SL> or not, or if there are, whether they encode all of experience or
SL> only some of it.

There is no clear meaning of "representation". Different people take
it to mean different things.

SL> They cannot make testable predictions of future experiments that
SL> would resolve the matter one way or the other, nor can they build
SL> a functioning robot to demonstrate the *principle* behind the
SL> concept.

Steven keeps repeating this mantra.

Both sides are in the same predicament with respect to predictions.
Both can make vague predictions about what will eventually be
discovered in neurophysiology, but neither side can give specifics
sufficient to guide the neuro-scientists in such a determination.

No functioning robot has yet persuasively demonstrated any cognitive
or conscious capabilities, so both sides are on a par here too.

SL> The motivation behind direct percepton on the other hand is
SL> perfectly clear, it is motivated by the very vivid naive
SL> impression that what we experience is the world itself, rather
SL> than an internal representation.

This is where Steven makes a serious mistake. He is not a mind
reader, and he should stop trying to attribute motives to others.

In my case, I am not all that concerned with conscious experience.
My main interest has been cognition -- how we acquire and use
knowledge. My original view was the received view of
representationalism, which is the view most likely to arise out of
naive realism. I dropped that view when I realized how implausible
it is for a biological system.

I find nothing in representationalism that contributes to a study of
cognition. When we examine robots with representationalist designs,
the source of knowledge is clear -- it is "knowledge" designed into
the system. These systems are built on nativist principles, and
cannot account for the acquisition of knowledge such as we see in
human children.

SL> Any robot model with a camera, computer, and servos, demonstrates
SL> the *principle* behind representationalism, and thus clarifies
SL> concepts such as "information", "representation", and "processing"
SL> in terms that are perfectly clear to anyone who has used a
SL> computer.

No, it doesn't do any of those things. I don't know of anybody who
really believes that a camera, computer and servos has any conscious
experience, nor that it demonstrates any cognitive abilities
comparable to the learning we see in human children. The terms
"information" and "representation" are still quite muddy, with a
great deal of disagreement on what they main.

The camera/computer system uses representations in the sense that it
represents data for us humans. It does not represent anything for
the computer itself, for the computer has no self.

What is represented in the computer is information only in the sense
that it informs us humans. The computer is not itself informed. It
is merely a mechanical data processor, following preprogrammed
procedures on our behalf.

If roboticists are ever able to produce something comparable to
Asimov's R. Daneel Olivaw, that might indeed clarify some of these
concepts. But I won't hold my breath while waiting.

I read Steve Lehar's 'summary' of the debate nearly last of the long
series of messages on direct realism, etc., in my inbox. A most
annoying document! Without rehashing all the details, let me say that
virtually nothing he says about direct realism is true of the position
that I have articulated, as I am sure he knows. In fact, in no sense
has he summarized the debate. What he does is merely to summarize is
all the charges he has levelled against us. I will leave it to others
to judge whether I have 'wiggled and squirmed' more than Steve. At
least I try to respond to his claims.

Rickert >
There is no clear meaning of "representation". Different people take
it to mean different things.
< Rickert

There's a lot more consensus on the meaning of "representation" than
there is of "direct perception". Now there's a term that means
different things to different people! But most anyone would recognize
a photodiode array as a representation that represents patterns of
brightness across an image with a pattern of voltages in certain
registers. It does not matter whether there is anyone there to "see"
or experience the representation, it is still a representation that
represents one thing with another.

SL> They cannot make testable predictions of future experiments that
SL> would resolve the matter one way or the other, nor can they build
SL> a functioning robot to demonstrate the *principle* behind the
SL> concept.

Rickert >
Steven keeps repeating this mantra. ...
Both sides are in the same predicament with respect to predictions.
Both can make vague predictions about what will eventually be
discovered in neurophysiology, but neither side can give specifics
sufficient to guide the neuro-scientists in such a determination.
< Rickert

Not so! Representationalism makes the very concrete prediction that
every aspect of the world that is experienced, will eventually be
identified as various states of the brain. That there is no aspect of
experience that is *not* explicitly represented as some state in the
brain. That thesis will one day be proven right, or proven wrong. It
cannot be both.

Direct perception on the other hand seems to make no testable
predictions. Even if we found representations for *every last*
experience in the brain, that would not disprove direct
perception. But then again, what would? If there is no imaginable
experiment that would distinguish between direct and indirect
perception, then they are indistinguishable. That is, direct
perception would be identical to representationalism.

So we toss a coin to choose between them? No! One way is a reasonable
explanation in terms that everyone understands, such as information
and representation and computation, that can be built in real
hardware, whereas the other is a vague statement of words including a
lot of redefinitions of ordinary words to new meanings, and a host of
"category errors" that forbid certain conceptualizations, but with
*no* specific implications for how to build a robot, or what kind of
representations (if any) we should expect to find in the brain.

SL> Any robot model with a camera, computer, and servos, demonstrates
SL> the *principle* behind representationalism, and thus clarifies
SL> concepts such as "information", "representation", and "processing"
SL> in terms that are perfectly clear to anyone who has used a
SL> computer.

Rickert >
No, it doesn't do any of those things. I don't know of anybody who
really believes that a camera, computer and servos has any conscious
experience,
< Rickert

Redefinition: "representation" now means "has conscious experience"

Rickert>
nor that it demonstrates any cognitive abilities
comparable to the learning we see in human children.
< Rickert

Rickert >
The terms "information" and "representation" are still quite muddy, with a
great deal of disagreement on what they main.
< Rickert

Only to direct perceptionists. To the rest of the world these terms
are perfectly clear and unambiguous. Now if you think "representation"
necessarily implies consciousness and cognitive ability comparable to
children, then I can see how it would seem to be muddy.

Rickert >
The camera/computer system uses representations in the sense that it
represents data for us humans. It does not represent anything for
the computer itself, for the computer has no self.
< Rickert

That is one narrow and non-standard re-definition of a commonly
understood concept. Most any reasonable person would consider a
camera/computer system a representation whether or not there was a
human around for it to represent to. Instead of redefining every
other word to new and obscure meanings, it would be helpful if direct
perceptionists used the ordinary meanings of words, and explain their
theory in words we already understand.

The principle of representationalism *is* demonstrated in robot models,
whereas the principle of direct perception is not even specified sufficient
to devise such a model.

SL> There's a lot more consensus on the meaning of "representation"
SL> than there is of "direct perception". Now there's a term that
SL> means different things to different people!

There is a computer science meaning of represent, and there is a
cognitive meaning of represent. They are not the same.

It is one thing to say that the output of individual diodes
represents the intensity of light striking that diode. It is a quite
different thing to say that the array as a whole represents an
image.

SL> They cannot make testable predictions of future experiments that
SL> would resolve the matter one way or the other, nor can they build
SL> a functioning robot to demonstrate the *principle* behind the
SL> concept.

Rickert >>
NR>> Steven keeps repeating this mantra. ...
NR>> Both sides are in the same predicament with respect to predictions.
<< Rickert

SL> Not so! Representationalism makes the very concrete prediction
SL> that every aspect of the world that is experienced, will
SL> eventually be identified as various states of the brain.

There is nothing concrete about that prediction. There is no
suggested way of testing it. There is no known metric for "every
aspect of the world that is experienced". The prediction claims an
identity between two things. One of these is knowable in principle,
but unknowable in practice, due to the complexity of the brain. The
other is unknowable in principle for it refers to what is private and
subjective.

SL>> Any robot model with a camera, computer, and servos, demonstrates
SL>> the *principle* behind representationalism, and thus clarifies
SL>> concepts such as "information", "representation", and
SL>> "processing" in terms that are perfectly clear to anyone who has
SL>> used a computer.

Rickert >>
NR>> No, it doesn't do any of those things. I don't know of anybody
NR>> who really believes that a camera, computer and servos has any
NR>> conscious experience,
<< Rickert

The redefinition is due to Steven. A few paragraphs up, he
makes what he calls a concrete prediction, which equates what
he calls representations with experience. But now he apparently
wants to deny that representation has anything to do with
experience.

I can only repeat my point. Representationalism, at least according
to some of the claims of its proponents, is purported to be an
explanation of conscious experience. Yet the robots that are said to
demonstrate the principle of representationalism do not actually
demonstrate any conscious experience at all.

SL>> Representationalism makes the very concrete prediction that every
SL>> aspect of the world that is experienced, will eventually be
SL>> identified as various states of the brain.

Rickert >
There is nothing concrete about that prediction. There is no
suggested way of testing it.
< Rickert

That's unduly pessimistic. For example, I'm aware of the colours
of various objects. That's reasonably knowable. Sure, there are
skeptical reasons for saying we don't really know it, but skepticism
applies much more generally. We needn't make it especially important
in this case.

Then, it's unlikely that the brain is so complex that we can
never work out whether colours are represented (in the CS sense)
in the brain.

Rickert >
I can only repeat my point. Representationalism, at least according
to some of the claims of its proponents, is purported to be an
explanation of conscious experience. Yet the robots that are said to
demonstrate the principle of representationalism do not actually
demonstrate any conscious experience at all.
< Rickert

Most representationalist (at least) don't think they're explaining
conscious experience. They're explaining the content of conscuious
experience (or something like that). How / why various goings-on
in the brain are conscious experience is left as a mystery by
their representationalism.

Do the advocates of direct perception think that conscious
experience is somehow (magic?) able to directly connect with
the world, without involving the sorts of mechanisms that
would be implemented in a robot?

I don't claim to speak for all direct perceptionists. So I will speak
only for myself.

I don't expect that there is any magic. On the other hand, I also
don't expect that we will build robots capable of having experience,
or at least I don't expect that we will build them any time soon.

For sure, I do not deny that there are mechanisms, and I don't expect
any new physics to be required.

The trouble with the mechanisms used by robots, as we build them
today, is that they lead to a brittleness of behavior. I expect
biological systems will use different mechanisms more resistant to
this problem.

At least part of the disagreement between me and a representationalist
such as Steven, is conceptual. We disagree on how things should be
described.

Let me start by stating a couple of positions where I think Steven
would disagree.

1: Computers don't actually compute. People compute, and use
computers as tools to assist them in that computation.

My reason for this view is that I take computation to be action
on numbers (or other mathematical objects), rather than action on
numerals (the marks we make to represent numbers). To be sure,
we carry out our computation by operating on numerals, which we
use as proxies for the numbers. But the term "computation"
refers to the action on the numbers, not to the proxy action on
the numerals.

2: An ordinary mercury thermometer does not carry a
representation of the temperature. Nor is it a symbol that
denotes the current temperature.

The thermometer reading is strongly correlated with temperature,
and we use that correlation in our determination of the
temperature. But we also have to consider whether the thermometer
is miscalibrated, or whether there is a local source of heat that
makes the thermometer reading unreliable for a particular
reading.

This possibility of error in the thermometer is one of the
considerations that leads me to the view that the temperature does
not represent or denote.

I expect that the representationalist does take the thermometer
reading as a representation. He assumes that all we need is to copy
that external representation, to form an internal representation.

The direct perceptionist view, or at least my own view, is that we
cannot copy. Rather, we must *read* the thermometer. And reading is
not just copying, for it involves judgement as to whether conditions
(such as a local source of heat) could affect the reliability of the
thermometer.

There, I think, is the main difference. The representationalist sees
perception as a mainly passive activity, copying representions that
exist in the sensors. The direct perceptionist view is that
perception is not passive, but involves activity (as in reading the
thermometer). The direct perceptionist is emphasizing the
interactive aspects of perception.

Rickert >
Let me start by stating a couple of positions where I think Steven
would disagree.
< Rickert

You are absolutely right in that assessment.

Rickert >
1: Computers don't actually compute. People compute, and use
computers as tools to assist them in that computation.

1(a): Computers don't make inferences.

2: An ordinary mercury thermometer does not carry a
representation of the temperature. Nor is it a symbol that
denotes the current temperature.
< Rickert

Very revealing pattern! Do you believe that computers will *ever* be able
to compute, make inferences, and represent? Or are these functions in
principle beyond the capacity of mere machines?

And if some future computer *will* one day be able to perform these
functions, what essential element or vital function would it be in these
super advanced computers that would allow them to perform those functions?
What is the missing ingredient, or magic component, that would allow them
to do what current computers cannot?

It's become reasonably clear that the debate between Andy Brook and
Steve Lehar is terminological. That is, they don't disagree in any
relevant respects about what's going on in the brain, in experience,
or in the world. They just differ in how they use the terms "direct
perception", "representationalism", and maybe "perceive". As far as I
can tell, what Andy calls "directly perceiving the world through the
use of a representation", and what Steve calls "indirectly perceiving
the world through perceiving a representation" are exactly the same
thing.

So it's better to get past the terminological issues and concentrate
on matters of substance. It may be that there is some other
substantive, non-terminological respect in which the two views
disagree. But if so, I haven't seen it in this discussion.

Suggestion: to find a substantive thesis on which the sides differ,
one must state it without using any of the contested terms above. If
one can find a neutrally-stated thesis about which the two sides
disagree (for non-terminological reasons), that will be progress. If
one can't, so that the two sides turn out to agree on matters of
substance, that will be progress too.

Chalmers >
It's become reasonably clear that the debate between Andy Brook and
Steve Lehar is terminological. ... As far as I
can tell, what Andy calls "directly perceiving the world through the
use of a representation", and what Steve calls "indirectly perceiving
the world through perceiving a representation" are exactly the same
thing.
< Chalmers

No, there is something more fundamental at stake here than mere
terminology. In an off-list discussion I have discovered that
Brook believes that the spatial structure of our
experience is the structure of the world itself, whereas
representationalism states that the structure of visual experience is
a structure in our brain, and only in secondary fashion is that
structure also representative of a more remote external world.

It comes down to the question of whether it is valid to make
observations about the principles of visual representation by
examining the nature of visual experience. According to Brook, visual
experience gives us knowledge *of* the external world. According to
representationalism the dimensions of conscious experience necessarily
map directly to the dimensions of the representational machinery in
the brain, and therefore it is valid to make observations on the
representational machinery of the brain by observation of experience.
For example the first and most obvious observation is that the visual
representation is *analogical*, that is, objects and surfaces are
represented explicitly by objects and surfaces in a spatial
representation.

Chalmers >
Suggestion: to find a substantive thesis on which the sides differ,
one must state it without using any of the contested terms above. If
one can find a neutrally-stated thesis about which the two sides
disagree (for non-terminological reasons), that will be progress. If
one can't, so that the two sides turn out to agree on matters of
substance, that will be progress too.
< Chalmers

I think the core issue behind this disagreement is whether or not
there are analogical or pictorial representations in the brain that
explicitly encode every aspect of our experience. And whether or not
the principles of the organism interacting with the environment
necessarily involves an analog replica of the organism, its
environment, and the forces in the world, as suggested here.

It is clear that it is *that* idea which the direct perceptionists find to
be incredible. And it is not just a question of terminology; the visual
brain is either analogical/representational, or it is not.

I think I have spotted what leads Steve to keep insisting that I hold
that part of the structure of experience is out in the world. He holds
as self-evident that if X represents Y, then X has to have the same
structure as Y. As he said yesterday, if X represents corners and
colours, then X has to have corners and colours. (So Steve was right
when he said that this is one issue between us. Not the crucial one
but it is an issue.) Then when I insist that representations are of
things in the world, he thinks it follows from this that part of the
structure of the representation is not only *of* but *in* the world, a
patently crazy view and a reductio of any form of direct
realism. Well, a reductio it would be -- if we held any such view. But
we don't. Nor do most philosophers. As Bill Seager said, the word
'red' does not have to *be* red to *mean* redness.

Certainly information about the world is encoded in some way in the
brain, information falling on the foveal spot on the retina at least
(and analogously for the other senses), but it would be astonishing if
it is encoded in any analogue form. When I see a tangerine, nothing in
my brain is tangerine (and if it were, I could not sense that,
everything being pitch-black in my brain).

Response to Andrew Brook, in "Direct percepton and terminological disputes"
thread, I thought it appropriate to re-name the thread, as it seems that we
have identified the key difference between our viewpoints, and it is not a
mere question of terminology, but a profound paradigmatic issue. A way we
view the world. People have difficulty even imagining the world of the
alternative paradigm, so they talk at cross-purposes, every word being
interpreted in subtly different ways. We've seen enough of that here, but I
believe we have broken through that cycle.

Brook >
He holds as self-evident that if X represents Y, then X has to have the
same structure as Y. As he said yesterday, if X represents corners and
colours, then X has to have corners and colours.
< Brook

This here, I believe, is the center of our disagreement. But I would
state it this way: If you accept that perception is indirect (as some
of us can't help believing) then it follows that the dimensions of
conscious experience must necessarily reflect the dimensions of the
information encoded in the brain. I know you direct perceptionists
don't believe this, but just imagine for a minute *if* it were
actually true, that the world around you was a kind of holographic 3-D
image created by resonances in your visual brain. If that were your
understanding of your experience, would it not *then* (hypothetically
speaking) be self-evident that the representational principle of the
brain is analogical?

Our *experience* of the world is analogical. Only a bizarre
re-definition of experience would deny to it its spatial aspect. You
direct perceptionists believe that the reason that your experience is
analogical (spatially extended, volumetric manifold) is because it is
an experience *of* an analogical world. We representationalists
believe that experience is analogical because the representational
mechanism of our brain is analogical, and the truth of that statement
can be verified by inspection, as long as one does not get confused
about what one is actually "seeing". (Note the ""quote"" marks) Of
course we believe the world to be analogical also. But that is why an
analogical imaging mechanism is the best representation for spatial
interaction with a spatial world.

If the representation were not analogical, how would the
"analogicality" of the world get through to our experience of it?
Wouldn't it be filtered out by the first non-analogical sensor, just
as a photodiode array filters out the polarity and phase of detected
light?

And why do direct perceptionists insist on denying the spatial aspect
of experience? Why do they have to "define it out of existence"?
Because if I talk about the spatial aspect **of your experience** (not
of the world) everyone but a direct perceptionist knows exactly what
you are talking about. It is that three-dimensional volumetric space
full of colored objects, that follows you around wherever you go,
filling in a 3-D picture of the world for you wherever you point your
eyes. Why do we have to deny its existence? Can direct percepton not
survive if our experience *is* spatial? Can't we admit "Yeah, we have
a problem with spatial experience, but it is still more credible than
representationalism". Wouldn't that be the honest response? Why does
one get a sense that direct perceptions are forever redefining words
and banning concepts? Is that really necessary?

Brook >
but it would be astonishing if it is encoded in any analogue form. When I
see a tangerine, nothing in my brain is tangerine (and if it were, I could
not sense that, everything being pitch-black in my brain).
< Brook

Au contraire! It would be astonishing if this rich spatially
structured analog experience were represented in the brain any other
than by a three- dimensional volumetric imaging system with as much
fidelity and resolution as you see in the world around you now. To
deny that mechanism is to leave the rich spatial information in our
experience in a peculiar kind of limbo, where it is experienced, but
does not actually exist as spatial structure anywhere, especially in
the case of dreams and hallucinations. But information cannot exist
without a physical substrate to store or register that information,
and spatial experience requires a spatial representation.

The fact that the world of visual experience appears in the form of a
spatial structure is itself conclusive evidence that information in
the visual brain is spatially structured.

This is not an esoteric philosophical question, but a very significant
concrete question that will one day be proven one way or the other
experimentally. I propose we start looking *for* an imaging mechanism
in the brain. Its probably right before our eyes, if only we knew what
to be looking for in the brain.

Steve, you don't get it. We *deny* that experience has a structure
analogous to the structure of what it represents -- as do philosophers
of all realist and irrealist stripes, both connectionist and classical
AI researchers, most cognitivists, and at least a great many
neuroscientists. Do you really believe that the word 'red' has to *be*
red to *represent* redness?

As to your claim that analogical structure is obvious, um, a bit of
evidence that we are doing more than representing structure, that our
representations have the structure they represent, would be nice. Not
that there is -- or perhaps even could be -- any.

Brook >
Steve, you don't get it. We *deny* that experience has a structure
analogous to the structure of what it represents -- as do philosophers of
all realist and irrealist stripes, both connectionist and classical AI
researchers, most cognitivists, and at least a great many neuroscientists.
< Brook

And perhaps all those people were mistaken! And perhaps the
Gestaltists were right all along!

Brook >
Do you really believe that the word 'red' has to *be* red to *represent*
redness?
< Brook

Yes, thats a powerful argument. But if you view it from the other
paradigm suddenly it looks very different. For although the word "red"
does not have to *be* red to *represent* redness, the COLOR red DOES
have to *be* red to *represent* redness. It sounds absurd to you that
a portion of your brain should "turn red" when you perceive red
things. But when you realize that everything in your experience is
just a picture in your brain, suddenly it makes sense again, because
certain portions of that picture are "red", and that redness expresses
its own content and meaning right there by simply being red.

Everything looks completely different from the epistemologically inverted
world! For one thing, many of the most profound paradoxes simply vanish.

Lehar >
And perhaps all those people were mistaken! And perhaps the Gestaltists
were right all along!
< Lehar

Steve, stop being obtuse. The point is, our view is an extremely common
one so there is no basis for foisting a view that we *don't* hold on us
and presenting it as self-evident truth as you do over and over.

Lehar >
Yes, thats a powerful argument. But if you view it from the other paradigm
suddenly it looks very different. For although the word "red" does not have
to *be* red to *represent* redness, the COLOR red DOES have to *be* red to
*represent* redness.
< Lehar

The colour red does not represent redness, not normally anyway.
Something gets to represent something else by being selected by a
cognitive system or a system of cognitive systems (a society) to have
that function (plus, maybe, by satisfying other conditions). Surface
reflectances are seldom thus selected. It is also true, however, that
virtually anything *can* be selected for that role, including things
that do not resemble what they have been selected to represent in the
slightest.

Brook >
Steve, stop being obtuse. The point is, our view is an extremely common
one so there is no basis for foisting a view that we *don't* hold on us
and presenting it as self-evident truth as you do over and over.
< Brook

I don't mean to be obtuse, I am merely pointing out that even though your
view is indeed an extremely common one, paradigms do change occasionally,
and when they do, the extremely common view suddenly becomes the rejected
view. My point is that we should decide the issue on the facts and the
arguments, not on the basis that it is a common view.

As for "foisting a view that you don't hold", I have done no such thing. I
have merely pointed out that your concept of direct representationalism is
*either* indistinguishable from regular representationalism, and thus
vacuous, *or* it involves having experience of things which are not
explicitly represented in your brain, which is impossible in principle.

We have already established that our former misunderstanding on this point
is due to the fact that the part of the world that is experienced without
being represented is its volumetric spatial extendeness, which you and
others redefine to not be experience at all, but an aspect of the
world "viewed directly". So my response was that the aspect of the world
that you believe to be viewing directly *through* your experience is in
fact part of your experience (which is why you experience it) and is not
the world itself.

The onus is on you to explain how any aspect of the world, including its
analogical spatial extendedness, can possibly penetrate into our experience
*without* being represented in our brains. Can you explain that to us?

As I have said over and over, being represented and being an object of
direct perception are not alternatives to one another. When you insist
on saying or implying that for me, they are, you are foisting onto me a
view that I do not hold. Of course everything of which we are directly
aware is represented. How else could we become aware of these things? We
are aware of then *via*, *through* representations. But we are aware of
the things, not merely of our representations of them. This too I have
said over and over. And argued. And presented evidence for. Steve, you
are welcome to reject this view. But you are not welcome to make out
that I don't hold it.

On the 'detailed representation' issues, I have said -- in detail and
more than once -- what is represented in detail in the brain: in vision,
the things in the world that the foveal spot is representing and
analogously for the other senses.

Dalton >
Steven Lehar holds that everything in experience
has to be represented in the brain, and that is close(?) to being
a self-evident truth.
But he also holds that the representation is analogical; and
that is at least far less clearly true.
< Dalton

Yes, good, lets talk about that one!

First, generally speaking, if the representation in our brain was *not*
analogical, for example if it were abstract-symbolic of some form, as
spatial data is often represented in computers, then why would our
experience of the world not also be an abstract-symbolic experience? How
would we even *know* that the world is analogical? Why would our experience
of the world not be something like our experience of numbers, or equations,
or logic? Those are abstract-symbolic experiences, so we are capable of
having them, and so is your experience of my statement when I say, for
example, that my couch at home is in front of my TV with a coffee table in
between. You *can* form a spatial mental image of the situation, but you
don't know the specific shapes and distances and configurations of my
living room, so you probably have a very abstract fuzzy image of my
statement. But if you were here (as I am) you would have in addition this
very vivid spatial experience. Why would that be if not for a spatial
representation?

Second: consider the *information content* of a spatial experience. It is
a "filled-in" or "reified" data structure, that is, some process has gone
to the trouble of painting in every point on every visible surface, and we
can even perceive every point in the empty space around those solid
objects, and every point on every surface, and in empty space, is
experienced simultaneously and in parallel, where they are experienced as
continuous surfaces and volumes. The experience is a "manifold" in the
mathematical sense, so you simply could not store or register that
information without having the full information content of a volumetric
scene.

This point is elaborated in detail in this cartoon on the
Neurophysiological Implications of Spatial Perception.

Brook >
Steve, you don't get it. We *deny* that experience has a structure
analogous to the structure of what it represents -- as do philosophers
of all realist and irrealist stripes, both connectionist and classical
AI researchers, most cognitivists, and at least a great many
neuroscientists. Do you really believe that the word 'red' has to *be*
red to *represent* redness?
< Brook

Now, there is clearly something wrong here. It seems that "experience" in
this statement has been used to mean the neural or cognitive correlates of
experience. What is said here may well be true of representations in the
brain (assuming they exist) but it isn't true of subjective experience
itself, which does have spattially extended structure, as we can determine
by introspection?

Reason >
What is said here may well be true of representations in the
brain (assuming they exist) but it isn't true of subjective experience
itself, which does have spattially extended structure, as we can determine
by introspection?
< Reason

No we can't. All we can determine by introspection is that experience
*appears* to have spatial, etc., structure. For all we know, that
appearance could fail to correspond to how things are. This point has
been a commonplace of cognitive research since Kant first made it,
though many people have stoutly resisted it, usually by appeal to some
principle that experience is different. Here, how things seem is a
guarantee of how they are. Not so, as
the hundreds of confabulation experiments and a host of other work has
shown.

Brook >
No we can't. All we can determine by introspection is that experience
*appears* to have spatial, etc., structure. For all we know, that
appearance could fail to correspond to how things are.
< Brook

Once again, you seem to be confusing subjective experience with objective
perception. What you say would be perfectly true if we were talking about
introspection on to our brain-states. Objective perception involves
correlating an observing state with some observed state, and so for the
observing state itself to be observed, we would have to correlate some new
state with it. (This is the basis of the little paradox I mentioned in my
message to Anthony Sebastian.)

But I'm not talking about introspecting on to our brain-states. I'm talking
about introspection on to our subjective states. And our subjective
experience of something is, by definition, how that thing appears to us.
So there cannot be some feature of our subjective experience which is other
than how something appears to us, because they are the same thing. If
something appears to us to have spatial extension, then our subjective
experience of that thing also has spatial extension.

Cathy, repeating a point over and over does not make it any closer to true. You
get it exactly right when you say,

Reason >
But I'm not talking about introspecting on to our brain-states. I'm talking
about introspection on to our subjective states. And our subjective
experience of something is, by definition, how that thing appears to us.
< Reason

As Dennett put it, on how things appear to you, you are the final, ultimate
authority. But on the properties that an act of appearing has, you are no
authority at all. Who knows what an act of something appearing to you as X is
like? It is most unlikely that it has the properties of X. That something
appears to you to be a certain way tells you nothing whatsoever about what an
appearance of that sort is like. End of story. If I *interpret* a representation
as being about Y, does that interpretation automatically have the properties
that Y has?

Reason >
So there cannot be some feature of our subjective experience which is other
than how something appears to us, because they are the same thing. If
something appears to us to have spatial extension, then our subjective
experience of that thing also has spatial extension.
< Reason

Not so, no matter how many times you say it. For the reason I just gave.

Brook >
As Dennett put it, on how things appear to you, you are the final, ultimate
authority. But on the properties that an act of appearing has, you are no
authority at all. Who knows what an act of something appearing to you as X
is like? It is most unlikely that it has the properties of X. That something
appears to you to be a certain way tells you nothing whatsoever about what
an appearance of that sort is like. End of story.
< Brook

Well, Dennett is wrong about so many things that I'm not in the least
surprised he's wrong about this as well. ;-)

Andrew, all you are doing is constructing elaborate forms of words which,
when deconstructed, amount to claiming that subjective experience has
properties which subjective experience does not have. Our subjective
experience of something is how it appears to us. They are the same thing.
Your entire argument is predicated on trying to find some way of saying that
subjective experience of something, and how something appears to us, are
different things.

I would disagree that introspection reveals that experiences
have spatial properties. How big is your experience? Ten
cubic feet? When I look at the night sky I can see (with
naked eye) the Andromeda Galaxy some 2 million light
years away - so is the "spatially extended structure"
of the experience some 8 million cubic light years in
size.

What we can tell by introspection is that we are *seeing*
a world that has spatial structure. (To deal with the
possibility of hallucination, this can be amended to
"introspection reveals that we are having visual
experiences as of a world that has spatial structure.)
It is not logically valid to infer from this
that the experiences themselves have any spatial
structure. Of course, if experiences are identical to
certain brain processes then they probably have
spatial structure of their own in the brain - but
introspection does not reveal this. And such
neural spatial structure is completely distinct from
the spatial structure of the world which visual
experience reveals.

Seager >
How big is your experience? Ten cubic feet? When I look at the night
sky I can see (with naked eye) the Andromeda Galaxy some 2 million light
years away - so is the "spatially extended structure" of the experience
some 8 million cubic light years in size?
< Seager

Of course not! *Phenomenally* speaking the Andromeda galexy is experienced
to be *no farther* than the sun, moon, and stars, all of which appear as if
pasted on the dome of the sky, and appear barely farther than the farthest
mountains on the horizon. Without scientific knowledge we would have no way
of knowing that one celestial object is closer or farther than the others.

Obviously all sizes in experience are relative, although there are a couple
of benchmark distances available. One is the perceived size of our own
body. If you have a hallucination in which your own body is invisible (I
have had such a thing) there is no way to know the size of the things in
that experience. (Protagoras: "Man is the measure of all things")
The other benchmark is the perceived size of the dome of the sky, that
remains fixed no matter how big a space we are perceiving.

that makes the mapping from phenomenal to objective size nonlinear, and
maps an *infinite* objective distance within the finite sphere of
experience.

In fact, the phenomenon of perspective is itself one of the most convincing
proofs that experience is *not* direct, because it has a prominent
curvature and distortion that are clearly not a property of the objective
world "viewed directly", but an essential property of the phenomenal world
that allows the brain to encode an essentially infinite space in a finite
spatial model.

Steve, would you be willing to agree that we do not *perceive* the
representations in our brain, and accept the claim that these internal
representations *constitute* our phenomenal experience of the external
world? The television example is a poor analogy because it requires an
extra observer (sort of like a homunculus, perhaps?). I think Alex Green
would agree. If he doesn't agree, I'm sure he will correct me.

Steve, if you would agree to the suggested rewording, one major point of
dispute between you and Andrew might be resolved.

Trehub >
Steve, would you be willing to agree that we do not *perceive* the
representations in our brain, and accept the claim that these internal
representations *constitute* our phenomenal experience of the external
world? ...
< Trehub

Yes yes yes, representations *constitute* our phenomenal experience. I sometimes
use loose and casual wording such as "perceiving" or "seeing" our
representations. But every representationalist knows exactly what I mean
whichever way I phrase it, and every direct perceptionist *misunderstands* what
I mean whichever way I phrase it.

Trehub >
if you would agree to the suggested rewording, one major point of
dispute between you and Andrew might be resolved.
< Trehub

Not a chance! Because it really does *not* come down to a difference in
terminology, it is a profound paradigmatic difference in conceptualization that
manifests itself in different terminology. But no amount of re-wording will
resolve this question that easily!

The foundational difference at the core of the debate is that direct
perceptionists really and truly believe that the spatial structure that they
(constitute, perceive, experience, see, have aquaintance with, you name it!)
actually *IS* the world itself, outside their head, whereas representationalists
recognize that that vivid spatial structure *IS* an image of the world in their
head.

The reason why the debate goes round and round in futile circles is that this
difference is not a conclusion arrived at by logical reasoning, but it is an
*initial assumption* that we/they take as self-evidently true, and we/they are
willing to warp and morph the rest of understanding to fit in with that
"obviously true" initial assumption.

The only reasonable way to address paradigmatic differences of this sort (I said
"address", not "resolve", that would be impossible!) is to consider both
alternatives as if they could actually be true, and compare the two alternative
world views in their totality, to see which one leads to a more coherent picture
of perception with the least number of paradoxical anomalies or "incredible"
components.

The UN-reasonable way to not-address paradigmatic differences is to make the
choice based on the perceived credibility (or otherwise) of the *initial
hypotheses* themselves. It is *not* a fruitful argument to begin with "It is
*self evident* that perception is (direct/indirect)", because that is exactly
the issue in question!

As I see it, the paradoxical anomalies in representationalism are the following:

1: It is truly incredible that the brain, as we know it, is capable of
constructing vivid spatial structures in real time of the degree of detail and
complexity as the world we see around us.

2: Even if it could, it is truly incredible that those internal pictures should
somehow become conscious of their own spatial structure.

To accept those two points would require that we *totally and fundamentally
revise* our notions of how the brain represents and processes information, and
our notions of what consciousness is at a foundational level.

The paradoxical anomalies in direct perception are the following:

1: It is impossible in principle to become aware of, and thus gain information
about, objects outside of our self *except* by means of sensory input and
internal representations.

2: The information content of our experience cannot be greater than the
information explicitly represented in the brain.

To accept those points would require that we *totally and fundamentally revise*
our notions of information, representation, and processing, and (Rickert &
Sizemore) accept that those functions can somehow occur outside of our heads,
independent of the computational machinery of our brain, or (Andrew Brook) to
accept that our experience can somehow be an experience of the world directly,
unmediated by the representations of it in our brain.

The paradigmatic partizans on both sides can be heard endlessly repeating again
and again ... and AGAIN! the *initial assumptions* of their paradigm, as if the
other side has not heard or grasped them yet.

I do however detect an asymmetry in this debate. For my part I fully acknowledge
and recognize the profound paradoxes of the representationalist view, and have
pointed them out many times, although in the balance I consider them less of a
problem than the profound paradoxes of the direct perception view. But I have
*never once* heard the direct perceptionists acknowledge the profound paradoxes
of their position, they argue as if they were blissfully unaware of *any* kind
of problem there, and they repeatedly use familiar terminology in unfamiliar
senses as if they can somehow redefine themselves out of their problems. To me
that sounds like rationalizations to justify their initial paradigmatic
hypothesis, without ever seriously questioning that hypothesis itself. This is
why the debate goes uselessly round and round.

I would have a lot more respect for the direct perceptionist view if I heard
them acknowledge that both positions contain paradoxical components, but that in
their judgment they consider their paradoxical problems less problematic than
those of the representationalist view. We would still have to ultimately agree
to disagree, but it would put an end to the endless and futile debate, and place
focus where it belongs, on the *relative* incredibilities of the two opposing
views. Then we could have a rational exchange.

Lehar >
The reason why the debate goes round and round in futile circles is that
this difference is not a conclusion arrived at by logical reasoning, but
it is an *initial assumption* that we/they take as self-evidently true,
and we/they are willing to warp and morph the rest of understanding to
fit in with that "obviously true" initial assumption.
< Lehar

I usually agree with what Steven Lehar says, but I can't agree with that.

I do agree that representationalism "is not a conclusion arrived at
by logical reasoning" -- because there's an empirical component, and
because the reasoning isn't strictly deductive or absolutely conclusive.

But it's not an initial assumption either.

Where I started was, if anywhere, with a naive form of direct
perception. I moved away from that via an exposure to the relevant
science, by considering the arguments for the various philosophical
views, and by considering my own conscious experience.

I don't mean "initial" as necessarily precedent in time, but rather as
"foundational", an assumption upon which all the rest of one's reasoning stands.
You are right in temporal sequence, one usually begins with naive realism.

I don't think it's an assumption at all. The "assumption"
phrasing came in when you wrote:

The reason why the debate goes round and round in futile circles is
that this difference is not a conclusion arrived at by logical
reasoning, but it is an *initial assumption* that we/they take as
self-evidently true, and we/they are willing to warp and morph
the rest of understanding to fit in with that "obviously true"
initial assumption.

Just to be clear: I don't think it's self-evidently true either;
nor am I willing to warp or morph anything to fit with it.
Those ways of putting it are a bit too strong.

Paradigmatic assumptions are assumptions in the sense that they are not arrived
at by logical reasoning, but they are simply accepted as self-evident. All
science necessarily has foundational assumptions, for example the assumption
that there is an external world with independent objective existence. Without
that assumption, all science would be useless. But it is not something that can
be proven beyond a doubt, it is just an assumption that all scientists share.

Andrew Brook and the other direct perceptionists start from the assumption that
we experience the world itself directly, where it lies out in the world. They
say so again and again, as if the truth of this were *self-evident*, and thus
beyond question. That is a paradigmatic assumption. They are not willing to
negotiate on that question. To them, it is inconceivable that it should be
otherwise. And that in turn forces them to warp their definition of the meaning
of words like information and representation and processing, (and Andrew Brook
to deny that visual experience is spatially structured) in order to fit with
their initial assumptions.

My own paradigmatic assumption is that mind is a physical process taking place
in the physical mechanism of the brain. I am not willing to negotiate on that
point, because there is no way to prove that assumption. And that assumption in
turn forces me to warp my understanding to the point where I propose that
spatial structures in the brain can become conscious of their own spatial
structure. It is not a comfortable conclusion, but one to which I fell compelled
by the facts, given my initial assumptions.

My point is that there is a difference between such foundational assumptions and
the more ordinary conclusions arrived at by logical reasoning, such as that the
retina has a certain resolution, or that the optic nerve contains so many
fibers. These are issues that people are willing to modify and revise when faced
with new data. But people are not willing to revise their paradigmatic
assumptions, so it is useful to identify them as such, so as to alert us to the
fact that debating them further is futile.

We really are getting nowhere and it is time for me to bow out. But in doing so,
let me try one last time to set the record straight.

Lehar >
Andrew Brook and the other direct perceptionists start from the
assumption that we experience the world itself directly, where it lies
out in the world. They say so again and again, as if the truth of
this were *self-evident*, and thus beyond question.
< Lehar

Absolutely not. I have *argued* not assumed that direct perception is the case.

Lehar >
That is a paradigmatic assumption. They are not willing to negotiate
on that question. To them, it is inconceivable that it should be
otherwise. And that in turn forces them to warp their definition of
the meaning of words like information and representation and
processing, (and Andrew Brook to deny that visual experience is
spatially structured) in order to fit with their initial assumptions.
< Lehar

Most of this is simply wrong. In particular, far from denying that experience is
spatially structured, I have insisted that it is -- over and over. What I claim
are two things: that the spatial structure of an experience is usually nothing
like the spatial structure it represents objects to have, and the spatial
structures that it represents are no guide whatsoever to the spatial structure
it has.

Lehar >
My own paradigmatic assumption is that mind is a physical process
taking place in the physical mechanism of the brain. I am not willing
to negotiate on that point, because there is no way to prove that
assumption.
< Lehar

As you know, I entirely agree with you about the conclusion, though I of course
think that the point can be argued. Materialism is not a matter of faith, it is
a well-justified theory of the relationship of mind and brain, one that shows
(Cartesian) dualism to be false.

Lehar >
And that assumption in turn forces me to warp my understanding to the
point where I propose that spatial structures in the brain can become
conscious of their own spatial structure. It is not a comfortable
conclusion, but one to which I fell compelled by the facts, given my
initial assumptions.
< Lehar

Why does being a materialist about mind force you into any such position? What a
curious thing to say!

Brook>
I couldn't be more hostile to sense data theory if I tried.
< Brook

How can one possibly object to the concept of sense data?

Sense data is not really a theory, nor is it contrasted with direct perception.
It is merely a factoring, or separation, of two entities which are easily
confused: the object of perception (e.g. a chair), and the sensory *experience*
of that object (your perceptual experience of the chair). One might claim, as a
direct realist, that those two items are one and the same thing. But that does
not deny the distinction between the two. For example you experience only the
exposed surfaces of a chair, whereas the chair itself exists throughout its
volume, all the way to its hidden rear surfaces. The experience is composed of
color and surface, and is limited to a finite resolution, and it changes in
different lighting, and ceases to exist in darkness, whereas the chair itself is
composed of wood or metal or plastic, and exists to a very much finer resolution
to the scale of atoms and molecules, and it does not change in different
lighting, nor cease to exist in the dark. Direct perceptionists claim that your
experience of a chair is a *subset* of the chair, whereas representationalists
claim that it is a distinct entity that is not even spatially superimposed on
the real chair that it represents. But whether they are the same or distinct
entities, it is always valid to factor things conceptually so that we can
discuss their components separately, even if we ultimately conclude that the
factoring is illusory, and that the two are one and the same.

What Andrew Brook objects to in sense data theory is that it highlights the
*indisputable fact* that our visual experience is spatially structured.

Steve, I reject the idea of sense-data because the representational experiences
in my theory are nothing like sense-data. Sense-data are images, something like
pictures, at least in traditional sense-data theory (Mill, Russell). My
representations present the world itself, not images of any kind (well, some
tiny number of representations present images -- when I am looking at a picture,
for example). I know you find my view utterly weird, even though it accords with
commonsense and is compatible with vision science, but it is what I believe and
I have tried to explain and justify it a number of times, indeed show that it is
the only coherent point of view. As have others on the list.

Brook >
Steve, I reject the idea of sense-data because the representational
experiences in my theory are nothing like sense-data. ... My
representations present the world itself, not images of any kind
< Brook

Yes, I understand that you deny explicit spatial images in the brain,
and that your experience is an experience of the world. But even so,
the world you experience "directly" manifests itself to you in two
separable aspects, the directly perceived modal colored surfaces
exposed to your view, and the indirectly perceived amodal solid
volumes that you infer from those modal experiences. Whether or not
these experiences are in your head or out in the world, the former,
directly visible modal surfaces are *conceptually separable* from the
invisible solid volumes that they delimit, and "sense data" simply
refer to those conceptually separable modal surfaces.

How could one possibly object to making that distinction, whatever one's
theoretical conclusions
might be?

Of course one can make some kind of experience/world distinction; but to call
the experience side in my approach 'sense-data' would just sow conceptual
confusion. Experiences in my theory are nothing remotely like sense-data as that
term has been understood in the tradition. The first step to doing good science
is constructing clear, precise concepts.

Brook >
Experiences in my theory are nothing remotely like sense-data as that
term has been understood in the tradition.
< Brook

Yes, and that is exactly what is lacking in your theory! There is no accounting
in it for the existence of sense data anywhere, whereas we know sense-data to
exist because we experience them to exist. That is the most direct first-person
verification possible in principle! We know this with more certainty than we can
possibly know anything else! Its an epistemological fact.

Far from sowing conceptual confusion, the concept of sense-data highlights a
specific aspect of experience, that is, its spatially extended nature, and
thereby demands an explanation for how our expericence can become spatially
extended except by means of an analogical spatially extended representation.
Direct perception solves this problem by denial, flat-out denying that most
certain of all truths, that my experience is spatially extended!

I know for a fact that I have an experience, and I know for a fact that it is
spatially structured.

I infer from that experience that I live in a spatially structured world, and it
is that world that I am seeing in my experience. But I cannot be certain that
that inference is right. It may be that solipsism is true, and that nothing
exists except me and my structured experience. More plausibly, it may be that
there is in fact an external world, but I am not seeing it directly, but rather
I am directly experiencing a spatial replica of that spatial world, which is
thus perceived indirectly. Andrew Brook objects vehemently to this very
possibility, as if it were so self-evidently false that it does not even have to
be disproven. But epistemologically speaking, there is no way for Andrew to
prove this to himself or anyone else, it is just an assumption that he is
making, and making it dogmatically as if you would have to be crazy not to see
this as obviously true.

But in the process, Andrew denies that his *experience* is spatially structured,
a denial that *I* consider to be so self-evidently false that it does not even
have to be disproven.

Epistemologically I stand on firmer ground insisting that my *experience* is
spatially structured, than Andrew with his insistance that perception is
obviously and necessarily direct, a fact that is neither self-evident nor
proven.

Because whether or not perception is direct, it is in fact undeniable that
visual experience is spatially structured, even if we cannot tell whether that
experience is a structure in the world or in our brain.

Steve, you are certain that certain events in you *appear* to have these
properties but, as I have suggested a number of times, you have no evidence
whatsoever that these appearances, sense-data, call them what you will, *have*
such properties. That is an article of faith. Introspection tells us only how
things appear to be, not how they are.

When I see a chair, and I experience it as a volumetric structure, it
may be an illusion, and the chair is not volumetric at all, there
could even be no chair there at all. But I *can* say with absolute
certainty that my *experience* is volumetric. It is impossible for my
experience to mis-represent itself. An experience is *by definition*
how it appears to me, not how it is represented in my brain, or what
it represents in the external world.

If the theory of direct perception (or direct representationalism)
requires that we deny that our experiences have the properties that we
experience them to have, then that is itself a fatal blow to the
theory, because it is a blatant violation of epistemology.

Lehar >
An experience is *by definition* how it appears to me, not how it is
represented in my brain, or what it represents in the external world.
< Lehar

What makes experience of experience so different. An experience can misrepresent
itself, or not represent most or even all of its properties, same as it can
misrepresent everything else. Now, you may deny this as an article of faith but
you have and could have no evidence that what I say here is false.

Lehar >
If the theory of direct perception (or direct representationalism)
requires that we deny that our experiences have the properties that we
experience them to have, then that is itself a fatal blow to the
theory, because it is a blatant violation of epistemology.
< Lehar

Wrong on both counts. Direct perception is silent on the relationship of
experience to itself (as is indirect representationalism) and saying that
experiences can misrepresent or not represent themselves is not only no
violation of epistemology, it is obviously correct if you think about it. My
experience of red need not be coloured and my experience of a box need not be
cube-shaped. The idea that we have little patches of colour and little
three-dimensional shapes in our brain (lit up or dark? -- remember, everything
is pitch black in the brain but to be aware of a colour, it has to be lit) makes
the mistake of thinking that properties of what is represented have to be
properties of the representing, that representations have to have the properties
they represent things as having. Sure, they *appear* to have certain properties
-- but appearance no more locks in reality here than it does anywhere else.

Brook >
An experience can misrepresent itself, ... Now, you may deny this as
an article of faith but you have and could have no evidence that what
I say here is false.
< Brook

It is not a question of evidence, it is a question of definitions! Experience is
that which is primary, before analysis sets to work. It interprets nothing, it
simply is as it is experienced to be. To claim otherwise is to redefine a
familiar term to mean something completely different.

Brook >
saying that experiences can misrepresent ... themselves is not only no
violation of epistemology, it is obviously correct if you think about it. My
experience of red need not be coloured...
< Brook

The neurons in your brain need not be red. The electrical voltage in your brain
need not be red. But the *experience* is what we are talking about when we say
the word "red"! The *experience* of red absolutely *MUST* be red, it is the very
definition of redness. To deny this would be to say that a red experience can at
the same time be actually green. Or an experience of black is actually white! An
experience is exactly what it is experienced to be, and no different.

This is not an article of faith, but simple plain logic! To claim otherwise is a
contradiction in terms!

Brook >
But experiences need not have the properties that they appear to have
< Brook

They most certainly do, unless you mean to critically redefine the dictionary
defined meaning of the word. Experiences *are* appearances, that is why they
appear to be as they appear, necessarily and inescapably!

This is not an article of faith on my part, but a simple definition of a word.
To say that appearances appear different than they appear to appear is just a
flat-out contradiction in terms!

A few comments below and then I really will say no more on this topic.

Lehar >
It is not a question of evidence, it is a question of definitions!
... To claim otherwise is to redefine a familiar term to mean
something completely different.
< Lehar

You have a strange theory of definition. Of course an experience is and
of course things appear in it. But it need not be what appears in it.

Lehar >
The *experience* of red absolutely *MUST* be red, it is the very
definition of redness.
< Lehar

This is such a strange statement that I actually do not understand what
you think it says.

From one of your other messages:

Lehar >
we know sense-data to exist because we experience them to exist.
< Lehar

We do not. We know experiences to exist and experience them and they are
front and centre in my theory. But sense-data theory is, well, a theory,
a theory of what experiences are like -- and can be wrong like any other
theory. That is presumably why it is called a *theory*. Or are you
redefining the word 'sense-data'?

Brook >
A few comments below and then I really will say no more on this topic.
< Brook

Likewise for me, we have taken this as far as it will go.

But it has not been a waste of time. Not at all! We have identified the kernel
of our disagreement.

The theory of direct percepton begins with the assumption that perception is
direct, and that initial assumption is not open to question, it is taken as
self-evidently true. Then the rest of our knowledge of perception is warped and
contorted beyond all recognition in order to conform to that initial assumption.

This warping includes denying the plainly manifest properties of experience, in
particular its spatial extendedness. This is achieved by simply pushing part of
experience out of the definition of experience, and asserting that the spatial
extendedness of experience is not a property of the experience itself, it is a
property of that which the experience is *of*. Even when the experience is
hallucinatory! So experience has been split into two entities, that which
experience *is*, and that which experience *appears* to be. But this is nothing
other than a fundamental re-definition of experience as not-experience, because
experience is already that which it *appears* to be, it cannot *be* something
other than appearance itself while still being experience. Experiences do
indeed have "magical self-certifying powers", they prove to us their own
existence, and they appear to us with their apparent properties.

Brook attempts to throw us off the scent by suggesting that this relates to the
distinction between the *vehicles* and the *contents* of experience, as if the
vehicle (neurons etc. in the brain) is what an experience *is*, whereas the
contents are what the vehicles *appear* to be. But this too is a blatant
re-definition, because experience is already by definition the *content*, it has
nothing whatsoever to do with the *vehicle* of experience.

Brook objects vehemently to the notion of sense-data, the factoring of the
experience of an object from the object itself. But it is always justifiable to
separate things into separable components, at least conceptually, even if only
to later show that the separated concepts are one and the same. But the reason
Andrew Brook objects to this perfectly legitimate factoring is that it draws
attention to an item which Andrew wishes passionately to ignore, which is the
spatial component of experience, something that is clearly distinct from that
which it is an experience *of*. In fact, Brook denies the very *existence* of
that thing we call the sense-data, because it is an embarassment to the theory
of direct perception. Sense data is not a theory, it is a factoring. However it
seems that direct perception cannot survive if sense data are allowed to exist.

The choice is ours. Either we accept the apparently incredible fact that the
world of experience is a spatial data structure within our own brain, or we are
permanently condemned to live in a world where experience is not as it appears
to appear, spatial experience is not experience, representations don't
represent, indirect perception is direct, the experience of red is not actually
red, and experience can somehow bypass the causal chain of sensory processing to
give us awareness of spatial extendedness directly out in the world without
(necessarily) any spatially extended representations in the brain. Even when
that world is hallucinatory!

Its a straight paradigmatic choice: pick the paradox you feel least
uncomfortable with.

But it is *NOT* a pseudo-problem. This question will be settled eventually, and
I believe it will be settled in our lifetime. Representationalism will be
confirmed as soon as we discover the volumetric spatial imaging mechanism in the
brain that accounts for our spatial experience.

And it will be discovered all the sooner if we know what it is that we are
looking for in the brain!

I said I wasn't going to say anything more on the appearance/reality of
experience issue but I have to respond to Steve's 'summary' of my
position because he gets it wrong on almost every point. Most of the
mistakes will be obvious to anyone who has been following the debate but
there is a new one. Contrary to what Steve says, I do not align the way
our experiences appear to us and what they are really like with the
vehicle/content distinction. Both the vehicle and the content merely
appear to us and neither appearance need tell us anything about what the
respective elements of experience actually are like.

Btw, hallucinations do have two components, the event of hallucinating
and what is hallucinated, i.e., the content or object of the hallucination.

Lehar >
The theory of direct percepton begins with the assumption that perception is
direct, and that initial assumption is not open to question, it is taken as
self-evidently true.
< Lehar

Steven keeps repeating this. I cannot speak for Andrew. It is
surely false for me.

My starting point was that perception is a deep mystery. It seems
impossible. Moreover, representationism does nothing to help. If
anything, representationalism makes the problem harder. For it would
seem that a representationalist has to solve two different perceptual
problems:

1: the problem of perceiving the world, in order to gather the
information needed to generate the internal representation

2: the problem of perceiving the internal representation.

Steven has yet to explain how representationalism solves anything.

Hmm, I did have another starting assumption, and that was some sort
of empiricism. I took it as implausible that the DNA could carry all
of the information and the assumptions that would be required by a
nativist/ innatist account of perception.

So how do we perceive the world? On my empiricist assumptions, this
has to require learning. We would have to learn the way that the
world is. The only way I can see that we could learn how to perceive
the world, is if that learning involves interactions with the world.
That way, our knowledge of the three dimensional structure of the
world can come from those geometric appliances we carry around with
us, namely our arms and legs (and, of course, our eyes which also
move).

That my perception is *direct* is due to my having learned how to
perceive via *direct* interaction with the world. Whether or not
there happen to be internal representations, my interaction was with
the world, not with those representations. That there may be several
layers of neural processing in no way affects the directness of my
interaction with the world, and therefore the directness of my
perception as it results from that interaction.

Lehar >
Then the rest of our knowledge of perception is warped and
contorted beyond all recognition in order to conform to that initial assumption.
< Lehar

I have no idea what warping, if any, Steven is referring to here. In
any case, since I don't make the initial assumptions that Steven is
talking about, those assumptions I do not make would surely not be
able to warp anything.

Lehar >
This warping includes denying the plainly manifest properties of experience, in
particular its spatial extendedness.
< Lehar

On second thoughts, after rereading your sentence above, I do deny
that spatial extendedness is a property of experience. Rather, my
experience is of the spatial extendedness of the world.

I guess you could say that as I travel, I carry my experience with
me. And I suppose that could be said to give experience a spatial
extendedness. Yet even that is doubtful, for my experience is only
at one place at a time, if it can even be said to be at any place at
all.

I guess we have quite different meanings for "experience". For me,
the word "experience" refers to my interactive relations with the
world. For Steven, "experience" seems to refer to that 3-dimension
movie now playing in the Cartesian theater that he presumes to be in
his head.

Lehar >
Brook objects vehemently to the notion of sense-data, the factoring of the
experience of an object from the object itself.
< Lehar

Again, I cannot answer for Andrew.

According to the sense-data theories, we supposedly infer things in
the world from sense-data. I cannot fathom what this sense-data is
supposed to be. For there is nothing in my experience from which I
infer things in the world. It has long seemed to me that sense-data
is a philosopher's invention, and not anything based on evidence.
Or, to put it differently, "sense-data" appears to be a theoretical
term invented to serve the needs of a particular philosophical theory
of perception.