In preparation for the talk given by Michael Shadlen April 25th 2007 ...

M. Shadlen, and others (e.g. Roger Carpenter's LATER model) propose that a perceptual decision is made when sufficient evidence has built up and crossed a threshold (diffusion to bound model). This supposes that there is a "moment of perception" that distinguishes not seeing sth from seeing it. If I recall correctly this idea of a moment of perception is at odds with SMC theory but I have not yet grasped exactly why that is. I get that linking this "moment of perception" to neural activity is at odds with the theory, but what's wrong with the idea of a moment of perception in itself?

on: Apr 23, 2007 [17:00]

score: 0.00

reads: 78790

Posted messages

Thanks for starting this, Therese. I also had a few thoughts on this...

Let's assume I accept the sensory-motor theory of perception. A perceptual event, e.g. me feeling an object touching my hand, is constituted not by neural activity in somatosensory cortex, nor anywhere else in the brain, but by my whole organisms' knowledge of the way sensory input will change in response to, e.g., moving my hand.

In psychology we 'measure' perception by requiring our subjects to make a behavioural response, e.g. a button press in response to a tactile stimulus. Concerning the 'moment of perception', it seems to be clear to me that the perceptual event happens at some time point (or period) between the object touching my hand and my response button press. Even if we can't pin it down to the millisecond, it does make sense to ask the question. To use the same analogy, France elected Sarkozy at some point or period between the voting offices opening and the official result being published. At least we know these outer limits.

In less controlled situations, when I'm not in an experiment, but (more or less) passively or introspectively perceiving, one of those limits is missing. We can't be sure when the perceptual process is completed, so it gets a bit more difficult.

Now, in my understanding of the sensory-motor account, the answer to the question of the 'moment of perception' would be: The perceptual event occurs when (or for the duration) the object touches my hand, because that is when I, as an organism who possesses the required knowledge, interact with the object. What other time point or time period would make sense in the sensory-motor account? To identify any point in time when some brain activity occurs would make the homuncular error.

This seems plausible to some extent. But if we consider using a very brief stimulus and additionally the delays in the nervous system from the skin to the brain, the claim would be that the perceptual event happens at a time, when not a single action potential has reached the brain. I don't think anybody seriously wants to make the claim, that the brain does not play any role in the perception of the stimulus.

Do I have to allow time for my organism to master my sensory-motor skills, possibly involving some neural processing in the brain, before I perceive? How much time? Do I make a 'temporal homuncular error' in asking this?

How does my organism deal with the problem of neural delays in sensory pathways? Is my perception of an event temporally veridical, or does it lag by any amount of time after the physical event occurs?

If we state that perception in general is an 'a posteriori description of what happens', then it seems to be out of the causal loop of events when interacting with the world.

>
> Let's assume I accept the sensory-motor theory of perception. A perceptual event, e.g. me feeling an object touching my hand, is constituted not by neural activity in somatosensory cortex, nor anywhere else in the brain, but by my whole organisms' knowledge of the way sensory input will change in response to, e.g., moving my hand.
>
> In psychology we 'measure' perception by requiring our subjects to make a behavioural response, e.g. a button press in response to a tactile stimulus. Concerning the 'moment of perception', it seems to be clear to me that the perceptual event happens at some time point (or period) between the object touching my hand and my response button press. Even if we can't pin it down to the millisecond, it does make sense to ask the question. To use the same analogy, France elected Sarkozy at some point or period between the voting offices opening and the official result being published. At least we know these outer limits.

I don't quite agree on this: you say: it makes sense to ask the question about when the perceptual event happened. But there are two events: the brain events and the perceptual event. These two events are two levels of description of what happened. The brain events can be described at the millisecond level. The perceptual event only makes sense in the mental conception of the observer. There is no direct link between the two, unfortunately.

Suppose the voting offices were open from 8 am to 8 pm, but that for some reason everybody in France voted between 8 am and 9 am. Did France elect Sarkozy between 8 am and 9 am then? We could say that yes, in some sense it did. But from an *administrative* point of view, the election took place from 8 am to 8 pm, and the administrators would say it was during the whole period that the election occurred.

Having no access to our internal neural states, we are only "administrators" of our behavior.

>
> In less controlled situations, when I'm not in an experiment, but (more or less) passively or introspectively perceiving, one of those limits is missing. We can't be sure when the perceptual process is completed, so it gets a bit more difficult.
>
> Now, in my understanding of the sensory-motor account, the answer to the question of the 'moment of perception' would be: The perceptual event occurs when (or for the duration) the object touches my hand, because that is when I, as an organism who possesses the required knowledge, interact with the object. What other time point or time period would make sense in the sensory-motor account? To identify any point in time when some brain activity occurs would make the homuncular error.
>

that's right

> This seems plausible to some extent. But if we consider using a very brief stimulus and additionally the delays in the nervous system from the skin to the brain, the claim would be that the perceptual event happens at a time, when not a single action potential has reached the brain. I don't think anybody seriously wants to make the claim, that the brain does not play any role in the perception of the stimulus.

but that argument is confusing the brain view and the perceptual ("administrative") view.

>
> Do I have to allow time for my organism to master my sensory-motor skills, possibly involving some neural processing in the brain, before I perceive? How much time? Do I make a 'temporal homuncular error' in asking this?

yes!!

>
> How does my organism deal with the problem of neural delays in sensory pathways? Is my perception of an event temporally veridical, or does it lag by any amount of time after the physical event occurs?

asking that question is again to make the error. There is no simple link between the delays in the nervous system and the "moment of perception". This is because the notion of "moment of perception" is something that can only be defined from the "administrative" point of view of the observer's thoughts and (potential) behavior.

Said in another way, there is no "fact of the matter" (i.e. the question has no answer because it makes no sense -- it is asked at the wrong level of description) about temporal veridicality, and temporal lags with regard to brain events.

(I would very much like to find a convincing analogy to explain this better. I admit that I have difficulty making this really clear to people)

>
> If we state that perception in general is an 'a posteriori description of what happens', then it seems to be out of the causal loop of events when interacting with the world.

Yes, I think that the link between the physical, neural events, and the mental and behavioural events is complicated. It must of course be causal since there is no magic. But the concept of "moment of perception" and "moment of brain event" cannot be put into direct correspondence.

Maybe a good example is the color phi phenomenon, where a red shape jumps to a nearby position and turns green at the same time. What we see is a red dot becoming green. What we see is continuous motion and continuous color change. We know the underlying neurophysiological mechanism pretty well. The question of where and when exactly we "perceive" the change from red to green makes no sense at the perceptual level of description: there is no "fact of the matter".

Gerritt: thank you for your contribution. I feel my reply is not wholly satisfactory. Further discussion might be helpful in trying to find a clearer understanding of these points.

G:
> > In psychology we 'measure' perception by requiring our subjects to make a behavioural response, e.g. a button press in response to a tactile stimulus. Concerning the 'moment of perception', it seems to be clear to me that the perceptual event happens at some time point (or period) between the object touching my hand and my response button press. Even if we can't pin it down to the millisecond, it does make sense to ask the question.
K:
> I don't quite agree on this: you say: it makes sense to ask the question about when the perceptual event happened. But there are two events: the brain events and the perceptual event. These two events are two levels of description of what happened. The brain events can be described at the millisecond level. The perceptual event only makes sense in the mental conception of the observer. There is no direct link between the two, unfortunately.

There may be no direct link between the two, but that does not mean there is no moment of perception. There can be A moment in time that separates perceiving from not perceiving, be it that the definition of perceiving is related to a brain state or whether it is related to the sensory-motor contigencies we have with the object at hand. Sensory-motor contingency theory stresses the role of attention towards the object and towards the action. There is a moment when that engagement of attention gives rise to a perception.

> Suppose the voting offices were open from 8 am to 8 pm, but that for some reason everybody in France voted between 8 am and 9 am. Did France elect Sarkozy between 8 am and 9 am then? We could say that yes, in some sense it did. But from an *administrative* point of view, the election took place from 8 am to 8 pm, and the administrators would say it was during the whole period that the election occurred.

Actually, there was one point in time where there were more votes for Sarkozy than voters left. There was one point where there were no more degrees of freedom. Could one not argue that an equivalent "point of no return" occurs for perception?
(And, as an aside, Sarkozy has not been elected (yet) ... and may never be!!!)

G:
> > Do I have to allow time for my organism to master my sensory-motor skills, possibly involving some neural processing in the brain, before I perceive? How much time? Do I make a 'temporal homuncular error' in asking this?
K:
> yes!!

The homuncular error comes from using the word "brain"!
But it does make sense to say that if the mastery of sensory-motor skills takes time, if the acces to the knowledge about the s-m contingencies takes time, if the attention that has to be directed to the object takes time then you have to say that perceiving is a process that has temporal boundaries, and that they may therefore be measurable.

G:
> > How does my organism deal with the problem of neural delays in sensory pathways? Is my perception of an event temporally veridical, or does it lag by any amount of time after the physical event occurs?
K:
> asking that question is again to make the error. There is no simple link between the delays in the nervous system and the "moment of perception". This is because the notion of "moment of perception" is something that can only be defined from the "administrative" point of view of the observer's thoughts and (potential) behavior.

I think the error comes not from asking the question, but from positing a causal relationship between brain and perceptual states. There can be a correlation (and I would bet there probably is).

K:
> Maybe a good example is the color phi phenomenon, where a red shape jumps to a nearby position and turns green at the same time. What we see is a red dot becoming green. What we see is continuous motion and continuous color change. We know the underlying neurophysiological mechanism pretty well. The question of where and when exactly we "perceive" the change from red to green makes no sense at the perceptual level of description: there is no "fact of the matter".

I understand your point, but if we have the feeling that we perceive the change occurring at some position in space between the two squares, and experimenters can manipulate what that position is, don't you think we've gone one step further in explaining something about perception if we can show correlated changes in brain states? We do not answer the question of why red feels red or green green but we do show that a change in perception is accompanied by a change in a brain state. Even if we do not want to make that link causal, and definitely not sufficient, we have to admit that we are one step closer to identifying one of the possible contributors to the complex phenomenon of perception.

1) maybe the Sarkozy example was not as good as the examples given by Dennett & Kinsbourne. Their idea is that when you read newspaper reports about countries, you read things like "France elected a new president on such and such a day". This is a compact way of describing the way the organism, France, behaves in a political context. But what is France other than an abstract concept which is useful in describing the way a certain group of people living in a certain part of the earth behave in an, amongst others, socio-political-economic context.

The notion of self or "I" could be a similar thing. It is a social construct which "we" find useful to describe how we interact with other organisms similar to us that we are interacting with. When we refer to the moment that "we" perceive something or take a decision, this is a construct which we we find useful in describing (to others but also to ourselves) our behavior within our environments. But this "we" is no single internal something, just as there is no single thing inside France that decided to elect this president, and no moment in time when France elected him (or much less probably, her!). Nevertheless saying things like France elected the president on this day is a useful concept when discussing France. Similarly, saying I saw that flash now is a useful way of describing what happened and what I was then able to do. But inside the brain there is no corresponding single event or moment that it makes sense to say it happened.

2) I dont like the term "postdiction" at all, because it also suggests some internal mechanism that reconstructs a picture or moment in time. More precisely, it suggests the existence of a little homunculus inside the brain looking at a film of what happens, moment by moment. Then, after the event happens, the film is patched up and reconstructed to postdict what happened, and presented to the homunculus. Anyway that's the idea that "postdiction" evokes in my mind (and judging from what the people who use the term say, I fear that that is their view).

" The notion of self ... is a social construct ... useful in describing (to others but also to ourselves) our behavior within our environments."

That's quite a Buddhist idea! The self does not exist in itself but is just a collection of events (see e.g. S. Collins, "Selfless Persons", Cambridge University Press, 1982/2004). One of the problems westerners have with Buddhism is the problem of how it it possible to reconcile philosophical selflessness with personal identity and psychological continuity.

"But this "we" is no single internal something"

Some (e.g. S. Pinker, "The Blank Slate", Penguin Books, 2002) argue that there is no self at all, single or plural. Pinker argues against the notion of a "ghost in the machine" (but it's a "pronoun in the machine"!). I agree with you and with him (on that point) but you have to admit that while it may be the case philosophically, as a psychological fact the notion of selflessness is difficult to adhere to. That's why I believe thinking about a unified self is a valid - and necessary - line of research in psychology.

I guess the idea here is that there are be two distinct domains of the self. In itself, as a area of philosphical inquiry, I see that your approach makes a very persuasive philosophical argument. However that does not have to encroach onto the other domain of the self, which is psychological. Research in psychology therefore has to explain that.

"saying I saw that flash now is a useful way of describing what happened and what I was then able to do. But inside the brain there is no corresponding single event or moment that it makes sense to say it happened."

I disagree. There is a difference in brain states between before and after the flash. There is a moment when that difference arises. (That moment of course does not have to be equivalent to physical time.) I take your point that this change in brain states is not causally related to the feeling of seeing, but would you not say that it is at least permissive?

"the term "postdiction" ... suggests the existence of a little homunculus inside the brain (and judging from what the people who use the term say, I fear that that is their view)."

Taken that way, the term postdiction indeed seems of little use. Would you ascribe to it as just a way to describe a potential relationship between physical and psychological time?

>> " The notion of self ... is a social construct ... useful in describing (to others but also to ourselves) our behavior within our environments."
>
> That's quite a Buddhist idea! The self does not exist in itself but is just a collection of events (see e.g. S. Collins, "Selfless Persons", Cambridge University Press, 1982/2004). One of the problems westerners have with Buddhism is the problem of how it it possible to reconcile philosophical selflessness with personal identity and psychological continuity.

ah interesting. I guess I should look at that. Do you have that book?

>
>> "But this "we" is no single internal something"
>
> Some (e.g. S. Pinker, "The Blank Slate", Penguin Books, 2002) argue that there is no self at all, single or plural. Pinker argues against the notion of a "ghost in the machine" (but it's a "pronoun in the machine"!).

Ah thank you , I didnt know that Pinker said that. Thanks for pointing me to him on this!

>I agree with you and with him (on that point) but you have to admit that while it may be the case philosophically, as a psychological fact the notion of selflessness is difficult to adhere to. That's why I believe thinking about a unified self is a valid - and necessary - line of research in psychology.

But in my view there is no contradiction. The notion of self, as a social construct, is a very real psychological fact (like love) and perhaps related to other social concepts like beauty and fame and richness. But such psychological and social facts are facts concerning the way the human agent behaves and reflects and considers him or herself, within the social environment. These facts must be considered at a level of description which involves the agent in its environment, and not a brain-level of description.

It would be what the philosophers call a vehicle-content confusion to think that what goes on inside the brain of an agent must somehow be isomorphic to what a description of its behavior requires. Behavior (and potential behaviors, including social constructs like love, shame, pride, and the self) are ways of describing what the whole organism does or tends to do or can do, as seen from a scientific perspective taken looking at the organism from the outside. There need be no internal replica of these behaviors, dispositions or notions. And if there were internal replicas (e.g. brain areas that become activated when you are in love), their activation would explain nothing at all. You would always have to explain further what it is about those brain areas that engenders the love. All this is not to say that the brain plays no role. Obviously the brain underlies the behaviours and potential behaviours. But it is a mistake to search for the notions themselves in the brain.

So, indeed, the psychologist can and should think about the unified self as a valid line of research. But he should not search for something that "generates" it in the brain. And just as neurons dont have to ooze out a red substance for you to see red, neurons dont have to respond NOW for you to see something NOW. Thinking this would be to make the mistake of confusing the neural code and what it encodes.

>
> I guess the idea here is that there are be two distinct domains of the self. In itself, as a area of philosphical inquiry, I see that your approach makes a very persuasive philosophical argument. However that does not have to encroach onto the other domain of the self, which is psychological. Research in psychology therefore has to explain that.
>
>> "saying I saw that flash now is a useful way of describing what happened and what I was then able to do. But inside the brain there is no corresponding single event or moment that it makes sense to say it happened."
>
> I disagree. There is a difference in brain states between before and after the flash. There is a moment when that difference arises. (That moment of course does not have to be equivalent to physical time.) I take your point that this change in brain states is not causally related to the feeling of seeing, but would you not say that it is at least permissive?

Yes brain events occur and changes in the brain register the flash. But because there is no "I" in the brain (if there were we would have to explain how it worked and we would be in a very sticky, infinite regress situation again), the notion that that "I" perceives at any particular moment in time makes no sense.

The brain changes accompanying the flash, and there are many, distributed over many moments in time, put the brain in a state where the agent is willing to say "I saw the flash". But the moment at which that happens may be different from the moment that the agent might press the button, or that the agent might move his or her eye. All these potential behaviours might occur temporally in a quite independent and uncorrelated fashion. When is the "real" moment of perception? To think there is such a thing is to make the mistake that there is an "I" *inside* the brain. But the "I" is not inside it, it is simply a useful way of talking about what the whole agent did or will do.

>
> "the term "postdiction" ... suggests the existence of a little homunculus inside the brain (and judging from what the people who use the term say, I fear that that is their view)."
>
> Taken that way, the term postdiction indeed seems of little use. Would you ascribe to it as just a way to describe a potential relationship between physical and psychological time?

I guess that would be ok. But for me the term is couched in a whole literature which makes the vehicle - content confusion, so it is better avoided!

Kevin:"I dont like the term "postdiction" at all, because it also suggests some internal mechanism that reconstructs a picture or moment in time. More precisely, it suggests the existence of a little homunculus inside the brain looking at a film of what happens, moment by moment. Then, after the event happens, the film is patched up and reconstructed to postdict what happened, and presented to the homunculus. Anyway that's the idea that "postdiction" evokes in my mind (and judging from what the people who use the term say, I fear that that is their view)."

My response: "Would you ascribe to the term postdiction as just a way to describe a potential relationship between physical and psychological time?"

Kevin:"I guess that would be ok. But for me the term is couched in a whole literature which makes the vehicle - content confusion, so it is better avoided!"

So my question to you Gerrit is: do people who study the flash-lag effect and explain it as an instance of postdiction believe that the flash lag effect is the result of perception being reconstructed from a representation (albeit imperfect)? Is there not a way to interpret (or word the interpretation of) the flash lag with the "soft" definition of postdiction that I just proposed?

I understand, but convinced... perhaps some more evidence needs to accumulate in my brain! 2 responses:

1) I think the Sarkozy analogy is unfair, because "France gave 30% to Sarkozy" is not only a generalizing statement, but it is also an event spread out in time - so of course it's difficult decide which of the constituent parts and which moment define the whole. Of course, none and all of them do. But take the simpler example that a subject has to make a saccade towards a red object. There is a point in time where the planning of the saccade starts because the red object has been perceived (not necessarily consciously, but somehow registered by the visual system). And there is another point in time where the planning of a hand movement would have started, if the task had been to move the hand to the red object. Are these not moments of perception? (Maybe that's part of your point: the moments of perception depend entirely on what action is to be planned?) I don't think it is the case that any other moment in the series of events that leads from a sensory stimulus to a response can be labeled a "moment of perception".

2) If the "moment of perception" is something that we make a posteriori as a description of what happens, then do you agree that perception can be the result of "post-dictive" processes?

There's nothing wrong with a model that proposes that evidence builds up and at some moment enough information becomes available for a decision to be taken. And that at some moment a decision is taken and a response programmed. And at some moment the programming of the response is finished and the actual response action is initiated.

But in this series of events, which was the actual moment of perception? When did the observer perceive the stimulus?

I suggest that there is no answer to this question. In Dennett & Kinsbourne's BBS article on this issue, they propose the analogy with a country. When did France actually decide to give 30% to Sarkozy? Was it at 20H when the voting offices closed? Was it when the official result was published at 22h? Was it some average of when the voters made their decision before voting? There is no answer.

To propose that there is a moment of perception in the brain supposes that there is a homunculus in the brain who then looks at whatever is perceived and perceives it at that moment. There is no necessity for things to work in this way, and for information to be brought together at one place and then redistributed for further actions to be taken. And even if that's how it worked, how would having information at that one place magically make it "perceived".

Another point:

to think that time in the brain is coded in the same sequential way that it seems to happen in our consciousness, is to make the same conceptual mistake as to think that neurons have to ooze out a red substance for us to see something red. It is the classical confusion between the "content" of perception and the "vehicle" that codes it.

the solution to all these problems is to realize that the notion of "moment of perception" is something that we as organisms fabricate a posteriori as a description of what happens. It is a a very useful description which is helpful to ourselves in accounting for and rationalizing our own behavior. But it is something which must be seen as a useful description seen from outside the organism. It is not something to be found IN the brain.