Menu

26) Some complex object categories, such as faces, have dedicated areas of cortex for processing them, but are also represented in a distributed fashion (Kanwisher – 1997, Haxby – 2001)

Early in her career Nancy Kanwisher used functional MRI (fMRI) to seek modules for perceptual and semantic processing. She was fortunate enough to discover what she termed the fusiform face area; an area of extrastriate cortex specialized for face perception.

This finding was immediately controversial. It was soon shown that other object categories also activate this area. Being the adept scientist that she is, Kanwisher showed that the area was nonetheless more active for faces than any other major object category.

Then came a slew of arguments purporting that the face area was in fact an ‘expertise area’. This hypothesis states that any visual category with sufficient expertise should activate the fusiform face area.

This argument is based on findings in cognitive psychology showing that many aspects of face perception once thought to be unique are in fact due to expertise (Diamond et al., 1986). Thus, a car can show many of the same perceptual effects as faces for a car expert. The jury is still out on this issue, but it appears that there is in fact a small area in the right fusiform gyrus dedicated to face perception (see Kanwisher’s evidence).

James Haxby entered the fray in 2001, showing that even after taking out the face area from his fMRI data he could predict the presence of faces based on distributed and overlapping activity patterns across visual cortex. Thus it was shown that face perception, like visual perception of other kinds of objects, is distributed across visual cortex.

It used to be that the main thing anyone "knew" about the dopamine system was that it is important for motor control.
Parkinson's disease, which visibly manifests itself as motor tremors, is caused by disruption of the dopamine system (specifically, the substantia nigra), so this was an understandable conclusion.

When Wolfram Schultz began recording from dopamine neurons in mice and monkeys he was having trouble finding correlations with his motor task. Was he doing something wrong? Was he recording from the right cells?

Instead of towing the line of dopamine = motor control he set out to find out what this system really does. It turns out that it is related to reward.

Schultz observed dopamine cell bursting at the onset of unexpected reward. He also observed that this bursting shifts to a cue (e.g., a bell sound) indicating a reward is forthcoming. When the reward cue occurs but no reward follows he saw that the dopamine cells go silent (below resting firing rate).

This pattern is quite interesting computationally. The dopamine signal mimics the error signal in a form of reinforcement learning called temporal difference learning.

This form of learning was originally developed by Sutton. It is a powerful algorithm for learning to predict reward and learn from errors in attaining reward.

Temporal difference learning basically propagates reward prediction back in time as far as possible, thus facilitating the process of attaining reward in the future.

Figure: (Top) No conditioned stimulus cue is given, so the reward is unexpected and there is a big dopamine burst. (Middle) The animal learns to predict the reward based on the cue and the dopamine burst moves to the cue. (Bottom) The reward is predicted, but since no reward occurs there is a depression in dopamine release.Source: Figure 2 of Schultz, 1999. (News in Physiological Sciences, Vol. 14, No. 6, 249-255, December 1999)

This request amounts to underhanded legal intimidation as using these graphs clearly falls under fair use. Shelley had clearly cited the source of the graph and accurately reported the results.

What might be their motivation for the legal threat? According to this comment by Shelley: “I think perhaps what the real issue here is that they were afraid I might bust their ‘press bubble.’ This study has been used as a justification for ‘fruity alcoholic drinks are health food’ and the spin was so ubiquitious throughout news venues it obviously was released that way. The real results do not support that conclusion.”
Many across the blogosphere have chimed in to support Shelley, and I’d like to be counted as one of them.

Science is about openness and sharing knowledge for the higher good of scientific progress. It’s perhaps telling that the legal threats included the parent company’s slogan “SCI – where science meets business”. They’re apparently too caught up in the business side of things, where legal threats for knowledge sharing are the norm.

Researchers investigating eye movements and attention recorded from different parts of the primate brain and found several regions showing very similar neural activity. Goldman-Rakic proposed the existence of a specialized network for the control of attention.

Many computational modelers emphasize the emergence of attention from the local organization of sensory cortex (e.g., local competition). However, when a shift in attention is task-driven (i.e., top-down) then it appears that a specialized system for attentional control drives activity in sensory cortex. Many properties of attention likely arise from the organization of sensory cortex, but empirical data indicate that this is not sufficient.

With the advent of neuroimaging in humans (PET and fMRI), Posner et al. found very similar regions as those reported by Goldman-Rakic. He found that some regions are related more to orienting to stimuli, while others are related more to cognitive control (i.e., controlled processing).

After many fMRI studies of cognitive control were published, Wager et al. performed a meta-analysis looking at most of this research. They found a set of cortical regions active in nearly all cognitive control tasks.

My own work with Schneider (in press) indicates that these regions form an innate network, which is better connected than the rest of cortex on average. We used resting state correlations of fMRI BOLD activity to determine this. This cognitive control network is involved in controlled processing in that it has greater activity early in practice relative to late in practice, and has greater activity for conflicting responses (e.g., the Stroop task).

Though these regions have similar responses, they are not redundant. Our study showed that lateral prefrontal cortex is involved in maintaining relevant task information, while medial prefrontal cortex is involved in preparing and making response decisions. In most cases these two cognitive demands are invoked at the same time; only by separating them in time were we able to show specialization within the cognitive control network. We expect that other regional specializations will be found with more work.

I’ll be covering my latest study in more detail once it is published (it has been accepted for publication at NeuroImage and should be published soon). The above figure is from that publication. It lists the six regions within the human cognitive control network. These regions include dorsolateral prefrontal cortex (DLPFC), inferior frontal junction (IFJ), dorsal pre-motor cortex (dPMC), anterior cingulate / pre-supplementary motor area (ACC/pSMA), anterior insula cortex (AIC), and posterior parietal cortex (PPC).

A general computational insight arising from this work (starting with Goldman-Rakic) is that cortex is composed of specialized regions that form specialized networks. This new paradigm for viewing brain function weds the old warring concepts of localized specialization and distributed function.

Implication: The mind, largely governed by reward-seeking behavior on a continuum between controlled and automatic processing, is implemented in an electro-chemical organ with distributed and modular function consisting of excitatory and inhibitory neurons communicating via ion-induced action potentials over convergent and divergent synaptic connections altered by timing-dependent correlated activity often driven by expectation errors. The cortex, a part of that organ organized via local competition and composed of functional column units whose spatial dedication determines representational resolution, is composed of many specialized regions forming specialized networks involved in perception (e.g., touch: parietal, vision: occipital), action (e.g., frontal), and memory (e.g., short-term: prefrontal, long-term: temporal), which depend on inter-regional connectivity for functional integration, population vector summation for representational specificity, and recurrent connectivity for sequential learning.

Penfield had shown that motor cortex is organized in a somatotopic map. However, it was unclear how individual neurons are organized. What does each neuron’s activity represent? The final location of a movement, or the direction of that movement?

Shwartz & Georgopoulos found that movement direction correlated best with neural activity. Further, they discovered that the neurons use a population code to specify a given movement. Thus, as illustrated above, a single neuron responds to a variety of movement direction but has one preferred direction (PD).

The preferred direction code across a large population of neurons thus sums to specify each movement.

Schwartz has since demonstrated how these population vectors can be interpreted by a computer to control a prosthetic arm. He has used this to imbue monkeys with Jedi powers; able to move a prosthetic limb in another room (or attached) with only a thought. Using this technology the Schwartz team hopes to allow amputee humans to control artificial limbs as they once did their own.

A general computational insight one might take from the Schwartz & Georgopoulos work is the possibility of population coding across cortex. It appears that all perception, semantics, and action may be coded as distributed population vectors.

Representational specificity within these vectors likely arises from conjunctions of receptive fields, and are dominated by those receptive fields most specific to each representation.

Implication: The mind, largely governed by reward-seeking behavior on a continuum between controlled and automatic processing, is implemented in an electro-chemical organ with distributed and modular function consisting of excitatory and inhibitory neurons communicating via ion-induced action potentials over convergent and divergent synaptic connections altered by timing-dependent correlated activity often driven by expectation errors. The cortex, a part of that organ organized via local competition and composed of functional column units whose spatial dedication determines representational resolution, is composed of many specialized regions involved in perception (e.g., touch: parietal, vision: occipital), action (e.g., frontal), and memory (e.g., short-term: prefrontal, long-term: temporal), which depend on inter-regional connectivity for functional integration, population vector summation for representational specificity, and recurrent connectivity for sequential learning.

In 1986 Michael Jordan (the computer scientist, not the basketball player) developed a network of neuron-like units that fed back upon itself. Jeff Elman expanded on this, showing how these recurrent networks can learn to recognize sequences of ordered stimuli.

Elman applied his recurrent networks to the problem of language perception. He concluded that language relies heavily on recurrent connectivity in cortex; an unproven but well-accepted statement among many scientists today.

The year after Elman's demonstration of sequence learning with language, Walter Schneider (Schneider & Oliver, 1991) used a recurrent network to implement what he termed a 'goal processor'. This network can learn arbitrary task sequences, effectively expanding recurrent networks beyond language learning to learning new tasks of any type.

See this article for a review of a model implementing a goal processor.

The goal processor has been likened to a part of neocortex (dorsolateral prefrontal cortex) shown to be involved in maintaining goal information in working memory. Also, this maintenance is believed to occur via local (and/or via long-range fronto-parietal connections) recurrent connectivity.

Implication: The mind, largely governed by reward-seeking behavior on a continuum between controlled and automatic processing, is implemented in an electro-chemical organ with distributed and modular function consisting of excitatory and inhibitory neurons communicating via ion-induced action potentials over convergent and divergent synaptic connections altered by timing-dependent correlated activity often driven by expectation errors. The cortex, a part of that organ organized via local competitionand composed of functional column units whose spatial dedication determines representational resolution, is composed of many specialized regions involved in perception (e.g., touch: parietal, vision: occipital), action (e.g., frontal), and memory (e.g., short-term: prefrontal, long-term: temporal), which depend on inter-regional connectivity for functional integration and recurrent connectivity for sequential learning.

Pitts & McCullochprovided amazing insight into how brain computations take place. However, their two-layer perceptrons were limited. For instance, they could not implement the logic gate XOR (i.e., 'one but not both'). An extra layer was added to solve this problem, but it became clear that the Pitts & McCulloch perceptrons could not learn anything requiring more than two layers.

Rumelhart solved this problem with two insights.

First, he implemented a non-linear sigmoid function (approximating a neuronal threshold), which turned out to be essential for the next insight.

Second, he developed an algorithm called 'backpropagation of error', which allows the output layer to propagate its error back across all the layers such that the error can be corrected in a distributed fashion. See P.L.'s previous post on the topic for further details.

Rumelhart & McClelland used this new learning algorithm to explore how cognition might be implemented in a parallel and distributed fashion in neuron-like units. Many of their insights are documented in the two-volume PDP series.

Unfortunately, the backpropagation of error algorithm is not very biologically plausible. Signals have never been shown to flow backward across synapses in the manner necessary for this algorithm to be implemented in actual neural tissue.

However, O'Reilly (whose thesis advisor was McClelland) expanded on Hinton & McClelland (1988) to implement a biologically plausible version of backpropagation of error. This is called the generalized recirculation algorithm, and is based on the contrastive-Hebbian learning algorithm.

O'Reilly and McClelland view the backpropagating error signal as the difference between the expected outcome and the perceived outcome. Under this interpretation these algorithms are quite general, applying to perception as well as action.

Hebb's original proposal was worded as such: "When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased." [emphasis added]

The phrase "takes part in firing" implies causation of B's activity via A's activity, not simply a correlation of the two.

There are several ways to go beyond correlation to infer causation. One method is to observe that one event (e.g., cell A's activity) comes just before the caused event (e.g., cell B's activity).

In 1983 Levy showed with hippocampal slices that electrically stimulating cell A to fire before cell B will cause long-lasting strengthening of the synapse from cell A to cell B. However, when the opposite occurs, and cell A is made to fire after cell B, there is depotentiation of the same synapse. In other words, timing is essential for synaptic learning. Today, this form of learning is called spike-timing dependent plasticity (STDP).

Using this rule, Levy has created a variety of neural network models aimed at understanding memory in the brain (e.g., especially in the hippocampus; see this paper for a short review).

More recently, other researchers including Sakmann, Bi, Poo, and Dan have further characterized this phenomenon. They showed that it occurs in vivo, within a specific time window (~8 msec timing difference is optimal), in neocortex, and (using behavioral evidence) in humans.

Figure caption: A) Figure from Bi & Poo (1998) showing the effects of STDP in potentiation and depotentiation with optimal results ~8-10ms in either direction. B) Figure from Markram et al. (1997) showing the timing of the stimulation relative to the post-synaptic cell's EPSP. C) Another figure from Markram et al. (1997) showing the resulting long-term changes in synaptic efficacy due to the manipulations in figure B.

Implication: The mind, largely governed by reward-seeking behavior on a continuum between controlled and automatic processing, is implemented in an electro-chemical organ with distributed and modular function consisting of excitatory and inhibitory neurons communicating via ion-induced action potentials over convergent and divergent synaptic connections strengthened by timing-dependent correlated activity. The cortex, a part of that organ organized via local competitionand composed of functional column units whose spatial dedication determines representational resolution, is composed of many specialized regionsinvolved in perception (e.g., touch: parietal, vision: occipital), action (e.g., frontal), and memory (e.g.,short-term: prefrontal, long-term: temporal),which depend on inter-regional communication for functional integration.

Hubel and Wiesel's work with the development of cortical columns (see previous post) hinted at it, but it wasn't until Grossberg and Kohonen built computational architectures explicitly exploring competition that its importance was made clear.

Grossberg was the first to illustrate the possibility of self-organization via competition. Several years later Kohonen created what is now termed a Kohonen network, or self-organizing map (SOM). This kind of network is composed of layers of neuron-like units connected with local excitation and, just outside that excitation, local inhibition. The above figure illustrates this 'Mexican hat' function in three dimensions, while the figure below represents it in two dimensions along with its inputs.

These networks, which implement Hebbian learning, will spontaneously organize into topographic maps.

For instance, line orientations that are similar to each other will tend to be represented by nearby neural units, while less similar line orientations will tend to be represented by more distant neural units. This occurs even when the map starts out with random synaptic weights. Also, this spontaneous organization will occur for even very complex stimuli (e.g., faces) as long as there are spatio-temporal regularities in the inputs.

Another interesting feature of Kohonen networks is that the more frequent input patterns are represented by larger areas in the map. This is consistent with findings in cortex, where more frequently used representations have larger cortical areas dedicated to them.

There are several computational advantages to having local competition between similar stimuli, which SOMs can provide.

One such advantage is that local competition can increase specificity of the representation by ruling out close alternatives via lateral inhibition. Using this computational trick, the retina can discern visual details better at the edges of objects (due to contrast enhancement).

Another computational advantage is enhancement of what's behaviorally important relative to what isn't. This works on a short time-scale with attention (what's not important is inhibited), and on a longer time-scale with increases in representational space in the map with repeated use, which increases representational resolution (e.g., the hand representation in the somatosensory homonculus).

You can explore SOMs using Topographica, a computational modeling environment for simulating topographic maps in cortex. Of special interest here is the SOM tutorial available at topographica.org.

Implication: The mind, largely governed by reward-seeking behavior on a continuum between controlled and automatic processing, is implemented in an electro-chemical organ with distributed and modular function consisting of excitatory and inhibitory neurons communicating via ion-induced action potentials over convergent and divergent synaptic connections strengthened by correlated activity. The cortex, a part of that organ organized via local competition and composed of functional column units whose spatial dedication determines representational resolution, is composed of many specialized regionsinvolved in perception (e.g., touch: parietal, vision: occipital), action (e.g., frontal), and memory (e.g.,short-term: prefrontal, long-term: temporal),which depend on inter-regional communication for functional integration.

During the 1970s those studying the cognitive computations underlying visual search were at an impasse. One group of researchers claimed that visual search was a flat search function (i.e., adding more distracters doesn't increase search time), while another group claimed that the function was linear (i.e., adding more distracters increases search time linearly).

Both groups had solid evidence supporting their view. What were the two groups doing differently that could explain such different results?

As a graduate student working with Shiffrin, Schneider sat the two groups down during a scientific conference to have them figure out why their results differed so much. Needless to say, little was accomplished as both sides talked past one another.

Several years later Schneider & Shiffrin came to the realization that the two groups were practicing their subjects differently. The group with the flat search function allowed their subjects to practice the search task many times before collecting data. In contrast, the group with the linear search function began collecting data as soon as their subjects could perform the task.

This realization lead Schneider & Shiffrin to posit a distinction between automatic (flat search function) and controlled (linear search function) processing. In a landmark set of papers they clearly demonstrated this dual process distinction along with the boundary conditions of controlled and automatic task performance.

Patient H.M., with his lack of long term memory but largely intact working (short-term) memory, illustrated a dissociation between these two forms of memory. While long-term memory seemed to rely on hippocampus and the neocortical temporal lobes, in the 1960s it was not clear how working memory might be maintained.

Hebb had postulated a distributed and dynamic mechanism for working memory that was quite hard to test. However, a more testable hypothesis had emerged from observations of patients with prefrontal cortex damage. Such patients had trouble making and carrying out plans over time, possibly due to a working memory deficit. Previous work by Jacobsen lesioning primate prefrontal cortex supported this theory, but this work was far from conclusive.

In 1970 Joaquin Fuster cooled the monkey prefrontal cortex, showing a reversible deficit in working memory. The following year he recorded from monkey prefrontal cortex neurons and found cells that maintained activity over delay periods (‘memory cells’). These neurons respond not just to the stimulus presentation and the response, but also maintain activity between the two events (see figure for illustration).

A decade later Fuster found visual memory cells in inferior temporal cortex. Subsequent research has suggested that the prefrontal memory cells drive the temporal cortex memory cells to maintain their activity.

Patricia Goldman-Rakic, another monkey neurophysiologist, was instrumental in elucidating the network properties of working memory function. She showed in 2000 that lateral prefrontal and posterior parietal cortices mutually support the sustained working memory activity discovered by Fuster. She also showed the importance of the dopamine system and thalamus in working memory function.

Since the advent of PET and functional MRI in the 1990s a number of researchers have extended the primate working memory findings to humans. Some of these researchers include Jonathan Cohen, Mark D’Esposito, Michael Petrides, Julie Fiez, and John Jonides.

Implication: The mind, largely governed by reward-seeking behavior, is implemented in an electro-chemical organ with distributed and modular function consisting of excitatory and inhibitory neurons communicating via ion-induced action potentials over convergent and divergent synaptic connections strengthened by correlated activity. The cortex, a part of that organ composed of functional column units whose spatial dedication (determined via local competition) determines representational resolution, is composed of many specialized regions involved in perception (e.g., touch: parietal, vision: occipital), action (e.g., frontal), and memory (e.g., short-term: prefrontal, long-term: temporal), which depend on inter-regional communication for functional integration.
[This post is part of a series chronicling history’s top brain computation insights (see the first of the series for a detailed description). See the history category archive to see all of the entries thus far.]

Hubel & Wiesel showed that the ocular dominance columns they had discovered in cortex (see previous post) are organized during a critical period of development. Keeping one eye of an animal shut during the first few months of life made that animal blind in that eye for the rest of its life. Keeping the eye shut for the same amount of time later in life had no such effect.

Hubel & Wiesel found that the ocular dominance columns became lopsided in such a case: The functional eye tended to take over cortical space not used by the blind eye. This finding extended Penfield‘s notion that the amount of cortical space dedicated to a function determines its resolution. In this case, visual acuity was decreased through smaller cortical space for the unused eye.

Importantly, this experiment illustrated that the eyes compete for cortical space, with the most active eye claiming the most space. Generalizing this finding, it appears that many representations in cortex compete for space, whether it is in visual, motor, or somatosensory cortex.

V. S. Ramachandran applied these findings in an interesting case involving phantom limb pain. He found that many patients who still felt pain in their amputated hand also reported feeling a touch on that phantom hand when their face was touched. As can be seen in the second figure of the previous post on the sensory-motor homunculus, the hand and face representations are next to each other in cortex.

Ramachandran interpreted this to mean that the loss of input from the hand allows the face representation to win its competition with the hand representation, allowing it take over the cortical space previously dedicated to the hand. This also suggests that this competition continues into adulthood in some cases.

It is quite difficult to localize the epileptic origin in some seizure patients. Rather than removing the gray matter of origin, neurosurgeons sometimes remove white matter to restrict the seizure to one part of the brain.

One particularly invasive procedure (the callosotomy) restricts the seizure to one half of cortex by removing the connections between the two halves. This is normally very effective in reducing the intensity of epileptic events. However, Sperry & Gazzaniga found that it comes at a price.

They found that presenting a word to the right visual field of a patient without a corpus callosum allowed only the patient’s left hemisphere to become aware of that word (and vice versa). When only the side opposite the one which was presented the word was allowed to respond, it had no idea what word had been presented.

The two hemispheres of cortex could not communicate, and thus two independent consciousnesses emerged.

Sperry & Gazzaniga also found that the left hemisphere, and not the right, could typically respond linguistically. This suggested that language is largely localized in the left hemisphere. (See the above figure for illustration.)

The functional distinction between the left and right hemispheres is supported by many lesion studies. Generally, the left hemisphere is specialized for language and abstract reasoning, while the right hemisphere is specialized for spatial, body, emotional, and environment awareness. The boundary between these specializations has been trivialized in the popular media; it is actually quite complex and, in healthy brains, quite subtle.

Mountcastle found that nearby neurons in monkey somatosensory cortex tend to activate for similar sensory experiences. For example, a neuron might respond best to a vibration of the right index finger tip, while a neuron slightly deeper in might respond best to a vibration of the middle of that finger.

The neurons with these similar 'receptive fields' are organized vertically in cortical columns. Mountcastle distinguished between mini-columns, the basic functional unit of cortex, and hyper-columns, which are functional aggregates of about 100 mini-columns.

Hubel & Wiesel expanded Mountcastle's findings to visual cortex, discovering mini-columns showing line orientation selectivity and hyper-columns showing ocular dominance (i.e., receptive fields for one eye and not the other). The figure below illustrates a typical spatial organization of orientation columns in occipital cortex (viewed from above), along with the line orientations corresponding to each color patch.

Prior to performing surgery, Wilder Penfield electrically stimulated epileptic patients' brains while they were awake. He found the motor and somatosensory strips along the central sulcus, just as was found in dogs by Fitsch & Hitzig (see previous post). The amount of cortex dedicated to a particular part of the body was proportional not to that body part's size, but to its fine control (for motor cortex) or its sensitivity (for somatosensory cortex).

Thus, the hands and lips have very large cortical spaces relative to their size, while the back has a very small cortical space relative to its size. The graphical representation of this (see above) is called the 'homunculus', or 'little man'.

Penfield also found that stimulating portions of the temporal lobe caused the patients to vividly recall old memories, suggesting that memories are transfered from the archicortical hippocampus into neocortical temporal lobes over time.