Decision-making Code

Learning and Solving Associative Cognitive Tasks

Any thoughtful action takes into account prior knowledge and context.
I am implementing computational models to explain how cortical and
hippocampal networks can solve a range of tasks that require choices
based on learning and context.

I am using computational models to address the following questions:

(1) Why is the hippocampus necessary for solving some associative tasks,
but not other, similar ones?

(2) Is there a trade-off between persistent activity following an input,
as needed for short-term memory,
and the ability to associate one input with another?

(3) Does homeostasis explain the loss of ability to solve a
previously learned task upon removal of hippocampus,
when the task can be solved
more rapidly without the hippocampus?

(4) Can associations formed in the hippocampus bias a decision-making
network, enabling correct performance in a set of contextual tasks?

(5) Can the same associations in the hippocampus slow learning of
tasks that do not require hippocampal associations to form?

Parametric Working Memory and Sequential Discrimination

Working memory is the ability to hold information temporarily, `on-line'
in preparation for its imminent use in a decision. The mechanism of this
short-term memory is either through recurrent firing of neurons, particularly in
the prefrontal cortex, or through persistent currents which have been
measured in the entorhinal currents.

The task that I model requires monkeys to make a decision by
comparing two stimuli that are separated by a delay. Ranulfo Romo
and co-workers at UNAM in Mexico City train monkeys to
differentiate the vibrational frequency of stimuli to their finger tips.
The discrimination is sequential, with a delay between the two stimuli
(hence the need for working memory). Importantly the two stimuli are
to the same finger, and activate the primary sensory cortical neurons in
the same manner, just at different times. A model of the cortical network
hence requires a response based on whether the second stimulus is of
greater or lesser frequency than the first. As such, it must perform a
subtraction across time.

I am carrying out computer simulations of networks of neurons which maintain
persistent activity in response to a cue stimulus. As such, they exhibit the
critical features of working memory. The memory circuit is equivalent to
an integrator in the mathematical sense, as a greater frequency of vibration
for a fixed duration leads to greater activity in a memory store.
Similarly, an integrator does not change when its input ends, so
exhibits persistent activity during any delay between stimuli.
Much research is ongoing to address whether neuronal integrators are
essentially continuous, or more discrete, allowing for robustness.

In my model of the discrimination part of the task, I use a robust integrator
to hold the memory, but assume inhibitory connections from the integrator to
its inputs. This method of integral feedback control is common in engineering,
and allows for a robust cancellation of the input during the first stimulus,
leading to discrimination of the difference between two stimuli spaced in time.

Molecular basis of long-term memory

John Lisman and coworkers have put forward a model for long-term memory
based on the autophosphorylation of CaMKII holoenzymes and their
dephosphorylation by PP1.Influx of calcium into the synapse can cause
the process of autophosphorylation to dominate, leaving the CaMKII
holoenzymes in a highly phosphorylated "up" state. The system is bistable,
so that at resting calcium an unphosphorylated "down" state is stable,
as well as the "up" state. Bistability occurs, because (a) autophosphorylation
is cooperative within a holoenzyme and slow to get started, while (b)
dephosphorylation saturates, so the per molecule rate is slower in the "up"
state than "down" state.

The location of CaMKII molecules that can take part in
long-term potentiation (the strengthening of a synapse believed to
underly long-term memory) is the post-synaptic density. The number of
CaMKII holoenzymes in the postsynaptic density is between 5 and 30, depending
on its size. As in reality, all reactions occur stochastically, it is a
significant question as to how stable any memory storage can be. Fluctuations
will eventually cause a switch of the system between "up" and "down" states
or vice versa. Any such spontaneous switching is disruptive of memory, so it
is important to know the timescale of such behavior.
I have carried out both detailed simulations and analytical calculations
for the system of enzymes, verifying that the timescale for
spontaneous transitions rises exponentially with the number of enzymes
present. In a reasonable range of parameters, 20 holoenzymes are sufficient
to maintain memories on the timescale of human lifetimes, even when the
individual proteins are being replaced by turnover randomly on a timescale
of 30 hours.