Neural Networks Forget Information Quickly

Researchers have figured out the speed that neural networks in the cerebral cortex can delete sensory information is a bit of information per active neuron per second. The activity patterns of the neural network models are deleted nearly as soon as they are passed on from sensory neurons.

The scientists used neural network models based on real neuronal properties for the first time for these calculations. Neuronal spike properties were figured into the models which also helped show that the cerebral cortex processes were extremely chaotic.

Neural networks and this type of research in general are all helping researchers better understand learning and memory processes. With better knowledge about learning and memory, researchers can work toward treatments for Alzheimer’s disease, dementia, learning disabilities, PTSD related memory loss and many other problems.

More details are provided in the release below.

Out of mind in a matter of seconds

Surprising rate at which neuronal networks in the cerebral cortex delete sensory information

The dynamics behind signal transmission in the brain are extremely chaotic. This conclusion has been reached by scientists from the Max Planck Institute for Dynamics and Self-Organization at the University of Göttingen and the Bernstein Center for Computational Neuroscience Göttingen. In addition, the Göttingen-based researchers calculated, for the first time, how quickly information stored in the activity patterns of the cerebral cortex neurons is discarded. At one bit per active neuron per second, the speed at which this information is forgotten is surprisingly high. Physical Review Letters, 105, 268104 (2010)

The brain codes information in the form of electrical pulses, known as spikes. Each of the brain’s approximately 100 billion interconnected neurons acts as both a receiver and transmitter: these bundle all incoming electrical pulses and, under certain circumstances, forward a pulse of their own to their neighbours. In this way, each piece of information processed by the brain generates its own activity pattern. This indicates which neuron sent an impulse to its neighbours: in other words, which neuron was active, and when. Therefore, the activity pattern is a kind of communication protocol that records the exchange of information between neurons.

How reliable is such a pattern? Do even minor changes in the neuronal communication produce a completely different pattern in the same way that a modification to a single contribution in a conversation could alter the message completely? Such behaviour is defined by scientists as chaotic. In this case, the dynamic processes in the brain could not be predicted for long. In addition, the information stored in the activity pattern would be gradually lost as a result of small errors. As opposed to this, so-called stable, that is non-chaotic, dynamics would be far less error-prone. The behaviour of individual neurons would then have little or no influence on the overall picture.

The new results obtained by the scientists in Göttingen have revealed that the processes in the cerebral cortex, the brain’s main switching centre, are extremely chaotic. The fact that the researchers used a realistic model of the neurons in their calculations for the first time was crucial. When a spike enters a neuron, an additional electric potential forms on its cell membrane. The neuron only becomes active when this potential exceeds a critical value. “This process is very important”, says Fred Wolf, head of the Theoretical Neurophysics research group at the Max Planck Institute for Dynamics and Self-Organization. “This is the only way that the uncertainty as to when a neuron becomes active can be taken into account precisely in the calculations”.

Older models described the neurons in a very simplified form and did not take into account exactly how and under what conditions a spike arises. “This gave rise to stable dynamics in some cases but non-stable dynamics in others”, explains Michael Monteforte from the Max Planck Institute for Dynamics and Self-Organization, who is also a doctoral student at the Göttingen Graduate School for Neurosciences and Molecular Biosciences (GGNB). It was thus impossible to resolve the long-established disagreement as to whether the processes in the cerebral cortex are chaotic or not, using these models.

Thanks to their more differentiated approach, the Göttingen-based researchers were able to calculate, for the first time, how quickly an activity pattern is lost through tiny changes; in other words, how it is forgotten. Approximately one bit of information disappears per active neuron per second. “This extraordinarily high deletion rate came as a huge surprise to us”, says Wolf. It appears that information is lost in the brain as quickly as it can be “delivered” from the senses.

This has fundamental consequences for our understanding of the neural code of the cerebral cortex. Due to the high deletion rate, information about sensory input signals can only be maintained for a few spikes. These new findings therefore indicate that the dynamics of the cerebral cortex are specifically tailored to the processing of brief snapshots of the outside world.