Reinforcement Learning of Visual Decision Making in RNNs (Master Thesis)

In most primates, including humans, visual perception is only fine grained in a small patch of the visual field called the fovea. Therefore, the fine details of a visual scene can often only be sensed when directly looking at them, and the eyes must constantly move from fixating one part of a visual scene to fixating the next. Since this takes time and resources, the question of where to look next is crucial for survival and a sophisticated neural system has evolved to control visual attention. This system includes a network of frontal brain areas that are also critical for higher cognition. We study the function of these areas using simultaneous recordings from hundreds of neurons while subjects solve naturalistic, visual, cognitive tasks involving eye movements.

A great deal of insight about how different brain areas might implement solutions to such tasks has come from (1) training recurrent neural networks (RNNs) to perform the tasks [1] and then (2) reverse engineering [2] the trained RNNs to generate candidate hypothesis of how the brain might do it [3]. One encouraging observation is that activations in the trained RNNs resemble neural activity recorded in animals performing the respective tasks.

Goal of this project is to use reinforcement learning [4] to train a recurrent architecture to control eye-movements of an artificial agent in eye-movement guided visual decision tasks.