Anwar Nunez-Elizalde, Fatma Deniz, James Gao, Jack Gallant, University of California, Berkeley, United States

Abstract:

The human cerebral cortex comprises many functionally distinct areas that represent different information about the world. It has been challenging to map these areas efficiently. Here we present a new approach that addresses this problem. Subjects watch 30 interesting short films while their brain activity is measured with fMRI. The short films contain speech, video, music, environmental sounds, emotions, human interaction and narrative structure. Each film was labeled using more than a dozen different high-dimensional feature spaces reflecting the visual, auditory and conceptual content of the films. Brain activity elicited by the films is then modeled using a novel voxelwise encoding model based on simultaneous Tikhonov regularization of the labeled feature spaces using a multivariate normal prior. The resulting encoding model reveals which specific feature spaces are represented in each voxel, and how each voxel is tuned with respect to those features. To validate the approach we examined model predictions using 27 minutes of novel short films not used for model estimation. We find that the voxelwise encoding model significantly predicts activity of voxels distributed broadly across the cerebral cortex. Furthermore, we find that the pattern of feature selectivity across cortex is highly consistent across all five individual subjects. The recovered feature spaces can capture novel functional subdivisions, even within well-studied regions such as middle temporal cortex.