Our ability to categorize natural scenes is essential for visual tasks such as navigation or the recognition of objects in their natural environment. Although different classes of natural scenes often share similar image statistics, human subjects are extremely efficient at categorizing natural scenes. In order to map out the brain regions involved in scene categorization, we use multivariate pattern recognition to analyze the fMRI activation within a small spherical region (the searchlight, Kriegeskorte et al. 2006) that is positioned at every possible location in the brain. From local activity patterns in each searchlight, we attempt to predict the scene category that the subject viewed during the experiment. Such an analysis allows us to generate a spatial map of those brain regions producing the highest classification accuracy. Furthermore, we can generate similar maps of the correlation of the pattern of errors made by the classification algorithm at each searchlight location with the pattern of errors made by human subjects in an accompanying behavioral experiment. Lastly, we ask which searchlight locations show a decrement in prediction accuracy for up-down inverted images relative to upright images, to reveal brain regions that may participate in the inversion effect that we found in the behavioral experiment. Together, these maps implicate large regions of the ventral visual cortex in the categorization of natural scenes, including area V1, the parahippocampal place area (PPA), retrosplenial cortex (RSC), and lateral occipital complex (LOC), previously shown to be involved in natural scene categorization (Caddigan et al., VSS 2007 & VSS 2008; Walther et al. HBM 2007 & SfN 2008) as well as other intermediate-level visual areas. We further explore the functions of these regions with respect to natural scene categorization and attempt to find their specific contributions to the scene categorization process.