Functional hierarchies are a prevalent feature of brain organization. In high-level visual cortex, the “occipital face area” (OFA/IOG-faces) is thought to be the input to a specialized processing hierarchy subserving human face perception. However, evidence supporting or refuting the causal role of IOG-faces as a necessary input to the face network evades researchers because it necessitates a patient with a focal lesion of the right inferior occipital cortex, as well as functional measurements both before and after surgical removal of this region. Here, in a rare patient fulfilling both of these requirements, we show that the face network is surprisingly resilient in two ways following surgical removal of IOG-faces. First, the large-scale cortical layout and selectivity of the face network are stable after removal of IOG-faces. Second, following resection, face-selective responses in ventral temporal cortex surprisingly become more reliable in the resected hemisphere, but not in the intact hemisphere. Further investigations of the anatomical underpinnings of this resiliency using diffusion tensor imaging suggest the existence of additional white matter pathways connecting early visual cortex to downstream face-selective regions independent of IOG-faces. Thus, after resection, neural signals can still reach downstream regions via these pathways that are largely unconsidered by present neurofunctional models of face processing. Altogether, these measurements indicate that IOG-faces is not the key input to the face network. Furthermore, our results pose important constraints on hierarchical models in high-level sensory cortices and provide powerful insight into the resiliency of such networks after damage or cortical trauma.