Two regions in the occipito-temporal cortex respond more strongly to faces than to objects and are thought to be important for face perception: ‘the fusiform face area’ (‘FFA’) and the ‘occipital face area’ (‘OFA’). Whether these areas responding preferentially to faces play a dominant or exclusive role in face processing or if sub-maximal responses in other areas of the ventral stream such as the lateral occipital complex (LOC) are also involved is currently debated. To clarify this issue, we tested a brain-damaged patient presenting a face-selective deficit, prosopagnosia, with functional magnetic resonance imaging (fMRI). Using fMRI-adaptation, we found a dissociation between the coding of identity in the structurally intact ‘FFA’, which was impaired for faces but preserved for objects. This observation complements recent fMRI findings that the ‘FFA’ reflects averaging of heterogeneous highly selective neural populations for faces and objects, by showing here that the responses of these populations can be functionally independent. Most importantly, a larger response to different faces than repeated faces was found in the ventral part of the LOC both for normals and the patient, next to the right hemisphere lesion. Following prosopagnosia, areas that do not respond preferentially to faces such as the ventral part of the LOC (vLOC) may still be recruited to subtend residual individual face discrimination. Overall, these observations indicate that faces are processed through a network of visual areas in the human brain, with a subset of these areas responding preferentially to faces being critical for efficient face recognition.