People are worse at recognizing other-race faces than own-race faces. This other-race effect (ORE) has been attributed to worse discrimination of other-race faces, as reflected in the phrase "they all look the same to me". A neglected challenge in face recognition has been the ability to recognize a face's identity across superficial changes (e.g., expression, hairstyle). Indeed, even for own-race faces, photos of the same person can be perceived as belonging to different individuals, unless that person is familiar (Jenkins et al., 2011). We investigated how within-person variability affects our perception of identity for own and other-race faces. Caucasians (n=49) were given 40 photographs of two unfamiliar people (20 photographs/model) and asked to sort them into piles such that each pile had all of the pictures of one person. The photos were either of own-race (UK celebrities) or other-race (Chinese celebrities) faces. Participants had more difficulty discriminating other-race faces; more participants put two different people into the same pile for other-race (92%) than own-race (63%) faces. Notably, participants sorted the photographs into significantly more identities for other-race (M = 10.96; range = 4 to 31) than for own-race faces (M = 4.79; range = 2 to 16: Cohen's d =1.18). It is unlikely that the smaller number of piles for own-race faces reflects less variability among the Caucasian photographs. In an ongoing study, Chinese participants (to date, n = 6) sorted the Caucasian photographs into an average of 15 identities (range = 8 to 20). These findings suggest that studies in which the same image of a face is used for presentation and test may under-estimate the challenge of recognizing other-race faces. In the real world it may be the case that 'they all look different to me'.