Critical-band masking paradigm is a method that reveals the band of spatial frequencies used by human observers to identify a stimulus. Previous studies of letter recognition have shown that a) the critical band of frequencies is relatively narrow and b) the peak frequency in object frequency units changes with letter size, indicating scale-dependence and suggesting the existence of channels specialized for processing letters of various sizes (Majaj, Pelli, Kurshan, & Palomares, 2002; Oruc & Landy 2006). In this study, we investigated whether similar results are found for other types of visual stimuli. We characterized stimuli along two main dimensions: evolutionary relevance and amount of training. Letters are arbitrary shapes from an evolutionary perspective, but for which most observers are highly trained. Faces are not only well trained but may have long-standing evolutionary relevance in the human visual system. As arbitrary and untrained stimuli we used, first, a set of novel shapes, and second, mirror-image letters. We found, first, that all four types of patterns are recognized using a narrow band of frequencies, despite the fact that these are all broadband stimuli. Second, the critical frequency band shifted with changes in stimulus size in a similar manner for letters, reversed letters, and novel shapes. Faces on the other hand differed, in that there was a greater degree of scale invariance for larger stimuli. These results show that the critical frequencies found for letter processing are not unique to these linguistic symbols; face processing, however, may differ from other stimuli.