Recently, we have shown that phase-randomized natural scene images are more effective than white noise masks at disrupting rapid scene-categorization performance in a backward-masking paradigm (Loschky et al. 2007; Loschky, Hansen et al., 2010). This is interesting because phase-randomized scene masks possess identical second-order statistics to their non-phase randomized counter-parts, while white noise masks possess no second-order statistical relationships. Thus, the second-order information contained in phase-randomized masks is processed at a level that interferes with rapid scene categorization. However, phase-randomized scenes differ from white noise by possessing 1/fα (α = 1.0)amplitude spectra with biases at the vertical/horizontal (i.e., cardinal) orientations. Here, we examined which aspect of the amplitude spectrum is responsible for masking. Specifically, is it a) the slope of the amplitude spectrum in phase-randomized masks, or b) the biases at different orientations, or c) both, that produce interference in rapid scene categorization for 1/fα (α = 1.0)noise as opposed to white noise (α = 0.0)? Target images were backward-masked by random-phase noise differing in slope and orientation bias at several SOAs. Target stimuli were from 6 scene categories (beach, forest, airport, street, home interior, and store interior). Mask stimuli were randomized phase noise having one of four different amplitude spectrum slopes (0.0, 0.5, 1.0, and 1.5). Within each slope set, we varied the cardinal orientation bias (5 levels from no-bias to maximal bias). The results showed that the amplitude spectrum slope drove the masking effects, with a slope of 1.0 masking scenes most, followed by 0.5, 1.5, and 0.0 respectively, but with little influence of mask orientation bias. Furthermore, the effects of slope were greatest early in scene processing (33ms to 67ms SOA). Lastly, control experiments verified that the masking effects cannot be accounted for by differences in perceived contrast of the noise masks.