The act of seeing might be a temporally extended one where visual information that is not sufficiently attended can be overwritten by new or remaining information. Behavioral support for this view of perception comes from work on object substitution masking (OSM), which can occur when a target in a search array is surrounded by a mere 4 dots. If the dots linger in the display after the target disappears, perception of the target suffers relative to when the dots offset simultaneously with the target (Di Lollo, Enns, & Rensink, 2000). But can any lingering stimulus overwrite a developing target representation regardless of its similarity to the target, or is OSM instead modulated by a subject's top-down attentional set? Jiang and Chun (2001) found that bottom-up featural similarity between the mask and target did not affect OSM, yet work on inattentional blindness has shown that attentional set heavily influences subjective perception (Most et al., 2001). Here, we investigate OSM under conditions in which subjects are forced to adopt an attentional set to distinguish a target from distractors. In one study, one black ‘C’ and 1 white ‘C’ (each randomly facing forwards or backwards) appeared among other black and white distractors on each trial, and subjects reported the orientation of either the black or white ‘C’ for the duration of the experiment. Thus, they were required to establish an attentional set for luminance. Across trials, the luminance of the 4-dot masks either matched or did not match the luminance that subject were attending to. When the luminance of the mask matched that of the target set, OSM was substantially more pronounced than when the luminances of the mask and target did not match. Additional experiments suggest that this modulation of OSM emerges as an effect of attentional set, rather than simply the featural similarity between the mask and target.