The head of the table: Marking the ‘front’ of an object is tightly linked with selection

Department of Psychology, Northwestern University

For the archival version of this research, and the preferred citation, see:

Xu, L. & Franconeri, S. L. (2012). The head of the table: Marking the "front" of an object is tightly linked with selection. Journal of Neuroscience, 32(4), 1408-1412. [DOI]

One of the goals of SILC is to understand the structure of an object's parts - within SILC's framework of spatial skills, such structure would be classified as 'intrinsic static'. Objects in the world do not have objective structural designations, such as axes or orientations. Instead, we assign structural organization to objects as part of visuospatial processing. One fundamental designation within object structure is how we mark one object surface as being its ‘front’. This designation is typically guided by several cues, including which surface is closest to the observer, which surface is most task-relevant or salient (e.g., the face is the front of a head), the source of action (e.g., the infrared emitter is the front of a remote control), or the direction of motion of an object (e.g. the leading edge is the front of a skateboard). However, when these cues are competing, weak, or absent (e.g. if the skateboard stops moving), we can also flexibly assign one surface as the ‘front’. For example, Figure 1 depicts a box, a cheese grater, and a table, each with competing

Figure 1. For some objects, the location of the "front" is ambiguous. For the box, one can impose at least three organizations: surface (1) being the front (e.g., a remote control), surface (2) being the front (e.g. a clock radio), or surface (3) being the front (e.g. a security camera). For the cheese grater, the 'front' is determined by the given task. For the table, we can designate an arbitrary position as 'head' of the table. In these cases, the 'front' of an object may be guided by the position of the attentional 'spotlight''.

surfaces that might be labeled the ‘front’. Here we explore a surprisingly simple mechanism that may allow the visual system to flexibly represent this core component of object structure. This abstract structural assignment may be guided by the location of the ‘spotlight’ of selection, where the selected region becomes the front. In a new paper from our lab (Xu & Franconeri, 2012, Journal of Neuroscience), we used an electrophysiological 'attention tracker' to show that seeing a surface as the ‘front’ has a tight temporal link to the position of the attentional 'spotlight'.

Figure 2. In the ambiguous figure task, the display is a modified version of a Necker Cube. The cubes on the right of the figure depict the two possible percepts.

In the first experiment, we tracked attention while participants rapidly switch their percepts between the two possible structures of a 'Necker' cube with ambiguous structure (Figure 2). We used an eye tracker to ensure that only their attention moved, and not their eyes. Either the left side of the cube was in front and the right in back, or vice versa (they always started with one percept and switched to the other). We gave them an auditory 'metronome' every second so that they'd see one percept for 1s, and then the other for another 1s. These results indicate that during the first 1s, participants shifted selection toward the perceived front of the cube, and in the second 1s they shifted to the new perceived front of the cube (Figure 3).

Figure 3. The difference waveforms in the ambiguous figure task, indicating relative selection of the 'front' or 'back' side of the cube. SeeXu & Franconeri, 2012, Journal of Neuroscience for details.

In the second experiment, we tracked attention during spontaneous reversals of the cube's structure (Figure 4a). The ambiguous cube was displayed for 8 seconds, while participants pressed a corresponding button each time their percept changed. Figure 4(b) depicts what they did with their attention before and after each reported switch. On average, participants shifted to the new front of the cube for the time period spanning from 900ms before till 600ms after the switch report.

Figure 4.(a) A schematic version of the analysis technique. Within an 8-second trial there could be several reports of a perceptual switch in the structure of the cube. We took response-locked ERPs at each report of a switch (see methods for details), and collapsed the two types of percept reports into a difference wave showing activity contralateral to the new perceived front of the cube. (b) The grand average of this difference wave across subjects. The magnitude of this difference wave does not reflect the actual position of spatial selection on the screen. The results show selection of the front face of the cube 900ms before and 600ms after the switch report, suggesting a shift of attention toward the new front side.

These experiments shows a tight temporal link between selective attention and the perceived structural organization of an ambiguous figure, during both directed and spontaneous changes of percepts. These results are consistent with similar fMRI evidence that attention is linked with object structure designations (Slotnick & Yantis, 2005), as well as results from previous behavioral studies have found that spatial selection can cause changes to interpretations of ambiguous images (e.g. Tsal & Kolbert, 1985; Peterson & Gibson, 1991). Manipulating an observer’s initial gaze can also bias percepts of the Necker cube (e.g. Elis & Stark, 1978) and the ambiguous old lady/young lady illusion (e.g. Georgiades, & Harris, 1997). Similar effects also occur within the interpretation of ambiguous scenes. In one study, participants were presented with images depicting events that can be described in either an active way or a passive way (e.g. “The dog is chasing the man.” or “The man is running away from the dog.”). Directing participants’ attention to different locations of the scene altered the sentence structure and the word choice when the scene was later described (Gleitman, January, Nappa, & Trueswell, 2007). The locus of spatial selection might serve to guide object structure because the features that drive a surface to be designated the ‘front’ – the closest surface, most task-relevant or salient surface, the source of action, or the direction of motion – are all features that also reliably cause a surface to be selected. Through experience, this relationship could serve to train a correlation between selection and the ‘front’. This correlation could then serve as a tool that allows an observer to control their interpretation of an object’s structure.

Our new work along this line extends to explore the role of selection in guiding dynamic processing of object structure, such as mental rotation (Figure 5). Rotating an object in one’s mind’s eye requires the observer to store a structural representation of the object in working memory. It is still unclear how exactly such structural representation is created and controlled in mental rotation (see Zacks, 2008, for a review). It is possible that the ‘spotlight’ of spatial attention can mark certain surface as special relative to others, which enables the visual system to “push” the object flexibly around an axis during mental imagery. If that's true, then perhaps some students who have difficulty with mental rotation just need to be taught how to 'push' in the right way - we can't wait to test this possibility in our lab!

Figure 5. Selection might also play a role in guiding dynamic processing of object structure, such as mental rotation. It is possible that the 'spotlight' of spatial attention can mark certain surface as special relative to others, which enables the visual system to "push" the object flexibly around an axis during mental imagery.