AbstractMultiplelines of evidence indicate that visual attention's temporal properties differ between the left and right visual fields (LVF and RVF).Notably, recent electroencephalograph recordings indicate that event-related potentials peak earlier for LVF than for RVF targets on bilateral-stream rapid serial visual presentation (RSVP) identification tasks. Might this hastened neural response render LVF targets perceptually available sooner than RVF targets? If so, how might the visual system reconcile these timing differences to estimate simultaneity across the LVF and RVF? We approached these questions by presenting bilateral-stream RSVP displays that contained opposite-hemifield targets and requiring participants to judge both the targets' temporal order and simultaneity. The temporal order judgments (TOJs) revealed that participants perceived LVF targets ∼134 ms sooner than RVF targets. This LVF hastening approximates a full cycle of visual attention's canonical ∼10 Hz (∼100 ms) temporal resolution. In contrast, performance on the simultaneity task did not exhibit the LVF hastening observed on the TOJ task, despite identical retinal stimulation across the two tasks. This finding rules out a stimulus-driven “bottom-up” explanation for the task-specific behavior. Moreover, error patterns across the two tasks revealed that, within the decision stage of simultaneity judgments, participants remapped LVF targets, but not RVF targets, to a later time in the RSVP sequence. Such hemifield-specific remapping would effectively compensate for the cross-hemifield asymmetries in neural response latencies that could otherwise impair simultaneity estimates.

Introduction

Neural computational work by John Tsotsos (1990) prompted Jeremy Wolfe to conclude that “processing everything, everywhere, all at once requires a brain that will not fit in the human head” (Wolfe et al., 2006, p. 177). Evolution's response to this neural computational limit is attention—the selection of sensory information. Attention selects—or samples—sensory information more slowly than our visual system samples the environment. For example, humans can discriminate a continuous light from one flickering at 30–50 Hz (Kelly, 1961; Rovamo & Raninen, 1984; Andrews, White, Binder, & Purves, 1996), yet our ability to select a flickering source's individual phase components for further analysis is limited to just ∼10 Hz (Rogers-Ramachandran & Ramachandran, 1998; Verstraten, Cavanagh, & Labianca, 2000; Battelli, Cavanagh, Martini, & Barton, 2003; Aghdaee & Cavanagh, 2007). This difference highlights the distinction between the temporal resolution of stimulus-driven vision versus that of visual attention. Visual attention's comparatively low temporal resolution (∼10 Hz) has important consequences in everyday life. For instance, while driving at 70 miles per hour (112.65 km/h) visual attention's ∼10 Hz temporal resolution permits only one sample every 10.27 feet (3.13 m), potentially a life and death difference in a crash. Here, we investigated visual attention's temporal resolution by requiring participants to judge the relative timing of two targets embedded amid distractors in rapid serial visual presentation (RSVP) displays.

Prior RSVP experiments have demonstrated that participants often incorrectly report the temporal order of two targets separated by ∼100 ms, i.e., approximately one cycle of visual attention's 10 Hz temporal resolution (Reeves & Sperling, 1986; Chun & Potter, 1995; Chun, 1997; Isaak, Shapiro, & Martin, 1999; Spalek, Falcon, & Di Lollo, 2006; Akyürek, Riddell, Toffanin, & Hommel, 2007; Olivers, Hilkenmeier, & Scharlau, 2011; Spalek, Lagroix, Yanko, & Di Lollo, 2012). Accuracy on these temporal order judgment (TOJ) tasks improves at larger target asynchronies. The relatively low TOJ accuracy at the ∼100 ms asynchrony could be explained by either of two hypotheses that Olivers et al. (2011) recently suggested. One hypothesis maintains that at brief (∼100 ms) asynchronies the two targets merge into a single attentional episode, rendering the physically sequential targets perceptually simultaneous. An alternative hypothesis allows for the possibility that participants perceive the targets sequentially but that the perceived temporal order can be reversed if the second target receives more attention than the first. This alternative hypothesis is physiologically plausible as recent single-cell and local field potentials recorded from the primate visual cortex demonstrate that attention reduces neural response latencies and increases neural contrast gain and response gain (Hudson, Schiff, Victor, & Purpura, 2009; Sundberg, Mitchell, Gawne, & Reynolds, 2012). Sundberg et al. (2012) “speculate that latencies may be reduced at higher contrast because stronger stimulus inputs drive neurons more rapidly to spiking threshold, while attention may reduce latencies by placing neurons in a more depolarized state closer to threshold before stimulus onset” (p. 16040). Although further experiments are needed to determine which of Olivers et al.'s hypotheses better explains TOJ errors at brief target asynchronies, both hypotheses posit that this phenomenon reflects the influence of visual attention.

Verleger et al.'s (2011) and Verleger et al.'s (2013) hastened ERPs to LVF targets raise important perceptual questions. Are LVF targets perceived sooner than RVF targets? If so, how might the visual system reconcile these timing differences to estimate simultaneity across the LVF and RVF? We approached these questions by presenting dual-stream RSVP displays similar to Verleger et al.'s (2011) and Verleger et al.'s (2013) and requiring each participant to judge both the targets' temporal order and simultaneity. Two surprising findings emerged. First, the TOJ task revealed that participants perceived the LVF targets ∼134 ms sooner than the RVF targets. This LVF hastening approximates a full cycle of visual attention's canonical ∼10 Hz (∼100 ms) temporal resolution (Verstraten et al., 2000; VanRullen & Dubois, 2011). Second, the simultaneity judgments exhibited error patterns that indicated participants remapped LVF targets, but not RVF targets, to a later time in the RSVP sequence. Such hemifield-specific remapping would effectively compensate for the cross-hemifield asymmetries in neural response latencies that could otherwise impair simultaneity estimates.

Method

Participants

Denison University's Human Subject Committee approved the experiments in this study, which were conducted with the understanding and written consent of each participant. Twenty-three Denison University undergraduates with normal or corrected-to-normal vision completed the study. All were naïve regarding the purpose of the experiment, and several expressed surprise when lateral hemifield effects were mentioned during the debriefing. We did not ask our participants to indicate handedness. Although Efron (1963) reported that handedness was correlated with hemifield differences in simultaneity and TOJs, subsequent and more complete psychophysical (Newman & Albino, 1977) and fMRI (Badzakova-Trajkov, Häberling, Roberts, & Corballis, 2010) findings have argued against this handedness effect.

Each RSVP sequence contained two targets (one in each lateral hemifield) and 38 distractors (19 in each lateral hemifield). One of the targets was an endogenous2 (less salient) black digit, randomly selected in each trial from the set 1, 2, 3, 4, 5, and 6. The other target was an exogenous red letter, randomly selected in each trial from the set D, F, G, J, K, and L. The distractors comprised all other letters, randomly sequenced, and always presented in black.

Tasks

Each participant completed a simultaneity task and a TOJ task. The simultaneity task required participants to report whether the exogenous (red letter) target and the endogenous (black number) target flashed at the “same” time or at “different” times. The TOJ task required participants to report which of the two targets flashed first, “letter” versus “number.” Note that a motor response bias favoring either response option would be unrelated to the targets' lateral hemifield configuration—the variable of interest. This precaution eliminated motor response biases as an explanation for any subsequently observed lateral hemifield effects. Participants responded on a standard computer keyboard with no restrictions on which finger or hand to use.

Procedure

TOJs occurred on one day, and simultaneity judgments occurred on a different day, counter-balanced across participants. Retinal stimulation remained identical across the two sessions. Within each daily session, each participant completed 22 practice trials, followed by five 120-trial blocks (600 trials for analysis). Each 120-trial block comprised 40 synchronized trials and 80 asynchronized trials. The 80 asynchronized trials comprised four instances each of 20 experimental conditions. These comprised five target asynchronies (133, 267, 400, 533, and 667 ms) crossed with two exogenous-target-hemifield configurations (hereafter, LVF-exo vs. RVF-exo) crossed with two target orders (hereafter, Exo-first vs. Endo-first). Among the 40 synchronized trials, half contained LVF-exo (RVF-endo) targets, and half contained RVF-exo (LVF-endo) targets. All trials contained at least one target in the 11th stimulus pair, which occurred 1.333 s into the RSVP sequence. All conditions were randomized anew within each 120-trial block.

On the simultaneity task, participants were informed initially—and subsequently reminded between trials blocks—that asynchronized and synchronized trials would occur in a 2:1 ratio. On the TOJ task, participants were informed that “letter-first” and “number-first” trials would occur equally often. To maintain motivation on each task, immediate visual feedback identified each response as correct or incorrect. “Letter-first” and “number-first” responses on TOJ trials with synchronized targets were not objectively classifiable as correct or incorrect. Consequently, half of the synchronized TOJ trials were predesignated arbitrarily as “letter first” and half as “number first” to avoid biasing participants' responses with feedback.

Data analysis

On the TOJ task, we assessed how hemifield affected the perception of relative target timing by constructing psychometric functions separately for the LVF- and RVF-exo target conditions. Within each of those conditions, the psychometric function's ordinate reflected the proportion of “letter-first” responses, i.e., Exo-target-first responses. The abscissa comprised target asynchronies ranging between −667 (Endo-target first) and +667 (Exo-target first) ms in 133-ms steps. A least-squares procedure was then used to fit the data with a sigmoid of the form

where K and Xo determine the slope and midpoint of the sigmoid, respectively. The correlation between the best fitting sigmoid and the data, as indexed by the Pearson correlation coefficient (r), was statistically significant (p < 0.05) within each target-hemifield condition for each participant. Because each fit was significant, it was possible to use Xo from the sigmoidal fits to determine each participant's point of subjective equality (PSE). The PSE corresponds to the temporal asynchrony at which the participant made “letter-first” and “number-first” responses equally often, i.e., 0.5 on the ordinate.3 The temporal resolution of our PSE algorithm was 0.53 ms (1886 Hz). We derived this value in a simulation that assumed 51 rather than 50 “letter-first” responses in the 100 synchronized trials and nonbiased responding on the 200 asynchronized trials. In addition to the PSEs, which indexed time perception, we also computed the 75% just noticeable differences, which indexed time discrimination. These discrimination thresholds were defined as half the stimulus (target-asynchrony) change required to alter the response rate from 0.25 to 0.75.

On the simultaneity task, we assessed how hemifield affected the perception of relative target timing by computing, separately for the LVF- and RVF-exo target conditions, the criterion value according to signal detection procedures (Green & Swets, 1966). Conceptually, the criterion value reflects the decision boundary between the two response options (the black vertical line in Figure 2). “Same” versus “different” responses on our simultaneity task occurred, respectively, when the level of sensory evidence for asynchronized targets fell below versus exceeded the criterion value. Computationally, the criterion value was derived from the proportion of hits and false alarms, respectively defined here as correct and incorrect “different” responses. Specifically, we z-transformed the proportions of hits and false alarms, then computed the following likelihood ratio:

The criterion from signal detection theory. On our simultaneity task, asynchronized targets occurred twice as often as synchronized targets, reflected here by the relative heights of the corresponding probability distributions (Wickens, 2002). The neutral criterion (vertical black line) occurs at the sensory evidence level at which the likelihood ratio of the asynchronized-to-synchronized distributions (the ratio of y-axis values) matched our 2:1 asynchronized-to-synchronized target ratio. Criterion values shifted left and right of this neutral point respectively generate lower and higher likelihood ratios, which correspond to biases favoring the “different” and “same” responses, respectively.

Figure 2

The criterion from signal detection theory. On our simultaneity task, asynchronized targets occurred twice as often as synchronized targets, reflected here by the relative heights of the corresponding probability distributions (Wickens, 2002). The neutral criterion (vertical black line) occurs at the sensory evidence level at which the likelihood ratio of the asynchronized-to-synchronized distributions (the ratio of y-axis values) matched our 2:1 asynchronized-to-synchronized target ratio. Criterion values shifted left and right of this neutral point respectively generate lower and higher likelihood ratios, which correspond to biases favoring the “different” and “same” responses, respectively.

The 134 ms LVF hastening is a large effect, approximating a full cycle of visual attention's canonical ∼10 Hz (∼100 ms) temporal resolution (Verstraten et al., 2000; VanRullen & Dubois, 2011). A temporal discrepancy of this magnitude raises an important question. How might the 134-ms LVF hastening influence simultaneity judgments across the lateral hemifields? The most parsimonious possibility would be to assume that simultaneity judgments and TOJs are governed by the same neural events. To the extent this were true, one would expect simultaneity judgments to exhibit the same LVF hastening observed on the TOJ task. This predicts that the proportion of “different” responses on the simultaneity task would be lowest, i.e., perceived simultaneity would be greatest, whenever LVF targets physically lag 133 ms behind RVF targets. The corresponding target asynchronies would be +133 ms on RVF-exo trials (when LVF-endo targets come second) and −133 ms on LVF-exo trials (when LVF-exo targets come second). Pink and light blue arrows, respectively, for RVF- and LVF-exo trials, show these predicted target asynchronies in Figure 5, to which we now turn.

Simultaneity task results. Participants reported whether the exogenous letter target and the endogenous number target appeared at the same time or at different times. The symbols and x-axis values are the same as in Figure 3. Arrows in pink (RVF-exo trials) and light blue (LVF-exo trials) indicate the predicted target asynchronies at which “different” responses would be lowest if the LVF hastening that affected temporal-order judgments also affected simultaneity judgments. The black arrow indicates the target asynchrony at which “different” responses were actually lowest. Error bars reflect +1 SEM (N = 23).

Figure 5

Simultaneity task results. Participants reported whether the exogenous letter target and the endogenous number target appeared at the same time or at different times. The symbols and x-axis values are the same as in Figure 3. Arrows in pink (RVF-exo trials) and light blue (LVF-exo trials) indicate the predicted target asynchronies at which “different” responses would be lowest if the LVF hastening that affected temporal-order judgments also affected simultaneity judgments. The black arrow indicates the target asynchrony at which “different” responses were actually lowest. Error bars reflect +1 SEM (N = 23).

Data from the simultaneity task disconfirm the LVF hastening predicted from the TOJ task. As Figure 5 reveals, the proportion of “different” responses was lowest when the two targets were physically synchronized (black arrow, 0 ms asynchrony), not at the predicted asynchronies (pink and light blue arrows, ±133 ms asynchronies). Critically, because retinal stimulation remained identical across the TOJ and simultaneity tasks, factors beyond the stimulus-driven “bottom-up” neural response must have caused the difference between the observed and predicted outcomes. This implies that the TOJ and simultaneity tasks are not governed by the same decision rule.

Information about the decision rule on the simultaneity task is revealed by comparing performance across the two smallest target asynchronies. Pairwise comparison between the LVF-exo trials (blue circles) at plus versus minus 133 ms reveals significantly more “different” responses, t(22) = 4.376, p = 0.0002, ηp = 0.465, when LVF-exo targets came second (−133 ms; M = 68.478; SE = 2.837) than when LVF-exo targets came first (+133 ms; M = 51.087; SE = 3.913). This pattern is exactly opposite to what would be predicted from the LVF hastening observed on the TOJ task. That is, LVF hastening should have reduced “different” responses when LVF-exo targets came second (−133 ms) and should have increased “different” responses when LVF-exo targets came first (+133 ms). The observed pattern not only disconfirms LVF-exo target hastening, it also reveals that participants performed the equivalent of remapping LVF-exo targets to a later point in the RSVP sequence. Critically, this remapping was specific to LVF-exo targets and did not occur for RVF-exo targets. Pairwise comparison between the RVF-exo trials (red squares) at plus versus minus 133 ms reveals significantly more “different” responses, t(22) = 6.359, p < 0.0001, ηp = 0.648, when RVF-exo targets came second (−133 ms; M = 64.565; SE = 2.433) than when RVF-exo targets came first (+133 ms; M = 39.130; SE = 3.439). That difference is consistent with the LVF hastening observed on the TOJ task. Overall, the data shown in Figure 5 reveal that, within the decision stage of simultaneity judgments, participants effectively remapped LVF-exo targets, but not RVF-exo targets, to a later time in the RSVP sequence. In the General discussion, we model this remapping by adding a delay to the time at which LVF-exo targets are estimated to arrive at the decision stage of the simultaneity task.

Further information about the decision stage of the simultaneity task arises from another aspect seen in Figure 5. Specifically, LVF and RVF response rates (blue circles and red squares) generally overlapped except when the targets were synchronized (0 ms), t(22) = 3.152, p = 0.005, ηp = 0.311 and when the exogenous-letter target led by the briefest asynchrony (+133 ms), t(22) = 2.295, p = 0.032, ηp = 0.193. At each of those temporal intervals, participants made “different” responses significantly less often in RVF-exo trials (red squares) than in LVF-exo trials (blue circles). This suggests longer PSEs, i.e., requiring larger temporal asynchronies to make a “different” response in RVF-exo trials than in LVF-exo trials.

To more formally quantify such hemifield-specific criterion shifts, we computed PSEs separately for RVF- and LVF-exo trials. These appear in Figure 6, separately for endo-target–first trials (top panel) and exo-target–first trials (bottom panel). Here, the ordinates represent likelihood ratios based on signal detection theory (Green & Swets, 1966). On our simultaneity task, lower likelihood ratios reflect more frequent use of the “different” response; higher likelihood ratios reflect more frequent use of the “same” response (see Method). Figure 6 reveals that participants perceived the targets' relative timing as “same” more frequently (higher likelihood ratios) on RVF-exo trials than on LVF-exo trials with nonoverlapping likelihood ratios occurring at the briefest asynchrony (133 ms). The PSEs—shown by the arrows drawn downward from the point at which each Weibull function intersects the neutral likelihood ratio of two—were also longer for RVF- than for LVF-exo trials. These PSEs are plotted in Figure 7. Relative to PSEs in the LVF-exo condition (blue bars), RVF-exo PSEs (red bars) increased by ∼55 ms when the endo target physically led and by ∼75 ms when the exo target physically led. Therefore, regardless of whether endo or exo targets physically led, participants demanded greater sensory evidence (longer target asynchronies) on RVF- than LVF-exo trials to report that the targets differed in time. In short, Figures 6 and 7 demonstrate compelling evidence for hemifield-dependent PSE shifts in our simultaneity task. In the General discussion, we model this hemifield-dependent PSE shift on RVF-exo trials by increasing the criterion (i.e., the temporal asynchrony required to make a “different” response) within the simultaneity task's decision rule.

Simultaneity task likelihood ratios. The symbols are the same as in Figure 5. The point of subjective equality (PSE) on the simultaneity task occurred when the likelihood ratio equaled two because asynchronized targets were presented twice as often as simultaneous targets. Likelihood ratios greater and less than two respectively reflect biases favoring “same” and “different” responses. Error bars reflect ±1 SEM (N = 23).

Figure 6

Simultaneity task likelihood ratios. The symbols are the same as in Figure 5. The point of subjective equality (PSE) on the simultaneity task occurred when the likelihood ratio equaled two because asynchronized targets were presented twice as often as simultaneous targets. Likelihood ratios greater and less than two respectively reflect biases favoring “same” and “different” responses. Error bars reflect ±1 SEM (N = 23).

Movie 2 shows the percentage of errors that occurred in all experimental conditions tested here. Visual inspection reveals that error rates on both the TOJ and simultaneity tasks tended to decrease when target asynchronies exceeded 133 ms as one would expect. By contrast, two counterintuitive phenomena can be observed in the data near the abscissa's center. First, on the simultaneity task, there was a significant crossover interaction between exo-hemifield and target asynchrony, F(1, 22) = 7.715, p = 0.011, ηp = 0.260 (see diagonal green ovals within Movie 2). Specifically, LVF-exo errors exceeded RVF-exo errors when the targets were physically synchronized, but the reverse error pattern occurred when the exogenous target led by 133 ms. This interaction is what one would expect if, to make a “different” response on the simultaneity task, participants demanded significantly larger target asynchronies (i.e., longer PSEs) on RVF-exo trials than on LVF-exo trials. Second, when the exogenous target led by 133 ms (see vertical gray oval within Movie 2), simultaneity-task errors significantly exceeded TOJ-task errors within each hemifield condition, t(22) > 6.240, p < 0.001, ηp > 0.639. This is counterintuitive because discerning the targets' temporal order would seem to require greater time-specificity than discerning whether the targets were asynchronized at all. Additionally, this error pattern did not occur when the endogenous target led by 133 ms (data points at −133 ms).

Spatiotemporal distribution of errors on the TOJ and simultaneity tasks. Symbols are the same as in Movie 2. Dotted lines (simultaneity task) and solid lines (TOJ task) highlight error patterns that distinguish the two tasks from each other. Error bars reflect ±1 SEM (N = 23).

Figure 8

Spatiotemporal distribution of errors on the TOJ and simultaneity tasks. Symbols are the same as in Movie 2. Dotted lines (simultaneity task) and solid lines (TOJ task) highlight error patterns that distinguish the two tasks from each other. Error bars reflect ±1 SEM (N = 23).

Lastly, although our research question addressed time perception rather than time discrimination,Table 1 contains 75% discrimination thresholds for our TOJ and simultaneity tasks to inform future experiments or replication.

The present experiments assessed lateral hemifield differences in attention's temporal properties. Specifically, participants judged the temporal order and simultaneity of opposite-hemifield targets embedded in dual-stream RSVP displays. The experiments generated the following main findings:

A ∼134 ms hastened perception of LVF targets on the TOJ task (Figures 3 and 4);

Patterns of simultaneity-task responses that disconfirm those predicted from the hastened perception of LVF targets on the TOJ task (Figure 4 vs. Figure 5);

These outcomes occurred while retinal stimulation remained identical across the TOJ and simultaneity tasks. This finding rules out a stimulus-driven “bottom-up” explanation for the task-specific behavior. We next consider models that account for these findings via task-specific decision-stage algorithms that receive identical stimulus-driven “bottom-up” inputs. Subsequently, we discuss how the present findings relate to prior studies.

Modeling the TOJ and simultaneity task results

One might model the present TOJ task results with an algorithm or decision rule, its inputs, and its output. The inputs to the decision rule include an estimated time of arrival (ETA) for our dual-stream RSVP displays' left and right visual field targets, LVFETA and RVFETA. For these ETAs, no unusual neural activity would be needed. The early visual pathway's retinotopic architecture would readily register each target's lateral hemifield, and illusions like the classic Pulfrich effect4 demonstrate that the early visual pathway labels stimuli with the functional equivalent of ETAs. The ETAs could be combined with a bias to generate an output—a TOJ response (“LVF first” vs. “RVF first”)—according to the following decision rule:

If ([LVFETA – RVFETA] + bias) < 0, “LVF first,” else “RVF first.”

The bias would be expressed in milliseconds and would correspond to a participant's PSE. When the bias is zero, the output would be determined entirely by the ETAs assigned in the early visual pathway. Negatively and positively signed bias values would generate, respectively, inclinations to favor “LVF first” and “RVF first” responses. In principle then, the hastened perception of LVF targets on the present TOJ task (Figures 3 and 4) could reflect hastened early visual system responses to LVF targets, a decision-stage bias, or both. A decision-stage bias favoring the LVF-first response seems unlikely, however, because our TOJ task required participants to report “letter first” versus “number first,” not “LVF first” versus “RVF first.” By contrast, a hastened early visual pathway response to LVF targets seems plausible given prior physiological data showing ERPs peaking earlier for LVF than for RVF targets on a dual-stream RSVP identification task (Verleger et al., 2011; Verleger et al., 2013).

While a hastened early visual pathway response to LVF targets would be sufficient to explain our TOJ results, additional factors are needed to model our simultaneity results. This is because our simultaneity results disconfirmed two predictions that would be made if the LVF hastening on the TOJ task also occurred on the simultaneity task. First, on the simultaneity task, “different” responses were lowest (perceived simultaneity was greatest) at the 0 ms target asynchrony, not at the predicted ±133 ms asynchronies. (See Figure 5, black arrow vs. pink and light blue arrows.) Second, LVF-exo trials on our simultaneity task generated a performance asymmetry opposite to that predicted from the hastened perception of LVF targets on the TOJ task. Specifically, participants perceived brief asynchronies as “different” significantly more often when LVF-exo targets followed (−133 ms), rather than preceded (+133 ms), RVF-endo targets (Figure 5). Moreover, hastened early visual pathway responses to LVF targets would not explain why our participants exhibited greater PSEs when judging simultaneity on RVF-exo trials relative to LVF-exo trials (Figures 6 and 7). Taken together, these results require a model of the present simultaneity task that incorporates the exo-target's hemifield.

One such model of the present simultaneity task contains the following two-step decision rule:

If the exogenous target is in the LVF, add a delay to LVFETA, else increase the bias;

If | LVFETA – RVFETA | > bias, “different,” else “same.”

The delay added to LVFETAs in step one represents the remapping of time that is reflected in the title of this article. We previously noted this LVF-specific remapping in the results for the simultaneity task (see Figure 5, LVF-exo simultaneity trials). Similarly, the bias increase in step one represents the observed RVF-exo–specific PSE shift also previously noted in the results for the simultaneity task (see Figures 5 through 7, RVF-exo simultaneity trials). In step one, determining the exogenous target's hemifield would not require any unusual neural activity. Indeed, Buschman and Miller's (2007) electrophysiological recordings suggest that exogenous and endogenous targets are registered by distinct neural synchrony rates and distinct cortical sequences of synchronized neural firing. Further support for distinct neural responses to endogenous versus exogenous targets comes from hemispatial-neglect patients' dissociations between these target types (Esterman, McGlinchey-Berroth, & Milberg, 2000), and from TMS experiments linking the temporoparietal junction with exogenous but not endogenous attention (Chica, Bartolomeo, & Valero-Cabré, 2011). In step two, the bias would be expressed in milliseconds and would correspond to a participant's PSE as on the TOJ task. Unlike the bias on the TOJ task, however, the bias in step two of the simultaneity task would be nonnegative. This is because a sufficiently large target asynchrony in either direction (the absolute value of LVFETA – RVFETA) would constitute evidence for a “different” response on the simultaneity task. Larger biases would indicate that participants demand greater sensory evidence (longer PSEs) for a target asynchrony before making a “different” response.

This two-step model explains the above-noted significant three-way (task by Exo-hemifield by target-order) interaction in error rates at the briefest (133 ms) target asynchrony (solid vs. dotted lines in Figure 8). To appreciate this, consider how the error rates change across the two tasks within a given exo-hemifield condition. When the exogenous target is in the LVF on simultaneity trials, the model adds a delay to the LVFETA to counter the early visual pathway's hastened response to LVF targets. The consequence of this LVF delay, though, depends on target order. Specifically, the delay increases the signal that participants must detect (the target asynchrony) when LVF-exo targets come second but decreases the signal to be detected when the LVF-exo targets come first. This explains our empirical observation that in LVF-exo trials when the task changed from TOJ (blue triangles) to simultaneity (blue circles) error rates declined when exo-targets came second (−133 ms) but increased when exo-targets came first (+133 ms). On RVF-exo trials, our model increases the value of the bias within the simultaneity decision. This drives simultaneity error rates above TOJ error rates because the LVFETA – RVFETA difference remains identical in the two tasks but must overcome an elevated bias within the simultaneity decision. The elevated bias explains our empirical observation that in RVF-exo trials when the task changed from TOJ (red triangles) to simultaneity (red circles) RVF-exo errors increased regardless of target order and despite identical retinal stimulation across the two tasks.5

We emphasize that the present simultaneity data could be modeled in other ways as well. The model proposed above, however, offers three desirable features. First, it parsimoniously explains the diverse range of empirical findings from our simultaneity experiment. Second, it relies only on physiological properties empirically documented by single unit recordings in monkeys (Buschman & Miller, 2007) or in humans via EEG recordings (Verleger et al., 2011; Verleger et al., 2013), TMS (Chica et al., 2011), and clinical observations of hemispatial-neglect patients (Esterman et al., 2000). Perhaps most importantly, the proposed model demonstrates a simple computational principle by which the visual system could successfully recover simultaneity from an asymmetry it imposes on itself—a hastened response to LVF targets.

Our findings also demonstrate that the LVF advantage on RSVP tasks need not be related to the so-called attentional blink (AB). The AB is the phenomenon that occurs in RSVP experiments when the identification of one target impairs the identification of a second target (Raymond, Shapiro, & Arnell, 1992; Chun & Potter, 1995). The impairment in the second target's identification, however, typically occurs only when it follows the first target by 200 to 500 ms. If the second target follows the first by less than 200 ms, its identification often remains unimpaired—a phenomenon known as “lag 1 sparing” (Visser, Zuvic, Bischof, & Di Lollo, 1999). By contrast, the present TOJ task exhibited a salient LVF advantage (hastening) at 0 ms and 133 ms target asynchronies—durations well within the typical lag 1 sparing interval. Moreover, the present LVF advantage diminished at longer asynchronies (i.e., “lags”) when the AB is typically maximal. Therefore, the LVF advantage on the present RSVP task is likely a distinct phenomenon from the AB given the distinct time courses.

Broadly, the present findings confirm recent reports of a dissociation between the neural events that mediate judgments about simultaneity and temporal order (van Eijk, Kohlrausch, Juola, & van de Par, 2008; Weiss & Scharlau, 2011; Love, Petrini, Cheng, & Pollick, 2013). These task-specific neural events can be modeled in various ways. The task-specific decision rules modeled above parsimoniously explain the main features of the behavioral data in a physiologically plausible manner.

Conclusion

The present experiments assessed lateral hemifield differences in attention's temporal properties. Our TOJ experiment revealed that participants perceived LVF targets ∼134 ms sooner than RVF targets. This LVF hemifield hastening, which approximates a full cycle of visual attention's canonical ∼10 Hz (∼100 ms) temporal resolution, was not evident when the same participants viewed the same RSVP displays but judged simultaneity. Together, these experiments demonstrate that the visual system remaps a hastened neural response to LVF targets by using hemifield-specific rules within the decision stage of simultaneity judgments. Such hemifield-specific remapping would effectively compensate for the cross-hemifield asymmetries in neural response latencies that could otherwise impair simultaneity estimates.

1Pillow and Rubin (2002) reported a perceptual completion task that generated superior unilateral performance, which depended on perceptual grouping but not visual attention.

Footnotes

2“Endogenous” and “exogenous” here, respectively, correspond to the biology-related designations of inside versus outside the body. The black digit targets form a group distinct from the black letter distractors only within the neural networks of those who have learned the English letter and Arabic number categories—an endogenous distinction. The red letter targets differ spectrally from the black letter distractors—an exogenous distinction.

Footnotes

3Although we plotted the proportion of “letter-first” responses, the difference between the LVF and RVF PSEs would be the same if one instead plotted the proportion of “number-first” responses.

Footnotes

4In the classic Pulfrich effect, a pendulum swinging from left to right is perceived as having an illusory three-dimensional rotation when a light-attenuating filter is placed before one of the two eyes. The neural response to the stimulus in the filtered eye's view is slowed relative to that of the unfiltered eye's view. This time delay results in artificial interocular positional differences that generate responses in cortical neurons tuned to various binocular disparities, thereby giving rise to the illusory three-dimensional rotation.

Footnotes

5The significantly elevated RVF-exo error rate on the simultaneity task initially suggests a suboptimal decision rule across the 40 RVF-exo trials that each participant completed at the briefest asynchronies (±133 ms). Yet the same decision rule significantly reduced the error rate (incorrect “different” responses) on the 100 RVF-exo trials that each participant completed when the two targets were physically synchronous (see Figure 8). Overall then, the decision rule was advantageous at the smallest temporal intervals tested here.

The criterion from signal detection theory. On our simultaneity task, asynchronized targets occurred twice as often as synchronized targets, reflected here by the relative heights of the corresponding probability distributions (Wickens, 2002). The neutral criterion (vertical black line) occurs at the sensory evidence level at which the likelihood ratio of the asynchronized-to-synchronized distributions (the ratio of y-axis values) matched our 2:1 asynchronized-to-synchronized target ratio. Criterion values shifted left and right of this neutral point respectively generate lower and higher likelihood ratios, which correspond to biases favoring the “different” and “same” responses, respectively.

Figure 2

The criterion from signal detection theory. On our simultaneity task, asynchronized targets occurred twice as often as synchronized targets, reflected here by the relative heights of the corresponding probability distributions (Wickens, 2002). The neutral criterion (vertical black line) occurs at the sensory evidence level at which the likelihood ratio of the asynchronized-to-synchronized distributions (the ratio of y-axis values) matched our 2:1 asynchronized-to-synchronized target ratio. Criterion values shifted left and right of this neutral point respectively generate lower and higher likelihood ratios, which correspond to biases favoring the “different” and “same” responses, respectively.

Simultaneity task results. Participants reported whether the exogenous letter target and the endogenous number target appeared at the same time or at different times. The symbols and x-axis values are the same as in Figure 3. Arrows in pink (RVF-exo trials) and light blue (LVF-exo trials) indicate the predicted target asynchronies at which “different” responses would be lowest if the LVF hastening that affected temporal-order judgments also affected simultaneity judgments. The black arrow indicates the target asynchrony at which “different” responses were actually lowest. Error bars reflect +1 SEM (N = 23).

Figure 5

Simultaneity task results. Participants reported whether the exogenous letter target and the endogenous number target appeared at the same time or at different times. The symbols and x-axis values are the same as in Figure 3. Arrows in pink (RVF-exo trials) and light blue (LVF-exo trials) indicate the predicted target asynchronies at which “different” responses would be lowest if the LVF hastening that affected temporal-order judgments also affected simultaneity judgments. The black arrow indicates the target asynchrony at which “different” responses were actually lowest. Error bars reflect +1 SEM (N = 23).

Simultaneity task likelihood ratios. The symbols are the same as in Figure 5. The point of subjective equality (PSE) on the simultaneity task occurred when the likelihood ratio equaled two because asynchronized targets were presented twice as often as simultaneous targets. Likelihood ratios greater and less than two respectively reflect biases favoring “same” and “different” responses. Error bars reflect ±1 SEM (N = 23).

Figure 6

Simultaneity task likelihood ratios. The symbols are the same as in Figure 5. The point of subjective equality (PSE) on the simultaneity task occurred when the likelihood ratio equaled two because asynchronized targets were presented twice as often as simultaneous targets. Likelihood ratios greater and less than two respectively reflect biases favoring “same” and “different” responses. Error bars reflect ±1 SEM (N = 23).

Spatiotemporal distribution of errors on the TOJ and simultaneity tasks. Symbols are the same as in Movie 2. Dotted lines (simultaneity task) and solid lines (TOJ task) highlight error patterns that distinguish the two tasks from each other. Error bars reflect ±1 SEM (N = 23).

Figure 8

Spatiotemporal distribution of errors on the TOJ and simultaneity tasks. Symbols are the same as in Movie 2. Dotted lines (simultaneity task) and solid lines (TOJ task) highlight error patterns that distinguish the two tasks from each other. Error bars reflect ±1 SEM (N = 23).