Perceptual learning is usually assumed to occur within sensory areas or when sensory evidence is mapped onto decisions. Subsequent procedural and motor processes, involved in most perceptual learning experiments, are thought to play no role in the learning process. Here, we show that this is not the case. Observers trained with a standard three-line bisection task and indicated the offset direction of the central line by pressing either a left or right push button. Before and after training, observers adjusted the central line of the same bisection stimulus using a computer mouse. As expected, performance improved through training. Surprisingly, learning did not transfer to the untrained mouse adjustment condition. The same was true for the opposite, i.e., training with mouse adjustments did not transfer to the push button condition. We found partial transfer when observers adjusted the central line with two different adjustment procedures. We suggest that perceptual learning is specific to procedural motor aspects beyond visual processing. Our results support theories were visual stimuli are coded together with their corresponding actions.

Visual processing is usually described by a framework, in which visual processing precedes decision making. First the stimulus is processed in a series of visual areas, and then a binary decision is made whether the stimulus belongs to class 1 or 2 (e.g., left vs. right offset). This decision is then mapped onto an arbitrarily assigned motor response, e.g., a left versus right button press, a left or right saccade, or verbal responses. Usually, it is assumed that perceptual learning occurs within the visual areas (Adab & Vogels, 2011; Crist et al., 2001; Karni & Sagi, 1991; Raiguel, Vogels, Mysore, & Orban, 2006; Schoups, Vogels, Qian, & Orban, 2001; Yang & Maunsell, 2004) or when sensory evidence is mapped onto decisions (Dosher & Lu, 1998, 1999; Law & Gold, 2008; Petrov, Dosher, & Lu, 2005; Uka, Sasaki, & Kumano, 2012). Motor processing, related to the motor response, is thought to play no role for and in the learning process, simply because the stimulus-response mapping is arbitrary. Here to the contrary, we show that motor processing is associated to visual stimuli through extensive training. Hence, perception and motor processing cannot be treated as independent modules.

General methods

Subjects

57 naïve students and one author participated in the study (36 males, 21 females; mean age 21 years, range 18–30 years). In order to take part in the study, observers had to reach a value of 1.0 at least with one eye with the Freiburg visual acuity test (corresponding to a Snellen fraction of 20/20; Bach, 1996). Observers signed informed consents and except for the author, they were paid 20 CHF/hr for their participation. Procedures were conducted in accordance with the Declaration of Helsinki and were approved by the local ethics committee.

General setup and stimuli

Observers sat in a dimly illuminated room at 2 m from the monitor. A chin rest with a forehead bar was used to minimize participant's head movements. Stimuli were presented on a Tektronix 608 monitor with a 200 Hz refresh rate controlled by a PC via fast 16 bit D/A converters (1MHz pixel rate). The luminance was measured with a Minolta LS-100 luminance meter. Bisection stimuli were composed of lines which were composed of overlapping dots drawn with a dot pitch of 200 μm at a dot rate of 1MHz. Vertical or horizontal stimuli were composed of 20 arcmin (′) long bluish lines (≈ 80 cd/m2) presented on a dark background (< 1 cd/m2). The distance between the outer lines was either 20′ or 40′ (Figure 1A, B, and C). No fixation point was presented to prevent observers judging the offset by means of the fixation dot. Observers responded either by button presses or mouse adjustment.

Bisection task with button presses

Observers judged whether the central line was either offset to the left or to the right (or up and down for horizontal stimuli) by pressing either a left or a right push button. An adaptive staircase procedure and maximum likelihood estimation of the parameters of the psychometric function were used to determine thresholds of 75% correct responses (parameter estimation by sequential tracking, “PEST”; Taylor & Creelman, 1967). Each trial started with a blank screen (200 ms) after which the stimulus was presented for 150 ms. Observers had 3000 ms to respond. An auditory tone indicated erroneous responses. The next trial started after a delay of 500 ms.

Bisection task with mouse adjustment

A Logitech B58 optical mouse was used for collecting observers' responses. Each trial started with a blank screen for 200 ms. Then, a bisection stimulus appeared in the center of the screen. The position of the central line was offset either to the left or to the right (or up and down for horizontal stimuli) by 120 arcsec for vertical and horizontal stimuli and 240 arcsec for the vertical wide stimulus. The side of the offset was randomized from trial to trial and indicated the offset direction which should be adjusted. Observers adjusted the central line by moving the computer mouse horizontally. At the end of each adjustment, observers confirmed the position of the central line by pressing the left mouse button. Responses were followed by an auditory feedback tone indicating the side to which observers adjusted the central line. Adjustments placed on the left side of the center were followed by a 420 Hz tone and those on the right by a 580 Hz tone. Adjustments shorter than 500 ms or longer than 15 s were rejected and replaced by new trials within the same testing block. After the adjustment, a blank screen was presented for 500 ms and the next trial started.

Procedure

Experiments were conducted on two consecutive days (Figure 1D) to avoid fatigue and promote overnight consolidation (Censor, Sagi, & Cohen, 2012). On day 1, observers performed thirty practice trials to familiarize with the task. These practice trials were performed with a 40′ wide, horizontal bisection stimulus. Observers were asked to adjust the central line with the computer mouse either to the center (Experiment 2) or to the smallest left or right offset they could perceive (Experiments 2, 3, 4a, and 4b). During the warming-up phase, the experimenter ensured that observers understood the task. Then, pretraining performance was determined for a vertical (20′; Figure 1A), a horizontal (20′; Figure 1B) and a vertical wide (40′; Figure 1C) bisection stimulus. Next, observers trained with the vertical stimulus for seven blocks of 80 trials (560 trials in total). On the second day, observers performed the second half of the training (seven blocks), followed by the posttests which were identical to the pretests, but presented in the inverse order. Observers were allowed to take breaks in-between training blocks if they were feeling tired.

Experiment 4a procedure

First, the experimenter performed two blocks with the vertical bisection stimulus and adjusted the central line with the mouse to the smallest offset he could discriminate. The traces of the mouse adjustments on the screen were recorded during the two blocks of 80 trials each. Second, observers performed the warming up trials as in the experiments before. Third, we tested observers in four pretraining conditions: (a) Observers were shown the two blocks of the 80 adjustment trajectories recorded by the experimenter and asked to indicate by button presses whether the final adjustment of a trace was offset either to the left or right (playback condition); (b) Observers performed the task with the vertical bisection stimulus and the adaptive staircase procedure (PEST) using button presses; and then, observers adjusted with the mouse the smallest visible offset of the horizontal (c) and the vertical (d) bisection stimulus, respectively. Fourth, observers underwent a training session adjusting the central line to the smallest offset with the mouse. Finally, observers performed all four posttraining tests in the inverse order. In the trials involving mouse adjustments, the visual stimulus appeared on the screen simultaneously with a 150 ms long auditory cue indicating to which side (left vs. right) the offset should be adjusted. Tones of 4000 Hz or 500 Hz indicated to adjust the central line to the smallest left or right offset, respectively. Erroneous button presses were indicated by a tone after the response.

Experiment 4b procedure

Five new observers participated. After the warming up trials, observers performed (a) two blocks with the vertical bisection stimulus (20′) using the adaptive PEST procedure (first pretraining test). Observers responded by button presses. Based on this measurement, each participant was matched to a corresponding participant from Experiment 4a with a similar threshold. Thresholds between observers in Experiment 4a and 4b differed by 6.4 arcsec on average only. Then, three further pretests were performed. (b) Observers were shown the same vertical bisection stimulus (20′; Figure 1A) and adjusted the smallest offset using the computer mouse. (c) and (d) Observers performed two playback conditions with the horizontal and vertical wide bisection stimuli, respectively (20′, 40′); i.e., observers saw the adjustment traces of the matched observers from Experiment 4a and indicated the final adjusted position of the central line by a button press. During training, observers were shown the adjustment traces of the matched observers recorded during training in Experiment 4a (vertical 20′ bisection stimulus). Observers discriminated the final adjusted position of the central line by pressing either the left or the right response button. After the training, observers performed the four posttests in the inverse order.

Experiments 5 and 6

Twenty new observers were recruited, ten for each experiment. The observers' task was to adjust the central line to the smallest offset on the side on which the central line appeared on the monitor. Both experiments differed only in the type of the motor response during training. In both experiments, on the first day (Figure 1D) observers first performed the warming up trials during which they performed 80 trials (two blocks of 40 trials) with a 40′ wide, horizontal bisection stimulus, i.e., Figure 1C but rotated by 90°. One of these adjustment blocks was performed with a computer mouse (as previously) and the other with two keyboard buttons, namely the left and the right arrow buttons. The order of these two blocks was counterbalanced across observers. The initial offset value was randomly chosen between 195″ and 276″ in each trial either above or below the center of the stimulus. Second, observers performed four blocks (80 trials each) of the pretraining tests with a 20′ wide, vertical bisection stimulus (Figure 1A). Two of the blocks were performed with a computer mouse and the other two with the buttons. The order of these blocks was counterbalanced across observers. The initial left or right offset was randomly chosen from 98″ to 138″ in each trial. Third, observers underwent the first half of the training either by adjusting the central line with a computer mouse (Experiment 5) or with the computer keyboard buttons (Experiment 6). On the second day, observers first performed the second half of the training and the posttraining tests which were identical to the pretraining tests.

Outlier Rejection

The main interest of the study was about the transfer of learning. Because transfer of learning is not expected for nonlearners, nonlearners were rejected from the analysis (Experiment 1: four observers; Experiment 4a: three observers; Experiment 4b: three observers; and Experiment 6: two observers). Nonlearners were defined as observers whose performance deteriorated during training as determined by the slope of the regression lines. Including these observers does not change the results related to transfer but decreased the average amount of learning in the training sessions, but not significantly.

Results

Experiment 1

We used a classic bisection task, where a central line bisects the space delineated by two outer lines (Figure 1A). The central line was either slightly offset to the left or right outer line and observers indicated the offset using the push buttons. Six observers participated in the experiment.

Both, the second and the third experiment were as Experiment 1 except that the response types were reversed, i.e., observers responded by mouse adjustment during training and by button presses in the pre- and posttraining vertical condition. In Experiment 2, observers adjusted the central line to be exactly at the center during training and the pre- and the posttests and in Experiment 3 to the smallest offset they perceived. Five different observers participated in each experiment.

First, we repeated Experiment 3 with five new observers. During training, observers adjusted the central line to the smallest offset. (A) Improvement during training (p = 0.048) and in the (B) pre- and the posttests was very similar to that of Experiment 3. Second, we recruited five new observers. (C) During training, observers responded by button press whether the adjusted offset they saw was either to the left or right (play back condition). We quantified performance in terms of d′. Thus, higher values indicate better performance. Performance improved during training (p = 0.009). (D) Pre- versus posttraining conditions. Learning with played back stimuli was specific to the stimulus orientation (cyan, left). There was a trend for the vertical wide stimulus. Most importantly, learning did not transfer to the trained stimulus when observers adjusted the offset themselves with the mouse (red) or responded by button press (blue). Error bars denote ±SEM.

Figure 2

First, we repeated Experiment 3 with five new observers. During training, observers adjusted the central line to the smallest offset. (A) Improvement during training (p = 0.048) and in the (B) pre- and the posttests was very similar to that of Experiment 3. Second, we recruited five new observers. (C) During training, observers responded by button press whether the adjusted offset they saw was either to the left or right (play back condition). We quantified performance in terms of d′. Thus, higher values indicate better performance. Performance improved during training (p = 0.009). (D) Pre- versus posttraining conditions. Learning with played back stimuli was specific to the stimulus orientation (cyan, left). There was a trend for the vertical wide stimulus. Most importantly, learning did not transfer to the trained stimulus when observers adjusted the offset themselves with the mouse (red) or responded by button press (blue). Error bars denote ±SEM.

Hence, even though the vertical bisection stimulus was identical in the training and the pre- and the posttraining tests, there were unavoidable differences between the button press and adjustment conditions. For example, stimulus duration was much longer when observers performed mouse adjustments (average duration of an adjustment being 2.3 s) than when they responded by button presses (150 ms). Moreover, stimuli were “in motion” during mouse adjustments whereas they were static in the button press condition. We controlled for these differences in Experiment 4. First, we repeated Experiment 3 and recorded the traces of mouse adjustments in each trial during training and in the pre- and the posttraining tests for five new observers, i.e., 7,200 individual trials (Experiment 4a). Next, in Experiment 4b, these recorded traces adjustments were shown to five new observers during training. Observers indicated the offset direction of the last position of the central line by button press.

Despite the fact that we controlled for differences relative to the visual stimulation (Experiment 4), the tasks are still different with regard to decision making and task monitoring. Whereas the mouse adjustment condition was performed in a continuous and active manner, the playback condition required a binary decision and observers remained “passive” during the stimulation. In the next two experiments, we controlled for the abovementioned differences. Like in Experiments 2, 3, and 4a, in Experiments 5 and 6, observers adjusted the central line to the smallest perceptible offset by using either the computer mouse or two keyboard buttons (left and right arrows), respectively. Ten new observers participated in each experiment. Contrary to the previous experiments, the dependent variable was the same in both pre- and posttraining conditions. Experiment 5 and Experiment 6 differed only in the type of the motor response used during training.

During training, participants adjusted the central line of the vertical bisection stimulus to the smallest offset visible either by using the computer mouse (Experiment 5, N = 10) or by using the keyboard buttons (Experiment 6, N = 10). Training in both experiments yield strong improvements of performance (p = 0.0001 and p < 0.0001, respectively). Before and after training (pre- and posttraining tests), participants performed adjustments with both the mouse and the buttons. In Experiment 5, learning did not transfer to the button adjustment condition (p = 0.059). However, observers showed a high initial performance in the pretraining test that was significantly better for the buttons adjustment condition (blue pre vs. red pre, p = 0.003). In Experiment 6, training transferred significantly to the mouse adjustment condition (p = 0.0014); however, the posttraining performance remained significantly lower for the mouse than for key adjustment condition (post red vs. post blue, p = 0.0086), suggesting that the transfer was partial. Error bars represent ±SEM.

Figure 3

During training, participants adjusted the central line of the vertical bisection stimulus to the smallest offset visible either by using the computer mouse (Experiment 5, N = 10) or by using the keyboard buttons (Experiment 6, N = 10). Training in both experiments yield strong improvements of performance (p = 0.0001 and p < 0.0001, respectively). Before and after training (pre- and posttraining tests), participants performed adjustments with both the mouse and the buttons. In Experiment 5, learning did not transfer to the button adjustment condition (p = 0.059). However, observers showed a high initial performance in the pretraining test that was significantly better for the buttons adjustment condition (blue pre vs. red pre, p = 0.003). In Experiment 6, training transferred significantly to the mouse adjustment condition (p = 0.0014); however, the posttraining performance remained significantly lower for the mouse than for key adjustment condition (post red vs. post blue, p = 0.0086), suggesting that the transfer was partial. Error bars represent ±SEM.

One of the biggest debates in the field of perceptual learning concerns the locus where learning takes place. Usually, perceptual learning is thought to occur either in the early sensory areas (Adab & Vogels, 2011; Crist et al., 2001; Karni & Sagi, 1991; Raiguel et al., 2006; Schoups et al., 2001; Yang & Maunsell, 2004) or decision making stages (Dosher & Lu, 1998, 1999; Law & Gold, 2008; Petrov et al., 2005; Uka et al., 2012). Perceptual learning is usually very specific for the stimuli trained with. Here, we propose that perceptual learning is even specific to procedural motor processing. Observers trained with the vertical bisection task by responding with button presses or mouse adjustments. Performance improved. As expected, no transfer occurred to the horizontal stimulus proving orientation specificity. We used also a wide vertical bisection stimulus in the pre- and the posttraining tests to show that transfer is possible in principle. We found a significant transfer (Experiment 3) or trends to significance in this condition (Experiments 1, 2, and 4). Surprisingly, learning did not transfer when observers saw the same stimuli as used during adjustment training but responded by the untrained motor response (Experiment 4). Similar results were found by Green, Kattner, Siegel, Kersten, and Schrater (2015) where observers trained to discriminate a clockwise from a counterclockwise orientated Gabor patch or to adjust the orientation of the same Gabor patch with a computer mouse. Interestingly, the discrimination training did not transfer to the adjustment condition and vice versa.

To make sure that the lack of transfer is not caused by nonlearners, observers with no improvements of performance in the training session were rejected. Including these observers does not change results qualitatively but just decreases the average amount of learning in the training sessions.

There was no transfer to the untrained type of motor response in Experiments 1 to 4. Very little transfer was observed (Experiments 1, 2, and 4b) or even some deterioration of performance (Experiments 3 and 4a). The overall amount of transfer in the four experiments was only 3.7″ and 4.5″ for the adjustment and the button press responses, respectively. Optimally, one should relate the amount of transfer to the improvements obtained during training for each observer individually. However, this is impossible in our study (except from Experiments 5 and 6) because the measures in the different conditions were highly different. For example, in Experiments 4b, we determined sensitivity (d′) during training while pre- and posttraining measurements were determined as thresholds, and hence both measures cannot by compared. We did not apply Bonferroni correction because we are aiming to “prove” a null result, namely, the absence of transfer. Uncorrected p values are more conservative with this respect. In Experiment 5, we found no transfer to the key adjustment condition. In Experiment 6, we found a significant transfer from the key adjustment to the mouse adjustment condition. Nevertheless, this transfer was partial as the posttraining performance was significantly higher in the trained condition (key adjustment). Thus, it seems that there might be some specificity even beyond procedural processing, which needs to be addressed by future research.

At a first glance, our results are surprising because it is usually assumed that stimuli are first processed in the visual system, then a binary decision is made in the decision unit (e.g., left or right offset), which is then mapped onto an arbitrary motor response (e.g., button presses, saccades, or verbal responses). Hence, visual processing should be independent of procedural and motor processing. We suggest that, during intensive training, strong stimulus-response associations are formed as in riding a bike, where strong sensorimotor contingencies prevail. We do not claim that stimuli are in general coded together with actions. We propose rather that the coding of actions is coded together with stimuli when both are coupled through extensive learning. In this respect our results support theories where stimuli are coded together with the corresponding actions, such as in the ecological approach (Gibson, 1979), the sensorimotor theory (O'Regan & Noë, 2002), the common coding theory (Prinz, 1997), and the theory of event coding (Hommel, Müsseler, Aschersleben, & Prinz, 2001). Our results are in line with current studies showing strong and direct links between sensory and motor processing (Beets et al., 2010; Beets, Rosler, & Fiehler, 2010; Brown, Wilson, Goodale, & Gribble, 2007; Casile and Giese, 2006; Hecht, Vogt, & Prinz, 2001; Vahdat, Darainy, & Ostry, 2014; Vahdat, Darainy, Milner, & Ostry, 2011; for reviews, see Cisek & Kalaska, 2010. Ostry and Gribble, 2016; Schütz-Bosbach & Prinz, 2007). For example, Beets, Rösler, et al. (2010) trained participants in a cyclical arm movement task. Training improved performance for this motor task and transferred to a visual task consisting in discrimination of elliptical shapes. Likewise, electrophysiological studies have shown that neural responses in primary visual cortex (V1) can be strongly modified by ongoing motor activity in mice (Poort et al., 2015; Saleem, Ayaz, Jeffery, Harris, & Carandini, 2013) and that somatosensory cortex can directly control the muscles involved in whisker retraction (Matyas et al., 2010). Similarly, evidence from human imaging studies showed that the mere visual exposure to movements activates motor-related brain areas (Engel, Burke, Fiehler, Bien, & Rösler, 2008; Reithler, van Mier, Peters, & Goebel, 2007). Interestingly, these activations were higher when the movements in question were trained. Moreover, evidence for simultaneous changes in the sensory and motor cortices were found in monkey and human following sensorimotor learning (Arce-McShane et al., 2014; Vahdat et al., 2011, 2014) supporting claims (Censor et al., 2012) that perceptual and motor learning share analogous properties in terms of temporal dynamics and the engagement of higher order brain areas.

In future experiments, it remains to be studied to what extend learning is specific for procedural aspects related to motor responses. For example, does training with the right hand transfer to the left hand in the mouse adjustment tasks? As an alternative, it may be that learning is only specific to the adjustment procedure as suggested by Experiment 5. In this case, perceptual learning is specific only to the procedural aspects of the motor response and not for the exact motor execution, e.g., specific muscle responses.

We would like to thank Marc Repnow for technical support and Takeo Watanabe and Merav Ahissar for useful comments. Lukasz Grzeczkowski was funded by the Sinergia project “Learning from Delayed and Sparse Feedback” (Project Number: CRSII2_147636) of the Swiss National Science Foundation (SNFS).

First, we repeated Experiment 3 with five new observers. During training, observers adjusted the central line to the smallest offset. (A) Improvement during training (p = 0.048) and in the (B) pre- and the posttests was very similar to that of Experiment 3. Second, we recruited five new observers. (C) During training, observers responded by button press whether the adjusted offset they saw was either to the left or right (play back condition). We quantified performance in terms of d′. Thus, higher values indicate better performance. Performance improved during training (p = 0.009). (D) Pre- versus posttraining conditions. Learning with played back stimuli was specific to the stimulus orientation (cyan, left). There was a trend for the vertical wide stimulus. Most importantly, learning did not transfer to the trained stimulus when observers adjusted the offset themselves with the mouse (red) or responded by button press (blue). Error bars denote ±SEM.

Figure 2

First, we repeated Experiment 3 with five new observers. During training, observers adjusted the central line to the smallest offset. (A) Improvement during training (p = 0.048) and in the (B) pre- and the posttests was very similar to that of Experiment 3. Second, we recruited five new observers. (C) During training, observers responded by button press whether the adjusted offset they saw was either to the left or right (play back condition). We quantified performance in terms of d′. Thus, higher values indicate better performance. Performance improved during training (p = 0.009). (D) Pre- versus posttraining conditions. Learning with played back stimuli was specific to the stimulus orientation (cyan, left). There was a trend for the vertical wide stimulus. Most importantly, learning did not transfer to the trained stimulus when observers adjusted the offset themselves with the mouse (red) or responded by button press (blue). Error bars denote ±SEM.

During training, participants adjusted the central line of the vertical bisection stimulus to the smallest offset visible either by using the computer mouse (Experiment 5, N = 10) or by using the keyboard buttons (Experiment 6, N = 10). Training in both experiments yield strong improvements of performance (p = 0.0001 and p < 0.0001, respectively). Before and after training (pre- and posttraining tests), participants performed adjustments with both the mouse and the buttons. In Experiment 5, learning did not transfer to the button adjustment condition (p = 0.059). However, observers showed a high initial performance in the pretraining test that was significantly better for the buttons adjustment condition (blue pre vs. red pre, p = 0.003). In Experiment 6, training transferred significantly to the mouse adjustment condition (p = 0.0014); however, the posttraining performance remained significantly lower for the mouse than for key adjustment condition (post red vs. post blue, p = 0.0086), suggesting that the transfer was partial. Error bars represent ±SEM.

Figure 3

During training, participants adjusted the central line of the vertical bisection stimulus to the smallest offset visible either by using the computer mouse (Experiment 5, N = 10) or by using the keyboard buttons (Experiment 6, N = 10). Training in both experiments yield strong improvements of performance (p = 0.0001 and p < 0.0001, respectively). Before and after training (pre- and posttraining tests), participants performed adjustments with both the mouse and the buttons. In Experiment 5, learning did not transfer to the button adjustment condition (p = 0.059). However, observers showed a high initial performance in the pretraining test that was significantly better for the buttons adjustment condition (blue pre vs. red pre, p = 0.003). In Experiment 6, training transferred significantly to the mouse adjustment condition (p = 0.0014); however, the posttraining performance remained significantly lower for the mouse than for key adjustment condition (post red vs. post blue, p = 0.0086), suggesting that the transfer was partial. Error bars represent ±SEM.