We use cookies to enhance your experience on our website. By continuing to use our website, you are agreeing to our use of cookies. You can change your cookie settings at any time.Find out moreJump to
Content

Multisensory Integration and the Perception of Self-Motion

Summary and Keywords

As we go about our everyday activities, our brain computes accurate estimates of both our motion relative to the world, and of our orientation relative to gravity. Essential to this computation is the information provided by the vestibular system; it detects the rotational velocity and linear acceleration of our heads relative to space, making a fundamental contribution to our perception of self-motion and spatial orientation. Additionally, in everyday life, our perception of self-motion depends on the integration of both vestibular and nonvestibular cues, including visual and proprioceptive information. Furthermore, the integration of motor-related information is also required for perceptual stability, so that the brain can distinguish whether the experienced sensory inflow was a result of active self-motion through the world or if instead self-motion that was externally generated. To date, understanding how the brain encodes and integrates sensory cues with motor signals for the perception of self-motion during natural behaviors remains a major goal in neuroscience. Recent experiments have (i) provided new insights into the neural code used to represent sensory information in vestibular pathways, (ii) established that vestibular pathways are inherently multimodal at the earliest stages of processing, and (iii) revealed that self-motion information processing is adjusted to meet the needs of specific tasks. Our current level of understanding of how the brain integrates sensory information and motor-related signals to encode self-motion and ensure perceptual stability during everyday activities is reviewed.

As we go about our everyday activities, it is vital that our brain computes robust estimates of both of our orientation relative to gravity and our motion relative to the world. This computation depends on the integration of sensory cues as well as motor information in order to let us know whether we are moving through the world or if, instead, our visual surround is moving. The vestibular system encodes self-motion and spatial orientation by means of the five sensory organs within the vestibular apparatus of the inner ear: the utricle, saccule, and three semicircular canals. Specifically, the utricle and saccule detect gravity (information in a vertical orientation) and linear movement, and the semicircular canals detect rotational movement. However, it is well appreciated that in everyday life, our perception of self-motion and sense of spatial orientation is based on the integration of vestibular and extra-vestibular cues, including proprioceptive and visual information. For example, if motion is passively generated (consider a passenger in a car accelerating after a traffic light turns green), the visual system provides retinal-image motion cues (optic flow) in addition to the linear acceleration provided by the otolith organs of the vestibular system. Moreover, during self-generated motion, the brain has access to other sources of information because extra-vestibular information is provided not only by the visual system, but also by the proprioceptive sensors of the muscles, tendons, and joints, which sense the relative position of neighboring parts of the body. Further, when motion is self-generated, information related to the motor-command itself can be integrated with other existing information to contribute to the brain’s estimate of self-motion.

Our current understanding of how the brain integrates vestibular, proprioceptive, and motor-related signals to encode self-motion during everyday activities is reviewed. Recent findings have challenged the traditional view of neural code that is used to represent sensory information in vestibular pathways. In addition, they have provided strong support for the view that vestibular pathways are inherently multimodal at the earliest stages of processing. Finally, recent studies have firmly established that the encoding of self-motion information differs when motion is self-generated versus externally applied. This progress is highlighted and the implications are considered of this self-motion coding strategy for ensuring accurate motor control and stability during self-motion.

The Statistics of Natural Self-Motion and Coding by Early Vestibular Pathways

The Statistics of the Self-Motion Experienced in Everyday Life

Figure 1. Statistics of natural self-motion and early vestibular processing. (A) Representation of a subject wearing the MEMS module (gold box), and example linear acceleration signal during different everyday activities. (B) Population-averaged power spectra (black) of these signals with corresponding SDs (dark gray bands) and power law fits over the low-frequency (green line) versus high-frequency (blue line) ranges. The power law exponents (i.e., slopes) and the transition frequencies (i.e., the frequency at which the power law fits intersect) are also shown. (C, D) Gain and mutual information density for populations of irregular (blue) versus regular (green) vestibular afferents.

The prevailing view is that sensory systems are adapted to optimally process stimuli that are experienced in everyday life. The power spectra of naturally experienced visual and auditory stimuli decay as a power law (i.e., as 1/fα‎) (for review, see Simoncelli & Olshausen, 2001). However, this is not the case for natural vestibular stimuli. Instead, the power of vestibular input experienced during everyday life decreases more slowly at lower frequencies (< 5 Hz, in humans, and < 10 Hz in monkeys) and more rapidly at higher frequencies for all motion dimensions (Carriot, Jamali, Chacron, & Cullen, 2014; Figure 1A, B). This unique stimulus structure is the result of active motion, as well as passive biomechanical filtering occurring before any neural processing. If the view that sensory systems have developed coding strategies that are constrained by the statistics of natural stimuli is correct, this then raises the question of how do the neural coding strategies used by the vestibular system relate to the statistics of the self-motion experienced in everyday life?

Self-motion information is first detected by the receptor cells (i.e., type I and II hair cells) of the semicircular canals and otoliths and then carried by the vestibular-nerve afferents to the brain. Over the range of frequencies typically experienced during everyday behaviors (i.e., up to 20 Hz) (Armand & Minor, 2001; Huterer & Cullen, 2002), canal afferents encode head velocity, whereas otolith afferents encode linear acceleration (Angelaki & Cullen, 2008; Goldberg, 2000).

Vestibular afferents from both the semicircular canals and otoliths can each be divided into two distinct classes on the basis of the regularity of their resting discharge (Goldberg, Smith, & Fernandez, 1984). Notably, units displaying low variability in their interspike intervals are sensibly termed regular afferents, while those with higher variability are termed irregular afferents. These two classes of afferents further differ in their morphological properties. Regular afferents, projecting from both the otoliths and semicircular canals, have relatively small axon diameter, and provide bouton endings to type II hair cells located at the periphery of the vestibular neuroepithelium. In contrast, irregular units preferentially innervate the calyx endings of type I hair cells. The results of recent studies have revealed that this intrinsic variability across vestibular-nerve afferents plays an important role in the strategy used to encode self-motion, with important implications for the functional roles of regular versus irregular afferents.

The Representation of Rotational Self-Motion at the First Central Stage of Vestibular Processing

Figure 2. Early vestibular processing and the sensory coding of self-motion: afferents versus central neurons. (A) Vestibular afferents project to the vestibular nuclei, which in turn comprises three main classes of neurons: (i) PVPs and FTNs, which control and modulate the vestibulo-ocular reflex to ensure gaze stability during everyday life, and (ii) VO neurons that control posture and balance, and also project to higher order structures involved in the estimation of self-motion. (B) Comparison of average neural detection threshold values of VO neurons (gray), with those of irregular (blue) and regular (green) afferents. Superimposed are human behavioral thresholds (red; Grabherr et al., 2008), and an estimate of the information transmitted by a population of 12 VO neurons (black). Side bands show +/− SEM. Data replotted from Sadeghi, Chacron, et al. (2007). (C) Population-averaged performance for irregular (blue) and regular (green) afferents versus central vestibular neuron (gray) as a function of frequency (top). The shaded bands show the standard error. (Insets) Average mutual information transmitted for each population by precise spike timing (top) versus firing rate (bottom). Data replotted from Jamali, Chacron, and Cullen (2016).

How do the two channels of afferent input combine at the next stage of vestibular processing, namely the vestibular nuclei? A distinct population of neurons in the vestibular nuclei—termed Vestibular-Only (VO) and alternatively called non–eye movement neurons—constitutes a critical relay for neural computations that underlie the perception of self-motion. VO neurons receive direct afferent input, are reciprocally interconnected with the nodulus/uvula of the cerebellum (Reisine & Raphan, 1992), and also likely project to the vestibular-sensitive neurons in thalamus and cortex (Grüsser, Pause, & Schreiter, 1990a, 1990b; Lang, Büttner-Ennever, & Büttner, 1979). Accordingly, these neurons are a key stage of vestibular processing in the pathways that mediate higher-order vestibular functions such as the perception of self-motion and spatial orientation (Figure 2A).

Recent studies have provided insight into how self-motion information is encoded by the VO neurons of the vestibular nuclei over the physiologically relevant frequency range (Massot et al., 2011). Although their response gains are generally greater than those of individual afferents, VO neurons actually transmit less information than even irregular afferents. Consistent with this latter finding, the neural detection thresholds of VO neurons are significantly worse than those of irregular as well as regular afferents (Figure 2B). Indeed, in order to approach the detection thresholds measured in behavioral experiments (~2.5 vs. 0.5–1°/s; Massot et al., 2011), the responses of a population of > 20 VO neurons must be pooled. The apparent discrepancy between the precision of coding at sequential stages of vestibular processing and the brain’s ability to estimate self-motion suggests that vestibular processing has adapted to extract particular features from self-motion.

Consistent with this idea, VO neurons demonstrate two important nonlinear behaviors. First, VO neurons do not simply sum their afferent input due to a nonlinearity in the input-output relationship (Massot et al., 2011). This coding feature, termed a static boosting nonlinearity, is potentially beneficial in that it extends the coding range of central vestibular neurons such that they are less likely to demonstrate cut-off or saturation. As a result of this same nonlinearity, VO neurons preferentially extract the high-frequency features of self-motion when embedded with low frequency motion during natural movements. Second, a recent report demonstrated that VO neurons can also reliably discriminate between different stimulus waveforms through differential patterns of precise spike timing (i.e., a temporal code; Jamali, Chacron, & Cullen, 2016; Figure 2C). Temporal coding on the same timescale was reported for irregular but not regular afferents. While it remains to be seen whether this nonlinear signal is decoded by higher areas, it seems likely that vestibular pathways would use spike timing precision as well as firing rate to represent self-motion, given that VO firing rate responses appear less sensitive than the organism. Thus, taken together, these recent findings challenge the traditional notion that the vestibular system uses a linear rate code to transmit information and have important consequences for understanding how the representation of sensory information changes across sensory pathways.

The Encoding of Translational Motion

At the level of the vestibular periphery, translation appears to be encoded using a different strategy than rotation. In particular, irregular otolith afferents are exceedingly sensitive to self-motion when compared to regular otolith afferents. Indeed, the sensitivity of irregular otolith afferents is so high that it effectively compensates for their higher trial-to-trial variability, resulting in detection thresholds that are comparable to those of regular afferents (Jamali, Carriot, Chacron, & Cullen, 2013). It remains to be established why the canal and otolith systems have evolved two distinct strategies for encoding self-motion. Nevertheless, a key difference is that otoliths, unlike the semicircular canals, also sense the constant force of gravity. Accordingly, it is likely that the neural coding strategy used by regular afferents to represent translational motion is particularly well-adapted to encode small changes in the head’s orientation to this critical reference. Further studies focusing on the central processing of otolith information experienced in everyday life, including dynamic stimuli that produce nonlinear responses, as well as persistent stimulation due to gravity, are needed to further our understanding of the functional implications of this difference.

Natural Self-Motion Is Multidimensional

The natural self-motion experienced in everyday life is multidimensional and simultaneously stimulates both the semicircular canals and otoliths. Surprisingly, however, relatively few studies have addressed the question of how the brain integrates semicircular canal and otolith information during self-motion that stimulates both types of sensory organs. It has been shown that the responses of VO neurons during passively applied combined motion cannot be predicted by the linear addition of their responses to the rotational and translational component (Dickman & Angelaki, 2002; Carriot, Jamali, Brooks, & Cullen, 2015). Instead, VO neurons sub-additively integrate their semicircular canals and otoliths afferent inputs (Carriot et al., 2015). Specifically, neuronal responses reflect the weighted sum of their responses to rotation and translation. Moreover, this weighting is frequency dependent, with canal inputs more heavily weighted at low frequencies, and otolith inputs more heavily weighted at higher frequencies. Interestingly, prior psychophysical studies of self-motion perception in humans have shown that human subjects more accurately perceived angular than linear displacement at lower frequencies (Ivanenko, Grasso, Israël, & Berthoz, 1997; MacNeilage, Turner, & Angelaki, 2010). Thus, the responses of VO neurons provide a neural correlate for this perceptional bias, and further suggest that subjects perceive linear motion more accurately than rotational motion at higher frequency of self-motion.

Figure 3. The vestibular nucleus is a site of rich multisensory and motor convergence. In addition to receiving direct input from vestibular afferents of the VIII nerve, the vestibular nuclei receive inputs from numerous regions including (i) proprioceptive pathways, (ii) the vestibular cerebellum, (iii) oculomotor areas of the brainstem, and (iv) several areas of the cortex (e.g., parietoinsular vestibular cortex [PIVC], premotor areas 6, 6pa, somatosensory area 3a, and superior temporal cortex).

As we go about our daily activities, self-motion information is provided not only by the vestibular sensors, but also via inputs from other sensory modalities including the somatosensory, proprioceptive, and visual systems. In addition, during active behaviors, efference copies of the motor signals that produce movement must also be integrated so that the brain can discriminate self-generated versus passively-applied motion—a distinction that is vital for ensuring accurate postural control, as well as perceptual stability. Indeed, recent studies have emphasized that a characteristic property of the vestibular system is that it combines multimodal sensory and motor information early in processing. The schematic in Figure 3 shows the rich convergence of extra-vestibular sensory inputs as well as premotor signals related to the generation of eye and head movements that are relayed to early vestibular pathways.

Integration of Vestibular and Proprioceptive/Somatosensory Inputs

During everyday life, much of our self-motion is generated as the result of our own voluntary behavior. For example, when we walk across a room or turn to talk to someone, our head moves relative to space, activating the vestibular sensors. However, these behaviors also activate proprioceptors within the muscles that actually generate them, as well as somatosensory receptors in skin surrounding the moving joints (Lackner & DiZio, 2005; Carriot, Cian, Paillard, Denise, & Lackner, 2011). These extra-vestibular inputs reach vestibular nuclei and deep cerebellar nuclei by means of direct and indirect projections from the dorsal-root axons of the spinal cord (reviewed in Cullen & Roy, 2004). Projections from areas of the cerebellum and cortex that are sensitive to proprioceptive and somatosensory inputs also send direct projections to the vestibular nuclei (reviewed in Manzoni, 2007; Guldin & Grüsser, 1998; Wilson et al., 1999). Thus, at the first central stage of processing, the connectivity of the vestibular system strongly supports multimodal integration of vestibular with these extravestibular cues.

Historically, most studies that have focused on the integration of vestibular with proprioceptive or somatosensory inputs have employed decerebrate or anesthetized preparations. These investigations were classically performed in cats, and quantified the influence of passive stimulation of neck muscle proprioceptors on the activity of neurons in the vestibular and deep cerebellar nuclei (see Goldberg & Cullen, 2011; Wilson & Schor, 1999, for reviews). Interestingly, under such conditions, neuronal sensitivities to proprioceptive versus vestibular stimulation can be either antagonistic or agonistic. As a result, when both systems are activated concurrently by passive motion of the head-on-body, neurons show reduced and augmented modulation, respectively, as compared to their responses to vestibular stimulation alone (Anastasopoulos & Mergner, 1982; Boyle & Pompeiano, 1981). Moreover, passive stimulation of limb proprioceptors also influences the activity of a substantive proportion of neurons in the vestibular nuclei and deep cerebellar nuclei (Arshian et al., 2014; McCall, Miller, Catanzaro, Cotter, & Yates, 2015), suggesting that a substrate for the integration of vestibular and limb proprioception is required for the accurate control of posture and balance.

Recently, there has been an increased focus on investigating the integration of vestibular and proprioceptive signals in alert animals (Brooks & Cullen, 2009; Kasper, Schor, & Wilson, 1988; Luan, Gdowski, Newlands, & Gdowski, 2013; Roy & Cullen, 2004, 2001), rather than in decerebrate or anesthetized preparations. To date, one main finding is that the strategy underling the integration of vestibular and proprioceptive signals is species dependent. The passive stimulation of neck proprioceptors influences the responses of vestibular nuclei neurons in mice (Medrea & Cullen, 2013), as well as in squirrel monkey (Gdowski & McCrea, 2000) and cynomolgus monkey (Sadeghi, Mitchell, & Cullen, 2009). Additionally, vestibular nuclei neurons—at least in the alert cat—respond to passive stimulation of limb as well as neck proprioceptors (McCall, Miller, DeMayo, Bourdages, & Yates, 2016). Interestingly, in alert squirrel monkey and cynomolgus monkey, multisensory integration is usually antagonistic. Thus, when the head is moved relative to body (i.e., during a passive head turn of the head on the body), neurons generally fire less robustly than for comparable head motion produced by whole-body motion (i.e., a condition in which only the vestibular system is stimulated).

In contrast, passive stimulation of proprioceptors in normal alert rhesus monkey does not directly alter the responses of VO neurons responses in the vestibular nuclei (Roy & Cullen, 2001; Carriot, Brooks, & Cullen, 2013). Responses to proprioceptive inputs are only seen following vestibular loss, suggesting a form of homeostatic plasticity that compensates for the reduced reliability of the vestibular input (Sadeghi, Minor, & Cullen, 2010, 2011, 2012). Indeed, rhesus monkeys (and presumably humans) integrate vestibular and proprioceptive signals at the level of the vestibular nuclei only during active head movements. The sophisticated mechanism underlying this integration allows the distinction between self-generated and externally applied self-motion, and is a common strategy utilized by other species.

On the other hand, stimulation of proprioceptors in normal alert rhesus monkey alters the responses of neurons at the next levels of vestibular processing (i.e., the rostral fastigial nucleus of the cerebellum and vestibular thalamus). The rostral fastigial nucleus is the most medial of the deep cerebellar nuclei, and is reciprocally connected to the vestibular nucleus (Furuya, Kawano, & Shimazu, 1975; Shimazu & Smith, 1971). Indeed, approximately half of the neurons within the rostral fastigial nucleus encode proprioceptive inputs in normal alert rhesus monkey. These neurons, termed bimodal neurons, display comparable (but antagonistic) modulation in response to passive vestibular and proprioceptive stimulation (Brooks & Cullen, 2009). As a result, because the responses of bimodal neurons sum linearly during combined stimulation, they robustly encode passive “body-in-space motion.” Interestingly, the other main population of neurons within the rhesus rostral fastigial nucleus (termed unimodal neurons) is sensitive to vestibular but insensitive to proprioceptive stimulation. As a result, these neurons encode passive “head-in-space motion,” much like the VO neurons of the vestibular nuclei.

Figure 4A compares the responses of an example unimodal and bimodal neuron in the rostral fastigial nucleus in alert rhesus monkey. This integration of vestibular and proprioceptive information at the level of the cerebellum is essential for the accurate control of posture and balance. For example, the corrective movements produced by vestibulospinal reflexes must account for changes in the position of the head relative to the body (Kennedy & Inglis, 2002; Tokita, Ito, & Takagi, 1989; Tokita, Miyata, Takagi, & Ito, 1991). Accordingly, patients with lesions to the vestibular cerebellum show deficits in the production of compensatory postural responses that are normally evoked by galvanic stimulation (Kammermeier, Kleine, & Büttner, 2009). In addition, early-stage sensory convergence in the vestibular nuclei and cerebellum is also likely vital for generating perceptual constancy of self-motion required for higher-order functions such as self-motion perception (Miller et al., 2008), including that required to perceive body motion independently of head motion (Mergner, Siebold, Schweigart, & Becker, 1991).

Figure 4. In response to passive stimulation, proprioceptive and visual information is encoded at the level of the vestibular cerebellum but not within the vestibular nuclei in rhesus monkey. (A) Approximately half of the neurons within the rostral fastigial neurons respond to neck proprioceptive (center) as well as vestibular (left) stimulation (i.e., bimodal neurons). For these neurons, when the head moves relative to body (as it would during a voluntary orienting head turn) vestibular and neck proprioceptive inputs sum to produce complete response cancellation—consistent with these neurons encoding body motion. The other ~ half of the neurons within the rostral fastigial neurons respond exclusively to vestibular (left) stimulation, consistent with these neurons encoding head motion. (B) VO neurons in the vestibular nuclei are unresponsive to full field visual motion that induces a robust optokinetic eye movement response. Abbreviations: Ep = induced change in eye position, Vs = change in visual surround position, and FR = constant firing rate.

Ascending projections from both the vestibular nuclei and the rostral fastigial nucleus target the vestibular thalamus (reviewed in Lopez & Blanke, 2011). In this context, there is evidence for the integration of vestibular and proprioceptive information in the thalamic ventral posterior lateral nucleus in both squirrel and rhesus monkeys. Similar to what was described for the rostral fastigial nucleus, a substantial percentage of vestibular-sensitive neurons in this region also respond robustly to passive stimulation of neck proprioceptors (Marlinski & McCrea, 2008a, 2008b; Dale & Cullen, 2015). In rhesus monkeys, this multisensory integration is antagonistic (again similar to what is seen in the cerebellum), and as a result, neurons fire less robustly for passive motion of the head relative to the body than for comparable head motion produced by whole-body motion.

Regions of the vestibular thalamus, including the ventral posterior lateral thalamus, comprise an important relay by which vestibular and proprioceptive information reach numerous regions of cortex (for review, Shinder & Taube, 2010; Hitier, Besnard, & Smith, 2014; Brandt & Dieterich, 2015). Accordingly, the self-motion information encoded by these thalamic neurons is likely vital for the construction of our perceptual estimate of head and body motion relative to space. It is thus noteworthy that combined stimulation of vestibular and proprioceptive systems, similar to that which would occur during natural, active self-motion, actually continues to produce a reduction in overall neuronal modulation.

Integration of Vestibular and Visual Inputs

Motion of the visual world across the retina, commonly termed optic flow, is also an important sensory cue during self-motion. Indeed, optic flow stimuli are capable of generating powerful sensations of motion even when a subject is motionless. Optic flow stimuli also induce reflexive eye movements that function to help keep the visual world stable on the retina. The resultant eye movements are called the optokinetic response (OKR), and during actual self-motion these eye movements work together with the vestibular-ocular reflex to ensure stable gaze at lower frequencies.

For the last many decades, the prevailing view had been that neurons in the vestibular nuclei are driven by large-field visual input as well as vestibular stimulation (Reisine & Raphan, 1992; Waespe & Henn, 1977a). Indeed, eye movement sensitive neurons in the vestibular nuclei that drive the vestibulo-ocular reflex to stabilize gaze during head motion are also an important component of the pathways that drive the OKR (goldfish, Dichgans, Bizzi, Morasso, & Tagliasco, 1973; cat, Keller & Precht, 1979; rat, Lannou, Cazin, Precht, & Toupet, 1982; monkey, Waespe & Henn, 1977a). In contrast, the population of neurons in the vestibular nuclei most relevant to understanding the neural computations that underlie the perception of self-motion are the VO neurons, which are not sensitive to eye movements. Indeed, while it was initially thought that VO neurons of the vestibular nuclei are also driven by optokinetic as well as vestibular stimulation (Boyle, Büttner, & Markert, 1985; Buettner & Büttner, 1979; Reisine & Raphan, 1992; Waespe & Henn, 1977a, 1977b), the extent of visual-vestibular convergence is not as prevalent as initially assumed (Figure 4B; Beraneck & Cullen, 2007; Bryan & Angelaki, 2009).

Consequently, while the idea that the vestibular nuclei neurons that contribute to self-motion pathways are driven by large-field visual is theoretically appealing, the extent of visual-vestibular convergence at the first central level of vestibular processing is not as prevalent as initially assumed. These recent studies show that the convergence of visual and vestibular inputs at this level does not provide a robust neural substrate by which the brain integrates these two sensory signals to estimate self-motion. Instead, the brain integrates full-field visual and vestibular inputs for higher-level functions such as the computation and perception of self-motion at the level of the cerebellum and cortex.

Early Vestibular Processing Can Discriminate Active From Passive Self-Motion

During everyday activities, our vestibular systems are most often activated as a result of our own actively generated self-motion. However, neurophysiological experiments to date have generally focused on the coding of self-motion that is passively applied. The ability to discriminate self-generated versus passively applied motion is vital for ensuring accurate postural control, as well as perceptual stability.

Figure 5. Neural mechanism for the attenuation of vestibular reafference. In order to make an active head movement, the brain must send a motor command to the neck muscles. The resultant head movement in turn produces stimulation of the vestibular stimulation system—termed vestibular reafference. While vestibular afferents encode this information, vestibular reafference is canceled at the first central stage of processing in the vestibular nuclei (blue traces), while in contrast head motion produced by passive stimulation is robustly encoded (green traces). Specifically, evidence to date is consistent with a mechanism in which a cancellation signal is sent to VO neurons in the vestibular nuclei when neck proprioceptive feedback matches the expected sensory consequence of neck motor command (red shaded box).

The cancellation of vestibular input resulting from active self-motion is mediated by a mechanism that compares the expected consequences of self-generated movement (computed by an internal model located in the cerebellum) and the actual sensory feedback (reviewed in Cullen, 2011, 2012). Key insights into this mechanism have been obtained through studies in which the relationship between intended and actual self-motion was experimentally controlled (Roy & Cullen, 2001, 2004; Brooks & Cullen, 2014; Brooks, Carriot, & Cullen, 2015). Specifically, a cancellation signal is generated only in conditions where the activation of proprioceptors matches the motor-generated expectation (i.e., an internal model of the brain’s expectation of the sensory consequences of the motor behavior). As a result, this cancellation mechanism ensures the selective encoding of passive self-motion at the earliest central stage of vestibular processing when both occur simultaneously. Importantly, however, in the specific condition where simultaneous active motion (termed reafference) and passive motion (termed exafference) result in activation of the same muscle proprioceptors, neurons robustly encoded the total vestibular input (i.e., responses to vestibular reafference and exafference were equally strong), rather than exafference alone (Brooks & Cullen, 2014). This is because cancellation of vestibular reafference in early vestibular processing requires an explicit match between expected and actual proprioceptive feedback.

As reviewed, the suppression of active motion occurs at the level of the vestibular nuclei. While both canal (Cullen & Minor, 2002; Sadeghi, Chacron, et al., 2007) and otolith (Jamali et al., 2009) afferents similarly encode active and passive rotations and translations, respectively, their target neurons in the vestibular nuclei—VO neurons—selectively respond to passive self-motion. This raises the question: What is the source of the cancellation signal that suppresses the vestibular-nerve input to vestibular-only neurons during active self-motion? To date, the available evidence suggests that the suppression of actively generated vestibular responses is mediated by a mechanism that (i) compares the actual activation of proprioceptors with a motor-generated expectation (i.e., an internal model of the expected proprioceptor activation; for review, see Cullen, 2011, 2012) and (ii) involves the cerebellum (Brooks & Cullen, 2013; Brooks et al., 2015).

It has long been known that the vestibular nuclei, deep cerebellar nuclei, and anterior vermis of the cerebellum are reciprocally interconnected (Batton, Jayaraman, Ruggiero, & Carpenter, 1977). In response to passively applied stimulation, these regions of the cerebellum also integrate vestibular and proprioceptive-related signals. For example, as reviewed, neurons in the most medial of the deep cerebellar nuclei (i.e., the rostral fastigial nucleus) integrate these two inputs such that they encode externally applied head and body-in-space motion in two distinct streams (Brooks & Cullen, 2009). In addition, single neurons in the anterior region of the cerebellar vermis (lobules I–V), which sends strong descending projections to the rostral fastigial nucleus, also integrate these two sensory inputs (Manzoni, Pompeiano, & Andre, 1998a, 1998b; Manzoni, Pompeiano, Bruschini, & Andre, 1999; Manzoini, Andre, & Bruschini, 2004).

Figure 6. The sensitivities of cerebellar neurons dynamically track the comparison of predictive and feedback signals. (A) Single unit recordings were made in the rostral fastigial nucleus, and the relationship between the motor command and resultant movement was altered during active head movements. (B) Head movements (top row) and activity of an example rostral fastigial neuron (middle row) during a motor learning task. Head movements and neuronal responses were first recorded before learning in passive and active conditions. Second, a load was applied and held constant for the learning phase. Third, after the learning phase, the motor was randomly turned off for single “catch” trials. Bottom row: Neurons’ normalized sensitivity to head movement during different phases of the learning sequence. During the learning phase, neuronal sensitivity gradually decreased from that measured during passive head motion to the suppressed response observed during active motion. Note that neuronal sensitivity during catch trials is comparable to the neuronal sensitivity during early learning and passive head movements.

During voluntary self-motion, however, individual rostral fastigial nucleus neurons encode an explicit and selective representation of unexpected self-motion (Brooks & Cullen, 2013). Specifically, neurons with vestibular or proprioceptive responses to passive externally applied head and body movements are unresponsive when the same motion is self-generated. Experiments, in which the relationship between the motor command and resultant movement was altered, have provided further insight into the underlying mechanism. In particular, trial-by-trial analyses have established that neuronal sensitivities dynamically track the comparison of predictive and feedback signals (Figure 6; Brooks et al., 2015), indicating that the output of the cerebellum reflects a fast and elegant computation in which the brain’s internal model of the sensory consequences of active self-motion is rapidly updated.

The rostral fastigial nucleus is a critical component of the descending pathway controlling postural reflexes and orienting behaviors; it projects to brainstem structures that control these behaviors, including vestibular-only neurons of the vestibular nuclei and spinal cord. In addition, the rostral fastigial and vestibular nuclei send important ascending projections to the vestibular thalamus (reviewed in Lopez & Blanke, 2011) and thus also play a vital role in carrying information required to compute our perception of self-motion and spatial orientation during everyday activities. The implications of findings discussed, namely that vestibular responses are canceled during active motion early in vestibular processing, are further considered.

In addition to the anterior lobe and deep cerebellar nuclei, three other primary regions of the cerebellum receive either primary (i.e., from afferents) or secondary (i.e., from vestibular nuclei) vestibular input (Figure 7). These regions include the nodulus and ventral uvula (lobules X and IX), flocculus and ventral paraflocculus, and oculomotor vermis of posterior lobe (lobules VI–VII). Of these, the nodulus/uvula of the cerebellum (Wearne, Raphan, & Cohen, 1998) likely plays the most critical role in the computations that underlie self-motion perception. The nodulus/uvula has strong reciprocal connection to the vestibular nuclei, and lesion studies in monkeys have shown that damage to these cerebellar regions produce deficits in spatial processing of vestibular information (Angelaki & Hess, 1995; Wearne et al., 1998).

The computations performed within this cerebellar network are thought to play an important role in allowing the brain to distinguish head tilt from translation. Notably, based on the physics of the vestibular sensory organs, the otolith organs cannot distinguish linear accelerations that are due to head tilts (relative to gravity) from those that are the result of translational self-motion. On the other hand, the activation of the semicircular canals differs in these two conditions, because semicircular canals are stimulated by the rotations associated with head tilts but are not stimulated during translation. Accordingly, in order to distinguish between tilt and translation, the brain must theoretically integrate otolith and canal inputs (Mayne, 1974; Merfeld, 1995; Angelaki, McHenry, Dickman, Newlands, & Hess, 1999; Merfeld, Zupan, & Peterka, 1999; Merfeld, Zupan, & Gifford, 2001; Merfeld, Park, & Gianna-Poulin, 2005; Bos & Bles, 2002; Green, Shaikh, & Angelaki, 2005; Green & Angelaki, 2007; Laurens & Droulez, 2007; Laurens & Angelaki, 2011; Laurens, Strauman, & Hess, 2011; Zupan, Merfeld, & Darlot, 2002). Consistent with this view, neurons in the monkey nodulus/uvula reflect a computation consistent with the integration of otolith and semicircular canal inputs. Specifically, when head motion is passively applied, some neurons appear to preferentially encode translation (Yakusheva et al., 2007), while others appear to better encode tilt (Laurens, Meng, & Angelaki, 2013). Note, however, that to date, it is not yet known whether these same cerebellar neurons distinguish tilt versus translation when the activation of the vestibular system is self-generated, during voluntary self-motion.

Vestibular information is relayed from the vestibular and deep cerebellar nuclei to the cortex via the thalamus. Notably, most ascending inputs from the vestibular nuclei project to the ventroposterior (VP) complex of the thalamus, which is also the main somatosensory nucleus of the thalamus. In turn, this thalamic region projects to areas of cortex where somatosensory and proprioceptive sensory signals are both encoded (i.e., areas 3av and parieto-insular vestibular cortex [PIVC]; reviewed in Goldberg et al., 2012). Single unit recording experiments in awake monkeys have further shown that neurons in the VP thalamic complex can also respond to passive rotations and translations (Büttner & Henn, 1976; Büttner, Henn, & Oswald, 1977; Magnin & Fuchs, 1977; Marlinski & McCrea, 2008a; Meng, May, Dickman, & Angelaki, 2007), and that this self-motion encoding is highly nonlinear such that response gains decrease markedly with increasing stimulus amplitude (Marlinski & McCrea, 2008a). Moreover, recordings from single VP neurons reveal the integration of vestibular signals with other inputs including somatosensory, proprioceptive or visual sensory information as well as motor signals (Marlinski & McCrea, 2008a, 2008b; Meng & Angelaki, 2010). Thus, vestibular processing at the level of the vestibular thalamus is already strongly multisensory in nature.

At the next stage of processing in the cortex, there are four main areas in which single neurons respond robustly to vestibular stimulation in monkeys, namely: (i) the parieto-insular vestibular cortex (PIVC) and areas 2v and 3a; (ii) the ventral intraparietal area (area VIP), (iii) the middle superior temporal area of extrastriate visual cortex (area MST), and (iv) the visuomotor areas of frontal cortex (specifically, the frontal eye field (FEFs), and supplementary eye fields (SEFs). The schematic in Figure 8 illustrates the locations of each of these regions. The results of single-unit studies in alert monkeys have led to the view that each of these areas serves a distinct but complementary role in relation to the integration of multisensory information for self-motion perception.

Parieto-Insular Vestibular Cortex (PIVC) and Areas 2v and 3a

Cortical area PIVC is commonly thought to be essential for shaping our perception of self-movement, spatial orientation, and body representation. In patients, stimulation of area PIVC produces vestibular sensations (Penfield, 1957). Correspondingly, PIVC lesions are associated with impairments in patients’ perception of subjective vertical (Brandt, Dieterich, & Danek, 1994) and increased psychophysical thresholds for heading judgements based on vestibular cues in monkeys (Chen, Gu, Liu, DeAngelis, & Angelaki, 2016). As noted, the parieto-insular vestibular cortex (PIVC) receives direct input from the vestibular thalamus. Interestingly, single neurons within area PIVC not only respond to rotations, and during optokinetic stimulation, but also can be sensitive to stimulation of somatosensory or proprioceptive receptors in the neck and occasionally limbs (Grüsser, Pause, & Schreiter, 1990a, 1990b; Guldin, Akbarian, & Grüsser, 1992). Moreover, while PIVC neurons are not sensitive to eye movements, they do respond to external visual target motion (Shinder & Newlands, 2014).

Area PIVC, in turn, is strongly interconnected with areas 3a and 2v (Guldin et al., 1992). Neurons in the vestibular region of area 3a also respond to proprioceptive as well as vestibular stimulation (Guldin et al., 1992), and send projections to motor cortex (Zarzecki, Blum, Bakker, & Herman, 1983). The vestibular properties of area 2v neurons have been characterized (Guldin et al., 1992), but much more work is still needed to understand the sensory and motor convergence that occurs in this region. Based on the information available to date, the role of the PIVC-3a-2v is currently not fully understood. Nevertheless, it has been suggested that the rich convergence of multisensory cues and external visual target motion information experienced during self-motion could be used to effectively monitor the motion of the head or body motion relative to that of an object of interest in external space (Akbarian, Grüsser, & Guldin, 1992; Guldin, Akbarian, & Grüsser, 1992; Shinder & Newlands, 2014). This in turn could be vital for shaping commands to generate accurate tracking movements, for example of the head and body, in order to successfully make contact with an object of interest as we move through our world.

In addition to integrating vestibular, somatosensory, and proprioceptive inputs, VIP neurons can respond strongly to full field visual information. Notably, neurons show tuning to optic flow stimulation that corresponds to their vestibular sensitivity (Bremmer et al., 2002; Chen, Henry, DeAngelis, & Angelaki, 2007). Overall, vestibular responses in VIP are generally stronger than their optic flow responses (Chen et al., 2007). Moreover, the vestibular tuning observed in VIP neurons does not vary as a function of either eye or head positions, suggesting that these neurons include information in a body- (or world-) centered reference frame (Chen, DeAngelis, & Angelaki, 2013). Taken together these observations have led to the common view that area VIP plays a role in the computation of heading direction during self-motion. However, somewhat surprisingly, a recent report concluded that inactivation of area VIP does not produce perceptual deficits in heading discrimination task (Chen et al., 2016).

Future work will be required to understand if and how the visual-vestibular convergence observed in area VIP contributes to the perception of heading direction or instead some other feature of self-motion perception. Indeed, to date only a few studies have described the responses of neurons in area VIP (as well as neighboring area MIP) during active self-motion (Klam & Graf, 2003, 2006). While neurons in both areas respond to passive whole-body rotation, the magnitudes of their responses to active head movement are typically smaller than those to passive whole-body rotation (Klam & Graf, 2006). Interesting, this observed attenuation of self-generated vestibular inputs at the cortical level is consistent with the suppression of vestibular reafference observed at earlier levels of vestibular processing in the vestibular and cerebellar nuclei. It has been suggested that the suppression of vestibular reafference at such high levels of cortical processing is vital for perceptual stability during active self-motion. Specifically, this distinction is likely to be critical for distinguishing movement through space that is self-generated from that which is externally applied and unexpected.

Single unit recording studies in monkeys have further revealed that MSTd neurons show modulation during self-motion in darkness (Duffy, 1998). This led to the proposal that area MSTd plays a role in computing a representation of heading direction (Duffy, 1998; Page & Duffy, 2003), in addition to its role in visuomotor transformations. In this context, a series of recent studies in monkeys by Angelaki and DeAngelis specifically focused on understanding how single neurons integrate visual and vestibular inputs in relation to heading perception during self-motion (reviewed in DeAngelis & Angelaki, 2012). However, while these neurophysiological studies found a correlative relationship between neuronal responses and heading perception based on vestibular signals (Fetsch, Wang, Gu, Deangelis, & Angelaki, 2007; Gu, DeAngelis, & Angelaki, 2007), it seems unlikely that MSTd plays a major role in self-motion perception. Indeed, these same investigators have shown that inactivation of the MSTd has little influence on vestibular heading as compared to visual heading thresholds (Gu, DeAngelis, & Angelaki, 2012). Accordingly, the evidence to date is consistent with the original view that this cortical area plays a principal role in shaping the visuomotor transformations required for the generation of compensatory eye movements in response to visual motion. In this context, the responses observed in response to vestibular stimulation may be predominantly related to the generation of a smooth eye movement command through either a parametric or predictive mechanisms (Barnes, 1988; Barnes & Eason, 1988; Lanman, Bizzi, & Allum, 1978; Cullen, Belton, & McCrea, 1991; Lisberger, 1990) in order to suppress the vestibulo-ocular reflex responses evoked during head movements to fixate a target.

Finally, it has long been appreciated that the frontal eye fields (FEFs), located within the frontal lobe of the brain, make important contributions to control of voluntary eye movements and visual attention. The finding that electrical stimulation of the contralateral vestibular nerve can produce evoked potentials in the region of the arcuate sulcus within the FEF (Ebata, Sugiuchi, Izawa, Shinomiya, & Shinoda, 2004) led to the proposal that this area is also involved in visuo-vestibular interactions. In head-restrained monkeys, neurons within the fundus of the arcuate sulcus preferentially respond to pursuit eye movements. These same neurons are typically unresponsive to passive whole-body rotation or translations in the dark but do respond when monkeys suppress their vestibulo-ocular reflex by tracking a target that moves with the head (Fukushima, Sato, Fukushima, Shinmei, & Kaneko, 2000; Fukushima, Akao, Kurkin, & Fukushima, 2005). Thus, similar to area MSTd, the FEF may mostly play a role in gaze stabilization during self-motion, likely by fine-tuning pursuit-vestibular interactions, rather than motion perception per se.

Pursuit responsive neurons can also be found within supplementary eye fields (SEFs) of frontal cortex. These neurons have relatively complex responses, and neurons can be driven by passive self-motion in the dark (Fukushima et al., 2004). This finding suggests that the SEF may have a different functional role than the FEF. Studies in patients with SEF lesions (Israël, Rivaud, Berthoz, & Pierrot-Deseilligny, 1992; Israël, Rivaud, Gaymard, Berthoz, & Pierrot-Deseilligny, 1995; Pierrot-Deseilligny, Israël, Berthoz, Rivaud, & Gaymard, 1993) can show impaired performance in a vestibular-memory-contingent saccade task. In this task, subjects are asked to estimate the amplitude of a prior passive head/body rotation by producing a saccadic eye movement (Bloomberg, Jones, Segal, McFarlane, & Soul, 1988). This suggests that the integration of information about self-motion and target location in the SEF may contribute to our ability to accurately control eye movements during every day activities.

Voluntary Behavior: Steering and Navigation

The ability to keep track of where we are in relation to where we are currently heading is essential for accurate motor control and perception. As reviewed, neurons in the earliest stages of vestibular processing—within the vestibular nuclei and deep cerebellar nuclei—show reduced responses to self-motion that is the result of voluntary behavior as compared to passively applied motion. A similar distinction is also made by vestibular-sensitive neurons at the level of the cerebellum and thalamus, which also encode other self-motion sensory cues including proprioceptive, somatosensory, and visual inputs.

The results of imaging studies in humans and experiments in which the vestibular nerve of patients was directly activated (De Waele, Baudonnière, Lepecq, Tran Ba Huy, & Vidal, 2001) or activated via caloric vestibular stimulation (Bottini et al., 1994; Friberg, Olsen, Roland, Paulson, & Lassen, 1985; Fasold et al., 2002; Lobel et al., 1998, 1999; Suzuki et al., 2001; Vitte et al., 1996) have established that the cortical areas activated are similar to those that have been identified in monkey single-unit recordings. In addition, the results of human studies further suggest that the cortical vestibular representation of vestibular information may be more widespread than is commonly assumed. For example, the regions of the prefrontal lobe and anterior portions of the supplementary motor area also show relatively short activation latencies (De Waele et al., 2001). To date, single-unit studies in monkeys have been performed in only a limited range of cortical areas, and thus whether vestibular information influences the activity of neurons in a wider range of cortical areas as is suggested by the human studies remains an open question.

In particular, there has been much recent emphasis on understanding the representation of self-motion information within areas of the medial temporal lobe including the entorhinal/perirhinal cortices and hippocampus—areas that likely play a critical role in spatial cognition and navigation (reviewed in Hitier, Besnard, & Smith, 2014). In this context, a specific class of neurons, called head direction cells, have been identified in rodents that are thought to represent the neural substrate of the perceived directional heading in the environment and enable accurate navigation. These neurons are found in numerous brain areas including the presubiculum, anterodorsal thalamus, lateral mammillary nuclei, retrosplenial cortex, lateral dorsal thalamus, striatum, and entorhinal cortex (reviewed in Taube, 2007). Common wisdom is that angular head velocity information from the vestibular nuclei plays a principal role in the generation of the head direction signal. Specifically, the head direction cell system is often thought of as an attractor network, which is updated by information of vestibular origin (McNaughton, Chen, & Markus, 1991; Skaggs, Knierim, Kudrimoti, & McNaughton, 1995; reviewed in Clark & Taube, 2012). A schematic of the proposed pathway linking the vestibular system to the head direction cell network is shown in Figure 9.

As reviewed, however, self-generated vestibular inputs are canceled at the first central stage of vestibular processing in rodents as well as monkeys. Thus, a more accurate view of head direction cells is that their responses reflect the integration of multiple self-motion cues. In fact, recent studies have shown that early vestibular pathways encode strong proprioceptive as well as vestibular signals in rodents (Medrea & Cullen, 2013), which could contribute to the generation of the head direction signal. This idea is consistent with reports that motor and proprioceptive influences shape head direction cell responses (Taube & Basset, 2003; Wiener, Berthoz, & Zugaro, 2002). Thus, the integration of multimodal sensory input and motor-based anticipation, a common feature of early vestibular processing, likely underlies the robust representation of directional heading in head direction cells.

To date, the coding of self-motion (i.e., vestibular) information during active locomotion has been well studied in rodents but not in primates. In primates, experiments have instead largely focused on virtual navigation tasks in which monkeys control their movement through a virtual environment. These studies have recorded the responses of neurons in cortical areas such as MSTd, a cortical region that has also been implicated in the generation of pursuit tracking movements. Notably in monkeys trained in a steering paradigm to perform a virtual navigation task, MSTd neurons show task-dependent response changes to visual stimulation including enhanced responses to the current heading direction (Page & Duffy, 2008; Jacob & Duffy, 2015). How the brain combines task-related (i.e., steering motor/motor preparation) signals with self-motion (i.e., vestibular, proprioceptive, and visual) information is not yet understood. Studies of early vestibular processing show that, in contrast to active head movements, neurons respond robustly when monkeys were trained to perform a “voluntary driving” task in which they steered a wheel to produce goal-directed head rotations (Roy & Cullen, 2001, 2002). This task, like virtual navigation, differs from natural navigation in that proprioceptive and motor-related signals that would normally be present during natural locomotion are absent. Thus, it remains to be determined if and how the responses of neurons at different levels along the self-motion pathways will respond when there is congruency between self-motion cues and motor signals, as is the case during natural navigation (i.e., locomotion).

Open Questions and Conclusions

Overall, the finding that early vestibular pathways preferentially encode passive self-motion has important implications regarding self-motion perception. When visual information is not available, the perception of self-motion can be achieved via path integration or dead reckoning (for review, see Loomis, Blascovich, & Beall, 1999; McNaughton, Battaglia, Jensen, Moser, & Moser, 2006). Path integration does not require identification of visual features in the external environment (e.g., viewing a landmark) but instead takes advantage of the integration of actively generated signals including motor efference copy, proprioceptive cues, and vestibular information (Frissen, Campos, Souman, & Ernst, 2011; Jürgens & Becker, 2006). The same neurons within the vestibular nuclei and deep cerebellar nuclei that show reduced responses to self-motion that is the consequence of voluntary behavior rather than passively applied are also the origin of vestibular input to the vestibular thalamus. Accordingly, it seems likely that the self-motion signals relayed to the higher-level cortical areas responsible for self-motion perception areas are attenuated during active motion (for review, see Cullen, 2012).

If the self-motion signals relayed to the higher-level cortical areas are suppressed during active motion, how does the brain compute an internal estimate of self-motion? Behavioral studies have shown that subjects will perceive changes in heading during active motion even in the absence of vision (Mittelstaedt & Glasauer, 1991, 1992). It is likely that, during active self-motion, the brain integrates head motion information derived from extra-vestibular inputs, including proprioceptive and motor efference copy signals (Blouin, Labrousse, Simoneau, Vercher, & Gauthier, 1998; Blouin, Okada, Wolsley, & Bronstein, 1998; Blouin, Amade, Vercher, & Gauthier, 1999). While these extravestibular inputs play a critical role in our ability to estimate head orientation, further experiments will be required to establish how vestibular sensitive neurons at subsequent stages of processing (i.e., thalamo-cortical pathways) actually encode information about the rotational and translational components of natural self-motion during actively generated versus passive self-motion.

To date, prior studies describing the information encoded by cortical areas, which likely contribute to the perception of self-motion (e.g., parieto-insular vestibular cortex, VIP, MSTd, areas 2v and 3a; Grüsser, Pause, & Shreiter, 1990a, 1990b; Fasold, Heinau, Trenner, Villringer, & Wenzel, 2008; for review, see Angelaki, Gu, & DeAngelis, 2011; Lopez & Blanke, 2011), have for the most part only considered passive stimuli. (for exceptions see Klam & Graf, 2003, 2006; Shinder & Newlands, 2014). Thus, it remains to be determined whether and how these areas distinguish actively generated from passive head movements. If vestibular inputs are attenuated, further experiments will also be required to understand the mechanism by which the brain integrates extra-vestibular cues with the attenuated vestibular signal to compute a robust estimate of heading during active self-motion.

Acknowledgments

This research study was supported by the Canadian Institutes of Health Research and the National Institutes of Health (DC2390).

Boyle, R., & Pompeiano, O. (1981). Responses of vestibulospinal neurons to neck and macular vestibular inputs in the presence or absence of the paleocerebellum. Annals of the New York Academy of Sciences, 374, 373–394.Find this resource:

Brooks, J. X., & Cullen, K. E. (2014). Early vestibular processing does not discriminate active from passive self-motion if there is a discrepancy between predicted and actual proprioceptive feedback. Journal of Neurophysiology, 111(12), 2465–2478.Find this resource:

Fukushima, K., Akao, T., Kurkin, S., & Fukushima, J. (2005). Role of vestibular signals in the caudal part of the frontal eye fields in pursuit eye movements in three‐dimensional space. Annals of the New York Academy of Sciences, 1039(1), 272–282.Find this resource:

Klam, F., & Graf, W. (2003). Vestibular signals of posterior parietal cortex neurons during active and passive head movements in macaque monkeys. Annals of the New York Academy of Sciences, 1004, 271–282.Find this resource: