Multisensory processing in spatial orientation: an inverse probabilistic approach

Effect of dynamic visual motion on perception of postural vertical through the modulation of prior knowledge of gravity

To internally estimate gravitational direction and body orientation, the central nervous system considers several sensory inputs from the periphery and prior knowledge of gravity. It is hypothesized that the modulation of visual inputs, supplying indirect information of gravity, affects the prior knowledge established internally by other sensory inputs from vestibular and somatosensory systems, leading to the alteration of perceived body orientation relative to gravity. In order to test the hypothesis, we examined the effect of presenting a visual motion stimulus during a whole-body static tilt on the subsequent evaluation of the perceived postural vertical. Fifteen subjects watched a target moving along the body longitudinal axis directing from head to feet with constant downward acceleration (CA condition) or constant velocity (CV condition), or they did not receive any visual stimulation (NV condition) during the whole-body static tilt. Subsequently, the direction of the subjective postural vertical (SPV) was evaluated. The result showed that the SPV in the CA condition was significantly tilted toward the direction of the preceding tilt compared to that in the NV condition while those in the CV and NV conditions were not significantly different. The present result suggests that dynamic visual motion along body longitudinal axis with downward acceleration can modulate prior knowledge of gravity, and in turn this affects the perception of body verticality.

Altered Visual and Proprioceptive Spatial Perception in Individuals with Parkinson's Disease

Difficulties in the integration of visual, vestibular, and somatosensory information in individuals with Parkinson's disease (PD) may alter perception of verticality. Accordingly, in this cross-sectional study, we analyzed PD patients' (n = 13) subjective visual vertical (SVV) and subjective haptic vertical (SHV) perceptions and compared them to those of healthy controls (n = 14). We compared SVV and SHV findings among participants with PD, healthy controls, and cutoff points of normality based on prior research literature, using the parametric nonpaired t test (at p < .05) and Cohen's d (at d > 0.8) to determine clinical relevance. We analyzed SVV with the bucket test and SHV with the rod rotations task in clockwise and counterclockwise directions. We calculated Pearson correlations to analyze the association between verticality tests and the most clinically affected body side. We calculated both the percentage of A-effect (expression of body tilt underestimation to the midline) and E-effect (expression of body tilt overestimation in the upright position). Individuals with PD showed greater variability in right SHV supination compared to the healthy control participants (p = .002). There was greater clinical relevance in right (as opposed to left) SVV (d = 0.83), right (as opposed to left) SHV pronation (d = 0.91), and left (as opposed to right) SHV pronation (d = 0.88). We observed a higher proportion of E-effect in individuals with PD. A significantly higher proportion of patients with PD, compared to patients in past literature, had right SHV pronation (p = .001), left SHV pronation (p = .023), right SHV supination (p = .001), left SHV supination (p = .046), and left SHV pronation (p = .046). Thus, subjective visual and proprioceptive perception of verticality is altered in patients with PD, compared to individuals without PD.

Control of human gait stability through foot placement

During human walking, the centre of mass (CoM) is outside the base of support for most of the time, which poses a challenge to stabilizing the gait pattern. Nevertheless, most of us are able to walk without substantial problems. In this review, we aim to provide an integrative overview of how humans cope with an underactuated gait pattern. A central idea that emerges from the literature is that foot placement is crucial in maintaining a stable gait pattern. In this review, we explore this idea; we first describe mechanical models and concepts that have been used to predict how foot placement can be used to control gait stability. These concepts, such as for instance the extrapolated CoM concept, the foot placement estimator concept and the capture point concept, provide explicit predictions on where to place the foot relative to the body at each step, such that gait is stabilized. Next, we describe empirical findings on foot placement during human gait in unperturbed and perturbed conditions. We conclude that humans show behaviour that is largely in accordance with the aforementioned concepts, with foot placement being actively coordinated to body CoM kinematics during the preceding step. In this section, we also address the requirements for such control in terms of the sensory information and the motor strategies that can implement such control, as well as the parts of the central nervous system that may be involved. We show that visual, vestibular and proprioceptive information contribute to estimation of the state of the CoM. Foot placement is adjusted to variations in CoM state mainly by modulation of hip abductor muscle activity during the swing phase of gait, and this process appears to be under spinal and supraspinal, including cortical, control. We conclude with a description of how control of foot placement can be impaired in humans, using ageing as a primary example and with some reference to pathology, and we address alternative strategies available to stabilize gait, which include modulation of ankle moments in the stance leg and changes in body angular momentum, such as rapid trunk tilts. Finally, for future research, we believe that especially the integration of consideration of environmental constraints on foot placement with balance control deserves attention.

Time course of the subjective visual vertical during sustained optokinetic and galvanic vestibular stimulation

The brain is thought to use rotation cues from both the vestibular and optokinetic system to disambiguate the gravito-inertial force, as measured by the otoliths, into components of linear acceleration and gravity direction relative to the head. Hence, when the head is stationary and upright, an erroneous percept of tilt arises during optokinetic roll stimulation (OKS) or when an artificial canal-like signal is delivered by means of galvanic vestibular stimulation (GVS). It is still unknown how this percept is affected by the combined presence of both cues or how it develops over time. Here, we measured the time course of the subjective visual vertical (SVV), as a proxy of perceived head tilt, in human participants ( n = 16) exposed to constant-current GVS (1 and 2 mA, cathodal and anodal) and constant-velocity OKS (30°/s clockwise and counterclockwise) or their combination. In each trial, participants continuously adjusted the orientation of a visual line, which drifted randomly, to Earth vertical. We found that both GVS and OKS evoke an exponential time course of the SVV. These time courses have different amplitudes and different time constants, 4 and 7 s respectively, and combine linearly when the two stimulations are presented together. We discuss these results in the framework of observer theory and Bayesian state estimation.

NEW & NOTEWORTHY While it is known that both roll optokinetic stimuli and galvanic vestibular stimulation affect the percept of vertical, how their effects combine and develop over time is still unclear. Here we show that both effects combined linearly but are characterized by different time constants, which we discuss from a probabilistic perspective.

Spatial orientation: Model-based approach to multi-sensory mechanisms

Perception of spatial orientation is generated through multimodal sensory integration. In this process, there are systematic errors with changes in the head or body position, which reflect challenges for the brain in maintaining a common sensory reference frame for spatial orientation. Here, we focus on this multisensory aspect of spatial orientation. We review a Bayesian spatial perception model that can be used as a framework to study sensory contributions to spatial orientation during lateral head tilts and probe neural networks involved in this process.

Coding strategies in the otolith system differ for translational head motion vs. static orientation relative to gravity

The detection of gravito-inertial forces by the otolith system is essential for our sense of balance and accurate perception. To date, however, how this system encodes the self-motion stimuli that are experienced during everyday activities remains unknown. Here, we addressed this fundamental question directly by recording from single otolith afferents in monkeys during naturalistic translational self-motion and changes in static head orientation. Otolith afferents with higher intrinsic variability transmitted more information overall about translational self-motion than their regular counterparts, owing to stronger nonlinearities that enabled precise spike timing including phase locking. By contrast, more regular afferents better discriminated between different static head orientations relative to gravity. Using computational methods, we further demonstrated that coupled increases in intrinsic variability and sensitivity accounted for the observed functional differences between afferent classes. Together, our results indicate that irregular and regular otolith afferents use different strategies to encode naturalistic self-motion and static head orientation relative to gravity.

Transformation of vestibular signals for the decisions of hand choice during whole body motion

In daily life, we frequently reach toward objects while our body is in motion. We have recently shown that body accelerations influence the decision of which hand to use for the reach, possibly by modulating the body-centered computations of the expected reach costs. However, head orientation relative to the body was not manipulated, and hence it remains unclear whether vestibular signals contribute in their head-based sensory frame or in a transformed body-centered reference frame to these cost calculations. To test this, subjects performed a preferential reaching task to targets at various directions while they were sinusoidally translated along the lateral body axis, with their head either aligned with the body (straight ahead) or rotated 18° to the left. As a measure of hand preference, we determined the target direction that resulted in equiprobable right/left-hand choices. Results show that head orientation affects this balanced target angle when the body is stationary but does not further modulate hand preference when the body is in motion. Furthermore, reaction and movement times were larger for reaches to the balanced target angle, resembling a competitive selection process, and were modulated by head orientation when the body was stationary. During body translation, reaction and movement times depended on the phase of the motion, but this phase-dependent modulation had no interaction with head orientation. We conclude that the brain transforms vestibular signals to body-centered coordinates at the early stage of reach planning, when the decision of hand choice is computed. NEW & NOTEWORTHY The brain takes inertial acceleration into account in computing the anticipated biomechanical costs that guide hand selection during whole body motion. Whereas these costs are defined in a body-centered, muscle-based reference frame, the otoliths detect the inertial acceleration in head-centered coordinates. By systematically manipulating head position relative to the body, we show that the brain transforms otolith signals into body-centered coordinates at an early stage of reach planning, i.e., before the decision of hand choice is computed.

Perception of Verticality and Vestibular Disorders of Balance and Falls

Objective: To review current knowledge of the perception of verticality, its normal function and disorders. This is based on an integrative graviceptive input from the vertical semicircular canals and the otolith organs.

Methods: The special focus is on human psychophysics, neurophysiological and imaging data on the adjustments of subjective visual vertical (SVV) and the subjective postural vertical. Furthermore, examples of mathematical modeling of specific vestibular cell functions for orientation in space in rodents and in patients are briefly presented.

Results: Pathological tilts of the SVV in the roll plane are most sensitive and frequent clinical vestibular signs of unilateral lesions extending from the labyrinths via the brainstem and thalamus to the parieto-insular vestibular cortex. Due to crossings of ascending graviceptive fibers, peripheral vestibular and pontomedullary lesions cause ipsilateral tilts of the SVV; ponto-mesencephalic lesions cause contralateral tilts. In contrast, SVV tilts, which are measured in unilateral vestibular lesions at thalamic and cortical levels, have two different characteristic features: (i) they may be ipsi- or contralateral, and (ii) they are smaller than those found in lower brainstem or peripheral lesions. Motor signs such as head tilt and body lateropulsion, components of ocular tilt reaction, are typical for vestibular lesions of the peripheral vestibular organ and the pontomedullary brainstem (vestibular nucleus). They are less frequent in midbrain lesions (interstitial nucleus of Cajal) and rare in cortical lesions. Isolated body lateropulsion is chiefly found in caudal lateral medullary brainstem lesions. Vestibular function in the roll plane and its disorders can be mathematically modeled by an attractor model of angular head velocity cell and head direction cell function. Disorders manifesting with misperception of the body vertical are the pusher syndrome, the progressive supranuclear palsy, or the normal pressure hydrocephalus; they may affect roll and/or pitch plane.

Conclusion: Clinical determinations of the SVV are easy and reliable. They indicate acute unilateral vestibular dysfunctions, the causative lesion of which extends from labyrinth to cortex. They allow precise topographical diagnosis of side and level in unilateral brainstem or peripheral vestibular disorders. SVV tilts may coincide with or differ from the perception of body vertical, e.g., in isolated body lateropulsion.

Age-related reweighting of visual and vestibular cues for vertical perception

As we age, the acuity of our sensory organs declines, which may affect our lifestyle. Sensory deterioration in the vestibular system is typically bilateral and gradual, and could lead to problems with balance and spatial orientation. To compensate for the sensory deterioration, it has been suggested that the brain reweights the sensory information sources according to their relative noise characteristics. For rehabilitation and training programs, it is important to understand the consequences of this reweighting, preferably at the individual subject level. We psychometrically examined the age-dependent reweighting of visual and vestibular cues used in spatial orientation in a group of 32 subjects (age range: 19-76 yr). We asked subjects to indicate the orientation of a line (clockwise or counterclockwise relative to the gravitational vertical) presented within an oriented square visual frame when seated upright or with their head tilted 30° relative to the body. Results show that subjects' vertical perception is biased by the orientation of the visual frame. Both the magnitude of this bias and response variability become larger with increasing age. Deducing the underlying sensory noise characteristics, using Bayesian inference, suggests an age-dependent reweighting of sensory information, with an increasing weight of the visual contextual information. Further scrutiny of the model suggests that this shift in sensory weights is the result of an increase in the noise of the vestibular signal. Our approach quantifies how noise properties of visual and vestibular systems change over the life span, which helps to understand the aging process at the neurocomputational level. NEW & NOTEWORTHY Perception of visual vertical involves a weighted fusion of visual and vestibular tilt cues. Using a Bayesian approach and experimental psychophysics, we quantify how this fusion process changes with age. We show that, with age, the vestibular information is down-weighted whereas the visual weight is increased. This shift in sensory reweighting is primarily due to an age-related increase of the noise of vestibular signals.

Biases in the Visual and Haptic Subjective Vertical Reveal the Role of Proprioceptive/Vestibular Priors in Child Development

Investigation of the perception of verticality permits to disclose the perceptual mechanisms that underlie balance control and spatial navigation. Estimation of verticality in unusual body orientation with respect to gravity (e.g., laterally tilted in the roll plane) leads to biases that change depending on the encoding sensory modality and the amount of tilt. A well-known phenomenon is the A-effect, that is a bias toward the body tilt often interpreted in a Bayesian framework to be the byproduct of a prior peaked at the most common head and body orientation, i.e., upright. In this study, we took advantage of this phenomenon to study the interaction of visual, haptic sensory information with vestibular/proprioceptive priors across development. We tested children (5-13 y.o) and adults (>22 y.o.) in an orientation discrimination task laterally tilted 90° to their left-ear side. Experimental conditions differed for the tested sensory modality: visual-only, haptic-only, both modalities. Resulting accuracy depended on the developmental stage and the encoding sensory modality, showing A-effects in vision across all ages and in the haptic modality only for the youngest children whereas bimodal judgments show lack of multisensory integration in children. A Bayesian prior model nicely predicts the behavioral data when the peak of the prior distribution shifts across age groups. Our results suggest that vision is pivotal to acquire an idiotropic vector useful for improving precision when upright. The acquisition of such a prior might be related to the development of head and trunk coordination, a process that is fundamental for gaining successful spatial navigation.

Plantar cutaneous afferents influence the perception of Subjective Visual Vertical in quiet stance

The estimation of Subjective Visual Vertical (SVV) involves the allocentric, gravitational and egocentric references, which are built by visual, vestibular and somatosensory afferents. Our goals were to assess the influence of plantar cutaneous afferents on the perception of SVV, and to see if there is a difference according to the efficiency of plantar cutaneous afferents. We recruited 48 young and healthy subjects and assessed their SVV and postural performances in quiet stance with a force platform, at 40 or 200 cm, in four ground conditions: on firm ground, on foam, with a bilateral, or with a unilateral 3 mm arch support. We also assessed the efficiency of our subjects' plantar afferents with the plantar quotient method and divided them in two groups: subjects with a normal use of plantar afferents and subjects with Plantar Exteroceptive Inefficiency (PEI). The results showed significant decreases in the counter clockwise SVV deviation only with the unilateral arch support, at near distance, and among the typically behaving subjects. We conclude that asymmetric foot cutaneous afferents are able to bias the egocentric vertical reference and hence influence the perception of SVV. This influence disappears among subjects with PEI, probably because of a distortion of the plantar signal.

Relationships Between Accuracy in Predicting Direction of Gravitational Vertical and Academic Performance and Physical Fitness in Schoolchildren

Enhanced levels of cardio-respiratory fitness (CRF) and physical activity (PA) are both positively associated with health and academic outcomes, but less is known about the spatial processing and perceptual components of PA. Perception of vertical (PV) is a spatial orientation ability that is important for PA, and is usually measured as relative accuracy in aligning an object to gravitational vertical against a tilted background. However, evidence is inconclusive regarding the relationship of PV to educational outcomes – most importantly, numeracy. Students were recruited from primary schools in the Australian Capital Territory. A group of 341 (females n = 162, mean age 11.3 years) children performed all the tests required for this study. A computerised rod and frame test of PV employing a small (20°) visual angle was administered, and socio-economic status (SES), national education test results (NAPLAN, 2010), and CRF and PA data were collected. Correlation and hierarchical regression analysis were used to examine the inter-relationships between PV and CRF, PA, SES and NAPLAN results. The two extreme quartile score groups from the measures of PV, PA and CRF were examined in relation to NAPLAN scores. PV scores arising from testing with a small visual angle and SES were found to be significantly associated with overall academic scores, and with the Numeracy, Reading, and Writing components of academic performance. Female gender was significantly associated with Writing score, and male with Numeracy score. Being less influenced by the background tilted frame, and therefore having visual field independence (FI), was associated with significantly higher academic scores, with the largest effect in Numeracy scores (effect size, d = 0.82) and also associated with higher CRF and PA levels. FI was positively associated with all the academic modules examined, and most strongly with Numeracy test results, suggesting that FI provides an indicator of STEM ability. These findings suggest that further longitudinal research into strategies designed to enhance visual FI deserve consideration, with a focus on specialized PA programs for pre-pubescent children. It is possible that small visual angle spatial tasks during PA may stimulate neural networks involved in numerical cognition.

Variability in the Vestibulo-Ocular Reflex and Vestibular Perception

The vestibular system enables humans to estimate self-motion, stabilize gaze and maintain posture, but these behaviors are impacted by neural noise at all levels of processing (e.g., sensory, central, motor). Despite its essential importance, the behavioral impact of noise in human vestibular pathways is not completely understood. Here, we characterize the vestibular imprecision that results from neural noise by measuring trial-to-trial vestibulo-ocular reflex (VOR) variability and perceptual just-noticeable differences (JNDs) in the same human subjects as a function of stimulus intensity. We used head-centered yaw rotations about an Earth-vertical axis over a broad range of motion velocities (0-65°/s for VOR variability and 3-90°/s peak velocity for JNDs). We found that VOR variability increased from approximately 0.6°/s at a chair velocity of 1°/s to approximately 3°/s at 65°/s; it exhibited a stimulus-independent range below roughly 1°/s. Perceptual imprecision ("sigma") increased from 0.76°/s at 3°/s to 4.7°/s at 90°/s. Using stimuli that manipulated the relationship between velocity, displacement and acceleration, we found that velocity was the salient cue for VOR variability for our motion stimuli. VOR and perceptual imprecision both increased with stimulus intensity and were broadly similar over a range of stimulus velocities, consistent with a common noise source that affects motor and perceptual pathways. This contrasts with differing perceptual and motor stimulus-dependent imprecision in visual studies. Either stimulus-dependent noise or non-linear signal processing could explain our results, but we argue that afferent non-linearities alone are unlikely to be the source of the observed behavioral stimulus-dependent imprecision.

Neck muscle spindle noise biases reaches in a multisensory integration task

Reference frame transformations (RFTs) are crucial components of sensorimotor transformations in the brain. Stochasticity in RFTs has been suggested to add noise to the transformed signal due to variability in transformation parameter estimates (e.g., angle) as well as the stochastic nature of computations in spiking networks of neurons. Here, we varied the RFT angle together with the associated variability and evaluated the behavioral impact in a reaching task that required variability-dependent visual-proprioceptive multisensory integration. Crucially, reaches were performed with the head either straight or rolled 30° to either shoulder, and we also applied neck loads of 0 or 1.8 kg (left or right) in a 3 × 3 design, resulting in different combinations of estimated head roll angle magnitude and variance required in RFTs. A novel three-dimensional stochastic model of multisensory integration across reference frames was fitted to the data and captured our main behavioral findings: 1) neck load biased head angle estimation across all head roll orientations, resulting in systematic shifts in reach errors; 2) increased neck muscle tone led to increased reach variability due to signal-dependent noise; and 3) both head roll and neck load created larger angular errors in reaches to visual targets away from the body compared with reaches toward the body. These results show that noise in muscle spindles and stochasticity in general have a tangible effect on RFTs underlying reach planning. Since RFTs are omnipresent in the brain, our results could have implications for processes as diverse as motor control, decision making, posture/balance control, and perception. NEW & NOTEWORTHY We show that increasing neck muscle tone systematically biases reach movements. A novel three-dimensional multisensory integration across reference frames model captures the data well and provides evidence that the brain must have online knowledge of full-body geometry together with the associated variability to plan reach movements accurately.

The Weighting of Cues to Upright Following Stroke With and Without a History of Pushing

OBJECTIVE

Perceived upright depends on three main factors: vision, graviception, and the internal representation of the long axis of the body. We assessed the relative contributions of these factors in individuals with sub-acute and chronic stroke and controls using a novel tool; the Oriented Character Recognition Test (OCHART). We also considered whether individuals who displayed active pushing or had a history of pushing behaviours had different weightings than those with no signs of pushing.

METHOD

Three participants experienced a stroke 6 months prior: eight with a history of pushing. In total, 12 participants served as healthy aged-matched controls. Visual and graviceptive cues were dissociated by orienting the visual background left, right, or upright relative to the body, or by orienting the body left, right, or upright relative to gravity. A three-vector model was used to quantify the weightings of vision, graviception, and the body to the perceptual upright.

RESULTS

The control group showed weightings of 13% vision, 25% graviception, and 62% body. Some individuals with stroke showed a similar pattern; others, particularly those with recent stroke, showed different patterns, for example, being unaffected by one of the three factors. The participant with active pushing behaviour displayed an ipsilesional perceptual bias (>30°) and was not affected by visual cues to upright.

CONCLUSION

The results of OCHART may be used to quantify the weightings of multisensory inputs in individuals post-stroke and may help characterize perceptual sources of pushing behaviours.

Psychophysical Evaluation of Sensory Reweighting in Bilateral Vestibulopathy

Perception of spatial orientation is thought to rely on the brain's integration of visual, vestibular, proprioceptive, and somatosensory signals, as well as internal beliefs. When one of these signals breaks down, such as the vestibular signal in bilateral vestibulopathy, patients start compensating by relying more on the remaining cues. How these signals are reweighted in this integration process is difficult to establish, since they cannot be measured in isolation during natural tasks, are inherently noisy, and can be ambiguous or in conflict. Here, we review our recent work, combining experimental psychophysics with a reverse engineering approach, based on Bayesian inference principles, to quantify sensory noise levels and optimal (re)weighting at the individual subject level, in both patients with bilateral vestibular deficits and healthy controls. We show that these patients reweight the remaining sensory information, relying more on visual and other nonvestibular information than healthy controls in the perception of spatial orientation. This quantification approach could improve diagnostics and prognostics of multisensory integration deficits in vestibular patients, and contribute to an evaluation of rehabilitation therapies directed toward specific training programs.

Causal Inference in the Perception of Verticality

The perceptual upright is thought to be constructed by the central nervous system (CNS) as a vector sum; by combining estimates on the upright provided by the visual system and the body's inertial sensors with prior knowledge that upright is usually above the head. Recent findings furthermore show that the weighting of the respective sensory signals is proportional to their reliability, consistent with a Bayesian interpretation of a vector sum (Forced Fusion, FF). However, violations of FF have also been reported, suggesting that the CNS may rely on a single sensory system (Cue Capture, CC), or choose to process sensory signals based on inferred signal causality (Causal Inference, CI). We developed a novel alternative-reality system to manipulate visual and physical tilt independently. We tasked participants (n = 36) to indicate the perceived upright for various (in-)congruent combinations of visual-inertial stimuli, and compared models based on their agreement with the data. The results favor the CI model over FF, although this effect became unambiguous only for large discrepancies (±60°). We conclude that the notion of a vector sum does not provide a comprehensive explanation of the perception of the upright, and that CI offers a better alternative.

Is perception of visual verticality intact in patients with idiopathic cervical dystonia?

Idiopathic cervical dystonia (CD) is a focal dystonia characterized by an abnormal tilted or twisted head position. This abnormal head position could lead to a distorted perception of the visual vertical and spatial orientation. The aim of this cross-sectional study was to investigate whether the perception of the visual vertical is impaired in patients with CD. The subjective visual vertical test (SVV) was measured in 24 patients with CD and 30 controls. The SVV test is conducted in a completely darkened room. A laser bar is projected on an opposing white wall, which is deviated from the earth's gravitational vertical. Participants were seated with their head unrestrained and were instructed to position this bar vertically. The deviations in degrees (°) are corrected for the side of laterocollis in order to measure the E-effect. We found that patients were able to position the laser bar as equally close to the earth's gravitational vertical as controls (+ 0.67° SD ± 2.12 vs + 0.29° SD ± 1.08, p = 0.43). No E-effect was measured. Notwithstanding the abnormal position of the head, the perception of the visual vertical in patients with idiopathic CD is intact, possibly because of central neural compensatory mechanisms.

Role of Rostral Fastigial Neurons in Encoding a Body-Centered Representation of Translation in Three Dimensions

Many daily behaviors rely critically on estimates of our body motion. Such estimates must be computed by combining neck proprioceptive signals with vestibular signals that have been transformed from a head- to a body-centered reference frame. Recent studies showed that deep cerebellar neurons in the rostral fastigial nucleus (rFN) reflect these computations, but whether they explicitly encode estimates of body motion remains unclear. A key limitation in addressing this question is that, to date, cell tuning properties have only been characterized for a restricted set of motions across head-re-body orientations in the horizontal plane. Here we examined, for the first time, how 3D spatiotemporal tuning for translational motion varies with head-re-body orientation in both horizontal and vertical planes in the rFN of male macaques. While vestibular coding was profoundly influenced by head-re-body position in both planes, neurons typically reflected at most a partial transformation. However, their tuning shifts were not random but followed the specific spatial trajectories predicted for a 3D transformation. We show that these properties facilitate the linear decoding of fully body-centered motion representations in 3D with a broad range of temporal characteristics from small groups of 5-7 cells. These results demonstrate that the vestibular reference frame transformation required to compute body motion is indeed encoded by cerebellar neurons. We propose that maintaining partially transformed rFN responses with different spatiotemporal properties facilitates the creation of downstream body motion representations with a range of dynamic characteristics, consistent with the functional requirements for tasks such as postural control and reaching.SIGNIFICANCE STATEMENT Estimates of body motion are essential for many daily activities. Vestibular signals are important contributors to such estimates but must be transformed from a head- to a body-centered reference frame. Here, we provide the first direct demonstration that the cerebellum computes this transformation fully in 3D. We show that the output of these computations is reflected in the tuning properties of deep cerebellar rostral fastigial nucleus neurons in a specific distributed fashion that facilitates the efficient creation of body-centered translation estimates with a broad range of temporal properties (i.e., from acceleration to position). These findings support an important role for the rostral fastigial nucleus as a source of body translation estimates functionally relevant for behaviors ranging from postural control to perception.

Gravity estimation and verticality perception

Gravity is a defining force that governs the evolution of mechanical forms, shapes and anchors our perception of the environment, and imposes fundamental constraints on our interactions with the world. Within the animal kingdom, humans are relatively unique in having evolved a vertical, bipedal posture. Although a vertical posture confers numerous benefits, it also renders us less stable than quadrupeds, increasing susceptibility to falls. The ability to accurately and precisely estimate our orientation relative to gravity is therefore of utmost importance. Here we review sensory information and computational processes underlying gravity estimation and verticality perception. Central to gravity estimation and verticality perception is multisensory cue combination, which serves to improve the precision of perception and resolve ambiguities in sensory representations by combining information from across the visual, vestibular, and somatosensory systems. We additionally review experimental paradigms for evaluating verticality perception, and discuss how particular disorders affect the perception of upright. Together, the work reviewed here highlights the critical role of multisensory cue combination in gravity estimation, verticality perception, and creating stable gravity-centered representations of our environment.

State Estimation for Early Feedback Responses in Reaching: Intramodal or Multimodal?

Humans are highly skilled in controlling their reaching movements, making fast and task-dependent movement corrections to unforeseen perturbations. To guide these corrections, the neural control system requires a continuous, instantaneous estimate of the current state of the arm and body in the world. According to Optimal Feedback Control theory, this estimate is multimodal and constructed based on the integration of forward motor predictions and sensory feedback, such as proprioceptive, visual and vestibular information, modulated by context, and shaped by past experience. But how can a multimodal estimate drive fast movement corrections, given that the involved sensory modalities have different processing delays, different coordinate representations, and different noise levels? We develop the hypothesis that the earliest online movement corrections are based on multiple single modality state estimates rather than one combined multimodal estimate. We review studies that have investigated online multimodal integration for reach control and offer suggestions for experiments to test for the existence of intramodal state estimates. If proven true, the framework of Optimal Feedback Control needs to be extended with a stage of intramodal state estimation, serving to drive short-latency movement corrections.

Bayesian quantification of sensory reweighting in a familial bilateral vestibular disorder (DFNA9)

DFNA9 is a rare progressive autosomal dominantly inherited vestibulo-cochlear disorder, resulting in a homogeneous group of patients with hearing impairment and bilateral vestibular function loss. These patients suffer from a deteriorated sense of spatial orientation, leading to balance problems in darkness, especially on irregular surfaces. Both behavioral and functional imaging studies suggest that the remaining sensory cues could compensate for the loss of vestibular information. A thorough model-based quantification of this reweighting in individual patients is, however, missing. Here we psychometrically examined the individual patient's sensory reweighting of these cues after complete vestibular loss. We asked a group of DFNA9 patients and healthy control subjects to judge the orientation (clockwise or counterclockwise relative to gravity) of a rod presented within an oriented square frame (rod-in-frame task) in three different head-on-body tilt conditions. Our results show a cyclical frame-induced bias in perceived gravity direction across a 90° range of frame orientations. The magnitude of this bias was significantly increased in the patients compared with the healthy control subjects. Response variability, which increased with head-on-body tilt, was also larger for the patients. Reverse engineering of the underlying signal properties, using Bayesian inference principles, suggests a reweighting of sensory signals, with an increase in visual weight of 20-40% in the patients. Our approach of combining psychophysics and Bayesian reverse engineering is the first to quantify the weights associated with the different sensory modalities at an individual patient level, which could make it possible to develop personal rehabilitation programs based on the patient's sensory weight distribution. NEW & NOTEWORTHY It has been suggested that patients with vestibular deficits can compensate for this loss by increasing reliance on other sensory cues, although an actual quantification of this reweighting is lacking. We combine experimental psychophysics with a reverse engineering approach based on Bayesian inference principles to quantify sensory reweighting in individual vestibular patients. We discuss the suitability of this approach for developing personal rehabilitation programs based on the patient's sensory weight distribution.

Task demands affect spatial reference frame weighting during tactile localization in sighted and congenitally blind adults

Task demands modulate tactile localization in sighted humans, presumably through weight adjustments in the spatial integration of anatomical, skin-based, and external, posture-based information. In contrast, previous studies have suggested that congenitally blind humans, by default, refrain from automatic spatial integration and localize touch using only skin-based information. Here, sighted and congenitally blind participants localized tactile targets on the palm or back of one hand, while ignoring simultaneous tactile distractors at congruent or incongruent locations on the other hand. We probed the interplay of anatomical and external location codes for spatial congruency effects by varying hand posture: the palms either both faced down, or one faced down and one up. In the latter posture, externally congruent target and distractor locations were anatomically incongruent and vice versa. Target locations had to be reported either anatomically (“palm” or “back” of the hand), or externally (“up” or “down” in space). Under anatomical instructions, performance was more accurate for anatomically congruent than incongruent target-distractor pairs. In contrast, under external instructions, performance was more accurate for externally congruent than incongruent pairs. These modulations were evident in sighted and blind individuals. Notably, distractor effects were overall far smaller in blind than in sighted participants, despite comparable target-distractor identification performance. Thus, the absence of developmental vision seems to be associated with an increased ability to focus tactile attention towards a non-spatially defined target. Nevertheless, that blind individuals exhibited effects of hand posture and task instructions in their congruency effects suggests that, like the sighted, they automatically integrate anatomical and external information during tactile localization. Moreover, spatial integration in tactile processing is, thus, flexibly adapted by top-down information—here, task instruction—even in the absence of developmental vision.

The Effect of Visual Experience on Perceived Haptic Verticality When Tilted in the Roll Plane

The orientation of the body in space can influence perception of verticality leading sometimes to biases consistent with priors peaked at the most common head and body orientation, that is upright. In this study, we investigate haptic perception of verticality in sighted individuals and early and late blind adults when tilted counterclockwise in the roll plane. Participants were asked to perform a stimulus orientation discrimination task with their body tilted to their left ear side 90° relative to gravity. Stimuli were presented by using a motorized haptic bar. In order to test whether different reference frames relative to the head influenced perception of verticality, we varied the position of the stimulus on the body longitudinal axis. Depending on the stimulus position sighted participants tended to have biases away or toward their body tilt. Visually impaired individuals instead show a different pattern of verticality estimations. A bias toward head and body tilt (i.e., Aubert effect) was observed in late blind individuals. Interestingly, no strong biases were observed in early blind individuals. Overall, these results posit visual sensory information to be fundamental in influencing the haptic readout of proprioceptive and vestibular information about body orientation relative to gravity. The acquisition of an idiotropic vector signaling the upright might take place through vision during development. Regarding early blind individuals, independent spatial navigation experience likely enhanced by echolocation behavior might have a role in such acquisition. In the case of participants with late onset blindness, early experience of vision might lead them to anchor their visually acquired priors to the haptic modality with no disambiguation between head and body references as observed in sighted individuals (Fraser et al., 2015). With our study, we aim to investigate haptic perception of gravity direction in unusual body tilts when vision is absent due to visual impairment. Insofar, our findings throw light on the influence of proprioceptive/vestibular sensory information on haptic perceived verticality in blind individuals showing how this phenomenon is affected by visual experience.

Bayesian optimal adaptation explains age-related human sensorimotor changes

The brain uses information from different sensory systems to guide motor behavior, and aging is associated with simultaneous decline in the quality of sensory information provided to the brain and deterioration in motor control. Correlations between age-dependent decline in sensory anatomical structures and behavior have been demonstrated in many sensorimotor systems, and it has recently been suggested that a Bayesian framework could explain these relationships. Here we show that age-dependent changes in a human sensorimotor reflex, the vestibuloocular reflex, are explained by a Bayesian optimal adaptation in the brain occurring in response to death of motion-sensing hair cells. Specifically, we found that the temporal dynamics of the reflex as a function of age emerge from ( r = 0.93, P < 0.001) a Kalman filter model that determines the optimal behavioral output when the sensory signal-to-noise characteristics are degraded by death of the transducers. These findings demonstrate that the aging brain is capable of generating the ideal and statistically optimal behavioral response when provided with deteriorating sensory information. While the Bayesian framework has been shown to be a general neural principle for multimodal sensory integration and dynamic sensory estimation, these findings provide evidence of longitudinal Bayesian processing over the human life span. These results illuminate how the aging brain strives to optimize motor behavior when faced with deterioration in the peripheral and central nervous systems and have implications in the field of vestibular and balance disorders, as they will likely provide guidance for physical therapy and for prosthetic aids that aim to reduce falls in the elderly. NEW & NOTEWORTHY We showed that age-dependent changes in the vestibuloocular reflex are explained by a Bayesian optimal adaptation in the brain that occurs in response to age-dependent sensory anatomical changes. This demonstrates that the brain can longitudinally respond to age-related sensory loss in an ideal and statistically optimal way. This has implications for understanding and treating vestibular disorders caused by aging and provides insight into the structure-function relationship during aging.

Perception of Upright: Multisensory Convergence and the Role of Temporo-Parietal Cortex

We inherently maintain a stable perception of the world despite frequent changes in the head, eye, and body positions. Such "orientation constancy" is a prerequisite for coherent spatial perception and sensorimotor planning. As a multimodal sensory reference, perception of upright represents neural processes that subserve orientation constancy through integration of sensory information encoding the eye, head, and body positions. Although perception of upright is distinct from perception of body orientation, they share similar neural substrates within the cerebral cortical networks involved in perception of spatial orientation. These cortical networks, mainly within the temporo-parietal junction, are crucial for multisensory processing and integration that generate sensory reference frames for coherent perception of self-position and extrapersonal space transformations. In this review, we focus on these neural mechanisms and discuss (i) neurobehavioral aspects of orientation constancy, (ii) sensory models that address the neurophysiology underlying perception of upright, and (iii) the current evidence for the role of cerebral cortex in perception of upright and orientation constancy, including findings from the neurological disorders that affect cortical function.

Perception of the dynamic visual vertical during sinusoidal linear motion

The vestibular system provides information for spatial orientation. However, this information is ambiguous: because the otoliths sense the gravitoinertial force, they cannot distinguish gravitational and inertial components. As a consequence, prolonged linear acceleration of the head can be interpreted as tilt, referred to as the somatogravic effect. Previous modeling work suggests that the brain disambiguates the otolith signal according to the rules of Bayesian inference, combining noisy canal cues with the a priori assumption that prolonged linear accelerations are unlikely. Within this modeling framework the noise of the vestibular signals affects the dynamic characteristics of the tilt percept during linear whole-body motion. To test this prediction, we devised a novel paradigm to psychometrically characterize the dynamic visual vertical-as a proxy for the tilt percept-during passive sinusoidal linear motion along the interaural axis (0.33 Hz motion frequency, 1.75 m/s² peak acceleration, 80 cm displacement). While subjects (n=10) kept fixation on a central body-fixed light, a line was briefly flashed (5 ms) at different phases of the motion, the orientation of which had to be judged relative to gravity. Consistent with the model's prediction, subjects showed a phase-dependent modulation of the dynamic visual vertical, with a subject-specific phase shift with respect to the imposed acceleration signal. The magnitude of this modulation was smaller than predicted, suggesting a contribution of nonvestibular signals to the dynamic visual vertical. Despite their dampening effect, our findings may point to a link between the noise components in the vestibular system and the characteristics of dynamic visual vertical.NEW & NOTEWORTHY A fundamental question in neuroscience is how the brain processes vestibular signals to infer the orientation of the body and objects in space. We show that, under sinusoidal linear motion, systematic error patterns appear in the disambiguation of linear acceleration and spatial orientation. We discuss the dynamics of these illusory percepts in terms of a dynamic Bayesian model that combines uncertainty in the vestibular signals with priors based on the natural statistics of head motion.

Unbounded evidence accumulation characterizes subjective visual vertical forced-choice perceptual choice and confidence

Humans can subjectively yet quantitatively assess choice confidence based on perceptual precision even when a perceptual decision is made without an immediate reward or feedback. However, surprisingly little is known about choice confidence. Here we investigate the dynamics of choice confidence by merging two parallel conceptual frameworks of decision making, signal detection theory and sequential analyses (i.e., drift-diffusion modeling). Specifically, to capture end-point statistics of binary choice and confidence, we built on a previous study that defined choice confidence in terms of psychophysics derived from signal detection theory. At the same time, we augmented this mathematical model to include accumulator dynamics of a drift-diffusion model to characterize the time dependence of the choice behaviors in a standard forced-choice paradigm in which stimulus duration is controlled by the operator. Human subjects performed a subjective visual vertical task, simultaneously reporting binary orientation choice and probabilistic confidence. Both binary choice and confidence experimental data displayed statistics and dynamics consistent with both signal detection theory and evidence accumulation, respectively. Specifically, the computational simulations showed that the unbounded evidence accumulator model fits the confidence data better than the classical bounded model, while bounded and unbounded models were indistinguishable for binary choice data. These results suggest that the brain can utilize mechanisms consistent with signal detection theory-especially when judging confidence without time pressure.NEW & NOTEWORTHY We found that choice confidence data show dynamics consistent with evidence accumulation for a forced-choice subjective visual vertical task. We also found that the evidence accumulation appeared unbounded when judging confidence, which suggests that the brain utilizes mechanisms consistent with signal detection theory to determine choice confidence.

Gravity dependence of the effect of optokinetic stimulation on the subjective visual vertical

Accurate and precise estimates of direction of gravity are essential for spatial orientation. According to Bayesian theory, multisensory vestibular, visual, and proprioceptive input is centrally integrated in a weighted fashion based on the reliability of the component sensory signals. For otolithic input, a decreasing signal-to-noise ratio was demonstrated with increasing roll angle. We hypothesized that the weights of vestibular (otolithic) and extravestibular (visual/proprioceptive) sensors are roll-angle dependent and predicted an increased weight of extravestibular cues with increasing roll angle, potentially following the Bayesian hypothesis. To probe this concept, the subjective visual vertical (SVV) was assessed in different roll positions (≤ ± 120°, steps = 30°, n = 10) with/without presenting an optokinetic stimulus (velocity = ± 60°/s). The optokinetic stimulus biased the SVV toward the direction of stimulus rotation for roll angles ≥ ± 30° (P < 0.005). Offsets grew from 3.9 ± 1.8° (upright) to 22.1 ± 11.8° (±120° roll tilt, P < 0.001). Trial-to-trial variability increased with roll angle, demonstrating a nonsignificant increase when providing optokinetic stimulation. Variability and optokinetic bias were correlated (R² = 0.71, slope = 0.71, 95% confidence interval = 0.57-0.86). An optimal-observer model combining an optokinetic bias with vestibular input reproduced measured errors closely. These findings support the hypothesis of a weighted multisensory integration when estimating direction of gravity with optokinetic stimulation. Visual input was weighted more when vestibular input became less reliable, i.e., at larger roll-tilt angles. However, according to Bayesian theory, the variability of combined cues is always lower than the variability of each source cue. If the observed increase in variability, although nonsignificant, is true, either it must depend on an additional source of variability, added after SVV computation, or it would conflict with the Bayesian hypothesis.NEW & NOTEWORTHY Applying a rotating optokinetic stimulus while recording the subjective visual vertical in different whole body roll angles, we noted the optokinetic-induced bias to correlate with the roll angle. These findings allow the hypothesis that the established optimal weighting of single-sensory cues depending on their reliability to estimate direction of gravity could be extended to a bias caused by visual self-motion stimuli.

Eye Movements in Darkness Modulate Self-Motion Perception

During self-motion, humans typically move the eyes to maintain fixation on the stationary environment around them. These eye movements could in principle be used to estimate self-motion, but their impact on perception is unknown. We had participants judge self-motion during different eye-movement conditions in the absence of full-field optic flow. In a two-alternative forced choice task, participants indicated whether the second of two successive passive lateral whole-body translations was longer or shorter than the first. This task was used in two experiments. In the first (n = 8), eye movements were constrained differently in the two translation intervals by presenting either a world-fixed or body-fixed fixation point or no fixation point at all (allowing free gaze). Results show that perceived translations were shorter with a body-fixed than a world-fixed fixation point. A linear model indicated that eye-movement signals received a weight of ∼25% for the self-motion percept. This model was independently validated in the trials without a fixation point (free gaze). In the second experiment (n = 10), gaze was free during both translation intervals. Results show that the translation with the larger eye-movement excursion was judged more often to be larger than chance, based on an oculomotor choice probability analysis. We conclude that eye-movement signals influence self-motion perception, even in the absence of visual stimulation.

Upright Perception and Ocular Torsion Change Independently during Head Tilt

We maintain a stable perception of the visual world despite continuous movements of our eyes, head and body. Perception of upright is a key aspect of such orientation constancy. Here we investigated whether changes in upright perception during sustained head tilt were related to simultaneous changes in torsional position of the eyes. We used a subjective visual vertical (SVV) task, modified to track changes in upright perception over time, and a custom video method to measure ocular torsion simultaneously. We tested 12 subjects in upright position, during prolonged (~15 min) lateral head tilts of 20 degrees, and also after the head returned to upright position. While the head was tilted, SVV drifted in the same direction as the head tilt (left tilt: -5.4 ± 1.4° and right tilt: +2.2 ± 2.1°). After the head returned to upright position, there was an SVV aftereffect with respect to the pre-tilt baseline, which was also in the same direction as the head tilt (left tilt: -3.9 ± 0.6° and right tilt: +2.55 ± 1.0°). Neither the SVV drift nor the SVV aftereffect were correlated with the changes in ocular torsion. Using the Bayesian spatial-perception model we show that the pattern of SVV drift and aftereffect in our results could be explained by a drift and an adaptation in sensory inputs that encode head orientation. The fact that ocular torsion (mainly driven by the otoliths) could not account for the perceptual changes suggests that neck proprioception could be the primary source of drift in upright perception during head tilt, and subsequently the aftereffect in upright position.

Concurrent use of somatotopic and external reference frames in a tactile mislocalization task

Localizing tactile stimuli on our body requires sensory information to be represented in multiple frames of reference along the sensory pathways. These reference frames include the representation of sensory information in skin coordinates, in which the spatial relationship of skin regions is maintained. The organization of the primary somatosensory cortex matches such somatotopic reference frame. In contrast, higher-order representations are based on external coordinates, in which body posture and gaze direction are taken into account in order to localise touch in other meaningful ways according to task demands. Dominance of one representation or the other, or the use of multiple representations with different weights, is thought to depend on contextual factors of cognitive and/or sensory origins. However, it is unclear under which situations a reference frame takes over another or when different reference frames are jointly used at the same time. The study of tactile mislocalizations at the fingers has shown a key role of the somatotopic frame of reference, both when touches are delivered unilaterally to a single hand, and when they are delivered bilaterally to both hands. Here, we took advantage of a well-established tactile mislocalization paradigm to investigate whether the reference frame used to integrate bilateral tactile stimuli can change as a function of the spatial relationship between the two hands. Specifically, supra-threshold interference stimuli were applied to the index or little fingers of the left hand 200ms prior to the application of a test stimulus on a finger of the right hand. Crucially, different hands postures were adopted (uncrossed or crossed). Results show that introducing a change in hand-posture triggered the concurrent use of somatotopic and external reference frames when processing bilateral touch at the fingers. This demonstrates that both somatotopic and external reference frames can be concurrently used to localise tactile stimuli on the fingers.

A Bayesian Account of Visual-Vestibular Interactions in the Rod-and-Frame Task

Panoramic visual cues, as generated by the objects in the environment, provide the brain with important information about gravity direction. To derive an optimal, i.e., Bayesian, estimate of gravity direction, the brain must combine panoramic information with gravity information detected by the vestibular system. Here, we examined the individual sensory contributions to this estimate psychometrically. We asked human subjects to judge the orientation (clockwise or counterclockwise relative to gravity) of a briefly flashed luminous rod, presented within an oriented square frame (rod-in-frame). Vestibular contributions were manipulated by tilting the subject's head, whereas visual contributions were manipulated by changing the viewing distance of the rod and frame. Results show a cyclical modulation of the frame-induced bias in perceived verticality across a 90° range of frame orientations. The magnitude of this bias decreased significantly with larger viewing distance, as if visual reliability was reduced. Biases increased significantly when the head was tilted, as if vestibular reliability was reduced. A Bayesian optimal integration model, with distinct vertical and horizontal panoramic weights, a gain factor to allow for visual reliability changes, and ocular counterroll in response to head tilt, provided a good fit to the data. We conclude that subjects flexibly weigh visual panoramic and vestibular information based on their orientation-dependent reliability, resulting in the observed verticality biases and the associated response variabilities.

Optimal visuotactile integration for velocity discrimination of self-hand movements

Illusory hand movements can be elicited by a textured disk or a visual pattern rotating under one's hand, while proprioceptive inputs convey immobility information (Blanchard C, Roll R, Roll JP, Kavounoudias A. PLoS One 8: e62475, 2013). Here, we investigated whether visuotactile integration can optimize velocity discrimination of illusory hand movements in line with Bayesian predictions. We induced illusory movements in 15 volunteers by visual and/or tactile stimulation delivered at six angular velocities. Participants had to compare hand illusion velocities with a 5°/s hand reference movement in an alternative forced choice paradigm. Results showed that the discrimination threshold decreased in the visuotactile condition compared with unimodal (visual or tactile) conditions, reflecting better bimodal discrimination. The perceptual strength (gain) of the illusions also increased: the stimulation required to give rise to a 5°/s illusory movement was slower in the visuotactile condition compared with each of the two unimodal conditions. The maximum likelihood estimation model satisfactorily predicted the improved discrimination threshold but not the increase in gain. When we added a zero-centered prior, reflecting immobility information, the Bayesian model did actually predict the gain increase but systematically overestimated it. Interestingly, the predicted gains better fit the visuotactile performances when a proprioceptive noise was generated by covibrating antagonist wrist muscles. These findings show that kinesthetic information of visual and tactile origins is optimally integrated to improve velocity discrimination of self-hand movements. However, a Bayesian model alone could not fully describe the illusory phenomenon pointing to the crucial importance of the omnipresent muscle proprioceptive cues with respect to other sensory cues for kinesthesia.

Do Visual and Vestibular Inputs Compensate for Somatosensory Loss in the Perception of Spatial Orientation? Insights from a Deafferented Patient

The present study aimed at investigating the consequences of a massive loss of somatosensory inputs on the perception of spatial orientation. The occurrence of possible compensatory processes for external (i.e., object) orientation perception and self-orientation perception was examined by manipulating visual and/or vestibular cues. To that aim, we compared perceptual responses of a deafferented patient (GL) with respect to age-matched Controls in two tasks involving gravity-related judgments. In the first task, subjects had to align a visual rod with the gravitational vertical (i.e., Subjective Visual Vertical: SVV) when facing a tilted visual frame in a classic Rod-and-Frame Test. In the second task, subjects had to report whether they felt tilted when facing different visuo-postural conditions which consisted in very slow pitch tilts of the body and/or visual surroundings away from vertical. Results showed that, much more than Controls, the deafferented patient was fully dependent on spatial cues issued from the visual frame when judging the SVV. On the other hand, the deafferented patient did not rely at all on visual cues for self-tilt detection. Moreover, the patient never reported any sensation of tilt up to 18° contrary to Controls, hence showing that she did not rely on vestibular (i.e., otoliths) signals for the detection of very slow body tilts either. Overall, this study demonstrates that a massive somatosensory deficit substantially impairs the perception of spatial orientation, and that the use of the remaining sensory inputs available to a deafferented patient differs regarding whether the judgment concerns external vs. self-orientation.

Gravity in the Brain as a Reference for Space and Time Perception

Moving and interacting with the environment require a reference for orientation and a scale for calibration in space and time. There is a wide variety of environmental clues and calibrated frames at different locales, but the reference of gravity is ubiquitous on Earth. The pull of gravity on static objects provides a plummet which, together with the horizontal plane, defines a three-dimensional Cartesian frame for visual images. On the other hand, the gravitational acceleration of falling objects can provide a time-stamp on events, because the motion duration of an object accelerated by gravity over a given path is fixed. Indeed, since ancient times, man has been using plumb bobs for spatial surveying, and water clocks or pendulum clocks for time keeping. Here we review behavioral evidence in favor of the hypothesis that the brain is endowed with mechanisms that exploit the presence of gravity to estimate the spatial orientation and the passage of time. Several visual and non-visual (vestibular, haptic, visceral) cues are merged to estimate the orientation of the visual vertical. However, the relative weight of each cue is not fixed, but depends on the specific task. Next, we show that an internal model of the effects of gravity is combined with multisensory signals to time the interception of falling objects, to time the passage through spatial landmarks during virtual navigation, to assess the duration of a gravitational motion, and to judge the naturalness of periodic motion under gravity.

Dissociating vestibular and somatosensory contributions to spatial orientation

Inferring object orientation in the surroundings heavily depends on our internal sense of direction of gravity. Previous research showed that this sense is based on the integration of multiple information sources, including visual, vestibular (otolithic), and somatosensory signals. The individual noise characteristics and contributions of these sensors can be studied using spatial orientation tasks, such as the subjective visual vertical (SVV) task. A recent study reported that patients with complete bilateral vestibular loss perform similar as healthy controls on these tasks, from which it was conjectured that the noise levels of both otoliths and body somatosensors are roll-tilt dependent. Here, we tested this hypothesis in 10 healthy human subjects by roll tilting the head relative to the body to dissociate tilt-angle dependencies of otolith and somatosensory noise. Using a psychometric approach, we measured the perceived orientation, and its variability, of a briefly flashed line relative to the gravitational vertical (SVV). Measurements were taken at multiple body-in-space orientations (-90 to 90°, steps of 30°) and head-on-body roll tilts (30° left ear down, aligned, 30° right ear down). Results showed that verticality perception is processed in a head-in-space reference frame, with a systematic SVV error that increased with larger head-in-space orientations. Variability patterns indicated a larger contribution of the otolith organs around upright and a more substantial contribution of the body somatosensors at larger body-in-space roll tilts. Simulations show that these findings are consistent with a statistical model that involves tilt-dependent noise levels of both otolith and somatosensory signals, confirming dynamic shifts in the weights of sensory inputs with tilt angle.

Stimulus intensity modulates multisensory temporal processing

One of the more challenging feats that multisensory systems must perform is to determine which sensory signals originate from the same external event, and thus should be integrated or "bound" into a singular perceptual object or event, and which signals should be segregated. Two important stimulus properties impacting this process are the timing and effectiveness of the paired stimuli. It has been well established that the more temporally aligned two stimuli are, the greater the degree to which they influence one another's processing. In addition, the less effective the individual unisensory stimuli are in eliciting a response, the greater the benefit when they are combined. However, the interaction between stimulus timing and stimulus effectiveness in driving multisensory-mediated behaviors has never been explored - which was the purpose of the current study. Participants were presented with either high- or low-intensity audiovisual stimuli in which stimulus onset asynchronies (SOAs) were parametrically varied, and were asked to report on the perceived synchrony/asynchrony of the paired stimuli. Our results revealed an interaction between the temporal relationship (SOA) and intensity of the stimuli. Specifically, individuals were more tolerant of larger temporal offsets (i.e., more likely to call them synchronous) when the paired stimuli were less effective. This interaction was also seen in response time (RT) distributions. Behavioral gains in RTs were seen with synchronous relative to asynchronous presentations, but this effect was more pronounced with high-intensity stimuli. These data suggest that stimulus effectiveness plays an underappreciated role in the perception of the timing of multisensory events, and reinforces the interdependency of the principles of multisensory integration in determining behavior and shaping perception.

The Subjective Visual Vertical and the Subjective Haptic Vertical Access Different Gravity Estimates

The subjective visual vertical (SVV) and the subjective haptic vertical (SHV) both claim to probe the underlying perception of gravity. However, when the body is roll tilted these two measures evoke different patterns of errors with SVV generally becoming biased towards the body (A-effect, named for its discoverer, Hermann Rudolph Aubert) and SHV remaining accurate or becoming biased away from the body (E-effect, short for Entgegengesetzt-effect, meaning “opposite”, i.e., opposite to the A-effect). We compared the two methods in a series of five experiments and provide evidence that the two measures access two different but related estimates of gravitational vertical. Experiment 1 compared SVV and SHV across three levels of whole-body tilt and found that SVV showed an A-effect at larger tilts while SHV was accurate. Experiment 2 found that tilting either the head or the trunk independently produced an A-effect in SVV while SHV remained accurate when the head was tilted on an upright body but showed an A-effect when the body was tilted below an upright head. Experiment 3 repeated these head/body configurations in the presence of vestibular noise induced by using disruptive galvanic vestibular stimulation (dGVS). dGVS abolished both SVV and SHV A-effects while evoking a massive E-effect in the SHV head tilt condition. Experiments 4 and 5 show that SVV and SHV do not combine in an optimally statistical fashion, but when vibration is applied to the dorsal neck muscles, integration becomes optimal. Overall our results suggest that SVV and SHV access distinct underlying gravity percepts based primarily on head and body position information respectively, consistent with a model proposed by Clemens and colleagues.

Internal Models, Vestibular Cognition, and Mental Imagery: Conceptual Considerations

Vestibular cognition has recently gained attention. Despite numerous experimental and clinical demonstrations, it is not yet clear what vestibular cognition really is. For future research in vestibular cognition, adopting a computational approach will make it easier to explore the underlying mechanisms. Indeed, most modeling approaches in vestibular science include a top-down or a priori component. We review recent Bayesian optimal observer models, and discuss in detail the conceptual value of prior assumptions, likelihood and posterior estimates for research in vestibular cognition. We then consider forward models in vestibular processing, which are required in order to distinguish between sensory input that is induced by active self-motion, and sensory input that is due to passive self-motion. We suggest that forward models are used not only in the service of estimating sensory states but they can also be drawn upon in an offline mode (e.g., spatial perspective transformations), in which interaction with sensory input is not desired. A computational approach to vestibular cognition will help to discover connections across studies, and it will provide a more coherent framework for investigating vestibular cognition.

Sensory substitution in bilateral vestibular a-reflexic patients

Patients with bilateral vestibular loss have balance problems in darkness, but maintain spatial orientation rather effectively in the light. It has been suggested that these patients compensate for vestibular cues by relying on extravestibular signals, including visual and somatosensory cues, and integrating them with internal beliefs. How this integration comes about is unknown, but recent literature suggests the healthy brain remaps the various signals into a task-dependent reference frame, thereby weighting them according to their reliability. In this paper, we examined this account in six patients with bilateral vestibular a-reflexia, and compared them to six age-matched healthy controls. Subjects had to report the orientation of their body relative to a reference orientation or the orientation of a flashed luminous line relative to the gravitational vertical, by means of a two-alternative-forced-choice response. We tested both groups psychometrically in upright position (0°) and 90° sideways roll tilt. Perception of body tilt was unbiased in both patients and controls. Response variability, which was larger for 90° tilt, did not differ between groups, indicating that body somatosensory cues have tilt-dependent uncertainty. Perception of the visual vertical was unbiased when upright, but showed systematic undercompensation at 90° tilt. Variability, which was larger for 90° tilt than upright, did not differ between patients and controls. Our results suggest that extravestibular signals substitute for vestibular input in patients’ perception of spatial orientation. This is in line with the current status of rehabilitation programs in acute vestibular patients, targeting at recognizing body somatosensory signals as a reliable replacement for vestibular loss.

Rocking or rolling--perception of ambiguous motion after returning from space

The central nervous system must resolve the ambiguity of inertial motion sensory cues in order to derive an accurate representation of spatial orientation. Adaptive changes during spaceflight in how the brain integrates vestibular cues with other sensory information can lead to impaired movement coordination, vertigo, spatial disorientation, and perceptual illusions after return to Earth. The purpose of this study was to compare tilt and translation motion perception in astronauts before and after returning from spaceflight. We hypothesized that these stimuli would be the most ambiguous in the low-frequency range (i.e., at about 0.3 Hz) where the linear acceleration can be interpreted either as a translation or as a tilt relative to gravity. Verbal reports were obtained in eleven astronauts tested using a motion-based tilt-translation device and a variable radius centrifuge before and after flying for two weeks on board the Space Shuttle. Consistent with previous studies, roll tilt perception was overestimated shortly after spaceflight and then recovered with 1–2 days. During dynamic linear acceleration (0.15–0.6 Hz, ±1.7 m/s²) perception of translation was also overestimated immediately after flight. Recovery to baseline was observed after 2 days for lateral translation and 8 days for fore–aft translation. These results suggest that there was a shift in the frequency dynamic of tilt-translation motion perception after adaptation to weightlessness. These results have implications for manual control during landing of a space vehicle after exposure to microgravity, as it will be the case for human asteroid and Mars missions.

Static roll-tilt over 5 minutes locally distorts the internal estimate of direction of gravity

The subjective visual vertical (SVV) indicates perceived direction of gravity. Even in healthy human subjects, roll angle-dependent misestimations, roll overcompensation (A-effect, head-roll > 60° and <135°) and undercompensation (E-effect, head-roll < 60°), occur. Previously, we demonstrated that, after prolonged roll-tilt, SVV estimates when upright are biased toward the preceding roll position, which indicates that perceived vertical (PV) is shifted by the prior tilt (Tarnutzer AA, Bertolini G, Bockisch CJ, Straumann D, Marti S. PLoS One 8: e78079, 2013). Hypothetically, PV in any roll position could be biased toward the previous roll position. We asked whether such a "global" bias occurs or whether the bias is "local". The SVV of healthy human subjects (N = 9) was measured in nine roll positions (-120° to +120°, steps = 30°) after 5 min of roll-tilt in one of two adaptation positions (±90°) and compared with control trials without adaptation. After adapting, adjustments were shifted significantly (P < 0.05) toward the previous adaptation position for nearby roll-tilted positions (±30°, ±60°) and upright only. We computationally simulated errors based on the sum of a monotonically increasing function (producing roll undercompensation) and a mixture of Gaussian functions (representing roll overcompensation centered around PV). In combination, the pattern of A- and E-effects could be generated. By shifting the function representing local overcompensation toward the adaptation position, the experimental postadaptation data could be fitted successfully. We conclude that prolonged roll-tilt locally distorts PV rather than globally shifting it. Short-term adaptation of roll overcompensation may explain these shifts and could reflect the brain's strategy to optimize SVV estimates around recent roll positions. Thus postural stability can be improved by visually-mediated compensatory responses at any sustained body-roll orientation.