Mathematical models for dynamic, multisensory spatial orientation perception

A computational model of motion sickness dynamics during passive self-motion in the dark

Predicting the time course of motion sickness symptoms enables the evaluation of provocative stimuli and the development of countermeasures for reducing symptom severity. In pursuit of this goal, we present an Observer-driven model of motion sickness for passive motions in the dark. Constructed in two stages, this model predicts motion sickness symptoms by bridging sensory conflict (i.e., differences between actual and expected sensory signals) arising from the Observer model of spatial orientation perception (stage 1) to Oman's model of motion sickness symptom dynamics (stage 2; presented in 1982 and 1990) through a proposed "Normalized Innovation Squared" statistic. The model outputs the expected temporal development of human motion sickness symptom magnitudes (mapped to the Misery Scale) at a population level, due to arbitrary, 6-degree-of-freedom, self-motion stimuli. We trained model parameters using individual subject responses collected during fore-aft translations and off-vertical axis of rotation motions. Improving on prior efforts, we only used datasets with experimental conditions congruent with the perceptual stage (i.e., adequately provided passive motions without visual cues) to inform the model. We assessed model performance by predicting an unseen validation dataset, producing a Q2 value of 0.91. Demonstrating this model's broad applicability, we formulate predictions for a host of stimuli, including translations, earth-vertical rotations, and altered gravity, and we provide our implementation for other users. Finally, to guide future research efforts, we suggest how to rigorously advance this model (e.g., incorporating visual cues, active motion, responses to motion of different frequency, etc.).

The role of vision in sensory integration models for predicting motion perception and sickness

Users of automated vehicles will engage in other activities and take their eyes off the road, making them prone to motion sickness. To resolve this, the current paper validates models predicting sickness in response to motion and visual conditions. We validate published models of vestibular and visual sensory integration that have been used for predicting motion sickness through sensory conflict. We use naturalistic driving data and laboratory motion (and vection) paradigms, such as sinusoidal translation and rotation at different frequencies, Earth-Vertical Axis Rotation, Off-Vertical Axis Rotation, Centrifugation, Somatogravic Illusion, and Pseudo-Coriolis, to evaluate different models for both motion perception and motion sickness. We investigate the effects of visual motion perception in terms of rotational velocity (visual flow) and verticality. According to our findings, the SVCI model, a 6DOF model based on the Subjective Vertical Conflict (SVC) theory, with visual rotational velocity input is effective at estimating motion sickness. However, it does not correctly replicate motion perception in paradigms such as roll-tilt perception during centrifuge, pitch perception during somatogravic illusion, and pitch perception during pseudo-Coriolis motions. On the other hand, the Multi-Sensory Observer Model (MSOM) accurately models motion perception in all considered paradigms, but does not effectively capture the frequency sensitivity of motion sickness, and the effects of vision on sickness. For both models (SVCI and MSOM), the visual perception of rotational velocity strongly affects sickness and perception. Visual verticality perception does not (yet) contribute to sickness prediction, and contributes to perception prediction only for the somatogravic illusion. In conclusion, the SVCI model with visual rotation velocity feedback is the current preferred option to design vehicle control algorithms for motion sickness reduction, while the MSOM best predicts perception. A unified model that jointly captures perception and motion sickness remains to be developed.

Humans gradually integrate sudden gain or loss of visual information into spatial orientation perception

Introduction

Vestibular and visual information is used in determining spatial orientation. Existing computational models of orientation perception focus on the integration of visual and vestibular orientation information when both are available. It is well-known, and computational models capture, differences in spatial orientation perception with visual information or without (i.e., in the dark). For example, during earth vertical yaw rotation at constant angular velocity without visual information, humans perceive their rate of rotation to decay. However, during the same sustained rotation with visual information, humans can continue to more accurately perceive self-rotation. Prior to this study, there was no existing literature on human motion perception where visual information suddenly become available or unavailable during self-motion.

Methods

Via a well verified psychophysical task, we obtained perceptual reports of self-rotation during various profiles of Earth-vertical yaw rotation. The task involved transitions in the availability of visual information (and control conditions with visual information available throughout the motion or unavailable throughout).

Results

We found that when visual orientation information suddenly became available, subjects gradually integrated the new visual information over ~10 seconds. In the opposite scenario (visual information suddenly removed), past visual information continued to impact subject perception of self-rotation for ~30 seconds. We present a novel computational model of orientation perception that is consistent with the experimental results presented in this study.

Discussion

The gradual integration of sudden loss or gain of visual information is achieved via low pass filtering in the visual angular velocity sensory conflict pathway. In conclusion, humans gradually integrate sudden gain or loss of visual information into their existing perception of self-motion.

Neck stabilization through sensory integration of vestibular and visual motion cues

Background

To counteract gravity, trunk motion, and other perturbations, the human head-neck system requires continuous muscular stabilization. In this study, we combine a musculoskeletal neck model with models of sensory integration (SI) to unravel the role of vestibular, visual, and muscle sensory cues in head-neck stabilization and relate SI conflicts and postural instability to motion sickness.

Method

A 3D multisegment neck model with 258 Hill-type muscle elements was extended with postural stabilization using SI of vestibular (semicircular and otolith) and visual (rotation rate, verticality, and yaw) cues using the multisensory observer model (MSOM) and the subjective vertical conflict model (SVC). Dynamic head-neck stabilization was studied using empirical datasets, including 6D trunk perturbations and a 4 m/s2 slalom drive inducing motion sickness.

Results

Recorded head translation and rotation are well matched when using all feedback loops with MSOM or SVC or assuming perfect perception. A basic version of the model, including muscle, but omitting vestibular and visual perception, shows that muscular feedback can stabilize the neck in all conditions. However, this model predicts excessive head rotations in conditions with trunk rotation and in the slalom. Adding feedback of head rotational velocity sensed by the semicircular canals effectively reduces head rotations at mid-frequencies. Realistic head rotations at low frequencies are obtained by adding vestibular and visual feedback of head rotation based on the MSOM or SVC model or assuming perfect perception. The MSOM with full vision well captures all conditions, whereas the MSOM excluding vision well captures all conditions without vision. The SVC provides two estimates of verticality, with a vestibular estimate SVCvest, which is highly effective in controlling head verticality, and an integrated vestibular/visual estimate SVCint which can complement SVCvest in conditions with vision. As expected, in the sickening drive, SI models imprecisely estimate verticality, resulting in sensory conflict and postural instability.

Conclusion

The results support the validity of SI models in postural stabilization, where both MSOM and SVC provide credible results. The results in the sickening drive show imprecise sensory integration to enlarge head motion. This uniquely links the sensory conflict theory and the postural instability theory in motion sickness causation.

A computational model of motion sickness dynamics during passive self-motion in the dark

Predicting the time course of motion sickness symptoms enables the evaluation of provocative stimuli and the development of countermeasures for reducing symptom severity. In pursuit of this goal, we present an observer-driven model of motion sickness for passive motions in the dark. Constructed in two stages, this model predicts motion sickness symptoms by bridging sensory conflict (i.e., differences between actual and expected sensory signals) arising from the observer model of spatial orientation perception (stage 1) to Oman's model of motion sickness symptom dynamics (stage 2; presented in 1982 and 1990) through a proposed "Normalized innovation squared" statistic. The model outputs the expected temporal development of human motion sickness symptom magnitudes (mapped to the Misery Scale) at a population level, due to arbitrary, 6-degree-of-freedom, self-motion stimuli. We trained model parameters using individual subject responses collected during fore-aft translations and off-vertical axis of rotation motions. Improving on prior efforts, we only used datasets with experimental conditions congruent with the perceptual stage (i.e., adequately provided passive motions without visual cues) to inform the model. We assessed model performance by predicting an unseen validation dataset, producing a Q2 value of 0.86. Demonstrating this model's broad applicability, we formulate predictions for a host of stimuli, including translations, earth-vertical rotations, and altered gravity, and we provide our implementation for other users. Finally, to guide future research efforts, we suggest how to rigorously advance this model (e.g., incorporating visual cues, active motion, responses to motion of different frequency, etc.).

Modeling orientation perception adaptation to altered gravity environments with memory of past sensorimotor states

Transitioning between gravitational environments results in a central reinterpretation of sensory information, producing an adapted sensorimotor state suitable for motor actions and perceptions in the new environment. Critically, this central adaptation is not instantaneous, and complete adaptation may require weeks of prolonged exposure to novel environments. To mitigate risks associated with the lagging time course of adaptation (e.g., spatial orientation misperceptions, alterations in locomotor and postural control, and motion sickness), it is critical that we better understand sensorimotor states during adaptation. Recently, efforts have emerged to model human perception of orientation and self-motion during sensorimotor adaptation to new gravity stimuli. While these nascent computational frameworks are well suited for modeling exposure to novel gravitational stimuli, they have yet to distinguish how the central nervous system (CNS) reinterprets sensory information from familiar environmental stimuli (i.e., readaptation). Here, we present a theoretical framework and resulting computational model of vestibular adaptation to gravity transitions which captures the role of implicit memory. This advancement enables faster readaptation to familiar gravitational stimuli, which has been observed in repeat flyers, by considering vestibular signals dependent on the new gravity environment, through Bayesian inference. The evolution and weighting of hypotheses considered by the CNS is modeled via a Rao-Blackwellized particle filter algorithm. Sensorimotor adaptation learning is facilitated by retaining a memory of past harmonious states, represented by a conditional state transition probability density function, which allows the model to consider previously experienced gravity levels (while also dynamically learning new states) when formulating new alternative hypotheses of gravity. In order to demonstrate our theoretical framework and motivate future experiments, we perform a variety of simulations. These simulations demonstrate the effectiveness of this model and its potential to advance our understanding of transitory states during which central reinterpretation occurs, ultimately mitigating the risks associated with the lagging time course of adaptation to gravitational environments.

Natural statistics of human head orientation constrain models of vestibular processing

Head orientation relative to gravity determines how gravity-dependent environmental structure is sampled by the visual system, as well as how gravity itself is sampled by the vestibular system. Therefore, both visual and vestibular sensory processing should be shaped by the statistics of head orientation relative to gravity. Here we report the statistics of human head orientation during unconstrained natural activities in humans for the first time, and we explore implications for models of vestibular processing. We find that the distribution of head pitch is more variable than head roll and that the head pitch distribution is asymmetrical with an over-representation of downward head pitch, consistent with ground-looking behavior. We further suggest that pitch and roll distributions can be used as empirical priors in a Bayesian framework to explain previously measured biases in perception of both roll and pitch. Gravitational and inertial acceleration stimulate the otoliths in an equivalent manner, so we also analyze the dynamics of human head orientation to better understand how knowledge of these dynamics can constrain solutions to the problem of gravitoinertial ambiguity. Gravitational acceleration dominates at low frequencies and inertial acceleration dominates at higher frequencies. The change in relative power of gravitational and inertial components as a function of frequency places empirical constraints on dynamic models of vestibular processing, including both frequency segregation and probabilistic internal model accounts. We conclude with a discussion of methodological considerations and scientific and applied domains that will benefit from continued measurement and analysis of natural head movements moving forward.

Objective Evaluation of the Somatogravic Illusion from Flight Data of an Airplane Accident

(1) Background: It is difficult for accident investigators to objectively determine whether spatial disorientation may have contributed to a fatal airplane accident. In this paper, we evaluate three methods to reconstruct the possible occurrence of the somatogravic illusion based on flight data recordings from an airplane accident. (2) Methods: The outputs of two vestibular models were compared with the “standard” method, which uses the unprocessed gravito-inertial acceleration (GIA). (3) Results: All three methods predicted that the changing orientation of the GIA would lead to a somatogravic illusion when no visual references were available. However, the methods were not able to explain the first pitch-down control input by the pilot flying, which may have been triggered by the inadvertent activation of the go-around mode and a corresponding pitch-up moment. Both vestibular models predicted a few seconds delay in the illusory tilt from GIA due to central processing and sensory integration. (4) Conclusions: While it is difficult to determine which method best predicted the somatogravic illusion perceived during the accident without data on the pilot’s pitch perception, both vestibular models go beyond the GIA analysis in taking into account validated vestibular dynamics, and they also account for other vestibular illusions. In that respect, accident investigators would benefit from a unified and validated vestibular model to better explain pilot actions in accidents related to spatial disorientation.

The otolith vermis: A systems neuroscience theory of the Nodulus and Uvula

The Nodulus and Uvula (NU) (lobules X and IX of the cerebellar vermis) form a prominent center of vestibular information processing. Over decades, fundamental and clinical research on the NU has uncovered many aspects of its function. Those include the resolution of a sensory ambiguity inherent to inertial sensors in the inner ear, the otolith organs; the use of gravity signals to sense head rotations; and the differential processing of self-generated and externally imposed head motion. Here, I review these works in the context of a theoretical framework of information processing called the internal model hypothesis. I propose that the NU implements a forward internal model to predict the activation of the otoliths, and outputs sensory predictions errors to correct internal estimates of self-motion or to drive learning. I show that a Kalman filter based on this framework accounts for various functions of the NU, neurophysiological findings, as well as the clinical consequences of NU lesions. This highlights the role of the NU in processing information from the otoliths and supports its denomination as the "otolith" vermis.

COMPASS: Computations for Orientation and Motion Perception in Altered Sensorimotor States

Reliable perception of self-motion and orientation requires the central nervous system (CNS) to adapt to changing environments, stimuli, and sensory organ function. The proposed computations required of neural systems for this adaptation process remain conceptual, limiting our understanding and ability to quantitatively predict adaptation and mitigate any resulting impairment prior to completing adaptation. Here, we have implemented a computational model of the internal calculations involved in the orientation perception system’s adaptation to changes in the magnitude of gravity. In summary, we propose that the CNS considers parallel, alternative hypotheses of the parameter of interest (in this case, the CNS’s internal estimate of the magnitude of gravity) and uses the associated sensory conflict signals (i.e., difference between sensory measurements and the expectation of them) to sequentially update the posterior probability of each hypothesis using Bayes rule. Over time, an updated central estimate of the internal magnitude of gravity emerges from the posterior probability distribution, which is then used to process sensory information and produce perceptions of self-motion and orientation. We have implemented these hypotheses in a computational model and performed various simulations to demonstrate quantitative model predictions of adaptation of the orientation perception system to changes in the magnitude of gravity, similar to those experienced by astronauts during space exploration missions. These model predictions serve as quantitative hypotheses to inspire future experimental assessments.

Individual motion perception parameters and motion sickness frequency sensitivity in fore-aft motion

Previous literature suggests a relationship between individual characteristics of motion perception and the peak frequency of motion sickness sensitivity. Here, we used well-established paradigms to relate motion perception and motion sickness on an individual level. We recruited 23 participants to complete a two-part experiment. In the first part, we determined individual velocity storage time constants from perceived rotation in response to Earth Vertical Axis Rotation (EVAR) and subjective vertical time constants from perceived tilt in response to centrifugation. The cross-over frequency for resolution of the gravito-inertial ambiguity was derived from our data using the Multi Sensory Observer Model (MSOM). In the second part of the experiment, we determined individual motion sickness frequency responses. Participants were exposed to 30-minute sinusoidal fore-aft motions at frequencies of 0.15, 0.2, 0.3, 0.4 and 0.5 Hz, with a peak amplitude of 2 m/s² in five separate sessions, approximately 1 week apart. Sickness responses were recorded using both the MIsery SCale (MISC) with 30 s intervals, and the Motion Sickness Assessment Questionnaire (MSAQ) at the end of the motion exposure. The average velocity storage and subjective vertical time constants were 17.2 s (STD = 6.8 s) and 9.2 s (STD = 7.17 s). The average cross-over frequency was 0.21 Hz (STD = 0.10 Hz). At the group level, there was no significant effect of frequency on motion sickness. However, considerable individual variability was observed in frequency sensitivities, with some participants being particularly sensitive to the lowest frequencies, whereas others were most sensitive to intermediate or higher frequencies. The frequency of peak sensitivity did not correlate with the velocity storage time constant (r = 0.32, p = 0.26) or the subjective vertical time constant (r = − 0.37, p = 0.29). Our prediction of a significant correlation between cross-over frequency and frequency sensitivity was not confirmed (r = 0.26, p = 0.44). However, we did observe a strong positive correlation between the subjective vertical time constant and general motion sickness sensitivity (r = 0.74, p = 0.0006). We conclude that frequency sensitivity is best considered a property unique to the individual. This has important consequences for existing models of motion sickness, which were fitted to group averaged sensitivities. The correlation between the subjective vertical time constant and motion sickness sensitivity supports the importance of verticality perception during exposure to translational sickness stimuli.

Loss of peripheral vestibular input alters the statistics of head movement experienced during natural self-motion

KEY POINTS

Sensory systems are adapted to the statistical structure of natural stimuli, thereby optimizing neural coding. Head motion during natural activities is first sensed and then processed by central vestibulo-motor pathways to influence subsequent behaviour, thereby establishing a feedback loop. To investigate the role of this vestibular feedback on the statistical structure of the head movements, we compared head movements in patients with unilateral vestibular loss and healthy controls. We show that the loss of vestibular feedback substantially alters the statistical structure of head motion for activities that require rapid online feedback control and predict this change by modelling the effects of increased movement variability. Our findings suggest that, following peripheral vestibular loss, changes in the reliability of the sensory input to central pathways impact the statistical structure of head motion during voluntary behaviours.

ABSTRACT

It is widely believed that sensory systems are adapted to optimize neural coding of their natural stimuli. Recent evidence suggests that this is the case for the vestibular system, which senses head movement and contributes to essential functions ranging from the most automatic reflexes to voluntary motor control. During everyday behaviours, head motion is sensed by the vestibular system. In turn, this sensory feedback influences subsequent behaviour, raising the questions of whether and how real-time feedback provided by the vestibular system alters the statistical structure of head movements. We predicted that a reduction in vestibular feedback would alter head movement statistics, particularly for tasks reliant on rapid vestibular feedback. To test this proposal, we recorded six-dimensional head motion in patients with variable degrees of unilateral vestibular loss during standard balance and gait tasks, as well as dynamic self-paced activities. While distributions of linear accelerations and rotational velocities were comparable for patients and age-matched healthy controls, comparison of power spectra revealed significant differences during more dynamic and challenging activities. Specifically, consistent with our prediction, head movement power spectra were significantly altered in patients during two tasks that required rapid online vestibular feedback: active repetitive jumping and walking on foam. Using computational methods, we analysed concurrently measured torso motion and identified increases in head-torso movement variability. Taken together, our results demonstrate that vestibular loss significantly alters head movement statistics and further suggest that increased variability and impaired feedback to internal models required for accurate motor control contribute to the observed changes.

Cathodal Transcranial Direct Current Stimulation Over the Right Temporoparietal Junction Suppresses Its Functional Connectivity and Reduces Contralateral Spatial and Temporal Perception

The temporoparietal junction plays key roles in vestibular function, motor-sensory ability, and attitude stability. Conventional approaches to studying the temporoparietal junction have drawbacks, and previous studies have focused on self-motion rather than on vestibular spatial perception. Using transcranial direct current stimulation, we explored the temporoparietal junction’s effects on vestibular-guided orientation for self-motion and vestibular spatial perception. Twenty participants underwent position, motion, and time tasks, as well as functional magnetic resonance imaging scans. In the position task, cathodal transcranial direct current stimulation yielded a significantly lower response in the −6, −7, −8, −9, −10, −11, and −12 stimulus conditions for leftward rotations (P < 0.05). In the time task, the temporal bias for real transcranial direct current stimulation significantly differed from that for sham stimulation (P < 0.01). Functional magnetic resonance imaging showed that cathodal transcranial direct current stimulation suppressed functional connectivity between the temporoparietal junction, right insular cortex, and right supplementary motor area. Moreover, the change in connectivity between the right temporoparietal junction seed and the right insular cortex was positively correlated with temporal bias under stimulation. The above mentioned results show that cathodal transcranial direct current stimulation induces immediate and extended vestibular effects, which could suppress the functional connectivity of the temporoparietal junction and in turn reduce contralateral spatial and temporal perception. The consistent variation in temporal and spatial bias suggested that the temporoparietal junction may be the cortical temporal integrator for the internal model. Moreover, transcranial direct current stimulation could modulate the integration process and may thus have potential clinical applications in vestibular disorders caused by temporoparietal junction dysfunction.

Proprioception and the predictive sensing of active self-motion

As we actively explore the environment, our motion relative to the world stimulates numerous sensory systems. Notably, proprioceptors provide feedback about body and limb position, while the vestibular system detects and encodes head motion. When the vestibular system is functioning normally, we are unaware of a distinct sensation because vestibular information is integrated with proprioceptive and other sensory inputs to generate our sense of motion. However, patients with vestibular sensory loss experience impairments that provide important insights into the function of this essential sensory system. For these patients, everyday activities such as walking become difficult because even small head movements can produce postural and perceptual instability. This review describes recent research demonstrating how the proprioceptive and vestibular systems effectively work together to provide us with our “6^th sense” during everyday activities, and in particular considers the neural computations underlying the brain’s predictive sensing of head movement during voluntary self-motion.