Integration of semicircular canal and otolith information for multisensory orientation stimuli

Tilt perception is different in the pitch and roll planes in human

Neurophysiological tests probing the vestibulo-ocular, colic and spinal pathways are the gold standard to evaluate the vestibular system in clinics. In contrast, vestibular perception is rarely tested despite its potential usefulness in professional training and for the longitudinal follow-up of professionals dealing with complex man-machine interfaces, such as aircraft pilots. This is explored here using a helicopter flight simulator to probe the vestibular perception of pilots. The vestibular perception of nine professional helicopter pilots was tested using a full flight helicopter simulator. The cabin was tilted six times in roll and six times in pitch (-15°, -10°, -5°, 5°, 10° and 15°) while the pilots had no visual cue. The velocities of the outbound displacement of the cabin were kept below the threshold of the semicircular canal perception. After the completion of each movement, the pilots were asked to put the cabin back in the horizontal plane (still without visual cues). The order of the 12 trials was randomized with two additional control trials where the cabin stayed in the horizontal plane but rotated in yaw (-10° and +10°). Pilots were significantly more precise in roll (average error in roll: 1.15 ± 0.67°) than in pitch (average error in pitch: 2.89 ± 1.06°) (Wilcoxon signed-rank test: p < 0.01). However, we did not find a significant difference either between left and right roll tilts (p = 0.51) or between forward and backward pitch tilts (p = 0.59). Furthermore, we found that the accuracies were significantly biased with respect to the initial tilt. The greater the initial tilt was, the less precise the pilots were, although maintaining the direction of the tilt, meaning that the error can be expressed as a vestibular error gain in the ability to perceive the modification in the orientation. This significant result was found in both roll (Friedman test: p < 0.01) and pitch (p < 0.001). However, the pitch trend error was more prominent (gain = 0.77 vs gain = 0.93) than roll. This study is a first step in the determination of the perceptive-motor profile of pilots, which could be of major use for their training and their longitudinal follow-up. A similar protocol may also be useful in clinics to monitor the aging process of the otolith system with a simplified testing device.

Mathematical models for dynamic, multisensory spatial orientation perception

Mathematical models have been proposed for how the brain interprets sensory information to produce estimates of self-orientation and self-motion. This process, spatial orientation perception, requires dynamically integrating multiple sensory modalities, including visual, vestibular, and somatosensory cues. Here, we review the progress in mathematical modeling of spatial orientation perception, focusing on dynamic multisensory models, and the experimental paradigms in which they have been validated. These models are primarily "black box" or "as if" models for how the brain processes spatial orientation cues. Yet, they have been effective scientifically, in making quantitative hypotheses that can be empirically assessed, and operationally, in investigating aircraft pilot disorientation, for example. The primary family of models considered, the observer model, implements estimation theory approaches, hypothesizing that internal models (i.e., neural systems replicating the behavior/dynamics of physical systems) are used to produce expected sensory measurements. Expected signals are then compared to actual sensory afference, yielding sensory conflict, which is weighted to drive central perceptions of gravity, angular velocity, and translation. This approach effectively predicts a wide range of experimental scenarios using a small set of fixed free parameters. We conclude with limitations and applications of existing mathematical models and important areas of future work.

A shared neural integrator for human posture control

Control of standing posture requires fusion of multiple inputs including visual, vestibular, somatosensory, and other sensors, each having distinct dynamics. The semicircular canals, for example, have a unique high-pass filter response to angular velocity, quickly sensing a step change in head rotational velocity followed by a decay. To stabilize gaze direction despite this decay, the central nervous system supplies a neural "velocity storage" integrator, a filter that extends the angular velocity signal. Similar filtering might contribute temporal dynamics to posture control, as suggested by some state estimation models. However, such filtering has not been tested explicitly. We propose that posture control indeed entails a neural integrator for sensory inputs, and we test its behavior with classic sensory perturbations: a rotating optokinetic stimulus to the visual system and a galvanic vestibular stimulus to the vestibular system. A simple model illustrates how these two inputs and body tilt sensors might produce a postural tilt response in the frontal plane. The model integrates these signals through a direct weighted sum of inputs, with or without an indirect pathway containing a neural integrator. Comparison with experimental data from healthy adult subjects (N = 16) reveals that the direct weighting model alone is insufficient to explain resulting postural transients, as measured by lateral tilt of the trunk. In contrast, the neural integrator, shared by sensory signals, produces the dynamics of both optokinetic and galvanic vestibular responses. These results suggest that posture control may involve both direct and indirect pathways, which filter sensory signals and make them compatible for sensory fusion.NEW & NOTEWORTHY Control of standing posture requires fusion of multiple inputs including visual, vestibular, somatosensory, and other sensors, each having distinct dynamics. We propose that postural control also entails a shared neural integrator. To test this theory, we perturbed standing subjects with classic sensory stimuli (optokinetic and galvanic vestibular stimulation) and found that our proposed shared filter reproduces the dynamics of subjects' postural responses.

Optimal estimator models for spatial orientation and vestibular nystagmus

Mathematical models have played an important role in research on the vestibular system over the past century, from the torsion pendulum analogies of the semicircular canal to the optimal estimator "observer" models of multisensory interaction and adaptation. This short review is limited to our own contributions in bringing the technology of feedback control theory to bear on the understanding of human spatial orientation, eye movements, and nystagmus, both on Earth and in space. It points to the importance of the "internal model" concept for treatment of the manner in which the brain constantly makes predictions about future sensory feedback, adjusts the weightings of sensors according to their signal-to-noise ratios, and adapts control according to the motion environment, and availability of sensory cues.

Phase-linking and the perceived motion during off-vertical axis rotation

Human off-vertical axis rotation (OVAR) in the dark typically produces perceived motion about a cone, the amplitude of which changes as a function of frequency. This perception is commonly attributed to the fact that both the OVAR and the conical motion have a gravity vector that rotates about the subject. Little-known, however, is that this rotating-gravity explanation for perceived conical motion is inconsistent with basic observations about self-motion perception: (a) that the perceived vertical moves toward alignment with the gravito-inertial acceleration (GIA) and (b) that perceived translation arises from perceived linear acceleration, as derived from the portion of the GIA not associated with gravity. Mathematically proved in this article is the fact that during OVAR these properties imply mismatched phase of perceived tilt and translation, in contrast to the common perception of matched phases which correspond to conical motion with pivot at the bottom. This result demonstrates that an additional perceptual rule is required to explain perception in OVAR. This study investigates, both analytically and computationally, the phase relationship between tilt and translation at different stimulus rates-slow (45 degrees /s) and fast (180 degrees /s), and the three-dimensional shape of predicted perceived motion, under different sets of hypotheses about self-motion perception. We propose that for human motion perception, there is a phase-linking of tilt and translation movements to construct a perception of one's overall motion path. Alternative hypotheses to achieve the phase match were tested with three-dimensional computational models, comparing the output with published experimental reports. The best fit with experimental data was the hypothesis that the phase of perceived translation was linked to perceived tilt, while the perceived tilt was determined by the GIA. This hypothesis successfully predicted the bottom-pivot cone commonly reported and a reduced sense of tilt during fast OVAR. Similar considerations apply to the hilltop illusion often reported during horizontal linear oscillation. Known response properties of central neurons are consistent with this ability to phase-link translation with tilt. In addition, the competing "standard" model was mathematically proved to be unable to predict the bottom-pivot cone regardless of the values used for parameters in the model.

Vestibular humanoid postural control

Many of our motor activities require stabilization against external disturbances. This especially applies to biped stance since it is inherently unstable. Disturbance compensation is mainly reactive, depending on sensory inputs and real-time sensor fusion. In humans, the vestibular system plays a major role. When there is no visual space reference, vestibular-loss clearly impairs stance stability. Most humanoid robots do not use a vestibular system, but stabilize upright body posture by means of center of pressure (COP) control. We here suggest using in addition a vestibular sensor and present a biologically inspired vestibular sensor along with a human-inspired stance control mechanism. We proceed in two steps. First, in an introductory review part, we report on relevant human sensors and their role in stance control, focusing on own models of transmitter fusion in the vestibular sensor and sensor fusion in stance control. In a second, experimental part, the models are used to construct an artificial vestibular system and to embed it into the stance control of a humanoid. The robot's performance is investigated using tilts of the support surface. The results are compared to those of humans. Functional significance of the vestibular sensor is highlighted by comparing vestibular-able with vestibular-loss states in robot and humans. We show that a kinematic body-space sensory feedback (vestibular) is advantageous over a kinetic one (force cues) for dynamic body-space balancing. Our embodiment of human sensorimotor control principles into a robot is more than just bionics. It inspired our biological work (neurorobotics: 'learning by building', proof of principle, and more). We envisage a future clinical use in the form of hardware-in-the-loop simulations of neurological symptoms for improving diagnosis and therapy and designing medical assistive devices.

An internal model of self-motion based on inertial signals

The question of how the central nervous system can distinguish tilt with respect to gravity from inertial acceleration due to translation in a horizontal plane using vestibular information has long been debated by the scientific community over the past ten years. Recently, it was hypothesized that such discrimination may be based on the multisensory integration of information provided by the otolith organs and the semicircular canals. Some evidence of such processing was found in the neural activity of cells in the fastigial nuclei and vestibular nuclei. To investigate the ability of the central nervous system to build an internal model of self motion based on vestibular signals, we developed an artificial vestibular sensor composed of accelerometers and gyroscopes providing movement data of the same nature as that transduced by the otoliths and canals, respectively. Here we show that the processing of these signals based on the multisensory integration hypothesis can be successfully used to discriminate tilt from translation and that the internal model based on such processing can successfully track angular and linear displacements over short periods of time.

Head tilt-translation combinations distinguished at the level of neurons

Angular and linear accelerations of the head occur throughout everyday life, whether from external forces such as in a vehicle or from volitional head movements. The relative timing of the angular and linear components of motion differs depending on the movement. The inner ear detects the angular and linear components with its semicircular canals and otolith organs, respectively, and secondary neurons in the vestibular nuclei receive input from these vestibular organs. Many secondary neurons receive both angular and linear input. Linear information alone does not distinguish between translational linear acceleration and angular tilt, with its gravity-induced change in the linear acceleration vector. Instead, motions are thought to be distinguished by use of both angular and linear information. However, for combined motions, composed of angular tilt and linear translation, the infinite range of possible relative timing of the angular and linear components gives an infinite set of motions among which to distinguish the various types of movement. The present research focuses on motions consisting of angular tilt and horizontal translation, both sinusoidal, where the relative timing, i.e. phase, of the tilt and translation can take any value in the range -180 degrees to 180 degrees . The results show how hypothetical neurons receiving convergent input can distinguish tilt from translation, and that each of these neurons has a preferred combined motion, to which the neuron responds maximally. Also shown are the values of angular and linear response amplitudes and phases that can cause a neuron to be tilt-only or translation-only. Such neurons turn out to be sufficient for distinguishing between combined motions, with all of the possible relative angular-linear phases. Combinations of other neurons, as well, are shown to distinguish motions. Relative response phases and in-phase firing-rate modulation are the key to identifying specific motions from within this infinite set of combined motions.

Mechanisms of human static spatial orientation

We have developed a tri-axial model of spatial orientation applicable to static 1g and non-1g environments. The model attempts to capture the mechanics of otolith organ transduction of static linear forces and the perceptual computations performed on these sensor signals to yield subjective orientation of the vertical direction relative to the head. Our model differs from other treatments that involve computing the gravitoinertial force (GIF) vector in three independent dimensions. The perceptual component of our model embodies the idea that the central nervous system processes utricular and saccular stimuli as if they were produced by a GIF vector equal to 1g, even when it differs in magnitude, because in the course of evolution living creatures have always experienced gravity as a constant. We determine just two independent angles of head orientation relative to the vertical that are GIF dependent, the third angle being derived from the first two and being GIF independent. Somatosensory stimulation is used to resolve our vestibular model's ambiguity of the up-down directions. Our otolith mechanical model takes into account recently established non-linear behavior of the force-displacement relationship of the otoconia, and possible otoconial deflections that are not co-linear with the direction of the input force (cross-talk). The free parameters of our model relate entirely to the mechanical otolith model. They were determined by fitting the integrated mechanical/perceptual model to subjective indications of the vertical obtained during pitch and roll body tilts in 1g and 2g force backgrounds and during recumbent yaw tilts in 1g. The complete data set was fit with very little residual error. A novel prediction of the model is that background force magnitude either lower or higher than 1g will not affect subjective vertical judgments during recumbent yaw tilt. These predictions have been confirmed in recent parabolic flight experiments.

An internal model of head kinematics predicts the influence of head orientation on reflexive eye movements

Our sense of self-motion and self-orientation results from combining information from different sources. We hypothesize that the central nervous system (CNS) uses internal models of the laws of physics to merge cues provided by different sensory systems. Different models that include internal models have been proposed; we focus herein on that referred to as the sensory weighting model. For simplicity, we isolate the portion of the sensory weighting model that estimates head angular velocity: it includes an inverse internal model of head kinematics and an 'idiotropic' vector aligned with the main body axis. Following a post-rotatory tilt in the dark, which is a rapid tilt following a constant-velocity rotation about an earth-vertical axis, the inverse internal model is applied to conflicting vestibular signals. Consequently, the CNS computes an inaccurate estimate of head angular velocity that shifts toward alignment with an estimate of gravity. Since reflexive eye movements known as vestibulo-ocular reflexes (VOR) compensate for this estimate of head angular velocity, the model predicts that the VOR rotation axis shifts toward alignment with this estimate of gravity and that the VOR time constant depends on final head orientation. These predictions are consistent with experimental data.

Neural processing of gravitoinertial cues in humans. III. Modeling tilt and translation responses

All linear accelerometers measure gravitoinertial force, which is the sum of gravitational force (tilt) and inertial force due to linear acceleration (translation). Neural strategies must exist to elicit tilt and translation responses from this ambiguous cue. To investigate these neural processes, we developed a model of human responses and simulated a number of motion paradigms used to investigate this tilt/translation ambiguity. In this model, the separation of GIF into neural estimates of gravity and linear acceleration is accomplished via an internal model made up of three principal components: 1) the influence of rotational cues (e.g., semicircular canals) on the neural representation of gravity, 2) the resolution of gravitoinertial force into neural representations of gravity and linear acceleration, and 3) the neural representation of the dynamics of the semicircular canals. By combining these simple hypotheses within the internal model framework, the model mimics human responses to a number of different paradigms, ranging from simple paradigms, like roll tilt, to complex paradigms, like postrotational tilt and centrifugation. It is important to note that the exact same mechanisms can explain responses induced by simple movements as well as by more complex paradigms; no additional elements or hypotheses are needed to match the data obtained during more complex paradigms. Therefore these modeled response characteristics are consistent with available data and with the hypothesis that the nervous system uses internal models to estimate tilt and translation in the presence of ambiguous sensory cues.

Replication of passive whole-body linear displacements from inertial cues. Facts and mechanisms

Using path integration, normal subjects should be able to compute the distance of a traveled path even from the sole inertial sensory input. Blindfolded subjects were submitted to a passive linear forward displacement along 2 to 10 m. Their task was to replicate the traveled distance, still blindfolded, by driving the vehicle they were seated upon using a joystick that controlled linear speed. Subjects replicated both the length and the velocity profile of the passive travel, suggesting that a dynamic record of experienced motion is stored in memory. Even when the replication of passive motion dynamics was made impossible, the subjects could still replicate the displacement. The results are explained by a dynamic feedback model that performs a running comparison between the perceived instantaneous displacement of the ongoing motion and the displacement derived from a spatiotemporal record of perceived passive motion. A multimodal acceleration-related sensory input is transformed into a displacement-related perception through double time-integration.

Humans use internal models to estimate gravity and linear acceleration

Because sensory systems often provide ambiguous information, neural processes must exist to resolve these ambiguities. It is likely that similar neural processes are used by different sensory systems. For example, many tasks require neural processing to distinguish linear acceleration from gravity, but Einstein's equivalence principle states that all linear accelerometers must measure both linear acceleration and gravity. Here we investigate whether the brain uses internal models, defined as neural systems that mimic physical principles, to help estimate linear acceleration and gravity. Internal models may be used in motor contro, sensorimotor integration and sensory processing, but direct experimental evidence for such models is limited. To determine how humans process ambiguous gravity and linear acceleration cues, subjects were tilted after being rotated at a constant velocity about an Earth-vertical axis. We show that the eye movements evoked by this post-rotational tilt include a response component that compensates for the estimated linear acceleration even when no actual linear acceleration occurs. These measured responses are consistent with our internal model predictions that the nervous system can develop a non-zero estimate of linear acceleration even when no true linear acceleration is present.

Spatial memory and path integration studied by self-driven passive linear displacement. I. Basic properties

According to path integration, the brain is able to compute the distance of a traveled path. In this research we applied our previously reported method for studying memory of linear distance, a crucial mechanism in path integration; our method is based on the overt reconstruction of a passive transport. Passive transport is a special case of navigation in which no active control is performed. Blindfolded subjects were first asked to travel 2 m forward, in darkness, by driving with a joystick the robot on which they were seated. The results show that all subjects but two undershot this distance, i.e., overestimated their own displacement. Then, subjects were submitted to a passive linear forward displacement along 2, 4, 6, 8, or 10 m, and had to reproduce the same distance, still blindfolded. The results show that the distance of the stimulus was accurately reproduced, as well as stimulus duration, peak velocity, and velocity profile. In this first condition, the imposed velocity profile was triangular and therefore stimulus distance and duration were correlated. In a second condition, it was shown that distance was correctly reproduced also when the information about stimulus duration was kept constant. Here, different velocity profiles were used as stimuli, and most subjects also reproduced the velocity profile. Statistical analyses indicated that distance was not reproduced as a consequence of duration, peak velocity, or velocity profile reproduction, but was uniquely correlated to stimulus distance. The previous hypothesis of a double integration of the otolith signal to provide a distance estimate can explain our results. There was a large discrepancy between the accuracy with which the subjects matched the velocity profiles and that of distance reproduction. It follows that, whereas the dynamics of passive motion are stored and available to further use, distance is independently estimated. It is concluded that vestibular and somatosensory signals excited by passive transport can be used to build a dynamic as well as a static representation of the traveled path. We found a close quantitative similarity between the present findings on distance reproduction and those obtained from active locomotion experiments in which the same paradigm was used. This resemblance suggests that the two types of navigation tasks draw on common physiological processes and extends the relevance of our results to naturally occurring path integration.

Motion sickness during off-vertical axis rotation: prediction by a model of sensory interactions and correlation with other forms of motion sickness

Motion sickness (MS) susceptibility of 108 normal subjects was measured during off-vertical axis rotation (OVAR) as a function of angular velocity (60-180 degrees/s). The chair rotated about a longitudinal axis tilted 30 degrees with respect to gravity. For each velocity, we measured the duration of exposure necessary to evoke a moderate malaise, with a limit of 30 min. MS appeared the fastest at a rotation velocity of 105 degrees/s; higher or lower velocities were less provocative. These results are in good agreement with predictions made by Zupan et al. [in ICANN'94, Springer-Verlag, 1995] by means of a MS mathematical model derived from a model of sensory interactions [Droulez and Darlot, in Attention and Performance, Vol. 13, Lawrence Erlbaum, Hillsdale, 1989]. We also found that MS susceptibility during OVAR is positively correlated with susceptibility to other forms of MS. Since OVAR induces sensory messages very different from those induced by other provocative stimulations, this could suggest that the sensitivity of a common final vegetative locus is an important factor of the individual differences in susceptibility to MS.

An optimal control model for analyzing human postural balance

The question posed in this study is whether optimal control and state estimation can explain selection of control strategies used by humans, in response to small perturbations to stable upright balance. To answer this question, a human sensorimotor control model, compatible with previous work by others, was assembled. This model incorporates linearized equations and full-state feedback with provision for state estimation. A form of gain-scheduling is employed to account for nonlinearities caused by control and biomechanical constraints. By decoupling the mechanics and transforming the controls into the space of experimentally observed strategies, the model is made amenable to the study of a number of possible control objectives. The objectives studied include cost functions on the state deviations, so as to control the center of mass, provide a stable platform for the head, or maintain upright stance, along with a cost function on control effort. Also studied was the effect of time delay on the stability of controls produced using various control strategies. An objective function weighting excursion of the center of mass and deviations from the upright stable position, while taking advantage of fast modes of the system, as dictated by inertial parameters and musculoskeletal geometry, produces a control that reasonably matches experimental data. Given estimates of sensor performance, the model is also suited for prediction of uncertainty in the response.

Fluid kinetics of endolymph during calorization

Various theories on the mechanism of caloric vestibular excitability are discussed and compared with our model, which involves two factors of endolymph movement: thermoconvection and fluid expansion. With this theory, caloric nystagmus in microgravity can be explained, as well as certain phenomena on earth, such as different vestibular responses depending on body position.

How to explain a constant subjective vertical at constant high speed rotation about an earth-horizontal axis

When rotated in darkness about an earth-horizontal axis at speeds above 0.2-0.5 Hz, subjects, instead of feeling rotated, experience a constant (though extrapersonally diverse) position in space and a constant visual vertical (SV). Computer simulation shows that this phenomenon cannot be explained by the extant models of Mayne (1) and Ormsby (2) about the interaction of otoliths and semicircular canals. It follows, however, from a static theory of the SV (3) if, as in the presently proposed dynamic model, the otolith afference is processed by a low-pass filter. At high speed rotation this filter can only be passed by the force-independent, temporally invariant components of the otolith information. Such force-independent components are bound to result from biassed resting discharges, and have previously been shown to affect the SV and the self-adopted horizontal position. The interaction of otoliths and canals proposed by the model does provide a veridical vertical in a working range of angular frequencies and hence a basis for inertial navigation.

Optimal estimator model for human spatial orientation

A model is presented to predict human dynamic spatial orientation in response to multisensory stimuli. Motion stimuli are first processed by dynamic models of the visual, vestibular, tactile, and proprioceptive sensors. Central nervous system function is modeled as a steady state Kalman filter that optimally blends information from the various sensors to form an estimate of spatial orientation. Where necessary, nonlinear elements preprocess inputs to the linear central estimator in order to reflect more accurately some nonlinear human response characteristics. Computer implementation of the model has shown agreement with several important qualitative characteristics of human spatial orientation.

The caloric vestibular reaction in space. Physiological considerations

Caloric stimulus testing was performed as part of the vestibular research program during the European Spacelab 1 mission in Nov/Dec 1983. Contrary to prediction according to the classical endolymph flow theory originally forwarded by Bárány, caloric nystagmus was elicited in both tested astronauts. The intensity of the response was found comparable to that measured on earth. The theoretical consequences of these findings are discussed and possible mechanisms are considered. The direct volume displacement hypothesis is favoured as the primary effect responsible for the observed vestibulo-ocular nystagmus. Estimated differential pressure conditions resulting in the endolymph canal support this hypothesis and are in agreement with the observed response intensity. It is further speculated that interaction in the central vestibular system between canal and otolith signals be responsible for the well-known body position modulation of the observed nystagmus.

The effect of a moving foveal target on the subjective sensation of motion

In this paper, we report on two experiments concerning the effect of the visual field of fovea on the subjective estimation of angular velocity. Experiment 1 investigates the effect of a slow moving target on the perception of self motion. The result of this experiment can be summarized as follows: a slow moving target seen in the visual field of fovea by a stationary person generates in this person a sensation of self rotation in the same direction as the motion of the target. This phenomenon will be called foveal induced ego motion. Experiment 2 investigates the latency for the detection of a self angular acceleration when the person focusses his fovea on a slowly moving target. From the results of this experiment we conclude that the latency for detection of a small self angular acceleration is shorter if the person sees a small foveal target moving with respect to the person in the direction of self rotation than if that small foveal target is moving (with respect to the person) in the opposite direction. The results of these experiments help us in refining existing models of visual-vestibular interaction, by providing a model which accounts for the phenomenon of oculogyral illusion.

A method for evaluating vestibular control of posture

Tests that measure the patient's ability to control his posture generally have failed to yield clinically useful information about vestibular function. The primary reason for this failure is that postural control does not depend on vestibular function when visual and proprioceptive cues are available. To test vestibular function, one should use a procedure that forces the patient to rely on vestibular cues by eliminating cues from these other sensory systems. Such a procedure is described. Vision is eliminated by eye closure, and proprioceptive cues are minimized by rotating the supporting surface to null changes in ankle angle. Data are presented which show that postural responses are complex, but that they appear to conform to the pattern originally observed by Nashner. A method potentially capable of separately evaluating semicircular canal and otolith function is described.