Effect of eye position during human visual-vestibular integration of heading perception

Psychometrics of inertial heading perception

BACKGROUND

Inertial self-motion perception is thought to depend primarily on otolith cues. Recent evidence demonstrated that vestibular perceptual thresholds (including inertial heading) are adaptable, suggesting novel clinical approaches for treating perceptual impairments resulting from vestibular disease.

OBJECTIVE

Little is known about the psychometric properties of perceptual estimates of inertial heading like test-retest reliability. Here we investigate the psychometric properties of a passive inertial heading perceptual test.

METHODS

Forty-seven healthy subjects participated across two visits, performing in an inertial heading discrimination task. The point of subjective equality (PSE) and thresholds for heading discrimination were identified for the same day and across day tests. Paired t-tests determined if the PSE or thresholds significantly changed and a mixed interclass correlation coefficient (ICC) model examined test-retest reliability. Minimum detectable change (MDC) was calculated for PSE and threshold for heading discrimination.

RESULTS

Within a testing session, the heading discrimination PSE score test-retest reliability was good (ICC = 0. 80) and did not change (t(1,36) = -1.23, p = 0.23). Heading discrimination thresholds were moderately reliable (ICC = 0.67) and also stable (t(1,36) = 0.10, p = 0.92). Across testing sessions, heading direction PSE scores were moderately correlated (ICC = 0.59) and stable (t(1,46) = -0.44, p = 0.66). Heading direction thresholds had poor reliability (ICC = 0.03) and were significantly smaller at the second visit (t(1,46) = 2.8, p = 0.008). MDC for heading direction PSE ranged from 6-9 degrees across tests.

CONCLUSION

The current results indicate moderate reliability for heading perception PSE and provide clinical context for interpreting change in inertial vestibular self-motion perception over time or after an intervention.

Diverse effects of gaze direction on heading perception in humans

Gaze change can misalign spatial reference frames encoding visual and vestibular signals in cortex, which may affect the heading discrimination. Here, by systematically manipulating the eye-in-head and head-on-body positions to change the gaze direction of subjects, the performance of heading discrimination was tested with visual, vestibular, and combined stimuli in a reaction-time task in which the reaction time is under the control of subjects. We found the gaze change induced substantial biases in perceived heading, increased the threshold of discrimination and reaction time of subjects in all stimulus conditions. For the visual stimulus, the gaze effects were induced by changing the eye-in-world position, and the perceived heading was biased in the opposite direction of gaze. In contrast, the vestibular gaze effects were induced by changing the eye-in-head position, and the perceived heading was biased in the same direction of gaze. Although the bias was reduced when the visual and vestibular stimuli were combined, integration of the 2 signals substantially deviated from predictions of an extended diffusion model that accumulates evidence optimally over time and across sensory modalities. These findings reveal diverse gaze effects on the heading discrimination and emphasize that the transformation of spatial reference frames may underlie the effects.

Cortical Mechanisms of Multisensory Linear Self-motion Perception

Accurate self-motion perception, which is critical for organisms to survive, is a process involving multiple sensory cues. The two most powerful cues are visual (optic flow) and vestibular (inertial motion). Psychophysical studies have indicated that humans and nonhuman primates integrate the two cues to improve the estimation of self-motion direction, often in a statistically Bayesian-optimal way. In the last decade, single-unit recordings in awake, behaving animals have provided valuable neurophysiological data with a high spatial and temporal resolution, giving insight into possible neural mechanisms underlying multisensory self-motion perception. Here, we review these findings, along with new evidence from the most recent studies focusing on the temporal dynamics of signals in different modalities. We show that, in light of new data, conventional thoughts about the cortical mechanisms underlying visuo-vestibular integration for linear self-motion are challenged. We propose that different temporal component signals may mediate different functions, a possibility that requires future studies.

Temporal synchrony effects of optic flow and vestibular inputs on multisensory heading perception

Precise heading perception requires integration of optic flow and vestibular cues, yet the two cues often carry distinct temporal dynamics that may confound cue integration benefit. Here, we varied temporal offset between the two sensory inputs while macaques discriminated headings around straight ahead. We find the best heading performance does not occur under natural condition of synchronous inputs with zero offset but rather when visual stimuli are artificially adjusted to lead vestibular by a few hundreds of milliseconds. This amount exactly matches the lag between the vestibular acceleration and visual speed signals as measured from single-unit-activity in frontal and posterior parietal cortices. Manually aligning cues in these areas best facilitates integration with some nonlinear gain modulation effects. These findings are consistent with predictions from a model by which the brain integrates optic flow speed with a faster vestibular acceleration signal for sensing instantaneous heading direction during self-motion in the environment.

Effect of timing delay between visual and vestibular stimuli on heading perception

Heading direction is perceived based on visual and inertial cues. The current study examined the effect of their relative timing on the ability of offset visual headings to influence inertial perception. Seven healthy human subjects experienced 2 s of translation along a heading of 0°, ±35°, ±70°, ±105°, or ±140°. These inertial headings were paired with 2-s duration visual headings that were presented at relative offsets of 0°, ±30°, ±60°, ±90°, or ±120°. The visual stimuli were also presented at 17 temporal delays ranging from -500 ms (visual lead) to 2,000 ms (visual delay) relative to the inertial stimulus. After each stimulus, subjects reported the direction of the inertial stimulus using a dial. The bias of the inertial heading toward the visual heading was robust at ±250 ms when examined across subjects during this period: 8.0° ± 0.5° with a 30° offset, 12.2° ± 0.5° with a 60° offset, 11.7° ± 0.6° with a 90° offset, and 9.8° ± 0.7° with a 120° offset (mean bias toward visual ± SE). The mean bias was much diminished with temporal misalignments of ±500 ms, and there was no longer any visual influence on the inertial heading when the visual stimulus was delayed by 1,000 ms or more. Although the amount of bias varied between subjects, the effect of delay was similar.NEW & NOTEWORTHY The effect of timing on visual-inertial integration on heading perception has not been previously examined. This study finds that visual direction influence inertial heading perception when timing differences are within 250 ms. This suggests visual-inertial stimuli can be integrated over a wider range than reported for visual-auditory integration and may be due to the unique nature of inertial sensation, which can only sense acceleration while the visual system senses position but encodes velocity.

Vestibular Precision at the Level of Perception, Eye Movements, Posture, and Neurons

Precision and accuracy are two fundamental properties of any system, including the nervous system. Reduced precision (i.e., imprecision) results from the presence of neural noise at each level of sensory, motor, and perceptual processing. This review has three objectives: (1) to show the importance of studying vestibular precision, and specifically that studying accuracy without studying precision ignores fundamental aspects of the vestibular system; (2) to synthesize key hypotheses about precision in vestibular perception, the vestibulo-ocular reflex, posture, and neurons; and (3) to show that groups of studies that are thoughts to be distinct (e.g., perceptual thresholds, subjective visual vertical variability, neuronal variability) are actually "two sides of the same coin" - because the methods used allow results to be related to the standard deviation of a Gaussian distribution describing the underlying neural noise. Vestibular precision varies with age, stimulus amplitude, stimulus frequency, body orientation, motion direction, pathology, medication, and electrical/mechanical vestibular stimulation, but does not vary with sex. The brain optimizes precision during integration of vestibular cues with visual, auditory, and/or somatosensory cues. Since a common concern with precision metrics is time required for testing, we describe approaches to optimize data collection and provide evidence that fatigue and session effects are minimal. Finally, we summarize how precision is an individual trait that is correlated with clinical outcomes in patients as well as with performance in functional tasks like balance. These findings highlight the importance of studying vestibular precision and accuracy, and that knowledge gaps remain.

The Effects of Depth Cues and Vestibular Translation Signals on the Rotation Tolerance of Heading Tuning in Macaque Area MSTd

When the eyes rotate during translational self-motion, the focus of expansion (FOE) in optic flow no longer indicates heading, yet heading judgements are largely unbiased. Much emphasis has been placed on the role of extraretinal signals in compensating for the visual consequences of eye rotation. However, recent studies also support a purely visual mechanism of rotation compensation in heading-selective neurons. Computational theories support a visual compensatory strategy but require different visual depth cues. We examined the rotation tolerance of heading tuning in macaque area MSTd using two different virtual environments, a frontoparallel (2D) wall and a 3D cloud of random dots. Both environments contained rotational optic flow cues (i.e., dynamic perspective), but only the 3D cloud stimulus contained local motion parallax cues, which are required by some models. The 3D cloud environment did not enhance the rotation tolerance of heading tuning for individual MSTd neurons, nor the accuracy of heading estimates decoded from population activity, suggesting a key role for dynamic perspective cues. We also added vestibular translation signals to optic flow, to test whether rotation tolerance is enhanced by non-visual cues to heading. We found no benefit of vestibular signals overall, but a modest effect for some neurons with significant vestibular heading tuning. We also find that neurons with more rotation tolerant heading tuning typically are less selective to pure visual rotation cues. Together, our findings help to clarify the types of information that are used to construct heading representations that are tolerant to eye rotations.

Common causation and offset effects in human visual-inertial heading direction integration

Movement direction can be determined from a combination of visual and inertial cues. Visual motion (optic flow) can represent self-motion through a fixed environment or environmental motion relative to an observer. Simultaneous visual and inertial heading cues present the question of whether the cues have a common cause (i.e., should be integrated) or whether they should be considered independent. This was studied in eight healthy human subjects who experienced 12 visual and inertial headings in the horizontal plane divided in 30° increments. The headings were estimated in two unisensory and six multisensory trial blocks. Each unisensory block included 72 stimulus presentations, while each multisensory block included 144 stimulus presentations, including every possible combination of visual and inertial headings in random order. After each multisensory stimulus, subjects reported their perception of visual and inertial headings as congruous (i.e., having common causation) or not. In the multisensory trial blocks, subjects also reported visual or inertial heading direction (3 trial blocks for each). For aligned visual-inertial headings, the rate of common causation was higher during alignment in cardinal than noncardinal directions. When visual and inertial stimuli were separated by 30°, the rate of reported common causation remained >50%, but it decreased to 15% or less for separation of ≥90°. The inertial heading was biased toward the visual heading by 11-20° for separations of 30-120°. Thus there was sensory integration even in conditions without reported common causation. The visual heading was minimally influenced by inertial direction. When trials with common causation perception were compared with those without, inertial heading perception had a stronger bias toward visual stimulus direction.NEW & NOTEWORTHY Optic flow ambiguously represents self-motion or environmental motion. When these are in different directions, it is uncertain whether these are integrated into a common perception or not. This study looks at that issue by determining whether the two modalities are consistent and by measuring their perceived directions to get a degree of influence. The visual stimulus can have significant influence on the inertial stimulus even when they are perceived as inconsistent.

Vision Does Not Necessarily Stabilize the Head in Space During Continuous Postural Perturbations

Vision favors head stabilization in space during perturbations of standing balance. This is particularly obvious under conditions of continuous predictable perturbations as during sinusoidal antero-posterior (A-P) translations of the supporting platform. We tested here the hypothesis that under this condition the head can instead undergo large A-P oscillations, when a precision visual task is concurrently performed. We compared the head oscillations across four conditions while standing on a continuously translating platform. Eyes open (EO, no visual task), EO while reading a text fixed to the moving platform (EO-TP), EO while reading a text fixed to earth-ground (EO-TG), eyes-closed (EC). The platform translated at 0.2 and 0.6 Hz. Participants were young adult subjects, who received no particular instruction except reading the text aloud when required. Markers fixed on head, platform and text-sheet were captured by an optoelectronic device. We found that head oscillations were larger with EC than under all EO conditions. The oscillations were the least with EO and EO-TG, and intermediate with EO-TP. This was true under both low and high translation frequency, in spite of broadly smaller head oscillations at high frequency, common to all visual conditions. The distance between the head and the text was quite constant with EO-TP but fluctuated with EO-TG. The basic whole-body coordination features were moderately similar under all conditions, as assessed by the head-platform correlation coefficients and time lags. It appears that vision does not produce head stabilization in space when a concurrent visual task requiring focusing on a reading-text moving with the platform is performed. Contrary to traditional views centered on the stabilizing effect of vision under both static and dynamic conditions, the results show that head stabilization, normally ensuring a reference for inertial guidance for body balance, can be revoked by the CNS to allow performance of a non-postural task. This novel paradigm can shift long-standing views on the effect of vision on equilibrium control and be considered a potential exercise treatment for enhancing the multisensory integration process in people with balance problems.

Effect of range of heading differences on human visual-inertial heading estimation

Both visual and inertial cues are salient in heading determination. However, optic flow can ambiguously represent self-motion or environmental motion. It is unclear how visual and inertial heading cues are determined to have common cause and integrated vs perceived independently. In four experiments visual and inertial headings were presented simultaneously with ten subjects reporting visual or inertial headings in separate trial blocks. Experiment 1 examined inertial headings within 30° of straight-ahead and visual headings that were offset by up to 60°. Perception of the inertial heading was shifted in the direction of the visual stimulus by as much as 35° by the 60° offset, while perception of the visual stimulus remained largely uninfluenced. Experiment 2 used ± 140° range of inertial headings with up to 120° visual offset. This experiment found variable behavior between subjects with most perceiving the sensory stimuli to be shifted towards an intermediate heading but a few perceiving the headings independently. The visual and inertial headings influenced each other even at the largest offsets. Experiments 3 and 4 had similar inertial headings to experiments 1 and 2, respectively, except subjects reported environmental motion direction. Experiment 4 displayed similar perceptual influences as experiment 2, but in experiment 3 percepts were independent. Results suggested that perception of visual and inertial stimuli tend to be perceived as having common causation in most subjects with offsets up to 90° although with significant variation in perception between individuals. Limiting the range of inertial headings caused the visual heading to dominate the perception.

Vestibular System and Self-Motion

Detection of the state of self-motion, such as the instantaneous heading direction, the traveled trajectory and traveled distance or time, is critical for efficient spatial navigation. Numerous psychophysical studies have indicated that the vestibular system, originating from the otolith and semicircular canals in our inner ears, provides robust signals for different aspects of self-motion perception. In addition, vestibular signals interact with other sensory signals such as visual optic flow to facilitate natural navigation. These behavioral results are consistent with recent findings in neurophysiological studies. In particular, vestibular activity in response to the translation or rotation of the head/body in darkness is revealed in a growing number of cortical regions, many of which are also sensitive to visual motion stimuli. The temporal dynamics of the vestibular activity in the central nervous system can vary widely, ranging from acceleration-dominant to velocity-dominant. Different temporal dynamic signals may be decoded by higher level areas for different functions. For example, the acceleration signals during the translation of body in the horizontal plane may be used by the brain to estimate the heading directions. Although translation and rotation signals arise from independent peripheral organs, that is, otolith and canals, respectively, they frequently converge onto single neurons in the central nervous system including both the brainstem and the cerebral cortex. The convergent neurons typically exhibit stronger responses during a combined curved motion trajectory which may serve as the neural correlate for complex path perception. During spatial navigation, traveled distance or time may be encoded by different population of neurons in multiple regions including hippocampal-entorhinal system, posterior parietal cortex, or frontal cortex.

Effect of vibration during visual-inertial integration on human heading perception during eccentric gaze

Heading direction is determined from visual and inertial cues. Visual headings use retinal coordinates while inertial headings use body coordinates. Thus during eccentric gaze the same heading may be perceived differently by visual and inertial modalities. Stimulus weights depend on the relative reliability of these stimuli, but previous work suggests that the inertial heading may be given more weight than predicted. These experiments only varied the visual stimulus reliability, and it is unclear what occurs with variation in inertial reliability. Five human subjects completed a heading discrimination task using 2s of translation with a peak velocity of 16cm/s. Eye position was ±25° left/right with visual, inertial, or combined motion. The visual motion coherence was 50%. Inertial stimuli included 6 Hz vertical vibration with 0, 0.10, 0.15, or 0.20cm amplitude. Subjects reported perceived heading relative to the midline. With an inertial heading, perception was biased 3.6° towards the gaze direction. Visual headings biased perception 9.6° opposite gaze. The inertial threshold without vibration was 4.8° which increased significantly to 8.8° with vibration but the amplitude of vibration did not influence reliability. With visual-inertial headings, empirical stimulus weights were calculated from the bias and compared with the optimal weight calculated from the threshold. In 2 subjects empirical weights were near optimal while in the remaining 3 subjects the inertial stimuli were weighted greater than optimal predictions. On average the inertial stimulus was weighted greater than predicted. These results indicate multisensory integration may not be a function of stimulus reliability when inertial stimulus reliability is varied.

Distinct spatial coordinate of visual and vestibular heading signals in macaque FEFsem and MSTd

Precise heading estimate requires integration of visual optic flow and vestibular inertial motion originating from distinct spatial coordinates (eye- and head-centered, respectively). To explore whether the two heading signals may share a common reference frame along the hierarchy of cortical stages, we explored two multisensory areas in macaques: the smooth pursuit area of the frontal eye field (FEFsem) closer to the motor side, and the dorsal portion of medial superior temporal area (MSTd) closer to the sensory side. In both areas, vestibular signals are head-centered, whereas visual signals are mainly eye-centered. However, visual signals in FEFsem are more shifted towards the head coordinate compared to MSTd. These results are robust being largely independent on: (1) smooth pursuit eye movement, (2) motion parallax cue, and (3) behavioral context for active heading estimation, indicating that the visual and vestibular heading signals may be represented in distinct spatial coordinate in sensory cortices.