Perceived timing of vestibular stimulation relative to touch, light and sound

Learning capabilities to resolve tilt-translation ambiguity in goldfish

Introduction

Spatial orientation refers to the perception of relative location and self-motion in space. The accurate formation of spatial orientation is essential for animals to survive and interact safely with their environment. The formation of spatial orientation involves the integration of sensory inputs from the vestibular, visual, and proprioceptive systems. Vestibular organs function as specialized head motion sensors, providing information regarding angular velocity and linear acceleration via the semicircular canals and otoliths, respectively. However, because forces arising from the linear acceleration (translation) and inclination relative to the gravitational axis (tilt) are equivalent, they are indistinguishable by accelerometers, including otoliths. This is commonly referred to as the tilt - translation ambiguity, which can occasionally lead to the misinterpretation of translation as a tilt. The major theoretical frameworks addressing this issue have proposed that the interpretation of tilt versus translation may be contingent on an animal's previous experiences of motion. However, empirical confirmation of this hypothesis is lacking.

Methods

In this study, we conducted a behavioral experiment using goldfish to investigate how an animal's motion experience influences its interpretation of tilt vs. translation. We examined a reflexive eye movement called the vestibulo-ocular reflex (VOR), which compensatory-rotates the eyes in response to head motion and is known to reflect an animal's three-dimensional head motion estimate.

Results

We demonstrated that the VORs of naïve goldfish do not differentiate between translation and tilt at 0.5 Hz. However, following prolonged visual-translation training, which provided appropriate visual stimulation in conjunction with translational head motion, the VORs were capable of distinguishing between the two types of head motion within 3 h. These results were replicated using the Kalman filter model of spatial orientation, which incorporated the variable variance of process noise corresponding to the accumulated motion experience.

Discussion

Based on these experimental and computational findings, we discuss the neural mechanism underlying the resolution of tilt-translation ambiguity within a context analogous to, yet distinct from, previous cross-axis VOR adaptations.

Practices for respecting the newborn's sleep-wake cycle: Interventional study in the neonatal intensive care unit

Objectives

Premature newborns are exposed to a great deal of over-stimulation, which can affect their cerebral development. For better sleep, certain practices should be recommended. The aim of this study was to evaluate the effect of a professional training program on the improvement of practices promoting respect for the newborn's sleep-wake cycle.

Materials and Methods

This was an interventional study with a longitudinal, single-group, and before-and-after design. The experimental design followed a three-stage time series: Eight months before, three months after, then eight months after intervention. It targeted a comprehensive sample of 66 professionals. It took place between October 2020 and March 2022 at the Neonatal Intensive Care Unit of the Hassan II University Hospital in Fez, Morocco. It was based on an observation grid and a self-administered questionnaire, validated and tested with a Cronbach's alpha reliability of 0.91.

Results

The light environment showed significant differences between the 1st and 3rd step (3.3% vs. 45.0%; P = 0.02; confidence interval [CI] = 13.644-10.456) for knowledge and (13.3% vs. 78.3%; P = 0.01; CI = 14.412-10.888) for practices; the noise environment showed a positive improvement between the 2nd and 3rd intervention (31.7% vs. 41.7%; P < 0.001; CI = 5.954-2.913) for knowledge and (65.0% vs. 73.3%; P < 0.001; CI = 3.597-1.236) for practices, with an average of (8.98 ± 0.30-28.15 ± 0.48; CI = 3.806-1.094) between the 1st and 3rd step. Practices surrounding sleep and wakefulness reported significant improvement between the three periods (14.35 ± 0.22 vs. 18.10 ± 0.35 vs. 19.90 ± 0.35; P P < 0.001; CI = 4.647-2.853) for sleep and (13.25 ± 0.48 vs. 22.27 ± 0.59; P < 0.001; CI = 10.563-7.471) for wakefulness with statistically significant correlations between knowledge and practices (0.426**) for sleep and (0.606**) for wakefulness.

Conclusion

The study demonstrated the positive impact of this sleep management and assessment program on the development of professional skills. Its implementation requires rigorous application of developmental support strategies for individualized care in neonatology.

Perceived timing of postural instability onset

BACKGROUND

This study investigates the perceived onset of postural instability, a critical aspect of balance. Prior research using Temporal Order Judgment (TOJ) tasks revealed that postural perturbations must occur significantly earlier than an auditory reference stimulus for individuals to perceive them as simultaneous. However, there are methodological concerns with this previous work, particularly an unbalanced stimulus onset asynchrony (SOA) distribution.

RESEARCH QUESTION

Does the point of subjective simultaneity (PSS) between postural perturbation onset and an auditory reference stimulus differ between SOA distributions unequally (distribution 1) and equally (distribution 2) distributed around true simultaneity (0 ms)?

METHODS

A repeated measures design was employed, presenting two different SOA distributions to 10 participants using a TOJ task during both distribution 1 (88 trials) and distribution 2 (72 trials) SOA distributions. Paired t-tests were used to determine if there was a significant difference between the PSS of distribution 1 and 2. One-sample t-tests were also performed on the PSS values of both conditions in comparison to 0 ms (defined as true simultaneity) to determine if perceptual responses were delayed.

RESULTS

Distribution 1 led to a perceived delay of postural instability onset by 20.34 ms, while distribution 2 resulted in a perceived delay of the auditory stimulus of 3.52 ms. However, neither condition was significantly different from each other nor from true simultaneity.

SIGNIFICANCE

These findings suggest that the perception of postural instability onset is not slow, contrary to previous beliefs, and emphasize the importance of controlling methodological parameters when examining sensory cues. This understanding will help inform falls prevention strategies.

Stimulus Onset Asynchrony Affects Weighting-related Event-related Spectral Power in Self-motion Perception

Self-motion perception relies primarily on the integration of the visual, vestibular, proprioceptive, and somatosensory systems. There is a gap in understanding how a temporal lag between visual and vestibular motion cues affects visual-vestibular weighting during self-motion perception. The beta band is an index of visual-vestibular weighting, in that robust beta event-related synchronization (ERS) is associated with visual weighting bias, and robust beta event-related desynchronization is associated with vestibular weighting bias. The present study examined modulation of event-related spectral power during a heading judgment task in which participants attended to either visual (optic flow) or physical (inertial cues stimulating the vestibular, proprioceptive and somatosensory systems) motion cues from a motion simulator mounted on a MOOG Stewart Platform. The temporal lag between the onset of visual and physical motion cues was manipulated to produce three lag conditions: simultaneous onset, visual before physical motion onset, and physical before visual motion onset. There were two main findings. First, we demonstrated that when the attended motion cue was presented before an ignored cue, the power of beta associated with the attended modality was greater than when visual-vestibular cues were presented simultaneously or when the ignored cue was presented first. This was the case for beta ERS when the visual-motion cue was attended to, and beta event-related desynchronization when the physical-motion cue was attended to. Second, we tested whether the power of feature-binding gamma ERS (demonstrated in audiovisual and visual-tactile integration studies) increased when the visual-vestibular cues were presented simultaneously versus with temporal asynchrony. We did not observe an increase in gamma ERS when cues were presented simultaneously, suggesting that electrophysiological markers of visual-vestibular binding differ from markers of audiovisual and visual-tactile integration. All event-related spectral power reported in this study were generated from dipoles projecting from the left and right motor areas, based on the results of Measure Projection Analysis.

Earth-vertical motion perception assessment using an elevator: a feasibility study

A feasible, inexpensive, rapid, and easy-to-use method to measure vestibular vertical movement perception is needed to assess the sacculus-mediated low-frequency otolith function of dizzy patients. To evaluate the feasibility of reaction time assessment in response to vertical motion induced by an elevator in healthy young individuals. We recorded linear acceleration/deceleration reaction times (LA-RT/LD-RT) of 20 healthy (13 female) subjects (mean age: 22 years ± 1 SD) as a measure of vertical vestibular motion perception. LA-RT/LD-RT were defined as the time elapsed from the start of elevator acceleration or deceleration to the time at which subjects in a sitting position indicated perceiving a change in velocity by pushing a button with their thumb. The light reaction time was measured as a reference. All 20 subjects tolerated the assessment with repeated elevator rides and reported no adverse events. Over all experiments, one upward and four downward rides had to be excluded for technical reasons (2.5%). The fraction of premature button presses varied among the four conditions, possibly related to elevator vibration (upward rides: LA-RT-up 66%, LD-RT-up 0%; downward rides: LA-RT-down 12%, LD-RT-down 4%). Thus LD-RT-up yielded the most robust results. The reaction time to earth-vertical deceleration elicited by an elevator provides a consistent indicator of linear vestibular motion perception in healthy humans. The testing procedure is inexpensive and easy to use. Deceleration on upward rides yielded the most robust measurements.

Delay and Acceleration Threshold of Movement Perception in Patients Suffering from Vertigo or Dizziness

Background: The objective was to evaluate the delay and the acceleration threshold (AT) of movement perception in a population of patients suffering from dizziness and analyze the factors influencing these parameters. Methods: This prospective study included 256 adult subjects: 16 control and 240 patients (5 acute unilateral vestibular loss, 13 compensated unilateral loss, 32 Meniere diseases, 48 persistent postural-perceptual dizziness (PPPD), 95 benign paroxysmal positional vertigo (BPPV), 10 central cases, 19 bilateral vestibulopathy, 14 vestibular migraine, and 4 age-related dizziness). Patients were evaluated for the sound–movement synchronicity perception (maximum delay between the bed oscillation peak and a beep perceived as synchronous, PST) and AT during a pendular movement on a swinging bed. Results: We observed higher PST in women and in senior patients regardless of etiology. AT was higher in senior patients. AT was not influenced by etiology except in patients with bilateral vestibulopathy who had higher thresholds. AT was related to unipodal stance performance, past history of fall, and stop-walking-when-talking test. Conclusions: Delay and acceleration thresholds appear to be coherent with clinical findings and open insights on the exploration of symptoms that cannot be explained by routine otoneurological tests.

The timing and amplitude of the muscular activity of the arms preceding impact in a forward fall is modulated with fall velocity

Protective arm reactions have been shown to be an important injury avoidance mechanism in unavoidable falls. Protective arm reactions have been shown to be modulated with fall height, however it is not clear if they are modulated with impact velocity. The aim of this study was to determine if protective arm reactions are modulated in response to a forward fall with an initially unpredictable impact velocity. Forward falls were evoked via sudden release of a standing pendulum support frame with adjustable counterweight to control fall acceleration and impact velocity. Thirteen younger adults (1 female) participated in this study. Counterweight load explained more than 89% of the variation of impact velocity. Angular velocity at impact decreased (p < 0.001), drop duration increased from 601 ms to 816 ms (p < 0.001), and the maximum vertical ground reaction force decreased from 64%BW to 46%BW (p < 0.001) between the small and large counterweight. Elbow angle at impact (129 degrees extension), triceps (119 ms) and biceps (98 ms) pre-impact time, and co-activation (57%) were not significantly affected by counterweight load (p-values > 0.08). Average triceps and biceps EMG amplitude decreased from 0.26 V/V to 0.19 V/V (p = 0.004) and 0.24 V/V to 0.11 V/V (p = 0.002) with increasing counterweight respectively. Protective arm reactions were modulated with fall velocity by reducing EMG amplitude with decreasing impact velocity. This demonstrates a neuromotor control strategy for managing evolving fall conditions. Future work is needed to further understand how the CNS deals with additional unpredictability (e.g., fall direction, perturbation magnitude, etc.) when deploying protective arm reactions.

Can the Perceived Timing of Multisensory Events Predict Cybersickness?

Humans are constantly presented with rich sensory information that the central nervous system (CNS) must process to form a coherent perception of the self and its relation to its surroundings. While the CNS is efficient in processing multisensory information in natural environments, virtual reality (VR) poses challenges of temporal discrepancies that the CNS must solve. These temporal discrepancies between information from different sensory modalities leads to inconsistencies in perception of the virtual environment which often causes cybersickness. Here, we investigate whether individual differences in the perceived relative timing of sensory events, specifically parameters of temporal-order judgement (TOJ), can predict cybersickness. Study 1 examined audiovisual (AV) TOJs while Study 2 examined audio-active head movement (AAHM) TOJs. We deduced metrics of the temporal binding window (TBW) and point of subjective simultaneity (PSS) for a total of 50 participants. Cybersickness was quantified using the Simulator Sickness Questionnaire (SSQ). Study 1 results (correlations and multiple regression) show that the oculomotor SSQ shares a significant yet positive correlation with AV PSS and TBW. While there is a positive correlation between the total SSQ scores and the TBW and PSS, these correlations are not significant. Therefore, although these results are promising, we did not find the same effect for AAHM TBW and PSS. We conclude that AV TOJ may serve as a potential tool to predict cybersickness in VR. Such findings will generate a better understanding of cybersickness which can be used for development of VR to help mitigate discomfort and maximize adoption.

Influence of Sensory Conflict on Perceived Timing of Passive Rotation in Virtual Reality

Integration of incoming sensory signals from multiple modalities is central in the determination of self-motion perception. With the emergence of consumer virtual reality (VR), it is becoming increasingly common to experience a mismatch in sensory feedback regarding motion when using immersive displays. In this study, we explored whether introducing various discrepancies between the vestibular and visual motion would influence the perceived timing of self-motion. Participants performed a series of temporal-order judgements between an auditory tone and a passive whole-body rotation on a motion platform accompanied by visual feedback using a virtual environment generated through a head-mounted display. Sensory conflict was induced by altering the speed and direction by which the movement of the visual scene updated relative to the observer's physical rotation. There were no differences in perceived timing of the rotation without vision, with congruent visual feedback and when the speed of the updating of the visual motion was slower. However, the perceived timing was significantly further from zero when the direction of the visual motion was incongruent with the rotation. These findings demonstrate the potential interaction between visual and vestibular signals in the temporal perception of self-motion. Additionally, we recorded cybersickness ratings and found that sickness severity was significantly greater when visual motion was present and incongruent with the physical motion. This supports previous research regarding cybersickness and the sensory conflict theory, where a mismatch between the visual and vestibular signals may lead to a greater likelihood for the occurrence of sickness symptoms.

Crash Prediction Using Deep Learning in a Disorienting Spaceflight Analog Balancing Task

Were astronauts forced to land on the surface of Mars using manual control of their vehicle, they would not have familiar gravitational cues because Mars’ gravity is only 0.38 g. They could become susceptible to spatial disorientation, potentially causing mission ending crashes. In our earlier studies, we secured blindfolded participants into a Multi-Axis Rotation System (MARS) device that was programmed to behave like an inverted pendulum. Participants used a joystick to stabilize around the balance point. We created a spaceflight analog condition by having participants dynamically balance in the horizontal roll plane, where they did not tilt relative to the gravitational vertical and therefore could not use gravitational cues to determine their position. We found 90% of participants in our spaceflight analog condition reported spatial disorientation and all of them showed it in their data. There was a high rate of crashing into boundaries that were set at ± 60° from the balance point. Our goal was to see whether we could use deep learning to predict the occurrence of crashes before they happened. We used stacked gated recurrent units (GRU) to predict crash events 800 ms in advance with an AUC (area under the curve) value of 99%. When we prioritized reducing false negatives we found it resulted in more false positives. We found that false negatives occurred when participants made destabilizing joystick deflections that rapidly moved the MARS away from the balance point. These unpredictable destabilizing joystick deflections, which occurred in the duration of time after the input data, are likely a result of spatial disorientation. If our model could work in real time, we calculated that immediate human action would result in the prevention of 80.7% of crashes, however, if we accounted for human reaction times (∼400 ms), only 30.3% of crashes could be prevented, suggesting that one solution could be an AI taking temporary control of the spacecraft during these moments.

A cerebellar internal model calibrates a feedback controller involved in sensorimotor control

Animals must adapt their behavior to survive in a changing environment. Behavioral adaptations can be evoked by two mechanisms: feedback control and internal-model-based control. Feedback controllers can maintain the sensory state of the animal at a desired level under different environmental conditions. In contrast, internal models learn the relationship between the motor output and its sensory consequences and can be used to recalibrate behaviors. Here, we present multiple unpredictable perturbations in visual feedback to larval zebrafish performing the optomotor response and show that they react to these perturbations through a feedback control mechanism. In contrast, if a perturbation is long-lasting, fish adapt their behavior by updating a cerebellum-dependent internal model. We use modelling and functional imaging to show that the neuronal requirements for these mechanisms are met in the larval zebrafish brain. Our results illustrate the role of the cerebellum in encoding internal models and how these can calibrate neuronal circuits involved in reactive behaviors depending on the interactions between animal and environment.

Multisensory coding of angular head velocity in the retrosplenial cortex

To successfully navigate the environment, animals depend on their ability to continuously track their heading direction and speed. Neurons that encode angular head velocity (AHV) are fundamental to this process, yet the contribution of various motion signals to AHV coding in the cortex remains elusive. By performing chronic single-unit recordings in the retrosplenial cortex (RSP) of the mouse and tracking the activity of individual AHV cells between freely moving and head-restrained conditions, we find that vestibular inputs dominate AHV signaling. Moreover, the addition of visual inputs onto these neurons increases the gain and signal-to-noise ratio of their tuning during active exploration. Psychophysical experiments and neural decoding further reveal that vestibular-visual integration increases the perceptual accuracy of angular self-motion and the fidelity of its representation by RSP ensembles. We conclude that while cortical AHV coding requires vestibular input, where possible, it also uses vision to optimize heading estimation during navigation.

•

Angular head velocity (AHV) coding is widespread in the retrosplenial cortex (RSP)

•

AHV cells maintain their tuning during passive motion and require vestibular input

•

The perception of angular self-motion is improved when visual cues are present

•

AHV coding is similarly improved when both vestibular and visual stimuli are used

Olfactory Virtual Reality: A New Frontier in the Treatment and Prevention of Posttraumatic Stress Disorder

This perspective piece reviews the clinical condition of posttraumatic stress disorder (PTSD), which is currently increasing due to the COVID-19 pandemic, and recent research illustrating how olfaction is being incorporated into virtual reality (VR) platforms. I then discuss the latest work examining the potential of olfactory virtual reality (OVR) for the treatment of PTSD. From this foundation I suggest novel ways in which OVR may be implemented in PTSD therapy and harnessed for preventing the development of PTSD. Perceptual and chemical features of olfaction that should be considered in OVR applications are also discussed.

Measuring threshold and latency of motion perception on a swinging bed

INTRODUCTION

Our objective was to develop and to evaluate a system to measure latency and threshold of pendular motion perception based on a swinging bed.

MATERIALS AND METHODS

This prospective study included 30 healthy adults (age: 32 ± 12 years). All subjects were tested twice with a 10 min. interval. A second trial was conducted 2 to 15 days after. A rehabilitation swinging bed was connected to an electronic device emitting a beep at the beginning of each oscillation phase with an adjustable time lag. Subjects were blindfolded and auditory cues other than the beep were minimized. The acceleration threshold was measured by letting the bed oscillate freely until a natural break and asking the patient when he did not perceive any motion. The perception latency was determined by asking the patient to indicate whether the beep and the peak of each oscillation were synchronous. The time lag between sound and peak of the head position was swept from -750 to +750 ms by 50 ms increments.

RESULTS

The mean acceleration threshold was 9.2±4.60 cm/s2. The range width of the synchronous perception interval was estimated as 535±190 ms. The point of subjective synchronicity defined as the center of this interval was -195±106 ms (n = 30). The test-retest evaluation in the same trial showed an acceptable reproducibility for the acceleration threshold and good to excellent for all parameters related to sound-movement latency.

CONCLUSION

Swinging bed combined to sound stimulation can provide reproducible information on movement perception in a simple and non-invasive manner with highly reproducible results.

Effect of timing delay between visual and vestibular stimuli on heading perception

Heading direction is perceived based on visual and inertial cues. The current study examined the effect of their relative timing on the ability of offset visual headings to influence inertial perception. Seven healthy human subjects experienced 2 s of translation along a heading of 0°, ±35°, ±70°, ±105°, or ±140°. These inertial headings were paired with 2-s duration visual headings that were presented at relative offsets of 0°, ±30°, ±60°, ±90°, or ±120°. The visual stimuli were also presented at 17 temporal delays ranging from -500 ms (visual lead) to 2,000 ms (visual delay) relative to the inertial stimulus. After each stimulus, subjects reported the direction of the inertial stimulus using a dial. The bias of the inertial heading toward the visual heading was robust at ±250 ms when examined across subjects during this period: 8.0° ± 0.5° with a 30° offset, 12.2° ± 0.5° with a 60° offset, 11.7° ± 0.6° with a 90° offset, and 9.8° ± 0.7° with a 120° offset (mean bias toward visual ± SE). The mean bias was much diminished with temporal misalignments of ±500 ms, and there was no longer any visual influence on the inertial heading when the visual stimulus was delayed by 1,000 ms or more. Although the amount of bias varied between subjects, the effect of delay was similar.NEW & NOTEWORTHY The effect of timing on visual-inertial integration on heading perception has not been previously examined. This study finds that visual direction influence inertial heading perception when timing differences are within 250 ms. This suggests visual-inertial stimuli can be integrated over a wider range than reported for visual-auditory integration and may be due to the unique nature of inertial sensation, which can only sense acceleration while the visual system senses position but encodes velocity.

Visual-vestibular integration is preserved with healthy aging in a simple acceleration detection task

Aging is associated with a gradual decline in the sensory systems and noisier sensory information. Some research has found that older adults compensate for this with enhanced multisensory integration. However, less is known about how aging influences visual-vestibular integration, an ability that underlies self-motion perception. We examined how visual-vestibular integration changes in participants from across the lifespan (18-79 years old) with a simple reaction time task. Participants were instructed to respond to visual (optic flow) and vestibular (inertial motion) acceleration cues, presented either alone or at a stimulus onset asynchrony. We measured reaction times and computed the violation area relative to the race model inequality as a measure of visual-vestibular integration. Across all ages, the greatest visual-vestibular integration occurred when the vestibular cue was presented first. Age was associated with longer reaction times and a significantly lower detection rate in the vestibular-only condition, a finding that is consistent with an age-related increase in vestibular noise. Although the relationship between age and visual-vestibular integration was positive, the effect size was very small and did not reach statistical significance. Our results suggest that although age is associated with a significant increase in vestibular perceptual threshold, the relative amount of visual-vestibular integration remains largely intact.

Oscillations in the auditory system and their possible role

GOURÉVITCH, B., C. Martin, O. Postal, J.J. Eggermont. Oscillations in the auditory system, their possible role. NEUROSCI BIOBEHAV REV XXX XXX-XXX, 2020. - Neural oscillations are thought to have various roles in brain processing such as, attention modulation, neuronal communication, motor coordination, memory consolidation, decision-making, or feature binding. The role of oscillations in the auditory system is less clear, especially due to the large discrepancy between human and animal studies. Here we describe many methodological issues that confound the results of oscillation studies in the auditory field. Moreover, we discuss the relationship between neural entrainment and oscillations that remains unclear. Finally, we aim to identify which kind of oscillations could be specific or salient to the auditory areas and their processing. We suggest that the role of oscillations might dramatically differ between the primary auditory cortex and the more associative auditory areas. Despite the moderate presence of intrinsic low frequency oscillations in the primary auditory cortex, rhythmic components in the input seem crucial for auditory processing. This allows the phase entrainment between the oscillatory phase and rhythmic input, which is an integral part of stimulus selection within the auditory system.

Temporal Metrics of Multisensory Processing Change in the Elderly

Older adults exhibit greater multisensory response time (RT) facilitation by violating the race model more than young adults; this is commonly interpreted as an enhancement in perception. Older adults typically exhibit wider temporal binding windows (TBWs) and points of subjective simultaneity (PSS) that typically lie farther from true simultaneity as compared to young adults when simultaneity judgment (SJ) and temporal-order judgment (TOJ) tasks are utilized; this is commonly interpreted as an impairment in perception. Here we explore the relation between the three tasks in order to better assess audiovisual multisensory temporal processing in both young and older adults. Our results confirm previous reports showing that audiovisual RT, TBWs and PSSs change with age; however, we show for the first time a significant positive relation between the magnitude of race model violation in young adults as a function of the PSS obtained from the audiovisual TOJ task (r: 0.49, p: 0.007), that is absent in older adults (r: 0.13, p: 0.58). Furthermore, we find no evidence for the relation between race model violation as a function of the PSS obtained from the audiovisual SJ task in both young (r: −0.01, p: 0.94) and older adults (r: 0.1, p: 0.66). Our results confirm previous reports that (i) audiovisual temporal processing changes with age; (ii) distinct processes are likely involved in simultaneity and temporal-order perception; and (iii) common processing between race model violation and temporal-order judgment is impaired in the elderly.

Vestibular and Multi-Sensory Influences Upon Self-Motion Perception and the Consequences for Human Behavior

In this manuscript, we comprehensively review both the human and animal literature regarding vestibular and multi-sensory contributions to self-motion perception. This covers the anatomical basis and how and where the signals are processed at all levels from the peripheral vestibular system to the brainstem and cerebellum and finally to the cortex. Further, we consider how and where these vestibular signals are integrated with other sensory cues to facilitate self-motion perception. We conclude by demonstrating the wide-ranging influences of the vestibular system and self-motion perception upon behavior, namely eye movement, postural control, and spatial awareness as well as new discoveries that such perception can impact upon numerical cognition, human affect, and bodily self-consciousness.

Perceived timing of active head movement at different speeds

The central nervous system must determine which sensory events occur at the same time. Actively moving the head corresponds with large changes in the relationship between the observer and the environment, sensorimotor processing, and spatiotemporal perception. Active head movement perception has been shown to be dependent on head movement velocity where participants who move their head fastest require the head to move earlier than comparison stimuli for perceived simultaneity more so than those who move their head slower. Such between-subject results cannot address whether active head movement perception changes with velocity. The present study used a within-subjects design to measure the point of subjective simultaneity (PSS) between active head movement speeds and a comparison sound stimulus to characterize the relationship between the velocity and perception of head movement onset. Our results clearly show that i) head movement perception is faster with faster head movements within-subjects, ii) active head movement onset must still precede the onset of other sensory events (average PSS: -123 ms to -52 ms; median PSS: -42 ms to -100 ms) in order to be perceived as occurring simultaneously even at the fastest speeds (average peak velocity: 76°/s-257°/s; median peak velocity 72 ms-257 ms). We conclude that head movement perception is slow, but that this delay is minimized with increased speed. These within-subject results are contrary to previous and present study between-subject results and are in agreement with literature where perception of auditory, visual and vestibular stimulus onset is less delayed with increased stimulus intensity.

Simultaneity and Temporal Order Judgments Are Coded Differently and Change With Age: An Event-Related Potential Study

Multisensory integration is required for a number of daily living tasks where the inability to accurately identify simultaneity and temporality of multisensory events results in errors in judgment leading to poor decision-making and dangerous behavior. Previously, our lab discovered that older adults exhibited impaired timing of audiovisual events, particularly when making temporal order judgments (TOJs). Simultaneity judgments (SJs), however, were preserved across the lifespan. Here, we investigate the difference between the TOJ and SJ tasks in younger and older adults to assess neural processing differences between these two tasks and across the lifespan. Event-related potentials (ERPs) were studied to determine between-task and between-age differences. Results revealed task specific differences in perceiving simultaneity and temporal order, suggesting that each task may be subserved via different neural mechanisms. Here, auditory N1 and visual P1 ERP amplitudes confirmed that unisensory processing of audiovisual stimuli did not differ between the two tasks within both younger and older groups, indicating that performance differences between tasks arise either from multisensory integration or higher-level decision-making. Compared to younger adults, older adults showed a sustained higher auditory N1 ERP amplitude response across SOAs, suggestive of broader response properties from an extended temporal binding window. Our work provides compelling evidence that different neural mechanisms subserve the SJ and TOJ tasks and that simultaneity and temporal order perception are coded differently and change with age.

Impaired perceived timing of falls in the elderly

Falls are the leading cause of injury-related deaths and hospitalizations, with older adults at an increased risk. As humans age, physical changes and health conditions make falls more likely. While we know how the body reflexively responds to prevent injury during a fall, we know little about how people perceive the fall itself. We previously found that young adults required a fall to precede a comparison sound stimulus by approximately 44ms to perceive the two events as simultaneous. This may relate to common anecdotal reports suggesting that humans often describe distortions in their perception of time - time seems to slow down during a fall - with very little recollection of how and when the fall began. Here we examine whether fall perception changes with age. Young (19-25y) and older (61-72y) healthy adults made temporal order judgments identifying whether the onset of their fall or the onset of a comparison sound came first to measure the point of subjective simultaneity. Results show that fall perception is nearly twice as slow for older adults, where perturbation onset has to precede sound onset by ∼88ms to appear coincident, compared to younger adults (∼44ms). We suggest that such age-related differences in fall perception may relate to increased fall rates in older adults. We conclude that a better understanding of how younger versus older adults perceive falls may identify important factors for innovative fall prevention strategies and rehabilitative training exercises to improve fall awareness.

Perturbation-evoked potentials: Significance and application in balance control research

Historically, balance control was thought to be mediated solely by subcortical structures based on animal research. However, recent findings provide compelling evidence of cortical involvement during balance reactions evoked by whole-body postural perturbations. In humans, an external perturbation elicits an evoked potential, termed the perturbation-evoked potential (PEP). PEPs are widely distributed over fronto-centro-parietal areas with maximal amplitude at the FCz/Cz electrode. From our literature review it is evident that the PEPs are comprised of a small positive potential (P1) that peaks around 30-90ms after perturbation onset, a large negative potential (N1) that peaks around 90-160ms, followed by positive (P2) and negative (N2) potentials between 200 and 400ms. Converging results across multiple studies suggest that these different PEP components are influenced by perturbation characteristics, postural set, environmental, and psychological factors. This review summarizes and integrates seminal research on the PEP, with a special emphasis on the PEP N1. Implications for future studies in PEP research are discussed to encourage further empirical investigation of PEP characteristics in healthy and patient populations.

Semicircular canal modeling in human perception

The human vestibular system is a sensory and equilibrium system that manages and controls the human sense of balance and movement. It is the main sensor humans use to perceive rotational and linear motions. Determining an accurate mathematical model of the human vestibular system is significant for research pertaining to motion perception, as the quality and effectiveness of the motion cueing algorithm (MCA) directly depends on the mathematical model used in its design. This paper describes the history and analyses the development process of mathematical semicircular canal models. The aim of this review is to determine the most consistent and reliable mathematical semicircular canal models that agree with experimental results and theoretical analyses, and offer reliable approximations for the semicircular canal functions based on the existing studies. Selecting and formulating accurate mathematical models of semicircular canals are essential for implementation into the MCA and for ensuring effective human motion perception modeling.

Bayesian Alternation during Tactile Augmentation

A large number of studies suggest that the integration of multisensory signals by humans is well-described by Bayesian principles. However, there are very few reports about cue combination between a native and an augmented sense. In particular, we asked the question whether adult participants are able to integrate an augmented sensory cue with existing native sensory information. Hence for the purpose of this study, we build a tactile augmentation device. Consequently, we compared different hypotheses of how untrained adult participants combine information from a native and an augmented sense. In a two-interval forced choice (2 IFC) task, while subjects were blindfolded and seated on a rotating platform, our sensory augmentation device translated information on whole body yaw rotation to tactile stimulation. Three conditions were realized: tactile stimulation only (augmented condition), rotation only (native condition), and both augmented and native information (bimodal condition). Participants had to choose one out of two consecutive rotations with higher angular rotation. For the analysis, we fitted the participants' responses with a probit model and calculated the just notable difference (JND). Then, we compared several models for predicting bimodal from unimodal responses. An objective Bayesian alternation model yielded a better prediction (χred2 = 1.67) than the Bayesian integration model (χred2 = 4.34). Slightly higher accuracy showed a non-Bayesian winner takes all (WTA) model (χred2 = 1.64), which either used only native or only augmented values per subject for prediction. However, the performance of the Bayesian alternation model could be substantially improved (χred2 = 1.09) utilizing subjective weights obtained by a questionnaire. As a result, the subjective Bayesian alternation model predicted bimodal performance most accurately among all tested models. These results suggest that information from augmented and existing sensory modalities in untrained humans is combined via a subjective Bayesian alternation process. Therefore, we conclude that behavior in our bimodal condition is explained better by top down-subjective weighting than by bottom-up weighting based upon objective cue reliability.

The Role of Visual and Haptic Feedback During Dynamically Coupled Bimanual Manipulation

The mechanisms that underlie the control of bimanual actions in which the two hands act separately to manipulate different objects (uncoupled independent control) has been well studied. In contrast, much less is known about how the central nervous system controls bimanual actions that require the two hands act cooperatively to manipulate a single object (dynamically coupled control). Furthermore, there is scant research into the manual lateralization and role assignment in the processing of visual and haptic feedback during dynamically coupled bimanual tasks. In this experiment, we examined the role of the dominant and non-dominant hands during a dynamically coupled bimanual task in which visual and haptic feedback regarding object penetration were manipulated. Twelve subjects performed a bimanual grasp and reach task towards different target locations in the workspace by using two identical wrist robotic devices. Results showed haptic feedback is necessary for task completion, and that hand specialization plays a fundamental role in spatial and temporal coordination between the two limbs.

The sense of body ownership relaxes temporal constraints for multisensory integration

Experimental work on body ownership illusions showed how simple multisensory manipulation can generate the illusory experience of an artificial limb as being part of the own-body. This work highlighted how own-body perception relies on a plastic brain representation emerging from multisensory integration. The flexibility of this representation is reflected in the short-term modulations of physiological states and perceptual processing observed during these illusions. Here, we explore the impact of ownership illusions on the temporal dimension of multisensory integration. We show that, during the illusion, the temporal window for integrating touch on the physical body with touch seen on a virtual body representation, increases with respect to integration with visual events seen close but separated from the virtual body. We show that this effect is mediated by the ownership illusion. Crucially, the temporal window for visuotactile integration was positively correlated with participants’ scores rating the illusory experience of owning the virtual body and touching the object seen in contact with it. Our results corroborate the recently proposed causal inference mechanism for illusory body ownership. As a novelty, they show that the ensuing illusory causal binding between stimuli from the real and fake body relaxes constraints for the integration of bodily signals.

A review on otolith models in human perception

The vestibular system, which consists of semicircular canals and otolith, are the main sensors mammals use to perceive rotational and linear motions. Identifying the most suitable and consistent mathematical model of the vestibular system is important for research related to driving perception. An appropriate vestibular model is essential for implementation of the Motion Cueing Algorithm (MCA) for motion simulation purposes, because the quality of the MCA is directly dependent on the vestibular model used. In this review, the history and development process of otolith models are presented and analyzed. The otolith organs can detect linear acceleration and transmit information about sensed applied specific forces on the human body. The main purpose of this review is to determine the appropriate otolith models that agree with theoretical analyses and experimental results as well as provide reliable estimation for the vestibular system functions. Formulating and selecting the most appropriate mathematical model of the vestibular system is important to ensure successful human perception modelling and simulation when implementing the model into the MCA for motion analysis.

A review of human sensory dynamics for application to models of driver steering and speed control

In comparison with the high level of knowledge about vehicle dynamics which exists nowadays, the role of the driver in the driver-vehicle system is still relatively poorly understood. A large variety of driver models exist for various applications; however, few of them take account of the driver's sensory dynamics, and those that do are limited in their scope and accuracy. A review of the literature has been carried out to consolidate information from previous studies which may be useful when incorporating human sensory systems into the design of a driver model. This includes information on sensory dynamics, delays, thresholds and integration of multiple sensory stimuli. This review should provide a basis for further study into sensory perception during driving.

The Effectiveness of Simulator Motion in the Transfer of Performance on a Tracking Task Is Influenced by Vision and Motion Disturbance Cues

OBJECTIVE

To examine the importance of platform motion to the transfer of performance in motion simulators.

BACKGROUND

The importance of platform motion in simulators for pilot training is strongly debated. We hypothesized that the type of motion (e.g., disturbance) contributes significantly to performance differences.

METHODS

Participants used a joystick to perform a target tracking task in a pod on top of a MOOG Stewart motion platform. Five conditions compared training without motion, with correlated motion, with disturbance motion, with disturbance motion isolated to the visual display, and with both correlated and disturbance motion. The test condition involved the full motion model with both correlated and disturbance motion. We analyzed speed and accuracy across training and test as well as strategic differences in joystick control.

RESULTS

Training with disturbance cues produced critical behavioral differences compared to training without disturbance; motion itself was less important.

CONCLUSION

Incorporation of disturbance cues is a potentially important source of variance between studies that do or do not show a benefit of motion platforms in the transfer of performance in simulators.

APPLICATION

Potential applications of this research include the assessment of the importance of motion platforms in flight simulators, with a focus on the efficacy of incorporating disturbance cues during training.

Impaired timing of audiovisual events in the elderly

Perceptual binding of multisensory events occurs within a limited time span known as the temporal binding window. Failure to correctly identify whether multisensory events occur simultaneously, what their temporal order is, or whether they should be causally bound can lead to inaccurate representations of the physical world, poor decision-making, and dangerous behavior. It has been shown that the ability to discriminate simultaneity, temporal order, and causal relationships among stimuli can become increasingly difficult as we age. In the present study, we assessed the relationship between these three attributes of temporally processing multisensory information in both younger and older adults. Performance on three tasks (temporal order judgment: TOJ, simultaneity judgment: SJ, and stream/bounce illusion) was compared using a large sample within-subjects design consisting of younger and older adults to determine aging effects as well as relationships between the three tasks. Older adults had more difficulty (larger temporal binding window) discriminating temporal order and perceived collision than younger adults. Simultaneity judgments in younger and older adults were indistinguishable. Positive correlations between TOJ and SJ as well as SJ and stream/bounce tasks were found in younger adults, which identify common (SJ) and distinct (TOJ, stream/bounce) neural mechanisms that sub-serve temporal processing of audiovisual information that is lost in older adults. We conclude that older adults have an extended temporal binding window for TOJ and stream/bounce tasks, but the temporal binding window in SJ is preserved, suggesting that age-related changes in multisensory integration are task specific and not a general trait of aging.

Advances in the diagnosis and treatment of vestibular disorders: psychophysics and prosthetics

Although vestibular disorders are common and often disabling, they remain difficult to diagnose and treat. For these reasons, considerable interest has been focused on developing new ways to identify peripheral and central vestibular abnormalities and on new therapeutic options that could benefit the numerous patients who remain symptomatic despite optimal therapy. In this review, I focus on the potential utility of psychophysical vestibular testing and vestibular prosthetics. The former offers a new diagnostic approach that may prove to be superior to the current tests in some circumstances; the latter may be a way to provide the brain with information about head motion that restores some elements of the information normally provided by the vestibular labyrinth.

Gain and phase of perceived virtual rotation evoked by electrical vestibular stimuli

Galvanic vestibular stimulation (GVS) evokes a perception of rotation; however, very few quantitative data exist on the matter. We performed psychophysical experiments on virtual rotations experienced when binaural bipolar electrical stimulation is applied over the mastoids. We also performed analogous real whole body yaw rotation experiments, allowing us to compare the frequency response of vestibular perception with (real) and without (virtual) natural mechanical stimulation of the semicircular canals. To estimate the gain of vestibular perception, we measured direction discrimination thresholds for virtual and real rotations. Real direction discrimination thresholds decreased at higher frequencies, confirming multiple previous studies. Conversely, virtual direction discrimination thresholds increased at higher frequencies, implying low-pass filtering of the virtual perception process occurring potentially anywhere between afferent transduction and cortical responses. To estimate the phase of vestibular perception, participants manually tracked their perceived position during sinusoidal virtual and real kinetic stimulation. For real rotations, perceived velocity was approximately in phase with actual velocity across all frequencies. Perceived virtual velocity was in phase with the GVS waveform at low frequencies (0.05 and 0.1 Hz). As frequency was increased to 1 Hz, the phase of perceived velocity advanced relative to the GVS waveform. Therefore, at low frequencies GVS is interpreted as an angular velocity signal and at higher frequencies GVS becomes interpreted increasingly as an angular position signal. These estimated gain and phase spectra for vestibular perception are a first step toward generating well-controlled virtual vestibular percepts, an endeavor that may reveal the usefulness of GVS in the areas of clinical assessment, neuroprosthetics, and virtual reality.

Vestibular implantation and longitudinal electrical stimulation of the semicircular canal afferents in human subjects

Animal experiments and limited data in humans suggest that electrical stimulation of the vestibular end organs could be used to treat loss of vestibular function. In this paper we demonstrate that canal-specific two-dimensionally (2D) measured eye velocities are elicited from intermittent brief 2 s biphasic pulse electrical stimulation in four human subjects implanted with a vestibular prosthesis. The 2D measured direction of the slow phase eye movements changed with the canal stimulated. Increasing pulse current over a 0-400 μA range typically produced a monotonic increase in slow phase eye velocity. The responses decremented or in some cases fluctuated over time in most implanted canals but could be partially restored by changing the return path of the stimulation current. Implantation of the device in Meniere's patients produced hearing and vestibular loss in the implanted ear. Electrical stimulation was well tolerated, producing no sensation of pain, nausea, or auditory percept with stimulation that elicited robust eye movements. There were changes in slow phase eye velocity with current and over time, and changes in electrically evoked compound action potentials produced by stimulation and recorded with the implanted device. Perceived rotation in subjects was consistent with the slow phase eye movements in direction and scaled with stimulation current in magnitude. These results suggest that electrical stimulation of the vestibular end organ in human subjects provided controlled vestibular inputs over time, but in Meniere's patients this apparently came at the cost of hearing and vestibular function in the implanted ear.

Differences in perceptual latency estimated from judgments of temporal order, simultaneity and duration are inconsistent

Differences in perceptual latency (ΔL) for two stimuli, such as an auditory and a visual stimulus, can be estimated from temporal order judgments (TOJ) and simultaneity judgments (SJ), but previous research has found evidence that ΔL estimated from these tasks do not coincide. Here, using an auditory and a visual stimulus we confirmed this and further show that ΔL as estimated from duration judgments also does not coincide with ΔL estimated from TOJ or SJ. These inconsistencies suggest that each judgment is subject to different processes that bias ΔL in different ways: TOJ might be affected by sensory interactions, a bias associated with the method of single stimuli and an order difficulty bias; SJ by sensory interactions and an asymmetrical criterion bias; duration judgments by an order duration bias.

Time-interval for integration of stabilizing haptic and visual information in subjects balancing under static and dynamic conditions

Maintaining equilibrium is basically a sensorimotor integration task. The central nervous system (CNS) continually and selectively weights and rapidly integrates sensory inputs from multiple sources, and coordinates multiple outputs. The weighting process is based on the availability and accuracy of afferent signals at a given instant, on the time-period required to process each input, and possibly on the plasticity of the relevant pathways. The likelihood that sensory inflow changes while balancing under static or dynamic conditions is high, because subjects can pass from a dark to a well-lit environment or from a tactile-guided stabilization to loss of haptic inflow. This review article presents recent data on the temporal events accompanying sensory transition, on which basic information is fragmentary. The processing time from sensory shift to reaching a new steady state includes the time to (a) subtract or integrate sensory inputs; (b) move from allocentric to egocentric reference or vice versa; and (c) adjust the calibration of motor activity in time and amplitude to the new sensory set. We present examples of processes of integration of posture-stabilizing information, and of the respective sensorimotor time-intervals while allowing or occluding vision or adding or subtracting tactile information. These intervals are short, in the order of 1–2 s for different postural conditions, modalities and deliberate or passive shift. They are just longer for haptic than visual shift, just shorter on withdrawal than on addition of stabilizing input, and on deliberate than unexpected mode. The delays are the shortest (for haptic shift) in blind subjects. Since automatic balance stabilization may be vulnerable to sensory-integration delays and to interference from concurrent cognitive tasks in patients with sensorimotor problems, insight into the processing time for balance control represents a critical step in the design of new balance- and locomotion training devices.

Evidence for a time-invariant phase variable in human ankle control

Human locomotion is a rhythmic task in which patterns of muscle activity are modulated by state-dependent feedback to accommodate perturbations. Two popular theories have been proposed for the underlying embodiment of phase in the human pattern generator: a time-dependent internal representation or a time-invariant feedback representation (i.e., reflex mechanisms). In either case the neuromuscular system must update or represent the phase of locomotor patterns based on the system state, which can include measurements of hundreds of variables. However, a much simpler representation of phase has emerged in recent designs for legged robots, which control joint patterns as functions of a single monotonic mechanical variable, termed a phase variable. We propose that human joint patterns may similarly depend on a physical phase variable, specifically the heel-to-toe movement of the Center of Pressure under the foot. We found that when the ankle is unexpectedly rotated to a position it would have encountered later in the step, the Center of Pressure also shifts forward to the corresponding later position, and the remaining portion of the gait pattern ensues. This phase shift suggests that the progression of the stance ankle is controlled by a biomechanical phase variable, motivating future investigations of phase variables in human locomotor control.

The role of differential delays in integrating transient visual and proprioceptive information

Many actions involve limb movements toward a target. Visual and proprioceptive estimates are available online, and by optimally combining (Ernst and Banks, 2002) both modalities during the movement, the system can increase the precision of the hand estimate. The notion that both sensory modalities are integrated is also motivated by the intuition that we do not consciously perceive any discrepancy between the felt and seen hand's positions. This coherence as a result of integration does not necessarily imply realignment between the two modalities (Smeets et al., 2006). For example, the two estimates (visual and proprioceptive) might be different without either of them (e.g., proprioception) ever being adjusted after recovering the other (e.g., vision). The implication that the felt and seen positions might be different has a temporal analog. Because the actual feedback from the hand at a given instantaneous position reaches brain areas at different times for proprioception and vision (shorter for proprioception), the corresponding instantaneous unisensory position estimates will be different, with the proprioceptive one being ahead of the visual one. Based on the assumption that the system integrates optimally and online the available evidence from both senses, we introduce a temporal mechanism that explains the reported overestimation of hand positions when vision is occluded for active and passive movements (Gritsenko et al., 2007) without the need to resort to initial feedforward estimates (Wolpert et al., 1995). We set up hypotheses to test the validity of the model, and we contrast simulation-based predictions with empirical data.

Catch the wave! Time-window sequential analysis of alertness stimulation in individuals with profound intellectual and multiple disabilities

BACKGROUND

While optimally activities are provided at those moments when the individual with profound intellectual and multiple disabilities (PIMD) is 'focused on the environment' or 'alert', detailed information about the impact that the design and timing of the activity has on alertness is lacking. Therefore, the aim of the present study is to shed light on the sequential relationship between different stimuli and alertness levels in individuals with PIMD.

METHOD

Video observations were conducted for 24 participants during one-on-one interactions with a direct support person in multisensory environments. Time-window sequential analyses were conducted for the 120 s following four different stimuli.

RESULTS

For the different stimuli, different patterns in terms of alertness became apparent. Following visual stimuli, the alertness levels of the individuals with PIMD changed in waves of about 20 s from 'active alert' to 'passive alert'. While auditory and tactile stimuli led to 'alert' reactions shortly after the stimulation, alertness levels decreased between seconds 20 and 120. Reactions to vestibular stimuli were only visible after 60 s; these were 'active alert' or 'withdrawn'.

CONCLUSIONS

The results of the present study show that individuals with PIMD show their reactions to stimuli only slightly, so that 'waves' might reflect the optimal alertness pattern for learning and development. Consequently, it is especially important that direct support persons follow and stimulate these individual 'waves' in the activities they provide to their clients.

Whole body motion-detection tasks can yield much lower thresholds than direction-recognition tasks: implications for the role of vibration

Earlier spatial orientation studies used both motion-detection (e.g., did I move?) and direction-recognition (e.g., did I move left/right?) paradigms. The purpose of our study was to compare thresholds measured with motion-detection and direction-recognition tasks on a standard Moog motion platform to see whether a substantial fraction of the reported threshold variation might be explained by the use of different discrimination tasks in the presence of vibrations that vary with motion. Thresholds for the perception of yaw rotation about an earth-vertical axis and for interaural translation in an earth-horizontal plane were determined for four healthy subjects with standard detection and recognition paradigms. For yaw rotation two-interval detection thresholds were, on average, 56 times smaller than two-interval recognition thresholds, and for interaural translation two-interval detection thresholds were, on average, 31 times smaller than two-interval recognition thresholds. This substantive difference between recognition thresholds and detection thresholds is one of our primary findings. For motions near our measured detection threshold, we measured vibrations that matched previously established vibration thresholds. This suggests that vibrations contribute to whole body motion detection. We also recorded yaw rotation thresholds on a second motion device with lower vibration and found direction-recognition and motion-detection thresholds that were not significantly different from one another or from the direction-recognition thresholds recorded on our Moog platform. Taken together, these various findings show that yaw rotation recognition thresholds are relatively unaffected by vibration when moderate (up to ≈ 0.08 m/s(2)) vibration cues are present.

Contextual control of audiovisual integration in low-level sensory cortices

Potential sources of multisensory influences on low-level sensory cortices include direct projections from sensory cortices of different modalities, as well as more indirect feedback inputs from higher order multisensory cortical regions. These multiple architectures may be functionally complementary, but the exact roles and inter-relationships of the circuits are unknown. Using a fully balanced context manipulation, we tested the hypotheses that: (1) feedforward and lateral pathways subserve speed functions, such as detecting peripheral stimuli. Multisensory integration effects in this context are predicted in peripheral fields of low-level sensory cortices. (2) Slower feedback pathways underpin accuracy functions, such as object discrimination. Integration effects in this context are predicted in higher-order association cortices and central/foveal fields of low-level sensory cortex. We used functional magnetic resonance imaging to compare the effects of central versus peripheral stimulation on audiovisual integration, while varying speed and accuracy requirements for behavioral responses. We found that interactions of task demands and stimulus eccentricity in low-level sensory cortices are more complex than would be predicted by a simple dichotomy such as our hypothesized peripheral/speed and foveal/accuracy functions. Additionally, our findings point to individual differences in integration that may be related to skills and strategy. Overall, our findings suggest that instead of using fixed, specialized pathways, the exact circuits and mechanisms that are used for low-level multisensory integration are much more flexible and contingent upon both individual and contextual factors than previously assumed.

Sight over sound in the judgment of music performance

Social judgments are made on the basis of both visual and auditory information, with consequential implications for our decisions. To examine the impact of visual information on expert judgment and its predictive validity for performance outcomes, this set of seven experiments in the domain of music offers a conservative test of the relative influence of vision versus audition. People consistently report that sound is the most important source of information in evaluating performance in music. However, the findings demonstrate that people actually depend primarily on visual information when making judgments about music performance. People reliably select the actual winners of live music competitions based on silent video recordings, but neither musical novices nor professional musicians were able to identify the winners based on sound recordings or recordings with both video and sound. The results highlight our natural, automatic, and nonconscious dependence on visual cues. The dominance of visual information emerges to the degree that it is overweighted relative to auditory information, even when sound is consciously valued as the core domain content.

Temporal processing of self-motion: modeling reaction times for rotations and translations

In this paper, we show that differences in reaction times (RT) to self-motion depend not only on the duration of the profile, but also on the actual time course of the acceleration. We previously proposed models that described direction discrimination thresholds for rotational and translational motions based on the dynamics of the vestibular sensory organs (otoliths and semi-circular canals). As these models have the potential to describe RT for different motion profiles (e.g., trapezoidal versus triangular acceleration profiles or varying profile durations), we validated these models by measuring RTs in human observers for a direction discrimination task using both translational and rotational motions varying in amplitude, duration and acceleration profile shape in a within-subjects design. In agreement with previous studies, amplitude and duration were found to affect RT, and importantly, we found an influence of the profile shape on RT. The models are able to fit the measured RTs with an accuracy of around 5 ms, and the best-fitting parameters are similar to those found from identifying the models based on threshold measurements. This confirms the validity of the modeling approach and links perceptual thresholds to RT. By establishing a link between vestibular thresholds for self-motion and RT, we show for the first time that RTs to purely inertial motion stimuli can be used as an alternative to threshold measurements for identifying self-motion perception models. This is advantageous, since RT tasks are less challenging for participants and make assessment of vestibular function less fatiguing. Further, our results provide strong evidence that the perceived timing of self-motion stimulation is largely influenced by the response dynamics of the vestibular sensory organs.