Audiovisual Simultaneity Judgment and Rapid Recalibration throughout the Lifespan

Neural correlates of audiovisual integration in schizophrenia - an ERP study

Introduction

Multisensory integration (MSI) enhances perception by combining information from different sensory modalities. In schizophrenia, individuals often exhibit impaired audiovisual processing, resulting in broader temporal binding windows (TBWs) which appear to be associated with symptom severity. Since the underlying mechanisms of these aberrations are not yet fully understood, the present study aims to investigate multisensory processing in schizophrenia in more detail.

Methods

Individuals with schizophrenia (SZ) and healthy controls (HC) performed a simultaneity judgement task, a paradigm that is suitable for the examination of multisensory integration processes. The paradigm was also conducted to allow for the comparison of perceptions under ecologically valid and invalid conditions. Additionally, EEG recordings were made to explore underlying neural mechanisms.

Results

In line with previous research, we replicated enlarged TBWs in SZ compared to HC, independent of ecological validity. Neurophysiological data further revealed reduced amplitudes in the early ERP complex N1/P2 in SZ compared to HC.

Discussion

Since amplitude reduction in the N1/P2 complex is often associated with audiovisual integration processes, the results highlight perceptual dysfunction in SZ, particularly concerning the disengagement of auditory and visual stimuli.

Audiovisual simultaneity windows reflect temporal sensory uncertainty

The ability to judge the temporal alignment of visual and auditory information is a prerequisite for multisensory integration and segregation. However, each temporal measurement is subject to error. Thus, when judging whether a visual and auditory stimulus were presented simultaneously, observers must rely on a subjective decision boundary to distinguish between measurement error and truly misaligned audiovisual signals. Here, we tested whether these decision boundaries are relaxed with increasing temporal sensory uncertainty, i.e., whether participants make the same type of adjustment an ideal observer would make. Participants judged the simultaneity of audiovisual stimulus pairs with varying temporal offset, while being immersed in different virtual environments. To obtain estimates of participants' temporal sensory uncertainty and simultaneity criteria in each environment, an independent-channels model was fitted to their simultaneity judgments. In two experiments, participants' simultaneity decision boundaries were predicted by their temporal uncertainty, which varied unsystematically with the environment. Hence, observers used a flexibly updated estimate of their own audiovisual temporal uncertainty to establish subjective criteria of simultaneity. This finding implies that, under typical circumstances, audiovisual simultaneity windows reflect an observer's cross-modal temporal uncertainty.

When 2 become 1: Autistic simultaneity judgements about asynchronous audiovisual speech

It has been proposed that autistic people experience a temporal distortion whereby the temporal binding window of multisensory integration is extended. Research to date has focused on autistic children so whether these differences persist into adulthood remains unknown. In addition, the possibility that the previous observations have arisen from between-group differences in response bias, rather than perceptual differences, has not been addressed. Participants completed simultaneity judgements of audiovisual speech stimuli across a range of stimulus-onset asynchronies. Response times and accuracy data were fitted to a drift-diffusion model so that the drift rate (a measure of processing efficiency) and starting point (response bias) could be estimated. In Experiment 1, we tested a sample of non-autistic adults who completed the Autism Quotient questionnaire. Autism Quotient score was not correlated with either drift rate or response bias, nor were there between-group differences when splitting based on the first and third quantiles of scores. In Experiment 2, we compared the performance of autistic with a group of non-autistic adults. There were no between-group differences in either drift rate or starting point. The results of this study do not support the previous suggestion that autistic people have an extended temporal binding window for audiovisual speech. In addition, exploratory analysis revealed that operationalising the temporal binding window in different ways influenced whether a group difference was observed, which is an important consideration for future work.

Synchrony perception across senses: A systematic review of temporal binding window changes from infancy to adolescence in typical and atypical development

Sensory integration is increasingly acknowledged as being crucial for the development of cognitive and social abilities. However, its developmental trajectory is still little understood. This systematic review delves into the topic by investigating the literature about the developmental changes from infancy through adolescence of the Temporal Binding Window (TBW) - the epoch of time within which sensory inputs are perceived as simultaneous and therefore integrated. Following comprehensive searches across PubMed, Elsevier, and PsycInfo databases, only experimental, behavioral, English-language, peer-reviewed studies on multisensory temporal processing in 0-17-year-olds have been included. Non-behavioral, non-multisensory, and non-human studies have been excluded as those that did not directly focus on the TBW. The selection process was independently performed by two Authors. The 39 selected studies involved 2859 participants in total. Findings indicate a predisposition towards cross-modal asynchrony sensitivity and a composite, still unclear, developmental trajectory, with atypical development associated to increased asynchrony tolerance. These results highlight the need for consistent and thorough research into TBW development to inform potential interventions.

The fusion point of temporal binding: Promises and perils of multisensory accounts

Performing an action to initiate a consequence in the environment triggers the perceptual illusion of temporal binding. This phenomenon entails that actions and following effects are perceived to occur closer in time than they do outside the action-effect relationship. Here we ask whether temporal binding can be explained in terms of multisensory integration, by assuming either multisensory fusion or partial integration of the two events. We gathered two datasets featuring a wide range of action-effect delays as a key factor influencing integration. We then tested the fit of a computational model for multisensory integration, the statistically optimal cue integration (SOCI) model. Indeed, qualitative aspects of the data on a group-level followed the principles of a multisensory account. By contrast, quantitative evidence from a comprehensive model evaluation indicated that temporal binding cannot be reduced to multisensory integration. Rather, multisensory integration should be seen as one of several component processes underlying temporal binding on an individual level.

Tactile cues are more intrinsically linked to motor timing than visual cues in visual-tactile sensorimotor synchronization

Many tasks require precise synchronization with external sensory stimuli, such as driving a car. This study investigates whether combined visual-tactile information provides additional benefits to movement synchrony over separate visual and tactile stimuli and explores the relationship with the temporal binding window for multisensory integration. In Experiment 1, participants completed a sensorimotor synchronization task to examine movement variability and a simultaneity judgment task to measure the temporal binding window. Results showed similar synchronization variability between visual-tactile and tactile-only stimuli, but significantly lower than visual only. In Experiment 2, participants completed a visual-tactile sensorimotor synchronization task with cross-modal stimuli presented inside (stimulus onset asynchrony 80 ms) and outside (stimulus-onset asynchrony 400 ms) the temporal binding window to examine temporal accuracy of movement execution. Participants synchronized their movement with the first stimulus in the cross-modal pair, either the visual or tactile stimulus. Results showed significantly greater temporal accuracy when only one stimulus was presented inside the window and the second stimulus was outside the window than when both stimuli were presented inside the window, with movement execution being more accurate when attending to the tactile stimulus. Overall, these findings indicate there may be a modality-specific benefit to sensorimotor synchronization performance, such that tactile cues are weighted more strongly than visual information as tactile information is more intrinsically linked to motor timing than visual information. Further, our findings indicate that the visual-tactile temporal binding window is related to the temporal accuracy of movement execution.

Multisensory Calibration: A Variety of Slow and Fast Brain Processes Throughout the Lifespan

From before we are born, throughout development, adulthood, and aging, we are immersed in a multisensory world. At each of these stages, our sensory cues are constantly changing, due to body, brain, and environmental changes. While integration of information from our different sensory cues improves precision, this only improves accuracy if the underlying cues are unbiased. Thus, multisensory calibration is a vital and ongoing process. To meet this grand challenge, our brains have evolved a variety of mechanisms. First, in response to a systematic discrepancy between sensory cues (without external feedback) the cues calibrate one another (unsupervised calibration). Second, multisensory function is calibrated to external feedback (supervised calibration). These two mechanisms superimpose. While the former likely reflects a lower level mechanism, the latter likely reflects a higher level cognitive mechanism. Indeed, neural correlates of supervised multisensory calibration in monkeys were found in higher level multisensory cortical area VIP, but not in the relatively lower level multisensory area MSTd. In addition, even without a cue discrepancy (e.g., when experiencing stimuli from different sensory cues in series) the brain monitors supra-modal statistics of events in the environment and adapts perception cross-modally. This too comprises a variety of mechanisms, including confirmation bias to prior choices, and lower level cross-sensory adaptation. Further research into the neuronal underpinnings of the broad and diverse functions of multisensory calibration, with improved synthesis of theories is needed to attain a more comprehensive understanding of multisensory brain function.

Development and experience-dependence of multisensory spatial processing

Multisensory spatial processes are fundamental for efficient interaction with the world. They include not only the integration of spatial cues across sensory modalities, but also the adjustment or recalibration of spatial representations to changing cue reliabilities, crossmodal correspondences, and causal structures. Yet how multisensory spatial functions emerge during ontogeny is poorly understood. New results suggest that temporal synchrony and enhanced multisensory associative learning capabilities first guide causal inference and initiate early coarse multisensory integration capabilities. These multisensory percepts are crucial for the alignment of spatial maps across sensory systems, and are used to derive more stable biases for adult crossmodal recalibration. The refinement of multisensory spatial integration with increasing age is further promoted by the inclusion of higher-order knowledge.

An imbalance of excitation and inhibition in the multisensory cortex impairs the temporal acuity of audiovisual processing and perception

The neural integration of closely timed auditory and visual stimuli can offer several behavioral advantages; however, an overly broad window of temporal integration-a phenomenon observed in various neurodevelopmental disorders-could have far-reaching perceptual consequences. Non-invasive studies in humans have suggested that the level of GABAergic inhibition in the multisensory cortex influences the temporal window over which auditory and visual stimuli are bound into a unified percept. Although this suggestion aligns with the theory that an imbalance of cortical excitation and inhibition alters multisensory processing, no prior studies have performed experimental manipulations to determine the causal effects of a reduction of GABAergic inhibition on audiovisual temporal perception. To that end, we used a combination of in vivo electrophysiology, neuropharmacology, and translational behavioral testing in rats to provide the first mechanistic evidence that a reduction of GABAergic inhibition in the audiovisual cortex is sufficient to disrupt unisensory and multisensory processing across the cortical layers, and ultimately impair the temporal acuity of audiovisual perception and its rapid adaptation to recent sensory experience. Looking forward, our findings provide support for using rat models to further investigate the neural mechanisms underlying the audiovisual perceptual alterations observed in neurodevelopmental disorders, such as autism, schizophrenia, and dyslexia.

Age-Related Changes to Multisensory Integration and Audiovisual Speech Perception

Multisensory integration is essential for the quick and accurate perception of our environment, particularly in everyday tasks like speech perception. Research has highlighted the importance of investigating bottom-up and top-down contributions to multisensory integration and how these change as a function of ageing. Specifically, perceptual factors like the temporal binding window and cognitive factors like attention and inhibition appear to be fundamental in the integration of visual and auditory information-integration that may become less efficient as we age. These factors have been linked to brain areas like the superior temporal sulcus, with neural oscillations in the alpha-band frequency also being implicated in multisensory processing. Age-related changes in multisensory integration may have significant consequences for the well-being of our increasingly ageing population, affecting their ability to communicate with others and safely move through their environment; it is crucial that the evidence surrounding this subject continues to be carefully investigated. This review will discuss research into age-related changes in the perceptual and cognitive mechanisms of multisensory integration and the impact that these changes have on speech perception and fall risk. The role of oscillatory alpha activity is of particular interest, as it may be key in the modulation of multisensory integration.

Change of rapid temporal recalibration magnitude for audiovisual asynchrony with modulation of temporal binding window width: A preliminary investigation

The subjective synchrony perception for audiovisual stimuli is affected by previous temporal information. The point of subjective simultaneity is shifted toward the same asynchronous direction of audiovisual stimuli in a previous trial. This phenomenon is called "rapid temporal recalibration." The factors that modulate the magnitude of rapid temporal recalibration have not been fully investigated. Previously, a positive correlation has been found between the magnitude of rapid temporal recalibration and the width of the temporal binding window (TBW). This preliminary study examined the causal relationship between TBW size and rapid recalibration magnitude using a single experimental group comparison design. In this experiment, the magnitude of rapid recalibration was compared before and after perceptual training, which narrowed the TBW width. The results indicated that the magnitude of rapid recalibration was reduced by perceptual training. Therefore, it was speculated that TBW size determined the magnitude of rapid recalibration. This causal relationship helps elucidate the mechanisms of the adaptation for temporal lags between visual and auditory sensations.

The development of audio-visual temporal precision precedes its rapid recalibration

Through development, multisensory systems reach a balance between stability and flexibility: the systems integrate optimally cross-modal signals from the same events, while remaining adaptive to environmental changes. Is continuous intersensory recalibration required to shape optimal integration mechanisms, or does multisensory integration develop prior to recalibration? Here, we examined the development of multisensory integration and rapid recalibration in the temporal domain by re-analyzing published datasets for audio-visual, audio-tactile, and visual-tactile combinations. Results showed that children reach an adult level of precision in audio-visual simultaneity perception and show the first sign of rapid recalibration at 9 years of age. In contrast, there was very weak rapid recalibration for other cross-modal combinations at all ages, even when adult levels of temporal precision had developed. Thus, the development of audio-visual rapid recalibration appears to require the maturation of temporal precision. It may serve to accommodate distance-dependent travel time differences between light and sound.

Musical training refines audiovisual integration but does not influence temporal recalibration

When the brain is exposed to a temporal asynchrony between the senses, it will shift its perception of simultaneity towards the previously experienced asynchrony (temporal recalibration). It is unknown whether recalibration depends on how accurately an individual integrates multisensory cues or on experiences they have had over their lifespan. Hence, we assessed whether musical training modulated audiovisual temporal recalibration. Musicians (n = 20) and non-musicians (n = 18) made simultaneity judgements to flash-tone stimuli before and after adaptation to asynchronous (± 200 ms) flash-tone stimuli. We analysed these judgements via an observer model that described the left and right boundaries of the temporal integration window (decisional criteria) and the amount of sensory noise that affected these judgements. Musicians’ boundaries were narrower (closer to true simultaneity) than non-musicians’, indicating stricter criteria for temporal integration, and they also exhibited enhanced sensory precision. However, while both musicians and non-musicians experienced cumulative and rapid recalibration, these recalibration effects did not differ between the groups. Unexpectedly, cumulative recalibration was caused by auditory-leading but not visual-leading adaptation. Overall, these findings suggest that the precision with which observers perceptually integrate audiovisual temporal cues does not predict their susceptibility to recalibration.

No differences between adults with and without autism in audiovisual synchrony perception

LAY ABSTRACT

It has been known for a long time that individuals diagnosed with autism spectrum disorder perceive the world differently. In this study, we investigated how people with or without autism perceive visual and auditory information. We know that an auditory and a visual stimulus do not have to be perfectly synchronous for us to perceive them as synchronous: first, when the two are within a certain time window (temporal binding window), the brain will tell us that they are synchronous. Second, the brain can also adapt quickly to audiovisual asynchronies (rapid recalibration). Although previous studies have shown that people with autism spectrum disorder have different temporal binding windows, and less rapid recalibration, we did not find these differences in our study. However, we did find that both processes develop over age, and since previous studies tested only young people (children, adolescents, and young adults), and we tested adults from 18 to 55 years, this might explain the different findings. In the end, there might be quite a complex story, where people with and without autism spectrum disorder perceive the world differently, even dependent on how old they are.

A Scoping Review of Audiovisual Integration Methodology: Screening for Auditory and Visual Impairment in Younger and Older Adults

With the rise of the aging population, many scientists studying multisensory integration have turned toward understanding how this process may change with age. This scoping review was conducted to understand and describe the scope and rigor with which researchers studying audiovisual sensory integration screen for hearing and vision impairment. A structured search in three licensed databases (Scopus, PubMed, and PsychInfo) using the key concepts of multisensory integration, audiovisual modality, and aging revealed 2,462 articles, which were screened for inclusion by two reviewers. Articles were included if they (1) tested healthy older adults (minimum mean or median age of 60) with younger adults as a comparison (mean or median age between 18 and 35), (2) measured auditory and visual integration, (3) were written in English, and (4) reported behavioral outcomes. Articles that included the following were excluded: (1) tested taste exclusively, (2) tested olfaction exclusively, (3) tested somatosensation exclusively, (4) tested emotion perception, (5) were not written in English, (6) were clinical commentaries, editorials, interviews, letters, newspaper articles, abstracts only, or non-peer reviewed literature (e.g., theses), and (7) focused on neuroimaging without a behavioral component. Data pertaining to the details of the study (e.g., country of publication, year of publication, etc.) were extracted, however, of higher importance to our research question, data pertaining to screening measures used for hearing and vision impairment (e.g., type of test used, whether hearing- and visual-aids were worn, thresholds used, etc.) were extracted, collated, and summarized. Our search revealed that only 64% of studies screened for age-abnormal hearing impairment, 51% screened for age-abnormal vision impairment, and that consistent definitions of normal or abnormal vision and hearing were not used among the studies that screened for sensory abilities. A total of 1,624 younger adults and 4,778 older participants were included in the scoping review with males composing approximately 44% and females composing 56% of the total sample and most of the data was obtained from only four countries. We recommend that studies investigating the effects of aging on multisensory integration should screen for normal vision and hearing by using the World Health Organization's (WHO) hearing loss and visual impairment cut-off scores in order to maintain consistency among other aging researchers. As mild cognitive impairment (MCI) has been defined as a "transitional" or a "transitory" stage between normal aging and dementia and because approximately 3-5% of the aging population will develop MCI each year, it is therefore important that when researchers aim to study a healthy aging population, that they appropriately screen for MCI. One of our secondary aims was to determine how often researchers were screening for cognitive impairment and the types of tests that were used to do so. Our results revealed that only 55 out of 72 studies tested for neurological and cognitive function, and only a subset used standardized tests. Additionally, among the studies that used standardized tests, the cut-off scores used were not always adequate for screening out mild cognitive impairment. An additional secondary aim of this scoping review was to determine the feasibility of whether a meta-analysis could be conducted in the future to further quantitatively evaluate the results (i.e., are the findings obtained from studies using self-reported vision and hearing impairment screening methods significantly different from those measuring vision and hearing impairment in the lab) and to assess the scope of this problem. We found that it may not be feasible to conduct a meta-analysis with the entire dataset of this scoping review. However, a meta-analysis can be conducted if stricter parameters are used (e.g., focusing on accuracy or response time data only). Systematic Review Registration: https://doi.org/10.17605/OSF.IO/GTUHD.

Impoverished Inhibitory Control Exacerbates Multisensory Impairments in Older Fallers

Impaired temporal perception of multisensory cues is a common phenomenon observed in older adults that can lead to unreliable percepts of the external world. For instance, the sound induced flash illusion (SIFI) can induce an illusory percept of a second flash by presenting a beep close in time to an initial flash-beep pair. Older adults that have enhanced susceptibility to a fall demonstrate significantly stronger illusion percepts during the SIFI task compared to those older adults without any history of falling. We hypothesize that a global inhibitory deficit may be driving the impairments across both postural stability and multisensory function in older adults with a fall history (FH). We investigated oscillatory activity and perceptual performance during the SIFI task, to understand how active sensory processing, measured by gamma (30–80 Hz) power, was regulated by alpha activity (8–13 Hz), oscillations that reflect inhibitory control. Compared to young adults (YA), the FH and non-faller (NF) groups demonstrated enhanced susceptibility to the SIFI. Further, the FH group had significantly greater illusion strength compared to the NF group. The FH group also showed significantly impaired performance relative to YA during congruent trials (2 flash-beep pairs resulting in veridical perception of 2 flashes). In illusion compared to non-illusion trials, the NF group demonstrated reduced alpha power (or diminished inhibitory control). Relative to YA and NF, the FH group showed reduced phase-amplitude coupling between alpha and gamma activity in non-illusion trials. This loss of inhibitory capacity over sensory processing in FH compared to NF suggests a more severe change than that consequent of natural aging.

Age-Related Changes in Audiovisual Simultaneity Perception and Their Relationship With Working Memory

OBJECTIVES

Perceiving simultaneity of a visual and an auditory signal is critical for humans to integrate these multisensory inputs effectively and respond properly. We examined age-related changes in audiovisual simultaneity perception, and the relationships between this perception and working memory performances with aging.

METHODS

Audiovisual simultaneity perception of young, middle-aged, and older adults was measured using a simultaneity judgment (SJ) task, in which a flash and a beep were presented at 1 of 11 stimulus-onset asynchronies (SOAs). Participants judged whether these two stimuli were perceived simultaneously. Precision of simultaneity perception, the SOA corresponding to the point of subjective simultaneity (PSS), and response errors at each SOA were estimated using model fitting. The precision and PSS are associated with multisensory perception per se, whereas the response error reflects executive ability when performing the SJ task. Visual working memory of the same middle-aged and older adults was measured using the Cambridge Neuropsychological Test Automated Battery (CANTAB) beforehand.

RESULTS

Compared to young adults' performances, middle-aged and older adults showed a decreased precision, a shift of PSS toward the visual-leading SOAs, and increased response errors at the visual-leading SOAs. Among these changes, only the increased response errors correlated with worse spatial recognition memory in middle-aged and older adults.

DISCUSSION

Age-related decrements in audiovisual simultaneity perception start from middle age and are manifested in both perceptual and executive parameters. Furthermore, higher-order executive ability is plausibly a common cause for age-related degenerations in the audiovisual simultaneity perception and visual working memory.

Audiomotor Temporal Recalibration Modulates Decision Criterion of Self-Agency but Not Perceptual Sensitivity

Exposure to delayed sensory feedback changes perceived simultaneity between action and feedback [temporal recalibration (TR)] and even modulates the sense of agency (SoA) over the feedback. To date, however, it is not clear whether the modulation of SoA by TR is caused by a change in perceptual sensitivity or decision criterion of self-agency. This experimental research aimed to tease apart these two by applying the signal detection theory (SDT) to the agency judgment over auditory feedback after voluntary action. Participants heard a short sequence of tone pips with equal inter-onset intervals, and they reproduced it by pressing a computer mouse. The delay of each tone pip after the mouse press was manipulated as 80 (baseline) or 180 ms (delayed). Subsequently, the participants reproduced it, in which the delay was fixed at 80 ms and there was a 50% chance that the computer took over the control of the tone pips from the participants. The participants’ task was to discriminate who controlled the tone pips and to judge synchrony between tone pips and mouse presses. Results showed that the modulation of the SoA by the TR is caused by a shift in the decision criterion but not in the perceptual sensitivity of agency.

Sensory- and memory-related drivers for altered ventriloquism effects and aftereffects in older adults

The manner in which humans exploit multisensory information for subsequent decisions changes with age. Multiple causes for such age-effects are being discussed, including a reduced precision in peripheral sensory representations, changes in cognitive inference about causal relations between sensory cues, and a decline in memory contributing to altered sequential patterns of multisensory behaviour. To dissociate these putative contributions, we investigated how healthy young and older adults integrate audio-visual spatial information within trials (the ventriloquism effect) and between trials (the ventriloquism aftereffect). With both a model-free and (Bayesian) model-based analyses we found that both biases differed between groups. Our results attribute the age-change in the ventriloquism bias to a decline in spatial hearing rather than a change in cognitive processes. This decline in peripheral function, combined with a more prominent influence from preceding responses rather than preceding stimuli in the elderly, can also explain the observed age-effect in the ventriloquism aftereffect. Our results suggest a transition from a sensory-to a behavior-driven influence of past multisensory experience on perceptual decisions with age, due to reduced sensory precision and change in memory capacity.

Multisensory integration involved in the body perception of community-dwelling older adults

This study investigates how the multisensory integration in body perception changes with increasing age, and whether it is associated with older adults’ risk of falling. For this, the rubber hand illusion (RHI) and rubber foot illusion (RFI) were used. Twenty-eight community-dwelling older adults and 25 university students were recruited. They viewed a rubber hand or foot that was stimulated in synchrony or asynchrony with their own hidden hand or foot. The illusion was assessed by using a questionnaire, and measuring the proprioceptive drift and latency. The Timed Up and Go Test was used to classify the older adults into lower and higher fall-risk groups. No difference was observed in the RHI between the younger and older adults. However, several differences were observed in the RFI. Specifically, the older adults with a lower fall-risk hardly experienced the illusion, whereas those with a higher fall-risk experienced it with a shorter latency and no weaker than the younger adults. These results suggest that in older adults, the mechanism of multisensory integration for constructing body perception can change depending on the stimulated body parts, and that the risk of falling is associated with multisensory integration.

Narrowing of the Audiovisual Temporal Binding Window Due To Perceptual Training Is Specific to High Visual Intensity Stimuli

The temporal binding window (TBW), which reflects the range of temporal offsets in which audiovisual stimuli are combined to form a singular percept, can be reduced through training. Our research aimed to investigate whether training-induced reductions in TBW size transfer across stimulus intensities. A total of 32 observers performed simultaneity judgements at two visual intensities with a fixed auditory intensity, before and after receiving audiovisual TBW training at just one of these two intensities. We show that training individuals with a high visual intensity reduces the size of the TBW for bright stimuli, but this improvement did not transfer to dim stimuli. The reduction in TBW can be explained by shifts in decision criteria. Those trained with the dim visual stimuli, however, showed no reduction in TBW. Our main finding is that perceptual improvements following training are specific for high-intensity stimuli, potentially highlighting limitations of proposed TBW training procedures.

Stimulus Specific to Age-Related Audio-Visual Integration in Discrimination Tasks

Age-related audio-visual integration (AVI) has been investigated extensively; however, AVI ability is either enhanced or reduced with ageing, and this matter is still controversial because of the lack of systematic investigations. To remove possible variates, 26 older adults and 26 younger adults were recruited to conduct meaningless and semantic audio-visual discrimination tasks to assess the ageing effect of AVI systematically. The results for the mean response times showed a significantly faster response to the audio-visual (AV) target than that to the auditory (A) or visual (V) target and a significantly faster response to all targets by the younger adults than that by the older adults (A, V, and AV) in all conditions. In addition, a further comparison of the differences between the probability of audio-visual cumulative distributive functions (CDFs) and race model CDFs showed delayed AVI effects and a longer time window for AVI in older adults than that in younger adults in all conditions. The AVI effect was lower in older adults than that in younger adults during simple meaningless image discrimination (63.0 ms vs. 108.8 ms), but the findings were inverse during semantic image discrimination (310.3 ms vs. 127.2 ms). In addition, there was no significant difference between older and younger adults during semantic character discrimination (98.1 ms vs. 117.2 ms). These results suggested that AVI ability was impaired in older adults, but a compensatory mechanism was established for processing sematic audio-visual stimuli.

Everything has Its Time: Narrow Temporal Windows are Associated with High Levels of Autistic Traits Via Weaknesses in Multisensory Integration

The present study examined whether fundamental sensory functions such as temporal processing and multisensory integration are related to autistic traits in the general population. Both a narrower temporal window (TW) for simultaneous perception, as measured by a temporal order judgement task, and a reduced ability to engage in multisensory integration during the sound-induced flash illusion task were related to higher levels of autistic traits. Additionally, a narrow TW is associated with high levels of autistic traits due to a deficiency in multisensory integration. Taken together, these findings suggest that alterations in fundamental functions produce a cascading effect on higher-order social and cognitive functions, such as those experienced by people with autism spectrum disorder.

Sensory experience during early sensitive periods shapes cross-modal temporal biases

Typical human perception features stable biases such as perceiving visual events as later than synchronous auditory events. The origin of such perceptual biases is unknown. To investigate the role of early sensory experience, we tested whether a congenital, transient loss of pattern vision, caused by bilateral dense cataracts, has sustained effects on audio-visual and tactile-visual temporal biases and resolution. Participants judged the temporal order of successively presented, spatially separated events within and across modalities. Individuals with reversed congenital cataracts showed a bias towards perceiving visual stimuli as occurring earlier than auditory (Expt. 1) and tactile (Expt. 2) stimuli. This finding stood in stark contrast to normally sighted controls and sight-recovery individuals who had developed cataracts later in childhood: both groups exhibited the typical bias of perceiving vision as delayed compared to audition. These findings provide strong evidence that cross-modal temporal biases depend on sensory experience during an early sensitive period.

Rapid Recalibration of Peri-Personal Space: Psychophysical, Electrophysiological, and Neural Network Modeling Evidence

Interactions between individuals and the environment occur within the peri-personal space (PPS). The encoding of this space plastically adapts to bodily constraints and stimuli features. However, these remapping effects have not been demonstrated on an adaptive time-scale, trial-to-trial. Here, we test this idea first via a visuo-tactile reaction time (RT) paradigm in augmented reality where participants are asked to respond as fast as possible to touch, as visual objects approach them. Results demonstrate that RTs to touch are facilitated as a function of visual proximity, and the sigmoidal function describing this facilitation shifts closer to the body if the immediately precedent trial had indexed a smaller visuo-tactile disparity. Next, we derive the electroencephalographic correlates of PPS and demonstrate that this multisensory measure is equally shaped by recent sensory history. Finally, we demonstrate that a validated neural network model of PPS is able to account for the present results via a simple Hebbian plasticity rule. The present findings suggest that PPS encoding remaps on a very rapid time-scale and, more generally, that it is sensitive to sensory history, a key feature for any process contextualizing subsequent incoming sensory information (e.g., a Bayesian prior).

Efficiency of Sensory Substitution Devices Alone and in Combination With Self-Motion for Spatial Navigation in Sighted and Visually Impaired

Human adults can optimally combine vision with self-motion to facilitate navigation. In the absence of visual input (e.g., dark environments and visual impairments), sensory substitution devices (SSDs), such as The vOICe or BrainPort, which translate visual information into auditory or tactile information, could be used to increase navigation precision when integrated together or with self-motion. In Experiment 1, we compared and assessed together The vOICe and BrainPort in aerial maps task performed by a group of sighted participants. In Experiment 2, we examined whether sighted individuals and a group of visually impaired (VI) individuals could benefit from using The vOICe, with and without self-motion, to accurately navigate a three-dimensional (3D) environment. In both studies, 3D motion tracking data were used to determine the level of precision with which participants performed two different tasks (an egocentric and an allocentric task) and three different conditions (two unisensory conditions and one multisensory condition). In Experiment 1, we found no benefit of using the devices together. In Experiment 2, the sighted performance during The vOICe was almost as good as that for self-motion despite a short training period, although we found no benefit (reduction in variability) of using The vOICe and self-motion in combination compared to the two in isolation. In contrast, the group of VI participants did benefit from combining The vOICe and self-motion despite the low number of trials. Finally, while both groups became more accurate in their use of The vOICe with increased trials, only the VI group showed an increased level of accuracy in the combined condition. Our findings highlight how exploiting non-visual multisensory integration to develop new assistive technologies could be key to help blind and VI persons, especially due to their difficulty in attaining allocentric information.

Processing of Real-World, Dynamic Natural Stimuli in Autism is Linked to Corticobasal Function

Many individuals with autism spectrum disorder (ASD) have been shown to perceive everyday sensory information differently compared to peers without autism. Research examining these sensory differences has primarily utilized nonnatural stimuli or natural stimuli using static photos with few having utilized dynamic, real-world nonverbal stimuli. Therefore, in this study, we used functional magnetic resonance imaging to characterize brain activation of individuals with high-functioning autism when viewing and listening to a video of a real-world scene (a person bouncing a ball) and anticipating the bounce. We investigated both multisensory and unisensory processing and hypothesized that individuals with ASD would show differential activation in (a) primary auditory and visual sensory cortical and association areas, and in (b) cortical and subcortical regions where auditory and visual information is integrated (e.g. temporal-parietal junction, pulvinar, superior colliculus). Contrary to our hypotheses, the whole-brain analysis revealed similar activation between the groups in these brain regions. However, compared to controls the ASD group showed significant hypoactivation in the left intraparietal sulcus and left putamen/globus pallidus. We theorize that this hypoactivation reflected underconnectivity for mediating spatiotemporal processing of the visual biological motion stimuli with the task demands of anticipating the timing of the bounce event. The paradigm thus may have tapped into a specific left-lateralized aberrant corticobasal circuit or loop involved in initiating or inhibiting motor responses. This was consistent with a dual "when versus where" psychophysical model of corticobasal function, which may reflect core differences in sensory processing of real-world, nonverbal natural stimuli in ASD. Autism Res 2020, 13: 539-549. © 2020 International Society for Autism Research, Wiley Periodicals, Inc. LAY SUMMARY: To understand how individuals with autism perceive the real-world, using magnetic resonance imaging we examined brain activation in individuals with autism while watching a video of someone bouncing a basketball. Those with autism had similar activation to controls in auditory and visual sensory brain regions, but less activation in an area that processes information about body movements and in a region involved in modulating movements. These areas are important for understanding the actions of others and developing social skills.

Audiovisual temporal integration: Cognitive processing, neural mechanisms, developmental trajectory and potential interventions

To integrate auditory and visual signals into a unified percept, the paired stimuli must co-occur within a limited time window known as the Temporal Binding Window (TBW). The width of the TBW, a proxy of audiovisual temporal integration ability, has been found to be correlated with higher-order cognitive and social functions. A comprehensive review of studies investigating audiovisual TBW reveals several findings: (1) a wide range of top-down processes and bottom-up features can modulate the width of the TBW, facilitating adaptation to the changing and multisensory external environment; (2) a large-scale brain network works in coordination to ensure successful detection of audiovisual (a)synchrony; (3) developmentally, audiovisual TBW follows a U-shaped pattern across the lifespan, with a protracted developmental course into late adolescence and rebounding in size again in late life; (4) an enlarged TBW is characteristic of a number of neurodevelopmental disorders; and (5) the TBW is highly flexible via perceptual and musical training. Interventions targeting the TBW may be able to improve multisensory function and ameliorate social communicative symptoms in clinical populations.

Time attracts auditory space representation during development

Vision is the most accurate sense for spatial representation, whereas audition is for temporal representation. However, how different sensory modalities shape the development of spatial and temporal representations is still unclear. Here, 45 children aged 11-13 years were tested to investigate the abilities to evaluate spatial features of auditory stimuli during bisection tasks, while conflicting or non-conflicting spatial and temporal information was delivered. Since audition is fundamental for temporal representation, the hypothesis was that temporal information could influence auditory spatial representation development. Results show a strong interaction between the temporal and the spatial domain. Younger children are not able to build complex spatial representations when the temporal domain is uninformative about space. However, when the spatial information is coherent with the temporal information children of all age are able to decode complex spatial relationships. When spatial and temporal cues are conflicting, younger children are strongly attracted by the temporal instead of spatial information, while older participants result unaffected by the cross-domain conflict. These findings suggest that during development temporal representation of events is used to infer spatial coordinates of the environment, offering important opportunities for new teaching and rehabilitation strategies.

Age-related sensory decline mediates the Sound-Induced Flash Illusion: Evidence for reliability weighting models of multisensory perception

Perception of our world is proposed to arise from combining multiple sensory inputs according to their relative reliability. We tested multisensory processes in a large sample of 2920 older adults to assess whether sensory ability mediates age-related changes in perception. Participants completed a test of audio-visual integration, the Sound Induced Flash Illusion (SIFI), alongside measures of visual (acuity, contrast sensitivity, self-reported vision and visual temporal discrimination (VTD)) and auditory (self-reported hearing and auditory temporal discrimination (ATD)) function. Structural equation modelling showed that SIFI susceptibility increased with age. This was mediated by visual acuity and self-reported hearing: better scores on these measures predicted reduced and stronger SIFI susceptibility, respectively. Unexpectedly, VTD improved with age and predicted increased SIFI susceptibility. Importantly, the relationship between age and SIFI susceptibility remained significant, even when considering mediators. A second model showed that, with age, visual 'gain' (the benefit of congruent auditory information on visual judgements) was predicted by ATD: better ATD predicted stronger visual gain. However, neither age nor SIFI susceptibility were directly associated with visual gain. Our findings illustrate, in the largest sample of older adults to date, how multisensory perception is influenced, but not fully accounted for, by age-related changes in unisensory abilities.

Audiovisual temporal integration and rapid temporal recalibration in adolescents and adults: Age-related changes and its correlation with autistic traits

Temporal structure is a key factor in determining the relatedness of multisensory stimuli. Stimuli that are close in time are more likely to be integrated into a unified perceptual representation. To investigate the age-related developmental differences in audiovisual temporal integration and rapid temporal recalibration, we administered simultaneity judgment (SJ) tasks to a group of adolescents (11-14 years) and young adults (18-28 years). No age-related changes were found in the width of the temporal binding window within which participants are highly likely to combine multisensory stimuli. The main distinction between adolescents and adults was audiovisual temporal recalibration. Although participants of both age groups could rapidly recalibrate based on the previous trial for speech stimuli (i.e., syllable utterances), only adults but not adolescents showed short-term recalibration for simple and non-speech stimuli. In both adolescents and adults, no significant correlation was found between audiovisual temporal integration ability and autistic or schizotypal traits. These findings provide new information on the developmental trajectory of basic multisensory function and may have implications for neurodevelopmental disorders (e.g., autism) with altered audiovisual temporal integration. Autism Res 2020, 13: 615-626. © 2019 International Society for Autism Research, Wiley Periodicals, Inc. LAY SUMMARY: Utilizing temporal cues to integrate and separate audiovisual information is a fundamental ability underlying higher order social communicative functions. This study examines the developmental changes of the ability to detect audiovisual asynchrony and rapidly adjust sensory decisions based on previous sensory input. In healthy adolescents and young adults, the correlation between autistic traits and audiovisual integration ability failed to reach a significant level. Therefore, more research is needed to examine whether impairment in basic sensory functions is correlated with broader autism phenotype in nonclinical populations. These results may help us understand altered multisensory integration in people with autism.

Consistent inter-individual differences in susceptibility to bodily illusions

Illusory senses of ownership and agency (that the hand or effector that we see belongs to us and moves at our will, respectively) support the embodiment of prosthetic limbs, tele-operated surgical devices, and human-machine interfaces. We exposed forty-eight individuals to four different procedures known to elicit illusory ownership or agency over a fake visible rubber hand or finger. The illusory ownership or agency arising from the hand correlated with that of the finger. For both body parts, sensory stimulation across different modalities (visual with tactile or visual with kinesthetic) produced illusions of similar strength. However, the strengths of the illusions of ownership and agency were unrelated within individuals, supporting the proposal that distinct neuropsychological processes underlie these two senses. Developing training programs to enhance susceptibility to illusions of agency or ownership for people with lower natural susceptibility could broaden the usefulness of the above technologies.

Dissociating the sequential dependency of subjective temporal order from subjective simultaneity

The physical simultaneity between two events can differ from our point of subjective simultaneity (PSS). Studies using simultaneity judgments (SJ) and temporal order judgments (TOJ) tasks have shown that whether two events are reported as simultaneous is highly context-dependent. It has been recently suggested that the interval between the two events in the previous trial can modulate judgments both in SJ and TOJ tasks, an effect named rapid recalibration. In this work, we investigated rapid recalibration in SJ and TOJ tasks and tested whether centering the range of presented intervals on perceived simultaneity modulated this effect. We found a rapid recalibration effect in TOJ, but not in SJ. Moreover, we found that centering the intervals on objective or subjective simultaneity did not change the pattern of results. Interestingly, we also found no correlations between an individual’s PSS in TOJ and in SJ tasks, which corroborates other studies in suggesting that these two psychophysical measures may capture different processes.

Cross-modal interference-control is reduced in childhood but maintained in aging: A cohort study of stimulus- and response-interference in cross-modal and unimodal Stroop tasks

Interference-control is the ability to exclude distractions and focus on a specific task or stimulus. However, it is currently unclear whether the same interference-control mechanisms underlie the ability to ignore unimodal and cross-modal distractions. In 2 experiments we assessed whether unimodal and cross-modal interference follow similar trajectories in development and aging and occur at similar processing levels. In Experiment 1, 42 children (6-11 years), 31 younger adults (18-25 years) and 32 older adults (60-84 years) identified color rectangles with either written (unimodal) or spoken (cross-modal) distractor-words. Stimuli could be congruent, incongruent but mapped to the same response (stimulus-incongruent), or incongruent and mapped to different responses (response-incongruent); thus, separating interference occurring at early (sensory) and late (response) processing levels. Unimodal interference was worst in childhood and old age; however, older adults maintained the ability to ignore cross-modal distraction. Unimodal but not cross-modal response-interference also reduced accuracy. In Experiment 2 we compared the effect of audition on vision and vice versa in 52 children (6-11 years), 30 young adults (22-33 years) and 30 older adults (60-84 years). As in Experiment 1, older adults maintained the ability to ignore cross-modal distraction arising from either modality, and neither type of cross-modal distraction limited accuracy in adults. However, cross-modal distraction still reduced accuracy in children and children were more slowed by stimulus-interference compared with adults. We conclude that; unimodal and cross-modal interference follow different life span trajectories and differences in stimulus- and response-interference may increase cross-modal distractibility in childhood. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Rapid recalibration to audiovisual asynchrony follows the physical-not the perceived-temporal order

In natural scenes, audiovisual events deriving from the same source are synchronized at their origin. However, from the perspective of the observer, there are likely to be significant multisensory delays due to physical and neural latencies. Fortunately, our brain appears to compensate for the resulting latency differences by rapidly adapting to asynchronous audiovisual events by shifting the point of subjective synchrony (PSS) in the direction of the leading modality of the most recent event. Here we examined whether it is the perceived modality order of this prior lag or its physical order that determines the direction of the subsequent rapid recalibration. On each experimental trial, a brief tone pip and flash were presented across a range of stimulus onset asynchronies (SOAs). The participants' task alternated over trials: On adaptor trials, audition either led or lagged vision with fixed SOAs, and participants judged the order of the audiovisual event; on test trials, the SOA as well as the modality order varied randomly, and participants judged whether or not the event was synchronized. For test trials, we showed that the PSS shifted in the direction of the physical rather than the perceived (reported) modality order of the preceding adaptor trial. These results suggest that rapid temporal recalibration is determined by the physical timing of the preceding events, not by one's prior perceptual decisions.

Audiovisual Temporal Perception in Aging: The Role of Multisensory Integration and Age-Related Sensory Loss

Within each sensory modality, age-related deficits in temporal perception contribute to the difficulties older adults experience when performing everyday tasks. Since perceptual experience is inherently multisensory, older adults also face the added challenge of appropriately integrating or segregating the auditory and visual cues present in our dynamic environment into coherent representations of distinct objects. As such, many studies have investigated how older adults perform when integrating temporal information across audition and vision. This review covers both direct judgments about temporal information (the sound-induced flash illusion, temporal order, perceived synchrony, and temporal rate discrimination) and judgments regarding stimuli containing temporal information (the audiovisual bounce effect and speech perception). Although an age-related increase in integration has been demonstrated on a variety of tasks, research specifically investigating the ability of older adults to integrate temporal auditory and visual cues has produced disparate results. In this short review, we explore what factors could underlie these divergent findings. We conclude that both task-specific differences and age-related sensory loss play a role in the reported disparity in age-related effects on the integration of auditory and visual temporal information.

Atypical audiovisual temporal function in autism and schizophrenia: similar phenotype, different cause

Binding across sensory modalities yields substantial perceptual benefits, including enhanced speech intelligibility. The coincidence of sensory inputs across time is a fundamental cue for this integration process. Recent work has suggested that individuals with diagnoses of schizophrenia (SZ) and autism spectrum disorder (ASD) will characterize auditory and visual events as synchronous over larger temporal disparities than their neurotypical counterparts. Namely, these clinical populations possess an enlarged temporal binding window (TBW). Although patients with SZ and ASD share aspects of their symptomatology, phenotypic similarities may result from distinct etiologies. To examine similarities and variances in audiovisual temporal function in these two populations, individuals diagnosed with ASD (n = 46; controls n = 40) and SZ (n = 16, controls = 16) completed an audiovisual simultaneity judgment task. In addition to standard psychometric analyses, synchrony judgments were assessed using Bayesian causal inference modeling. This approach permits distinguishing between distinct causes of an enlarged TBW: an a priori bias to bind sensory information and poor fidelity in the sensory representation. Findings indicate that both ASD and SZ populations show deficits in multisensory temporal acuity. Importantly, results suggest that while the wider TBWs in ASD most prominently results from atypical priors, the wider TBWs in SZ results from a trend toward changes in prior and weaknesses in the sensory representations. Results are discussed in light of current ASD and SZ theories and highlight that different perceptual training paradigms focused on improving multisensory integration may be most effective in these two clinical populations and emphasize that similar phenotypes may emanate from distinct mechanistic causes.

Audiovisual integration in depth: multisensory binding and gain as a function of distance

The integration of information across sensory modalities is dependent on the spatiotemporal characteristics of the stimuli that are paired. Despite large variation in the distance over which events occur in our environment, relatively little is known regarding how stimulus-observer distance affects multisensory integration. Prior work has suggested that exteroceptive stimuli are integrated over larger temporal intervals in near relative to far space, and that larger multisensory facilitations are evident in far relative to near space. Here, we sought to examine the interrelationship between these previously established distance-related features of multisensory processing. Participants performed an audiovisual simultaneity judgment and redundant target task in near and far space, while audiovisual stimuli were presented at a range of temporal delays (i.e., stimulus onset asynchronies). In line with the previous findings, temporal acuity was poorer in near relative to far space. Furthermore, reaction time to asynchronously presented audiovisual targets suggested a temporal window for fast detection-a range of stimuli asynchronies that was also larger in near as compared to far space. However, the range of reaction times over which multisensory response enhancement was observed was limited to a restricted range of relatively small (i.e., 150 ms) asynchronies, and did not differ significantly between near and far space. Furthermore, for synchronous presentations, these distance-related (i.e., near vs. far) modulations in temporal acuity and multisensory gain correlated negatively at an individual subject level. Thus, the findings support the conclusion that multisensory temporal binding and gain are asymmetrically modulated as a function of distance from the observer, and specifies that this relationship is specific for temporally synchronous audiovisual stimulus presentations.

Long-term music training modulates the recalibration of audiovisual simultaneity

To overcome differences in physical transmission time and neural processing, the brain adaptively recalibrates the point of simultaneity between auditory and visual signals by adapting to audiovisual asynchronies. Here, we examine whether the prolonged recalibration process of passively sensed visual and auditory signals is affected by naturally occurring multisensory training known to enhance audiovisual perceptual accuracy. Hence, we asked a group of drummers, of non-drummer musicians and of non-musicians to judge the audiovisual simultaneity of musical and non-musical audiovisual events, before and after adaptation with two fixed audiovisual asynchronies. We found that the recalibration for the musicians and drummers was in the opposite direction (sound leading vision) to that of non-musicians (vision leading sound), and change together with both increased music training and increased perceptual accuracy (i.e. ability to detect asynchrony). Our findings demonstrate that long-term musical training reshapes the way humans adaptively recalibrate simultaneity between auditory and visual signals.

Disrupted integration of exteroceptive and interoceptive signaling in autism spectrum disorder

In addition to deficits in social communication, individuals diagnosed with Autism Spectrum Disorder (ASD) frequently exhibit changes in sensory and multisensory function. Recent evidence has focused on changes in audiovisual temporal processing, and has sought to relate these sensory-based changes to weaknesses in social communication. These changes in audiovisual temporal function manifest as differences in the temporal epoch or "window" within which paired auditory and visual stimuli are integrated or bound, with those with ASD exhibiting expanded audiovisual temporal binding windows (TBWs). However, it is unknown whether this impairment is unique to audiovisual pairings, perhaps because of their relevance for speech processing, or whether it generalizes across pairings in different sensory modalities. In addition to the exteroceptive senses, there has been growing interest in ASD research in interoception (e.g., the monitoring of respiration, heartbeat, hunger, etc.), as these internally directed sensory processes appear to be altered as well in autism. In the current study, we sought to examine both exteroception and interoception in individuals with ASD and a group of typically developing (TD) matched controls, with an emphasis on temporal perception of audiovisual (exteroceptive) and cardiovisual (interoceptive to exteroceptive) cues. Results replicate prior findings showing expanded audiovisual TBWs in ASD in comparison to TD. In addition, strikingly, cardiovisual TBWs were fourfold larger in ASD than in TD, suggesting a putative complete lack of cardiovisual temporal acuity in ASD individuals. Results are discussed in light of recent evidence indicating a reduced tendency to rely on sensory priors in ASD. Autism Res 2018, 11: 194-205. © 2017 International Society for Autism Research, Wiley Periodicals, Inc.

LAY SUMMARY

Studies have shown that individuals with autism have difficulty in separating auditory and visual events in time. People with autism also weight sensory evidence originating from the external world and from their body differently. We measured simultaneity judgments regarding visual and auditory events and between visual and heartbeat events. Results suggest that while individuals with autism show unusual temporal function across the senses in a general manner, this deficit is greater when pairings bridged between the external world and the internal body.

Multisensory temporal function and EEG complexity in patients with epilepsy and psychogenic nonepileptic events

Cognitive and perceptual comorbidities frequently accompany epilepsy and psychogenic nonepileptic events (PNEE). However, and despite the fact that perceptual function is built upon a multisensory foundation, little knowledge exists concerning multisensory function in these populations. Here, we characterized facets of multisensory processing abilities in patients with epilepsy and PNEE, and probed the relationship between individual resting-state EEG complexity and these psychophysical measures in each patient. We prospectively studied a cohort of patients with epilepsy (N=18) and PNEE (N=20) patients who were admitted to Vanderbilt's Epilepsy Monitoring Unit (EMU) and weaned off of anticonvulsant drugs. Unaffected age-matched persons staying with the patients in the EMU (N=15) were also recruited as controls. All participants performed two tests of multisensory function: an audio-visual simultaneity judgment and an audio-visual redundant target task. Further, in the cohort of patients with epilepsy and PNEE we quantified resting state EEG gamma power and complexity. Compared with both patients with epilepsy and control subjects, patients with PNEE exhibited significantly poorer acuity in audiovisual temporal function as evidenced in significantly larger temporal binding windows (i.e., they perceived larger stimulus asynchronies as being presented simultaneously). These differences appeared to be specific for temporal function, as there was no difference among the three groups in a non-temporally based measure of multisensory function - the redundant target task. Further, patients with PNEE exhibited more complex resting state EEG patterns as compared to their patients with epilepsy, and EEG complexity correlated with multisensory temporal performance on a subject-by-subject manner. Taken together, findings seem to indicate that patients with PNEE bind information from audition and vision over larger temporal intervals when compared with control subjects as well as patients with epilepsy. This difference in multisensory function appears to be specific to the temporal domain, and may be a contributing factor to the behavioral and perceptual alterations seen in this population.

Time Order as Psychological Bias

Incorrectly perceiving the chronology of events can fundamentally alter people's understanding of the causal structure of the world. For example, when astronomers used the "eye and ear" method to locate stars, they showed systematic interindividual errors. In the current study, we showed that temporal-order perception may be considered a psychological bias that attention can modulate but not fully eradicate. According to Titchener's law of prior entry, attention prioritizes the perception of an event and thus can help compensate for possible interindividual differences in the perceived timing of an event by normalizing perception in time. In a longitudinal study, we tested the stability of participants' temporal-order perception across and within sensory modalities, together with the magnitude of the participants' prior-entry effect. All measurements showed the persistence of stable interindividual variability. Crucially, the magnitude of the prior-entry effect was insufficient to compensate for interindividual variability: Conscious time order was systematically subjective, and therefore traceable on an individual basis.

Event Related Potentials Index Rapid Recalibration to Audiovisual Temporal Asynchrony

Asynchronous arrival of multisensory information at the periphery is a ubiquitous property of signals in the natural environment due to differences in the propagation time of light and sound. Rapid adaptation to these asynchronies is crucial for the appropriate integration of these multisensory signals, which in turn is a fundamental neurobiological process in creating a coherent perceptual representation of our dynamic world. Indeed, multisensory temporal recalibration has been shown to occur at the single trial level, yet the mechanistic basis of this rapid adaptation is unknown. Here, we investigated the neural basis of rapid recalibration to audiovisual temporal asynchrony in human participants using a combination of psychophysics and electroencephalography (EEG). Consistent with previous reports, participant's perception of audiovisual temporal synchrony on a given trial (t) was influenced by the temporal structure of stimuli on the previous trial (t-1). When examined physiologically, event related potentials (ERPs) were found to be modulated by the temporal structure of the previous trial, manifesting as late differences (>125 ms post second-stimulus onset) in central and parietal positivity on trials with large stimulus onset asynchronies (SOAs). These findings indicate that single trial adaptation to audiovisual temporal asynchrony is reflected in modulations of late evoked components that have previously been linked to stimulus evaluation and decision-making.

The Impact of Feedback on the Different Time Courses of Multisensory Temporal Recalibration

The capacity to rapidly adjust perceptual representations confers a fundamental advantage when confronted with a constantly changing world. Unexplored is how feedback regarding sensory judgments (top-down factors) interacts with sensory statistics (bottom-up factors) to drive long- and short-term recalibration of multisensory perceptual representations. Here, we examined the time course of both cumulative and rapid temporal perceptual recalibration for individuals completing an audiovisual simultaneity judgment task in which they were provided with varying degrees of feedback. We find that in the presence of feedback (as opposed to simple sensory exposure) temporal recalibration is more robust. Additionally, differential time courses are seen for cumulative and rapid recalibration dependent upon the nature of the feedback provided. Whereas cumulative recalibration effects relied more heavily on feedback that informs (i.e., negative feedback) rather than confirms (i.e., positive feedback) the judgment, rapid recalibration shows the opposite tendency. Furthermore, differential effects on rapid and cumulative recalibration were seen when the reliability of feedback was altered. Collectively, our findings illustrate that feedback signals promote and sustain audiovisual recalibration over the course of cumulative learning and enhance rapid trial-to-trial learning. Furthermore, given the differential effects seen for cumulative and rapid recalibration, these processes may function via distinct mechanisms.

The spatial self in schizophrenia and autism spectrum disorder

Schizophrenia (SZ) and autism spectrum disorder (ASD) have been both described as disorders of the self. However, the manner in which the sense of self is impacted in these disorders is strikingly different. In the current review, we propose that SZ and ASD lay at opposite extremes of a particular component of the representation of self; namely, self-location and the construct of peripersonal space. We evaluate emerging literature suggesting that while SZ individuals possess an extremely weak or variable bodily boundary between self and other, ASD patients possess a sharper self-other boundary. Furthermore, based on recent behavioral and neural network modeling findings, we propose that multisensory training focused on either sharpening (for SZ) or making shallower (for ASD) the self-other boundary may hold promise as an interventional tool in the treatment of these disorders.