Why seeing is believing: merging auditory and visual worlds

The audiovisual competition effect induced by temporal asynchronous encoding weakened the visual dominance in working memory retrieval

Converging evidence suggests a facilitation effect of multisensory interactions on memory performance, reflected in higher accuracy or faster response time under a bimodal encoding condition than a unimodal condition. However, relatively little attention has been given to the effect of multisensory competition on memory. The present study adopted an adaptive staircase test to measure the point of subjective simultaneity (PSS), combined with a delayed matched-to-sample (DMS) task to probe the effect of audiovisual competition during the encoding stage on subsequent unisensory retrieval. The results showed that there was a robust visual dominance effect and multisensory interference effect in WM retrieval, regardless of the subjective synchronous or subjective asynchronous audiovisual presentation. However, a weakened visual dominance effect was observed when the auditory stimulus was presented before the visual stimulus in the encoding period, particularly in the semantically incongruent case. These findings revealed that the prior-entry of sensory information in the early perceptual stage could affect the processing in the late cognitive stage to some extent, and supported the evidence that there is a persistent advantage for visuospatial sketchpad in multisensory WM.

Insights of healthcare professionals regarding waterbirths and water immersion during labour: A mixed studies review

AIM

To consolidate healthcare professionals' insights about waterbirths and water immersion during labour.

DESIGN

Mixed studies review.

DATA SOURCES

Seven electronic databases were searched from their inception dates till June 2023: PubMed, Embase, CINAHL, PsycINFO, Web of Science, Scopus, ProQuest Dissertations and Theses Global.

METHODS

The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines and Pluye and Hong's mixed studies review framework guided this review. The quality of included studies was evaluated using the Mixed Methods Appraisal Tool. Findings were synthesized using the convergent qualitative synthesis method, and results were thematically analysed using Braun and Clarke's framework.

RESULTS

Three main themes were identified from the 22 included studies: (1) believing in waterbirths, (2) opposing forces and (3) plotting the course ahead.

CONCLUSION

Healthcare professionals reported different views about waterbirths and water immersion practices; midwives were most likely to support these practices, followed by nurses and lastly, few physicians supported them. Reasons for opposing waterbirths include insufficient training and support from colleagues as well as concerns about work efficiency, waterbirth safety and litigation issues.

IMPACT

The available evidence suggests the need to provide waterbirth training for healthcare professionals, equip healthcare facilities with necessary waterbirth-related infrastructure and develop appropriate waterbirth policies/guidelines. Healthcare professionals could also consider providing antenatal waterbirth education to women and obtain women's feedback to improve current policies/guidelines. Future research should explore the views of different types of healthcare professionals from more diverse cultures.

REPORTING METHOD

The PRISMA guidelines.

NO PATIENT OR PUBLIC CONTRIBUTION

Systematic review.

The development of audiovisual speech perception in Mandarin-speaking children: Evidence from the McGurk paradigm

The developmental trajectory of audiovisual speech perception in Mandarin-speaking children remains understudied. This cross-sectional study in Mandarin-speaking 3- to 4-year-old, 5- to 6-year-old, 7- to 8-year-old children, and adults from Xiamen, China (n = 87, 44 males) investigated this issue using the McGurk paradigm with three levels of auditory noise. For the identification of congruent stimuli, 3- to 4-year-olds underperformed older groups whose performances were comparable. For the perception of the incongruent stimuli, a developmental shift was observed as 3- to 4-year-olds made significantly more audio-dominant but fewer audiovisual-integrated responses to incongruent stimuli than older groups. With increasing auditory noise, the difference between children and adults widened in identifying congruent stimuli but narrowed in perceiving incongruent ones. The findings regarding noise effects agree with the statistically optimal hypothesis.

Auditory cortex conveys non-topographic sound localization signals to visual cortex

Spatiotemporally congruent sensory stimuli are fused into a unified percept. The auditory cortex (AC) sends projections to the primary visual cortex (V1), which could provide signals for binding spatially corresponding audio-visual stimuli. However, whether AC inputs in V1 encode sound location remains unknown. Using two-photon axonal calcium imaging and a speaker array, we measured the auditory spatial information transmitted from AC to layer 1 of V1. AC conveys information about the location of ipsilateral and contralateral sound sources to V1. Sound location could be accurately decoded by sampling AC axons in V1, providing a substrate for making location-specific audiovisual associations. However, AC inputs were not retinotopically arranged in V1, and audio-visual modulations of V1 neurons did not depend on the spatial congruency of the sound and light stimuli. The non-topographic sound localization signals provided by AC might allow the association of specific audiovisual spatial patterns in V1 neurons.

Multisensory Integration in Body Representation

To be aware of and to move one's body, the brain must maintain a coherent representation of the body. While the body and the brain are connected by dense ascending and descending sensory and motor pathways, representation of the body is not hardwired. This is demonstrated by the well-known rubber hand illusion in which a visible fake hand is erroneously felt as one's own hand when it is stroked in synchrony with the viewer's unseen actual hand. Thus, body representation in the brain is not mere maps of tactile and proprioceptive inputs, but a construct resulting from the interpretation and integration of inputs across sensory modalities.

Multisensory Calibration: A Variety of Slow and Fast Brain Processes Throughout the Lifespan

From before we are born, throughout development, adulthood, and aging, we are immersed in a multisensory world. At each of these stages, our sensory cues are constantly changing, due to body, brain, and environmental changes. While integration of information from our different sensory cues improves precision, this only improves accuracy if the underlying cues are unbiased. Thus, multisensory calibration is a vital and ongoing process. To meet this grand challenge, our brains have evolved a variety of mechanisms. First, in response to a systematic discrepancy between sensory cues (without external feedback) the cues calibrate one another (unsupervised calibration). Second, multisensory function is calibrated to external feedback (supervised calibration). These two mechanisms superimpose. While the former likely reflects a lower level mechanism, the latter likely reflects a higher level cognitive mechanism. Indeed, neural correlates of supervised multisensory calibration in monkeys were found in higher level multisensory cortical area VIP, but not in the relatively lower level multisensory area MSTd. In addition, even without a cue discrepancy (e.g., when experiencing stimuli from different sensory cues in series) the brain monitors supra-modal statistics of events in the environment and adapts perception cross-modally. This too comprises a variety of mechanisms, including confirmation bias to prior choices, and lower level cross-sensory adaptation. Further research into the neuronal underpinnings of the broad and diverse functions of multisensory calibration, with improved synthesis of theories is needed to attain a more comprehensive understanding of multisensory brain function.

Multisensory Integration and Causal Inference in Typical and Atypical Populations

Multisensory perception is critical for effective interaction with the environment, but human responses to multisensory stimuli vary across the lifespan and appear changed in some atypical populations. In this review chapter, we consider multisensory integration within a normative Bayesian framework. We begin by outlining the complex computational challenges of multisensory causal inference and reliability-weighted cue integration, and discuss whether healthy young adults behave in accordance with normative Bayesian models. We then compare their behaviour with various other human populations (children, older adults, and those with neurological or neuropsychiatric disorders). In particular, we consider whether the differences seen in these groups are due only to changes in their computational parameters (such as sensory noise or perceptual priors), or whether the fundamental computational principles (such as reliability weighting) underlying multisensory perception may also be altered. We conclude by arguing that future research should aim explicitly to differentiate between these possibilities.

The perception of auditory motion in sighted and early blind individuals

Motion perception is a fundamental sensory task that plays a critical evolutionary role. In vision, motion processing is classically described using a motion energy model with spatiotemporally nonseparable filters suited for capturing the smooth continuous changes in spatial position over time afforded by moving objects. However, it is still not clear whether the filters underlying auditory motion discrimination are also continuous motion detectors or infer motion from comparing discrete sound locations over time (spatiotemporally separable). We used a psychophysical reverse correlation paradigm, where participants discriminated the direction of a motion signal in the presence of spatiotemporal noise, to determine whether the filters underlying auditory motion discrimination were spatiotemporally separable or nonseparable. We then examined whether these auditory motion filters were altered as a result of early blindness. We found that both sighted and early blind individuals have separable filters. However, early blind individuals show increased sensitivity to auditory motion, with reduced susceptibility to noise and filters that were more accurate in detecting motion onsets/offsets. Model simulations suggest that this reliance on separable filters is optimal given the limited spatial resolution of auditory input.

The Visual Advantage Effect in Comparing Uni-Modal and Cross-Modal Probabilistic Category Learning

People rely on multiple learning systems to complete weather prediction (WP) tasks with visual cues. However, how people perform in audio and audiovisual modalities remains elusive. The present research investigated how the cue modality influences performance in probabilistic category learning and conscious awareness about the category knowledge acquired. A modified weather prediction task was adopted, in which the cues included two dimensions from visual, auditory, or audiovisual modalities. The results of all three experiments revealed better performances in the visual modality relative to the audio and audiovisual modalities. Moreover, participants primarily acquired unconscious knowledge in the audio and audiovisual modalities, while conscious knowledge was acquired in the visual modality. Interestingly, factors such as the amount of training, the complexity of visual stimuli, and the number of objects to which the two cues belonged influenced the amount of conscious knowledge acquired but did not change the visual advantage effect. These findings suggest that individuals can learn probabilistic cues and category associations across different modalities, but a robust visual advantage persists. Specifically, visual associations can be learned more effectively, and are more likely to become conscious. The possible causes and implications of these effects are discussed.

"Rear bias" in spatial auditory perception: Attentional and affective vigilance to sounds occurring outside the visual field

Presented studies explored the rear bias phenomenon, that is, the attentional and affective bias to sounds occurring behind the listener. Physiological and psychological reactions (i.e., fEMG, EDA/SCR, Simple Reaction Task-SRT, and self-assessments of affect-related states) were measured in response to tones of different frequencies (Study 1) and emotional vocalizations (Study 2) presented in rear and front spatial locations. Results showed that emotional vocalizations, when located in the back, facilitate reactions related to attention orientation (i.e., auricularis muscle response and simple reaction times) and evoke higher arousal-both physiological (as measured by SCR) and psychological (self-assessment scale). Importantly, observed asymmetries were larger for negative and threat-related signals (e.g., anger) than positive/nonthreatening ones (e.g., achievement). By contrast, there were only small differences for the relatively higher frequency tones. The observed relationships are discussed in terms of one of the postulated auditory system's functions, which is monitoring of the environment in order to quickly detect potential threats that occur outside of the visual field (e.g., behind one's back).

Mouse frontal cortex mediates additive multisensory decisions

The brain can combine auditory and visual information to localize objects. However, the cortical substrates underlying audiovisual integration remain uncertain. Here, we show that mouse frontal cortex combines auditory and visual evidence; that this combination is additive, mirroring behavior; and that it evolves with learning. We trained mice in an audiovisual localization task. Inactivating frontal cortex impaired responses to either sensory modality, while inactivating visual or parietal cortex affected only visual stimuli. Recordings from >14,000 neurons indicated that after task learning, activity in the anterior part of frontal area MOs (secondary motor cortex) additively encodes visual and auditory signals, consistent with the mice's behavioral strategy. An accumulator model applied to these sensory representations reproduced the observed choices and reaction times. These results suggest that frontal cortex adapts through learning to combine evidence across sensory cortices, providing a signal that is transformed into a binary decision by a downstream accumulator.

Intensive Training of Spatial Hearing Promotes Auditory Abilities of Bilateral Cochlear Implant Adults: A Pilot Study

OBJECTIVE

The aim of this study was to evaluate the feasibility of a virtual reality-based spatial hearing training protocol in bilateral cochlear implant (CI) users and to provide pilot data on the impact of this training on different qualities of hearing.

DESIGN

Twelve bilateral CI adults aged between 19 and 69 followed an intensive 10-week rehabilitation program comprised eight virtual reality training sessions (two per week) interspersed with several evaluation sessions (2 weeks before training started, after four and eight training sessions, and 1 month after the end of training). During each 45-minute training session, participants localized a sound source whose position varied in azimuth and/or in elevation. At the start of each trial, CI users received no information about sound location, but after each response, feedback was given to enable error correction. Participants were divided into two groups: a multisensory feedback group (audiovisual spatial cue) and an unisensory group (visual spatial cue) who only received feedback in a wholly intact sensory modality. Training benefits were measured at each evaluation point using three tests: 3D sound localization in virtual reality, the French Matrix test, and the Speech, Spatial and other Qualities of Hearing questionnaire.

RESULTS

The training was well accepted and all participants attended the whole rehabilitation program. Four training sessions spread across 2 weeks were insufficient to induce significant performance changes, whereas performance on all three tests improved after eight training sessions. Front-back confusions decreased from 32% to 14.1% ( p = 0.017); speech recognition threshold score from 1.5 dB to -0.7 dB signal-to-noise ratio ( p = 0.029) and eight CI users successfully achieved a negative signal-to-noise ratio. One month after the end of structured training, these performance improvements were still present, and quality of life was significantly improved for both self-reports of sound localization (from 5.3 to 6.7, p = 0.015) and speech understanding (from 5.2 to 5.9, p = 0.048).

CONCLUSIONS

This pilot study shows the feasibility and potential clinical relevance of this type of intervention involving a sensorial immersive environment and could pave the way for more systematic rehabilitation programs after cochlear implantation.

Variables influencing conditioning-evoked hallucinations: overview and future applications

Hallucinations occur in the absence of sensory stimulation and result in vivid perceptual experiences of nonexistent events that manifest across a range of sensory modalities. Approaches from the field of experimental and cognitive psychology have leveraged the idea that associative learning experiences can evoke conditioning-induced hallucinations in both animals and humans. In this review, we describe classical and contemporary findings and highlight the variables eliciting these experiences. We also provide an overview of the neurobiological mechanisms, along with the associative and computational factors that may explain hallucinations that are generated by representation-mediated conditioning phenomena. Through the integration of animal and human research, significant advances into the psychobiology of hallucinations are possible, which may ultimately translate to more effective clinical applications.

A Dynamic Neural Field Model of Multimodal Merging: Application to the Ventriloquist Effect

Multimodal merging encompasses the ability to localize stimuli based on imprecise information sampled through individual senses such as sight and hearing. Merging decisions are standardly described using Bayesian models that fit behaviors over many trials, encapsulated in a probability distribution. We introduce a novel computational model based on dynamic neural fields able to simulate decision dynamics and generate localization decisions, trial by trial, adapting to varying degrees of discrepancy between audio and visual stimulations. Neural fields are commonly used to model neural processes at a mesoscopic scale-for instance, neurophysiological activity in the superior colliculus. Our model is fit to human psychophysical data of the ventriloquist effect, additionally testing the influence of retinotopic projection onto the superior colliculus and providing a quantitative performance comparison to the Bayesian reference model. While models perform equally on average, a qualitative analysis of free parameters in our model allows insights into the dynamics of the decision and the individual variations in perception caused by noise. We finally show that the increase in the number of free parameters does not result in overfitting and that the parameter space may be either reduced to fit specific criteria or exploited to perform well on more demanding tasks in the future. Indeed, beyond decision or localization tasks, our model opens the door to the simulation of behavioral dynamics, as well as saccade generation driven by multimodal stimulation.

Robust Coding of Eye Position in Posterior Parietal Cortex despite Context-Dependent Tuning

Neurons in posterior parietal cortex (PPC) encode many aspects of the sensory world (e.g., scene structure), the posture of the body, and plans for action. For a downstream computation, however, only some of these dimensions are relevant; the rest are "nuisance variables" because their influence on neural activity changes with sensory and behavioral context, potentially corrupting the read-out of relevant information. Here we show that a key postural variable for vision (eye position) is represented robustly in male macaque PPC across a range of contexts, although the tuning of single neurons depended strongly on context. Contexts were defined by different stages of a visually guided reaching task, including (1) a visually sparse epoch, (2) a visually rich epoch, (3) a "go" epoch in which the reach was cued, and (4) during the reach itself. Eye position was constant within trials but varied across trials in a 3 × 3 grid spanning 24° × 24°. Using demixed principal component analysis of neural spike-counts, we found that the subspace of the population response encoding eye position is orthogonal to that encoding task context. Accordingly, a context-naive (fixed-parameter) decoder was nevertheless able to estimate eye position reliably across contexts. Errors were small given the sample size (∼1.78°) and would likely be even smaller with larger populations. Moreover, they were comparable to that of decoders that were optimized for each context. Our results suggest that population codes in PPC shield encoded signals from crosstalk to support robust sensorimotor transformations across contexts.SIGNIFICANCE STATEMENT Neurons in posterior parietal cortex (PPC) which are sensitive to gaze direction are thought to play a key role in spatial perception and behavior (e.g., reaching, navigation), and provide a potential substrate for brain-controlled prosthetics. Many, however, change their tuning under different sensory and behavioral contexts, raising the prospect that they provide unreliable representations of egocentric space. Here, we analyze the structure of encoding dimensions for gaze direction and context in PPC during different stages of a visually guided reaching task. We use demixed dimensionality reduction and decoding techniques to show that the coding of gaze direction in PPC is mostly invariant to context. This suggests that PPC can provide reliable spatial information across sensory and behavioral contexts.

Do Congruent Auditory Stimuli Facilitate Visual Search in Dynamic Environments? An Experimental Study Based on Multisensory Interaction

The purpose of this study was to investigate the cue congruency effect of auditory stimuli during visual search in dynamic environments. Twenty-eight participants were recruited to conduct a visual search experiment. The experiment applied auditory stimuli to understand whether they could facilitate visual search in different types of background. Additionally, target location and target orientation were manipulated to clarify their influences on visual search. Target location was related to horizontal visual search and target orientation was associated with visual search for an inverted target. The results regarding dynamic backgrounds reported that target-congruent auditory stimuli could speed up the visual search time. In addition, the cue congruency effect of auditory stimuli was critical for the center of the visual display but declined for the edge, indicating the inhibition of horizontal visual search behavior. Moreover, few improvements accompanying auditory stimuli were provided for the visual detection of non-inverted and inverted targets. The findings of this study suggested developing multisensory interaction with head-mounted displays, such as augmented reality glasses, in real life.

A Compact Two-Loudspeaker Virtual Sound Reproduction System for Clinical Testing of Spatial Hearing With Hearing-Assistive Devices

Exciting developments in hearing aid and cochlear implant technology for linking signal processing across the ears have improved spatial hearing outcomes. This has resulted in an increased emphasis on clinical assessment of the spatial hearing abilities of hearing-assistive device users. Effective assessment of spatial hearing currently requires a large and costly loudspeaker array system, housed in a heavily acoustically treated testing room. This imposes economic and logistical constraints that limit proliferation of array systems, particularly in developing nations. Despite their size and cost, the ability of current clinical array systems to reproduce realistic spatial sound fields is limited, which substantially reduces the range of realistic acoustic scenes that can be used for diagnostic testing. We propose an alternative low-cost, compact virtual acoustics system with just two loudspeakers. This system uses crosstalk cancelation to reproduce pressure signals at the device microphones that match those for real-world sound sources. Furthermore, in contrast to clinical array systems, the system can adapt to different room acoustics, removing the requirement for a heavily acoustically treated testing environment. We conducted a proof-of-concept study in two stages: in the first, we evaluated the physical performance of the system for a stationary listener in anechoic conditions and in a small audiological testing booth with moderate acoustic treatment. To do this, a head and torso simulator was fitted with specially adapted hearing-assistive devices that allowed direct access to the microphone signals. These microphone signals were compared for real and virtual sound sources at numerous source locations. In the second stage, we quantified the system’s robustness to head rotations with and without the system adapting for head position. In the stationary case, the system was found to be highly effective at reproducing signals, such as speech, at all tested source locations. When head rotation was added, it performed well for rotations of up to 2°, even without adapting. However, performance improved markedly for larger rotations when the system adapted. These findings suggest that a compact, low-cost virtual acoustics system can give wider access to advanced and ecologically valid audiological testing, which could substantially improve clinical assessment of hearing-assistive device users.

Fooled twice: People cannot detect deepfakes but think they can

Hyper-realistic manipulations of audio-visual content, i.e., deepfakes, present new challenges for establishing the veracity of online content. Research on the human impact of deepfakes remains sparse. In a pre-registered behavioral experiment (N = 210), we show that (1) people cannot reliably detect deepfakes and (2) neither raising awareness nor introducing financial incentives improves their detection accuracy. Zeroing in on the underlying cognitive processes, we find that (3) people are biased toward mistaking deepfakes as authentic videos (rather than vice versa) and (4) they overestimate their own detection abilities. Together, these results suggest that people adopt a “seeing-is-believing” heuristic for deepfake detection while being overconfident in their (low) detection abilities. The combination renders people particularly susceptible to be influenced by deepfake content.

•

People cannot reliably detect deepfakes

•

Raising awareness and financial incentives do not improve people's detection accuracy

•

People tend to mistake deepfakes as authentic videos (rather than vice versa)

•

People overestimate their own detection deepfake abilities

Visual cues enhance obstacle avoidance in echolocating bats

Studies have shown that bats are capable of using visual information for a variety of purposes, including navigation and foraging, but the relative contributions of visual and auditory modalities in obstacle avoidance has yet to be fully investigated, particularly in laryngeal echolocating bats. A first step requires the characterization of behavioral responses to different combinations of sensory cues. Here, we quantified the behavioral responses of the insectivorous big brown bat, Eptesicus fuscus, in an obstacle avoidance task offering different combinations of auditory and visual cues. To do so, we utilized a new method that eliminates the confounds typically associated with testing bat vision and precludes auditory cues. We found that the presence of visual and auditory cues together enhances bats' avoidance response to obstacles compared with cues requiring either vision or audition alone. Analyses of flight and echolocation behaviors, such as speed and call rate, did not vary significantly under different obstacle conditions, and thus are not informative indicators of a bat's response to obstacle stimulus type. These findings advance the understanding of the relative importance of visual and auditory sensory modalities in guiding obstacle avoidance behaviors.

Sensory Cueing of Autobiographical Memories in Normal Aging and Alzheimer's Disease: A Comparison Between Visual, Auditory, and Olfactory Information

BACKGROUND

Alzheimer's disease (AD) is a chronic, neurodegenerative disease resulting in a progressive decline of autobiographical memories (AMs) which favors the development of psycho-behavioral disorders. One of the most popular psychosocial interventions in dementia care, Reminiscence Therapy, commonly uses sensory cueing to stimulate AMs retrieval. However, few empirical studies have investigated the impact of sensory stimulation on AMs retrieval in AD.

OBJECTIVE

Our goal was to determine the most relevant cue for AMs retrieval in patients with early to mild AD when comparing odors, sounds and pictures.

METHODS

Sixty AD patients, 60 healthy older adults (OA), and 60 healthy young adults (YA) participated in our study. Participants were presented with either 4 odors, 4 sounds, or 4 pictures. For each stimulus, they were asked to retrieve a personal memory, to rate it across 3 dimensions (emotionality, vividness, rarity) and then to date it.

RESULTS

Overall, results showed no clear dominance of one sensory modality over the others in evoking higher-quality AMs. However, they show that using pictures is the better way to stimulate AD patients' AM, as it helps to retrieve a higher number of memories that are also less frequently retrieved, followed by odors. By contrast, auditory cueing with environmental sounds presented no true advantage.

CONCLUSION

Our data should help dementia care professionals to increase the efficiency of Reminiscence Therapy using sensory elicitors. Other clinical implications and future directions are also discussed.

Audio-visual integration in noise: Influence of auditory and visual stimulus degradation on eye movements and perception of the McGurk effect

Seeing a talker’s face can aid audiovisual (AV) integration when speech is presented in noise. However, few studies have simultaneously manipulated auditory and visual degradation. We aimed to establish how degrading the auditory and visual signal affected AV integration. Where people look on the face in this context is also of interest; Buchan, Paré and Munhall (Brain Research, 1242, 162–171, 2008) found fixations on the mouth increased in the presence of auditory noise whilst Wilson, Alsius, Paré and Munhall (Journal of Speech, Language, and Hearing Research, 59(4), 601–615, 2016) found mouth fixations decreased with decreasing visual resolution. In Condition 1, participants listened to clear speech, and in Condition 2, participants listened to vocoded speech designed to simulate the information provided by a cochlear implant. Speech was presented in three levels of auditory noise and three levels of visual blurring. Adding noise to the auditory signal increased McGurk responses, while blurring the visual signal decreased McGurk responses. Participants fixated the mouth more on trials when the McGurk effect was perceived. Adding auditory noise led to people fixating the mouth more, while visual degradation led to people fixating the mouth less. Combined, the results suggest that modality preference and where people look during AV integration of incongruent syllables varies according to the quality of information available.

Temporal recalibration in vision requires location-based binding

Occupying the same location and occurring at the same time are the essential spatial and temporal factors for different features of a natural event or object to be integrated. Audio-visual temporal recalibration, as a temporal integration mechanism, refers to the brain's capacity to perceive simultaneity by adjusting for differential delays in the transmission of auditory and visual signals. Co-localization of auditory and visual information, however, is found not to be necessary for audio-visual temporal recalibration to occur. Here, we show that after exposure to a time lag between a visual flash and a visual collision, simultaneity responses were shifted toward an adapt lag in a bound condition where the flash and collision belonged to the same object but not in a separate condition where the flash and collision belonged to spatially separated objects. The results demonstrate that location-based binding is a requisite for temporal recalibration within the visual modality. Our finding suggests that the brain takes the modality difference in object localization into consideration when integrating temporally asynchronous signals.

Monkeys and humans implement causal inference to simultaneously localize auditory and visual stimuli

The environment is sampled by multiple senses, which are woven together to produce a unified perceptual state. However, optimally unifying such signals requires assigning particular signals to the same or different underlying objects or events. Many prior studies (especially in animals) have assumed fusion of cross-modal information, whereas recent work in humans has begun to probe the appropriateness of this assumption. Here we present results from a novel behavioral task in which both monkeys (Macaca mulatta) and humans localized visual and auditory stimuli and reported their perceived sources through saccadic eye movements. When the locations of visual and auditory stimuli were widely separated, subjects made two saccades, while when the two stimuli were presented at the same location they made only a single saccade. Intermediate levels of separation produced mixed response patterns: a single saccade to an intermediate position on some trials or separate saccades to both locations on others. The distribution of responses was well described by a hierarchical causal inference model that accurately predicted both the explicit "same vs. different" source judgments as well as biases in localization of the source(s) under each of these conditions. The results from this task are broadly consistent with prior work in humans across a wide variety of analogous tasks, extending the study of multisensory causal inference to nonhuman primates and to a natural behavioral task with both a categorical assay of the number of perceived sources and a continuous report of the perceived position of the stimuli.NEW & NOTEWORTHY We developed a novel behavioral paradigm for the study of multisensory causal inference in both humans and monkeys and found that both species make causal judgments in the same Bayes-optimal fashion. To our knowledge, this is the first demonstration of behavioral causal inference in animals, and this cross-species comparison lays the groundwork for future experiments using neuronal recording techniques that are impractical or impossible in human subjects.

Visual and Auditory Information During Decision Making in Sport

In 2 experiments, the authors investigated the effects of bimodal integration in a sport-specific task. Beach volleyball players were required to make a tactical decision, responding either verbally or via a motor response, after being presented with visual, auditory, or both kinds of stimuli in a beach volleyball scenario. In Experiment 1, players made the correct decision in a game situation more often when visual and auditory information were congruent than in trials in which they experienced only one of the modalities or incongruent information. Decision-making accuracy was greater when motor, rather than verbal, responses were given. Experiment 2 replicated this congruence effect using different stimulus material and showed a decreasing effect of visual stimulation on decision making as a function of shorter visual stimulus durations. In conclusion, this study shows that bimodal integration of congruent visual and auditory information results in more accurate decision making in sport than unimodal information.

Crossmodal Language Comprehension-Psycholinguistic Insights and Computational Approaches

Crossmodal interaction in situated language comprehension is important for effective and efficient communication. The relationship between linguistic and visual stimuli provides mutual benefit: While vision contributes, for instance, information to improve language understanding, language in turn plays a role in driving the focus of attention in the visual environment. However, language and vision are two different representational modalities, which accommodate different aspects and granularities of conceptualizations. To integrate them into a single, coherent system solution is still a challenge, which could profit from inspiration by human crossmodal processing. Based on fundamental psycholinguistic insights into the nature of situated language comprehension, we derive a set of performance characteristics facilitating the robustness of language understanding, such as crossmodal reference resolution, attention guidance, or predictive processing. Artificial systems for language comprehension should meet these characteristics in order to be able to perform in a natural and smooth manner. We discuss how empirical findings on the crossmodal support of language comprehension in humans can be applied in computational solutions for situated language comprehension and how they can help to mitigate the shortcomings of current approaches.

Top-down attention modulates the direction and magnitude of sensory dominance

Bottom-up inputs from multiple sensory modalities compete to reach perceptual consciousness. The sensory dominance effect refers to the phenomenon that stimuli from one sensory modality are preferentially selected over the other modalities. Top-down attention helps us to select task-relevant information while filtering out task-irrelevant distracting information. To investigate how top-down attention towards one specific modality modulates the sensory dominance effect, we incorporated the endogenous cue-target paradigm and an adapted version of the Colavita paradigm in the present study. The visual responses could either precede or fall behind the auditory responses, i.e., the visual vs. auditory precedence trials. The direction of the sensory dominance was defined as the proportion of the visual vs. auditory precedence bimodal trials, and the magnitude of the sensory dominance was calculated as the difference in reaction times between the first and the second responses in the bimodal trials. Results from the present three experiments consistently showed that when attention was voluntarily directed to the visual modality, the visual dominance occurred more frequently than the auditory dominance, and the magnitude of the visual dominance was significantly larger than the auditory dominance. This pattern of results was independent of the delivery modality of the cue. The present results thus provide direct empirical evidence showing that endogenous attention towards one specific sensory modality modulates both the direction and the size of sensory dominance.

The evolution of wetness perception: A comparison of arachnid, insect and human models

Hygroreceptors are a type of humidity sensor that have been identified in several invertebrate classes including Insecta and Arachnida. While their structure has been well researched, the nature of the mechanisms behind their function is debated as being either mechanical, evaporative, or psychrometric in insects and potentially also olfactory in arachnids. There is evidence that can be used to support or oppose each of these concepts, which also invites the possibility of multiple unified mechanisms occurring together. The integration of multiple sensory modalities has also formed the foundation of wetness perception in humans, led by thermal and tactile cues with supplementary information from vision and sound. These inputs are integrated by a vast neural network in the brain, which also occurs on a smaller scale in insects and arachnids. It is possible that as cerebral capacity increased throughout human evolution, this facilitated a preferable system of wetness perception via multisensory integration and rendered hygroreceptors obsolete. While this cerebral development hypothesis is only speculative, it gives a framework for further investigation. Additional research needs to be conducted to correctly classify hygroreceptor types in invertebrates and their relative prevalence before evolutionary associations can be made with vertebrate species. This integratory premise also applies to the human system, as knowing the relative contribution and compounding effects of each sensory modality on wetness perception will aid the overall understanding of the system and help to uncover the evolutionary development pathways underpinning each sense.

A Case of Near-Optimal Sensory Integration Based on Kohonen Self-Organizing Maps

This letter shows by digital simulation that a simple rule applied to one-dimensional self-organized maps for integrating sensory perceptions from two identical sources yielding position information as integers, corrupted by independent noise sources, yields almost statistically optimal results for position estimation as determined by maximum likelihood estimation. There is no learning of the corrupting noise sources nor is any information about the statistics of the noise sources available to the integrating process. The simple rule employed yields a measure of the quality of the estimated position of the source. The letter also shows that if the Bayesian estimates, which are rational numbers, are rounded in order to comply with the stipulation that integers be identified, the Bayesian estimation will have a larger variance than the proposed integration.

Context-dependent signaling of coincident auditory and visual events in primary visual cortex

Detecting rapid, coincident changes across sensory modalities is essential for recognition of sudden threats or events. Using two-photon calcium imaging in identified cell types in awake, head-fixed mice, we show that, among the basic features of a sound envelope, loud sound onsets are a dominant feature coded by the auditory cortex neurons projecting to primary visual cortex (V1). In V1, a small number of layer 1 interneurons gates this cross-modal information flow in a context-dependent manner. In dark conditions, auditory cortex inputs lead to suppression of the V1 population. However, when sound input coincides with a visual stimulus, visual responses are boosted in V1, most strongly after loud sound onsets. Thus, a dynamic, asymmetric circuit connecting AC and V1 contributes to the encoding of visual events that are coincident with sounds.

Modulated stimuli demonstrate asymmetric interactions between hearing and vision

The nature of interactions between the senses is a topic of intense interest in neuroscience, but an unresolved question is how sensory information from hearing and vision are combined when the two senses interact. A problem for testing auditory-visual interactions is devising stimuli and tasks that are equivalent in both modalities. Here we report a novel paradigm in which we first equated the discriminability of the stimuli in each modality, then tested how a distractor in the other modality affected performance. Participants discriminated pairs of amplitude-modulated tones or size-modulated visual objects in the form of a cuboid shape, alone or when a similarly modulated distractor stimulus of the other modality occurred with one of the pair. Discrimination of sound modulation depth was affected by a modulated cuboid only when their modulation rates were the same. In contrast, discrimination of cuboid modulation depth was little affected by an equivalently modulated sound. Our results suggest that what observers perceive when auditory and visual signals interact is not simply determined by the discriminability of the individual sensory inputs, but also by factors that increase the perceptual binding of these inputs, such as temporal synchrony.

MRI acoustic noise-modulated computer animations for patient distraction and entertainment with application in pediatric psychiatric patients

PURPOSE

To reduce patient anxiety caused by the MRI scanner acoustic noise.

MATERIAL AND METHODS

We developed a simple and low-cost system for patient distraction using visual computer animations that were synchronized to the MRI scanner's acoustic noise during the MRI exam. The system was implemented on a 3T MRI system and tested in 28 pediatric patients with bipolar disorder. The patients were randomized to receive noise-synchronized animations in the form of abstract animations in addition to music (n = 13, F/M = 6/7, age = 10.9 ± 2.5 years) or, as a control, receive only music (n = 15, F/M = 7/8, age = 11.6 ± 2.3 years). After completion of the scans, all subjects answered a questionnaire about their scan experience and the perceived scan duration.

RESULTS

The scan duration with multisensory input (animations and music) was perceived to be ~15% shorter than in the control group (43 min vs. 50 min, P < 0.05). However, the overall scan experience was scored less favorably (3.9 vs. 4.6 in the control group, P < 0.04).

CONCLUSIONS

This simple system provided patient distraction and entertainment leading to perceived shorter scan times, but the provided visualization with abstract animations was not favored by this patient cohort.

Age-Related Effects on Cross-Modal Duration Perception

Reliable duration perception of external events is necessary to coordinate perception with action, precisely discriminate speech, and for other daily functions. Visual duration perception can be heavily influenced by concurrent auditory signals; however, age-related effects on this process have received minimal attention. In the present study, we examined the effect of aging on duration perception by quantifying (1) duration discrimination thresholds, (2) auditory temporal dominance, and (3) visual duration expansion/compression percepts induced by an accompanying auditory stimulus of longer/shorter duration. Duration discrimination thresholds were significantly greater for visual than auditory tasks in both age groups, however there was no effect of age. While the auditory modality retained dominance in duration perception with age, older adults still performed worse than young adults when comparing durations of two target stimuli (e.g., visual) in the presence of distractors from the other modality (e.g., auditory). Finally, both age groups perceived similar visual duration compression, whereas older adults exhibited visual duration expansion over a wider range of auditory durations compared to their younger counterparts. Results are discussed in terms of multisensory integration and possible decision strategies that change with age.

Multisensory Integration Is Modulated by Auditory Sound Frequency and Visual Spatial Frequency

Accurate integration of auditory and visual information is essential for our ability to communicate with others. Previous studies have shown that the temporal discrepancies over which audiovisual speech stimuli will be integrated into a coherent percept are much wider than those typically observed for simple stimuli like beeps and flashes of light. However, our sensitivity to the low-level features of simple stimuli is not constant. We hypothesized that part of the enhanced integration of audiovisual speech may be due to it consisting predominantly of the sound frequencies and visual spatial frequencies that humans are most sensitive to. Here, we examined integration behaviors for pure tones across the sound frequency spectrum and visual gratings across the spatial frequency spectrum to examine how these low-level features modulate integration. The temporal window of integration was modulated by both sound frequency and visual spatial frequency, with the widest integration window occurring when both stimuli fell within their respective peak sensitivity ranges. These results suggest that part of the increased tolerance for temporal asynchrony typically observed for audiovisual speech may be due to the differential integration of low-level stimulus features that are dominant within complex audiovisual speech.

Visual capture of gait during redirected walking

Redirected walking allows users of virtual reality applications to explore virtual environments larger than the available physical space. This is achieved by manipulating users’ walking trajectories through visual rotation of the virtual surroundings, without users noticing this manipulation. Apart from its applied relevance, redirected walking is an attractive paradigm to investigate human perception and locomotion. An important yet unsolved question concerns individual differences in the ability to detect redirection. Addressing this question, we administered several perceptual-cognitive tasks to healthy participants, whose thresholds of detecting redirection in a virtual environment were also determined. We report relations between individual thresholds and measures of multisensory weighting (visually-assisted postural stability (Romberg quotient), subjective visual vertical (rod-and-frame test) and illusory self-motion (vection)). The performance in the rod-and-frame test, a classical measure of visual dependency regarding postural information, showed the strongest relation to redirection detection thresholds: The higher the visual dependency, the higher the detection threshold. This supports the interpretation of users’ neglect of redirection manipulations as a “visual capture of gait”. We discuss how future interdisciplinary studies, merging the fields of virtual reality and psychology, may help improving virtual reality applications and simultaneously deepen our understanding of how humans process multisensory conflicts during locomotion.

Unidirectional monosynaptic connections from auditory areas to the primary visual cortex in the marmoset monkey

Until the late twentieth century, it was believed that different sensory modalities were processed by largely independent pathways in the primate cortex, with cross-modal integration only occurring in specialized polysensory areas. This model was challenged by the finding that the peripheral representation of the primary visual cortex (V1) receives monosynaptic connections from areas of the auditory cortex in the macaque. However, auditory projections to V1 have not been reported in other primates. We investigated the existence of direct interconnections between V1 and auditory areas in the marmoset, a New World monkey. Labelled neurons in auditory cortex were observed following 4 out of 10 retrograde tracer injections involving V1. These projections to V1 originated in the caudal subdivisions of auditory cortex (primary auditory cortex, caudal belt and parabelt areas), and targeted parts of V1 that represent parafoveal and peripheral vision. Injections near the representation of the vertical meridian of the visual field labelled few or no cells in auditory cortex. We also placed 8 retrograde tracer injections involving core, belt and parabelt auditory areas, none of which revealed direct projections from V1. These results confirm the existence of a direct, nonreciprocal projection from auditory areas to V1 in a different primate species, which has evolved separately from the macaque for over 30 million years. The essential similarity of these observations between marmoset and macaque indicate that early-stage audiovisual integration is a shared characteristic of primate sensory processing.

The Psychophysiology of Action: A Multidisciplinary Endeavor for Integrating Action and Cognition

There is a vast amount of literature concerning the integration of action and cognition. Although this broad research area is of great interest for many disciplines like sports, psychology and cognitive neuroscience, only a few attempts tried to bring together different perspectives so far. Our goal is to provide a perspective to spark a debate across theoretical borders and integration of different disciplines via psychophysiology. In order to boost advances in this research field it is not only necessary to become aware of the different areas that are relevant but also to consider methodological aspects and challenges. We briefly describe the most relevant theoretical accounts to the question of how internal and external information processes or factors interact and, based on this, argue that research programs should consider the three dimensions: (a) dynamics of movements; (b) multivariate measures and; (c) dynamic statistical parameters. Only with an extended perspective on theoretical and methodological accounts, one would be able to integrate the dynamics of actions into theoretical advances.

The threshold for the McGurk effect in audio-visual noise decreases with development

Across development, vision increasingly influences audio-visual perception. This is evidenced in illusions such as the McGurk effect, in which a seen mouth movement changes the perceived sound. The current paper assessed the effects of manipulating the clarity of the heard and seen signal upon the McGurk effect in children aged 3-6 (n = 29), 7-9 (n = 32) and 10-12 (n = 29) years, and adults aged 20-35 years (n = 32). Auditory noise increased, and visual blur decreased, the likelihood of vision changing auditory perception. Based upon a proposed developmental shift from auditory to visual dominance we predicted that younger children would be less susceptible to McGurk responses, and that adults would continue to be influenced by vision in higher levels of visual noise and with less auditory noise. Susceptibility to the McGurk effect was higher in adults compared with 3-6-year-olds and 7-9-year-olds but not 10-12-year-olds. Younger children required more auditory noise, and less visual noise, than adults to induce McGurk responses (i.e. adults and older children were more easily influenced by vision). Reduced susceptibility in childhood supports the theory that sensory dominance shifts across development and reaches adult-like levels by 10 years of age.

Emergence of an Adaptive Command for Orienting Behavior in Premotor Brainstem Neurons of Barn Owls

The midbrain map of auditory space commands sound-orienting responses in barn owls. Owls precisely localize sounds in frontal space but underestimate the direction of peripheral sound sources. This bias for central locations was proposed to be adaptive to the decreased reliability in the periphery of sensory cues used for sound localization by the owl. Understanding the neural pathway supporting this biased behavior provides a means to address how adaptive motor commands are implemented by neurons. Here we find that the sensory input for sound direction is weighted by its reliability in premotor neurons of the midbrain tegmentum of owls (male and female), such that the mean population firing rate approximates the head-orienting behavior. We provide evidence that this coding may emerge through convergence of upstream projections from the midbrain map of auditory space. We further show that manipulating the sensory input yields changes predicted by the convergent network in both premotor neural responses and behavior. This work demonstrates how a topographic sensory representation can be linearly read out to adjust behavioral responses by the reliability of the sensory input.SIGNIFICANCE STATEMENT This research shows how statistics of the sensory input can be integrated into a behavioral command by readout of a sensory representation. The firing rate of midbrain premotor neurons receiving sensory information from a topographic representation of auditory space is weighted by the reliability of sensory cues. We show that these premotor responses are consistent with a weighted convergence from the topographic sensory representation. This convergence was also tested behaviorally, where manipulation of stimulus properties led to bidirectional changes in sound localization errors. Thus a topographic representation of auditory space is translated into a premotor command for sound localization that is modulated by sensory reliability.

Perceptual Experience Norms for 506 Russian Nouns: Modality Rating, Spatial Localization, Manipulability, Imageability and Other Variables

A number of new psycholinguistic variables has been proposed during the last years within embodied cognition framework: modality experience rating (i.e., relationship between words and images of a particular perceptive modality-visual, auditory, haptic etc.), manipulability (the necessity for an object to interact with human hands in order to perform its function), vertical spatial localization. However, it is not clear how these new variables are related to each other and to such traditional variables as imageability, AoA and word frequency. In this article, normative data on the modality (visual, auditory, haptic, olfactory, and gustatory) ratings, vertical spatial localization of the object, manipulability, imageability, age of acquisition, and subjective frequency for 506 Russian nouns are presented. Strongest correlations were observed between olfactory and gustatory modalities (.81), visual modality and imageability (.78), haptic modality and manipulability (.7). Other modalities also significantly correlate with imageability: olfactory (.35), gustatory (.24), and haptic (.67). Factor analysis divided variables into four groups where visual and haptic modality ratings were combined with imageability, manipulability and AoA (the first factor); word length, frequency and AoA formed the second factor; olfactory modality was united with gustatory (the third factor); spatial localization only is included in the fourth factor. Present norms of imageability and AoA are consistent with previous as correlation analysis has revealed. The complete database can be downloaded from supplementary material.

Spatial vision in older adults: perceptual changes and neural bases

PURPOSE

The number of older adults is rapidly increasing internationally, leading to a significant increase in research on how healthy ageing impacts vision. Most clinical assessments of spatial vision involve simple detection (letter acuity, grating contrast sensitivity, perimetry). However, most natural visual environments are more spatially complicated, requiring contrast discrimination, and the delineation of object boundaries and contours, which are typically present on non-uniform backgrounds. In this review we discuss recent research that reports on the effects of normal ageing on these more complex visual functions, specifically in the context of recent neurophysiological studies.

RECENT FINDINGS

Recent research has concentrated on understanding the effects of healthy ageing on neural responses within the visual pathway in animal models. Such neurophysiological research has led to numerous, subsequently tested, hypotheses regarding the likely impact of healthy human ageing on specific aspects of spatial vision.

SUMMARY

Healthy normal ageing impacts significantly on spatial visual information processing from the retina through to visual cortex. Some human data validates that obtained from studies of animal physiology, however some findings indicate that rethinking of presumed neural substrates is required. Notably, not all spatial visual processes are altered by age. Healthy normal ageing impacts significantly on some spatial visual processes (in particular centre-surround tasks), but leaves contrast discrimination, contrast adaptation, and orientation discrimination relatively intact. The study of older adult vision contributes to knowledge of the brain mechanisms altered by the ageing process, can provide practical information regarding visual environments that older adults may find challenging, and may lead to new methods of assessing visual performance in clinical environments.

Audiovisual Temporal Perception in Aging: The Role of Multisensory Integration and Age-Related Sensory Loss

Within each sensory modality, age-related deficits in temporal perception contribute to the difficulties older adults experience when performing everyday tasks. Since perceptual experience is inherently multisensory, older adults also face the added challenge of appropriately integrating or segregating the auditory and visual cues present in our dynamic environment into coherent representations of distinct objects. As such, many studies have investigated how older adults perform when integrating temporal information across audition and vision. This review covers both direct judgments about temporal information (the sound-induced flash illusion, temporal order, perceived synchrony, and temporal rate discrimination) and judgments regarding stimuli containing temporal information (the audiovisual bounce effect and speech perception). Although an age-related increase in integration has been demonstrated on a variety of tasks, research specifically investigating the ability of older adults to integrate temporal auditory and visual cues has produced disparate results. In this short review, we explore what factors could underlie these divergent findings. We conclude that both task-specific differences and age-related sensory loss play a role in the reported disparity in age-related effects on the integration of auditory and visual temporal information.

Generic HRTFs May be Good Enough in Virtual Reality. Improving Source Localization through Cross-Modal Plasticity

Auditory spatial localization in humans is performed using a combination of interaural time differences, interaural level differences, as well as spectral cues provided by the geometry of the ear. To render spatialized sounds within a virtual reality (VR) headset, either individualized or generic Head Related Transfer Functions (HRTFs) are usually employed. The former require arduous calibrations, but enable accurate auditory source localization, which may lead to a heightened sense of presence within VR. The latter obviate the need for individualized calibrations, but result in less accurate auditory source localization. Previous research on auditory source localization in the real world suggests that our representation of acoustic space is highly plastic. In light of these findings, we investigated whether auditory source localization could be improved for users of generic HRTFs via cross-modal learning. The results show that pairing a dynamic auditory stimulus, with a spatio-temporally aligned visual counterpart, enabled users of generic HRTFs to improve subsequent auditory source localization. Exposure to the auditory stimulus alone or to asynchronous audiovisual stimuli did not improve auditory source localization. These findings have important implications for human perception as well as the development of VR systems as they indicate that generic HRTFs may be enough to enable good auditory source localization in VR.

Extrinsic feedback and upper limb motor skill learning in typically-developing children and children with cerebral palsy: Review

BACKGROUND

Improvment of upper limb motor skills occurs through motor learning that can be enhanced by providing extrinsic feedback. Different types and frequencies of feedback are discussed but specific guidelines for use of feedback for motor learning in typically-developing (TD) children and children with Cerebral Palsy (CP) are not available.

OBJECTIVE

Identify the most effective modalities and frequencies of feedback for improving upper limb motor skills in TD children and children with CP.

METHODS

Ovid MEDLINE, Cochrane, PEDro and PubMed-NCBI were searched from 1950 to December 2015 to identify English-language articles addressing the role of extrinsic feedback on upper limb motor learning in TD children and children with CP. Nine studies were selected with a total of 243 TD children and 102 children with CP. Study quality was evaluated using the Downs and Black scale and levels of evidence were determined with Sackett's quality ratings.

RESULTS

There was a lack of consistency in the modalities and frequencies of feedback delivery used to improve motor learning in TD children and in children with CP. Moreover, the complexity of the task to be learned influenced the degree of motor learning achieved.

CONCLUSION

A better understanding of the influence of feedback on motor learning is needed to optimize motor skill acquisition in children with CP.

Effects of Auditory Patterns on Judged Displacements of an Occluded Moving Object

Using displays in which a moving disk disappeared behind an occluder, we examined whether an accompanying auditory rhythm influenced the perceived displacement of the disk during occlusion. We manipulated a baseline rhythm, comprising a relatively fast alternation of equal sound and pause durations. We had two different manipulations to create auditory sequences with a slower rhythm: either the pause durations or the sound durations were increased. In the trial, a disk moved at a constant speed, and at a certain point moved behind an occluder during which an auditory rhythm was played. Participants were instructed to track the occluded disk, and judge the expected position of the disk at the moment that the auditory rhythm ended by touching the judged position on a touch screen. We investigated the influence of the auditory rhythm, i.e., ratio of sound to pause duration, and the influence of auditory density, i.e., the number of sound onsets per time unit, on the judged distance. The results showed that the temporal characteristics affected the spatial judgments. Overall, we found that in the current paradigm relatively slow rhythms led to shorter judged distance as compared to relatively fast rhythms for both pause and sound variations. There was no main effect of auditory density on the judged distance of an expected visual event. That is, whereas the speed of the auditory rhythm appears crucial, the number of sound onsets per time unit as such, i.e., the auditory density, appears a much weaker factor.

Still no Evidence for Sustained Effects of Multisensory Integration of Duration

In studies on temporal order perception, immediate as well as sustained effects of multisensory integration have been demonstrated repeatedly. Regarding duration perception, the corresponding literature reports clear immediate effects of multisensory integration, but evidence on sustained effects of multisensory duration integration is scarce. In fact, a single study [Heron, J. et al. (2013). A neural hierarchy for illusions of time: Duration adaptation precedes multisensory integration, J. Vis. 13, 1-12.] investigated adaptation to multisensory conflicting intervals, and found no sustained effects of the audiovisual conflict on perceived duration of subsequently presented unimodal visual intervals. In two experiments, we provide independent evidence in support of this finding. In Experiment 1, we demonstrate that adaptation to audiovisual conflict does not alter perceived duration of subsequently presented visual test intervals. Thus, replicating the results of Heron et al. (2013), we observed no sustained effect of multisensory duration integration. However, one might argue that the prolonged exposure to consistent multisensory conflict might have prevented or hampered multisensory integration per se. In Experiment 2, we rule out this alternative explanation by showing that multisensory integration of audiovisual conflicting intervals is still effective after exposure to audiovisual conflict. This further strengthens the conclusion that multisensory integration of interval duration affects perception in an immediate, but not in a sustained manner.

A short cut to the past: Cueing via concrete objects improves autobiographical memory retrieval in Alzheimer's disease patients

Older adults diagnosed with Alzheimer's disease (AD) have difficulties accessing autobiographical memories. However, this deficit tends to spare memories dated to earlier parts of their lives, and may partially reflect retrieval deficits rather than complete memory loss. Introducing a novel paradigm, the present study examines whether autobiographical memory recall can be improved in AD by manipulating the sensory richness, concreteness and cultural dating of the memory cues. Specifically, we examine whether concrete everyday objects historically dated to the participants' youth (e.g., a skipping rope), relative to verbal cues (i.e., the verbal signifiers for the objects) facilitate access to autobiographical memories. The study includes 49 AD patients, and 50 healthy, older matched control participants, all tested on word versus object-cued recall. Both groups recalled significantly more memories, when cued by objects relative to words, but the advantage was significantly larger in the AD group. In both groups, memory descriptions were longer and significantly more episodic in nature in response to object-cued recall. Together these findings suggest that the multimodal nature of the object cues (i.e. vision, olfaction, audition, somatic sensation) along with specific cue characteristics, such as time reference, texture, shape, may constrain the retrieval search, potentially minimizing executive function demands, and hence strategic processing requirements, thus easing access to autobiographical memories in AD.

From Behavioral Facilitation to Inhibition: The Neuronal Correlates of the Orienting and Reorienting of Auditory Attention

Successful adaptive behavior relies on the ability to automatically (bottom-up) orient attention to different locations in the environment. This results in a biphasic pattern in which reaction times (RT) are faster for stimuli that occur in the same spatial location (valid) for the first few hundred milliseconds, which is termed facilitation. This is followed by faster RT for stimuli that appear in novel locations (invalid) after longer delays, termed inhibition of return. The neuronal areas and networks involved in the transition between states of facilitation and inhibition remain poorly understood, especially for auditory stimuli. Functional magnetic resonance imaging (fMRI) data were therefore collected in a large sample of healthy volunteers (N = 52) at four separate auditory stimulus onset asynchronies (SOAs; 200, 400, 600, and 800 ms). Behavioral results indicated that facilitation (valid RT < invalid RT) occurred at the 200 ms SOA, with inhibition of return (valid RT > invalid RT) present at the three longer SOAs. fMRI results showed several brain areas varying their activation as a function of SOA, including bilateral superior temporal gyrus, anterior thalamus, cuneus, dorsal anterior cingulate gyrus, and right ventrolateral prefrontal cortex (VLPFC)/anterior insula. Right VLPFC was active during a behavioral state of facilitation, and its activation (invalid – valid trials) further correlated with behavioral reorienting at the 200 ms delay. These results suggest that right VLPFC plays a critical role when auditory attention must be quickly deployed or redeployed, demanding heightened cognitive and inhibitory control. In contrast to previous work, the ventral and dorsal frontoparietal attention networks were both active during valid and invalid trials across SOAs. These results suggest that the dorsal and ventral networks may not be as specialized during bottom-up auditory orienting as has been previously reported during visual orienting.