Perception of visual and audiovisual trajectories toward and away from the body in the first postnatal year

Perceiving motion in depth is important in everyday life, especially motion in relation to the body. Visual and auditory cues inform us about motion in space when presented in isolation from each other, but the most comprehensive information is obtained through the combination of both of these cues. We traced the development of infants' ability to discriminate between visual motion trajectories across peripersonal space and to match these with auditory cues specifying the same peripersonal motion. We measured 5-month-old (n = 20) and 9-month-old (n = 20) infants' visual preferences for visual motion toward or away from their body (presented simultaneously and side by side) across three conditions: (a) visual displays presented alone, (b) paired with a sound increasing in intensity, and (c) paired with a sound decreasing in intensity. Both groups preferred approaching motion in the visual-only condition. When the visual displays were paired with a sound increasing in intensity, neither group showed a visual preference. When a sound decreasing in intensity was played instead, the 5-month-olds preferred the receding (spatiotemporally congruent) visual stimulus, whereas the 9-month-olds preferred the approaching (spatiotemporally incongruent) visual stimulus. We speculate that in the approaching sound condition, the behavioral salience of the sound could have led infants to focus on the auditory information alone, in order to prepare a motor response, and to neglect the visual stimuli. In the receding sound condition, instead, the difference in response patterns in the two groups may have been driven by infants' emerging motor abilities and their developing predictive processing mechanisms supporting and influencing each other.

Abnormal multisensory temporal discrimination in Parkinson's disease

Cognitive deficits are prevalent in Parkinson's disease (PD), ranging from mild deficits in perception and executive function to severe dementia. Multisensory integration (MSI), the ability to pool information from different sensory modalities to form a combined, coherent perception of the environment, is known to be impaired in PD. This study investigated the disruption of audiovisual MSI in PD patients by evaluating temporal discrimination ability between auditory and visual stimuli with different stimulus onset asynchronies (SOAs). The experiment was conducted with Fifteen PD patients and fifteen age-matched healthy controls where participants were requested to report whether the audiovisual stimuli pairs were temporal simultaneous. The temporal binding window (TBW), the time during which sensory modalities are perceived as synchronous, was adapted as the comparison index between PD patients and healthy individuals. Our results showed that PD patients had a significantly wider TBW than healthy controls, indicating abnormal audiovisual temporal discrimination. Furthermore, PD patients had more difficulty in discriminating temporal asynchrony in visual-first, but not in auditory-first stimuli, compared to healthy controls. In contrast, no significant difference was observed for auditory-first stimuli. PD patients also had shorter reaction times than healthy controls regardless of stimulus priority. Together, our findings point to abnormal audiovisual temporal discrimination, a major component of MSI irregularity, in PD patients. These results have important implications for future models of MSI experiments and models that aim to uncover the underlying mechanism of MSI in patients afflicted with PD.

The Effects of Auditory Working Memory Task on Situation Awareness in Complex Dynamic Environments: An Eye-movement Study

OBJECTIVE

This study investigated the effect of auditory working memory task on situation awareness (SA) and eye-movement patterns in complex dynamic environments.

BACKGROUND

Many human errors in aviation are caused by a lack of SA, and distraction from auditory secondary tasks is a serious threat to SA. However, it remains unclear how auditory working memory tasks affect SA and eye-movement patterns.

METHOD

Participants (n = 28) were randomly allocated to two groups and received different periods of visual search training (short versus long). They subsequently completed a situation awareness measurement task in three auditory secondary task conditions (without secondary task, auditory calculation task, and auditory 2-back task). Eye-movement data were collected during the situation awareness measurement task.

RESULTS

The auditory 2-back task significantly reduced overall SA, Level 1 SA, dwell times, and total percentage of fixation time on task-related areas of interests in the SA measurement task. Overall SA and Level 3 SA were not reduced by the auditory 2-back task in individuals in the longer visual search training time condition.

CONCLUSION

Auditory working memory load impairs SA in the perception and projection stage; however, greater experience can overcome impairment of SA in the projection stage.

APPLICATION

This study provided possible approaches to preventing loss of SA: (1) improving crew members' communication skills to ensure the accurate and clear transmission of information, reducing the difficulty of processing information, and (2) providing targeted cognitive training tailored to each pilot's level of experience.

Psychological and physiological effects of soundscapes: A systematic review of 25 experiments in the English and Chinese literature

The aim of this systematic review was to conduct a comprehensive and rigorous investigation of both psychological and physiological responses to, and audio-visual interactions with, soundscapes to present an overview of the current status and to provide suggestions for future research. Our literature search focused on empirical and quantitative studies of journal articles and gray literature in English and Chinese. This systematic review excluded literature related to pure music, religious sounds, humanistic sounds, historical sounds, medical research, and differences in materials used. The Joanna Briggs Institute's Checklist for Randomized Controlled Trials was used to assess the risk of bias in the included studies. Twenty-five studies were included, involving 1950 participants. The major findings of this systematic review were that: (1) there were significant associations between the psychological and physiological responses; (2) the audio-visual interaction affected the psychological and physiological responses; and (3) because of the high risk of bias of the included studies, interpretation of their findings should be cautious. Nevertheless, given that this systematic review has a higher level of evidence than a single study and the synthesized evidence identified in this review is aligned with the results of other studies, the studies reviewed herein together provide consistent evidence. Replications are important in empirical research to build trustworthy results. Future research should focus on the psychological responses of pleasantness, preference, tranquility, the eight semantic dimensions (ISO 12913-2:2018), and the 11 pairs of adjectives describing the soundscape (Ba et al., 2023) and the physiological responses of heart rate variability and salivary, and follow the CONSORT guidelines to improve the research quality. An integration of sensory modalities, environmental factors, contextual indicators, temporal data, demographic variables, socio-cultural factors, and psychological and physiological responses may provide deeper insights into how people experience and understand the acoustic environment in context.

Audiovisual speech asynchrony asymmetrically modulates neural binding

Previous psychophysical and neurophysiological studies in young healthy adults have provided evidence that audiovisual speech integration occurs with a large degree of temporal tolerance around true simultaneity. To further determine whether audiovisual speech asynchrony modulates auditory cortical processing and neural binding in young healthy adults, N1/P2 auditory evoked responses were compared using an additive model during a syllable categorization task, without or with an audiovisual asynchrony ranging from 240 ms visual lead to 240 ms auditory lead. Consistent with previous psychophysical findings, the observed results converge in favor of an asymmetric temporal integration window. Three main findings were observed: 1) predictive temporal and phonetic cues from pre-phonatory visual movements before the acoustic onset appeared essential for neural binding to occur, 2) audiovisual synchrony, with visual pre-phonatory movements predictive of the onset of the acoustic signal, was a prerequisite for N1 latency facilitation, and 3) P2 amplitude suppression and latency facilitation occurred even when visual pre-phonatory movements were not predictive of the acoustic onset but the syllable to come. Taken together, these findings help further clarify how audiovisual speech integration partly operates through two stages of visually-based temporal and phonetic predictions.

Auditory oddball responses in the human subthalamic nucleus and substantia nigra pars reticulata

The auditory oddball is a mainstay in research on attention, novelty, and sensory prediction. How this task engages subcortical structures like the subthalamic nucleus and substantia nigra pars reticulata is unclear. We administered an auditory OB task while recording single unit activity (35 units) and local field potentials (57 recordings) from the subthalamic nucleus and substantia nigra pars reticulata of 30 patients with Parkinson's disease undergoing deep brain stimulation surgery. We found tone modulated and oddball modulated units in both regions. Population activity differentiated oddball from standard trials from 200 ms to 1000 ms after the tone in both regions. In the substantia nigra, beta band activity in the local field potential was decreased following oddball tones. The oddball related activity we observe may underlie attention, sensory prediction, or surprise-induced motor suppression.

The priming of stimulus-elicited involuntary mental imagery

Percepts, urges, and even high-level cognitions often enter the conscious field involuntarily. The Reflexive Imagery Task (RIT) was designed to investigate experimentally the nature of such entry into consciousness. In the most basic version of the task, participants are instructed not to subvocalize the names of visual objects. Involuntary subvocalizations arise on the majority of the trials. Can these effects be influenced by priming? In our experiment, participants were exposed to an auditory prime 300 ms before being presented with the RIT stimuli. For example, participants heard the word "FOOD" before seeing two RIT stimuli (e.g., line drawings of BANANA and CAT, with the former being the target of the prime). The short span between prime and target allowed us to assess whether the RIT effect is strategic or automatic. Before each trial, participants were instructed to disregard what they hear, and not to think of the name of any of the objects. On an average of 83% of the trials, the participants thought (involuntarily) of the name of the object associated with the prime. This is the first study to use a priming technique within the context of the RIT. The theoretical implications of these involuntary effects are discussed.

Auditory mismatch negativity in pre-manifest and manifest Huntington's disease

AIM

The aim of this study was to investigate the characteristics of the electrophysiological brain response elicited in a passive acoustic oddball paradigm, i.e. mismatch negativity (MMN), in patients with Huntington's disease (HD) in the premanifest (pHD) and manifest (mHD) phases. In this regard, we correlated the results of event-related potentials (ERP) with disease characteristics.

METHODS

This was an observational cross-sectional MMN study. In addition to the MMN recording of the passive oddball task, all subjects with first-degree inheritance for HD underwent genetic testing for mutant HTT, the Huntington's Disease Rating Scale, the Total Functional Capacity Scale, the Problem Behaviors Assessment short form, and the Mini-Mental State Examination.

RESULTS

We found that global field power (GFP) was reduced in the MMN time window in mHD patients compared to pHD and normal controls (NC). In the pHD group, MMN amplitude was only slightly and not significantly increased compared to mHD, while pHD patients showed increased theta coherence between trials compared to mHD. In the entire sample of HD gene carriers, the main MMN traits were not correlated with motor performance, cognitive impairment and functional disability.

CONCLUSION

These results suggest an initial and subtle deterioration of pre-attentive mechanisms in the presymptomatic phase of HD, with an increasing phase shift in the MMN time frame. This result could indicate initial functional changes with a possible compensatory effect.

SIGNIFICANCE

An initial and slight decrease in MMN associated with increased phase coherence in the corresponding EEG frequencies could indicate an early functional involvement of pre-attentive resources that could precede the clinical expression of HD.

Loudness Balancing Optimization for Better Speech Intelligibility, Music Perception, and Spectral Temporal Resolution in Cochlear Implant Users

HYPOTHESIS

The behaviorally based programming with loudness balancing (LB) would result in better speech understanding, spectral-temporal resolution, and music perception scores, and there would be a relationship between these scores.

BACKGROUND

Loudness imbalances at upper stimulation levels may cause sounds to be perceived as irregular, gravelly, or overly echoed and may negatively affect the listening performance of the cochlear implant (CI) user. LB should be performed after fitting to overcome these problems.

METHODS

The study included 26 unilateral Med-EL CI users. Two different CI programs based on the objective electrically evoked stapedial reflex threshold (P1) and the behaviorally program with LB (P2) were recorded for each participant. The Turkish Matrix Sentence Test (TMS) was applied to evaluate speech perception; the Random Gap Detection Test (RGDT) and Spectral-Temporally Modulated Ripple Test (SMRT) were applied to evaluate spectral temporal resolution skills; the Mini Profile of Music Perception Skills (mini-PROMS) and Melodic Contour Identification (MCI) tests were applied to evaluate music perception, and the results were compared.

RESULTS

Significantly better scores were obtained with P2 in TMS tests performed in noise and quiet. SMRT scores were significantly correlated with TMS in quiet and noise, and mini-PROMS sound perception results. Although better scores were obtained with P2 in the mini-PROMS total score and MCI, a significant difference was found only for MCI.

CONCLUSION

The data from the current study showed that equalization of loudness across CI electrodes leads to better perceptual acuity. It also revealed the relationship between speech perception, spectral-temporal resolution, and music perception.

No evidence in favor of the existence of "intentional" binding

Intentional binding refers to the subjective temporal compression between a voluntary action and its subsequent sensory outcome. Despite some studies challenging the link between temporal compression and intentional action, intentional binding is still widely used as an implicit measure for the sense of agency. The debate remains unsettled primarily because the experimental conditions used in previous studies were confounded with various alternative causes for temporal compression, and action intention has not yet been tested comprehensively against all potential alternative causes in a single study. Here, we solve this puzzle by jointly comparing participants' estimates of the interval between three types of triggering events with comparable predictability-voluntary movement, passive movement, and external sensory event-and an external sensory outcome (auditory or visual across experiments). The results failed to show intentional binding, that is, no shorter interval estimation for the voluntary than the passive movement conditions. Instead, we observed temporal (but not intentional) binding when comparing both movement conditions with the external sensory condition. Thus, temporal binding appears to originate from sensory integration and temporal prediction, not from action intention. As such, these findings underscore the need to reconsider the use of "intentional binding" as a reliable proxy of the sense of agency. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Enhancement of visual dominance effects at the response level in children with attention-deficit/hyperactivity disorder

Previous studies have widely demonstrated that individuals with attention-deficit/hyperactivity disorder (ADHD) exhibit deficits in conflict control tasks. However, there is limited evidence regarding the performance of children with ADHD in cross-modal conflict processing tasks. The current study aimed to investigate whether children with ADHD have poor conflict control, which has an impact on sensory dominance effects at different levels of information processing under the influence of visual similarity. A total of 82 children aged 7 to 14 years, including 41 children with ADHD and 41 age- and sex-matched typically developing (TD) children, were recruited. We used the 2:1 mapping paradigm to separate levels of conflict, and the congruency of the audiovisual stimuli was divided into three conditions. In C trials, the target stimulus and the distractor stimulus were identical, and the bimodal stimuli corresponded to the same response keys. In PRIC trials, the distractor stimulus differed from the target stimulus and did not correspond to any response keys. In RIC trials, the distractor stimulus differed from the target stimulus, and the bimodal stimuli corresponded to different response keys. Therefore, we explicitly differentiated cross-modal conflict into a preresponse level (PRIC > C), corresponding to the encoding process, and a response level (RIC > PRIC), corresponding to the response selection process. Our results suggested that auditory distractors caused more interference during visual processing than visual distractors caused during auditory processing (i.e., typical auditory dominance) at the preresponse level regardless of group. However, visual dominance effects were observed in the ADHD group, whereas no visual dominance effects were observed in the TD group at the response level. A possible explanation is that the increased interference effects due to visual similarity and children with ADHD made it more difficult to control conflict when simultaneously confronted with incongruent visual and auditory inputs. The current study highlights how children with ADHD process cross-modal conflicts at multiple levels of information processing, thereby shedding light on the mechanisms underlying ADHD.

Speeded classification of visual events is sensitive to crossmodal intensity correspondence

Crossmodal correspondences refer to systematic associations between stimulus attributes encountered in different sensory modalities. These correspondences can be probed in the speeded classification task where they tend to produce congruency effects. This study aimed to replicate and extend previous work conducted by Marks (1987, Experiment 3, Journal of Experimental Psychology: Human Perception and Performance, Vol. 13, No. 3, 384-394) which demonstrated a crossmodal correspondence between auditory and visual intensity attributes. Experiment 1 successfully replicates Marks' original finding that performance in a brightness classification task is affected by whether the loudness of a concurrently presented auditory distractor matches the brightness of the visual target. Furthermore, in line with the original study, we found that this effect was absent in a lightness classification task. In Experiment 2, we demonstrate that loudness-brightness correspondence is robust even when the exact stimulus input changes. This finding suggests that there is a context-dependent mapping between loudness and brightness levels, rather than an absolute mapping between any particular intensity levels. Finally, exploratory analysis using the diffusion model for conflict tasks indicated that evidence from the task-irrelevant modality generates a burst of weak, short-lived automatic activation that can bias decision-making in difficult tasks, but not in easy tasks. Our results provide further evidence for the existence of a flexible crossmodal correspondence between brightness and loudness, which might be helpful in determining one's distance to a stimulus source during the early stages of multisensory integration. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The Costs (and Benefits?) of Effortful Listening for Older Adults: Insights from Simultaneous Electrophysiology, Pupillometry, and Memory

Although the impact of acoustic challenge on speech processing and memory increases as a person ages, older adults may engage in strategies that help them compensate for these demands. In the current preregistered study, older adults (n = 48) listened to sentences-presented in quiet or in noise-that were high constraint with either expected or unexpected endings or were low constraint with unexpected endings. Pupillometry and EEG were simultaneously recorded, and subsequent sentence recognition and word recall were measured. Like young adults in prior work, we found that noise led to increases in pupil size, delayed and reduced ERP responses, and decreased recall for unexpected words. However, in contrast to prior work in young adults where a larger pupillary response predicted a recovery of the N400 at the cost of poorer memory performance in noise, older adults did not show an associated recovery of the N400 despite decreased memory performance. Instead, we found that in quiet, increases in pupil size were associated with delays in N400 onset latencies and increased recognition memory performance. In conclusion, we found that transient variation in pupil-linked arousal predicted trade-offs between real-time lexical processing and memory that emerged at lower levels of task demand in aging. Moreover, with increased acoustic challenge, older adults still exhibited costs associated with transient increases in arousal without the corresponding benefits.

The Cerebellum in Musicology: a Narrative Review

The cerebellum is involved in cognitive procressing including music perception and music production. This narrative review aims to summarize the current knowledge on the activation of the cerebellum by different musical stimuli, on the involvement of the cerebellum in cognitive loops underlying the analysis of music, and on the role of the cerebellum in the motor network underlying music production. A possible role of the cerebellum in therapeutic settings is also briefly discussed. In a second part, the cerebellum as object of musicology (i.e., in classical music, in contemporary music, cerebellar disorders of musicians) is described.

Systemic neurophysiological signals of auditory predictive coding

Predictive coding framework posits that our brain continuously monitors changes in the environment and updates its predictive models, minimizing prediction errors to efficiently adapt to environmental demands. However, the underlying neurophysiological mechanisms of these predictive phenomena remain unclear. The present study aimed to explore the systemic neurophysiological correlates of predictive coding processes during passive and active auditory processing. Electroencephalography (EEG), functional near-infrared spectroscopy (fNIRS), and autonomic nervous system (ANS) measures were analyzed using an auditory pattern-based novelty oddball paradigm. A sample of 32 healthy subjects was recruited. The results showed shared slow evoked potentials between passive and active conditions that could be interpreted as automatic predictive processes of anticipation and updating, independent of conscious attentional effort. A dissociated topography of the cortical hemodynamic activity and distinctive evoked potentials upon auditory pattern violation were also found between both conditions, whereas only conscious perception leading to imperative responses was accompanied by phasic ANS responses. These results suggest a systemic-level hierarchical reallocation of predictive coding neural resources as a function of contextual demands in the face of sensory stimulation. Principal component analysis permitted to associate the variability of some of the recorded signals.

Lower Childhood Socioeconomic Status Is Associated with Greater Neural Responses to Ambient Auditory Changes in Adulthood

Humans' early life experience varies by socioeconomic status (SES), raising the question of how this difference is reflected in the adult brain. An important aspect of brain function is the ability to detect salient ambient changes while focusing on a task. Here, we ask whether subjective social status during childhood is reflected by the way young adults' brain detecting changes in irrelevant information. In two studies (total n = 58), we examine electrical brain responses in the frontocentral region to a series of auditory tones, consisting of standard stimuli (80%) and deviant stimuli (20%) interspersed randomly, while participants were engaged in various visual tasks. Both studies showed stronger automatic change detection indexed by MMN in lower SES individuals, regardless of the unattended sound's feature, attended emotional content, or study type. Moreover, we observed a larger MMN in lower-SES participants, although they did not show differences in brain and behavior responses to the attended task. Lower-SES people also did not involuntarily orient more attention to sound changes (i.e., deviant stimuli), as indexed by the P3a. The study indicates that individuals with lower subjective social status may have an increased ability to automatically detect changes in their environment, which may suggest their adaptation to their childhood environments.

Auditory Processing of Intonational Rises and Falls in German: Rises Are Special in Attention Orienting

This article investigates the processing of intonational rises and falls when presented unexpectedly in a stream of repetitive auditory stimuli. It examines the neurophysiological correlates (ERPs) of attention to these unexpected stimuli through the use of an oddball paradigm where sequences of repetitive stimuli are occasionally interspersed with a deviant stimulus, allowing for elicitation of an MMN. Whereas previous oddball studies on attention toward unexpected sounds involving pitch rises were conducted on nonlinguistic stimuli, the present study uses as stimuli lexical items in German with naturalistic intonation contours. Results indicate that rising intonation plays a special role in attention orienting at a pre-attentive processing stage, whereas contextual meaning (here a list of items) is essential for activating attentional resources at a conscious processing stage. This is reflected in the activation of distinct brain responses: Rising intonation evokes the largest MMN, whereas falling intonation elicits a less pronounced MMN followed by a P3 (reflecting a conscious processing stage). Subsequently, we also find a complex interplay between the phonological status (i.e., accent/head marking vs. boundary/edge marking) and the direction of pitch change in their contribution to attention orienting: Attention is not oriented necessarily toward a specific position in prosodic structure (head or edge). Rather, we find that the intonation contour itself and the appropriateness of the contour in the linguistic context are the primary cues to two core mechanisms of attention orienting, pre-attentive and conscious orientation respectively, whereas the phonological status of the pitch event plays only a supplementary role.

Conditional deviant repetition in the oddball paradigm modulates processing at the level of P3a but not MMN

The auditory system has an amazing ability to rapidly encode auditory regularities. Evidence comes from the popular oddball paradigm, in which frequent (standard) sounds are occasionally exchanged for rare deviant sounds, which then elicit signs of prediction error based on their unexpectedness (e.g., MMN and P3a). Here, we examine the widely neglected characteristics of deviants being bearers of predictive information themselves; naive participants listened to sound sequences constructed according to a new, modified version of the oddball paradigm including two types of deviants that followed diametrically opposed rules: one deviant sound occurred mostly in pairs (repetition rule), the other deviant sound occurred mostly in isolation (non-repetition rule). Due to this manipulation, the sound following a first deviant (either the same deviant or a standard) was either predictable or unpredictable based on its conditional probability associated with the preceding deviant sound. Our behavioral results from an active deviant detection task replicate previous findings that deviant repetition rules (based on conditional probability) can be extracted when behaviorally relevant. Our electrophysiological findings obtained in a passive listening setting indicate that conditional probability also translates into differential processing at the P3a level. However, MMN was confined to global deviants and was not sensitive to conditional probability. This suggests that higher-level processing concerned with stimulus selection and/or evaluation (reflected in P3a) but not lower-level sensory processing (reflected in MMN) considers rarely encountered rules.

The influence of peripheral information on a proactive process during multitasking

The aim of this study was to examine whether peripheral information facilitates proactive processes during multitasking. For this purpose, peripheral information was presented regularly during multitasking and its effects on the performance of a tracking task (main task: reactive process) and a discrimination task (sub-task: proactive process) were examined. Experiment 1 presented peripheral information (white circles) in the same sensory modality (visual) as the information used for multitasking and the number of circle presentations was manipulated. In Experiment 2, a pure tone (auditory) was presented as peripheral information. We found that, in both experiments, the difficulty of the tracking task influenced discrimination performance, showing that as the difficulty of the tracking task (reactive process) increased, more cognitive resources were consumed in the tracking task, resulting in a decrease in cognitive resources available for the discrimination task (proactive process). In addition, regular presentation of peripheral information facilitated discrimination task performance in both experiments. Interestingly, this peripheral information also facilitated the tracking task performance (reactive process) even if the tracking task was difficult. Moreover, this promoting effect of the peripheral information occurred regardless of the sensory modality. This study revealed that processing of peripheral information facilitates the proactive process even if more cognitive resources are consumed, and that this facilitating effect does not conflict with multitasking and provides a margin of cognitive resources and also facilitates the reactive process. Our results provide evidence of how peripheral information and cognitive resources are used during multitasking.

Daily singing of adult songbirds functions to maintain song performance independently of auditory feedback and age

Many songbirds learn to produce songs through vocal practice in early life and continue to sing daily throughout their lifetime. While it is well-known that adult songbirds sing as part of their mating rituals, the functions of singing behavior outside of reproductive contexts remain unclear. Here, we investigated this issue in adult male zebra finches by suppressing their daily singing for two weeks and examining the effects on song performance. We found that singing suppression decreased the pitch, amplitude, and duration of songs, and that those song features substantially recovered through subsequent free singing. These reversible song changes were not dependent on auditory feedback or the age of the birds, contrasting with the adult song plasticity that has been reported previously. These results demonstrate that adult song structure is not stable without daily singing, and suggest that adult songbirds maintain song performance by preventing song changes through physical act of daily singing throughout their life. Such daily singing likely functions as vocal training to maintain the song production system in optimal conditions for song performance in reproductive contexts, similar to how human singers and athletes practice daily to maintain their performance.

Decoding predicted musical notes from omitted stimulus potentials

Electrophysiological studies have investigated predictive processing in music by examining event-related potentials (ERPs) elicited by the violation of musical expectations. While several studies have reported that the predictability of stimuli can modulate the amplitude of ERPs, it is unclear how specific the representation of the expected note is. The present study addressed this issue by recording the omitted stimulus potentials (OSPs) to avoid contamination of bottom-up sensory processing with top-down predictive processing. Decoding of the omitted content was attempted using a support vector machine, which is a type of machine learning. ERP responses to the omission of four target notes (E, F, A, and C) at the same position in familiar and unfamiliar melodies were recorded from 25 participants. The results showed that the omission N1 were larger in the familiar melody condition than in the unfamiliar melody condition. The decoding accuracy of the four omitted notes was significantly higher in the familiar melody condition than in the unfamiliar melody condition. These results suggest that the OSPs contain discriminable predictive information, and the higher the predictability, the more the specific representation of the expected note is generated.

The perceived catchiness of music affects the experience of groove

Catchiness and groove are common phenomena when listening to popular music. Catchiness may be a potential factor for experiencing groove but quantitative evidence for such a relationship is missing. To examine whether and how catchiness influences a key component of groove-the pleasurable urge to move to music (PLUMM)-we conducted a listening experiment with 450 participants and 240 short popular music clips of drum patterns, bass lines or keys/guitar parts. We found four main results: (1) catchiness as measured in a recognition task was only weakly associated with participants' perceived catchiness of music. We showed that perceived catchiness is multi-dimensional, subjective, and strongly associated with pleasure. (2) We found a sizeable positive relationship between PLUMM and perceived catchiness. (3) However, the relationship is complex, as further analysis showed that pleasure suppresses perceived catchiness' effect on the urge to move. (4) We compared common factors that promote perceived catchiness and PLUMM and found that listener-related variables contributed similarly, while the effects of musical content diverged. Overall, our data suggests music perceived as catchy is likely to foster groove experiences.

Investigating sensitivity to multi-domain prediction errors in chronic auditory phantom perception

The perception of a continuous phantom in a sensory domain in the absence of an external stimulus is explained as a maladaptive compensation of aberrant predictive coding, a proposed unified theory of brain functioning. If this were true, these changes would occur not only in the domain of the phantom percept but in other sensory domains as well. We confirm this hypothesis by using tinnitus (continuous phantom sound) as a model and probe the predictive coding mechanism using the established local-global oddball paradigm in both the auditory and visual domains. We observe that tinnitus patients are sensitive to changes in predictive coding not only in the auditory but also in the visual domain. We report changes in well-established components of event-related EEG such as the mismatch negativity. Furthermore, deviations in stimulus characteristics were correlated with the subjective tinnitus distress. These results provide an empirical confirmation that aberrant perceptions are a symptom of a higher-order systemic disorder transcending the domain of the percept.

EEGminer: discovering interpretable features of brain activity with learnable filters

Objective. The patterns of brain activity associated with different brain processes can be used to identify different brain states and make behavioural predictions. However, the relevant features are not readily apparent and accessible. Our aim is to design a system for learning informative latent representations from multichannel recordings of ongoing EEG activity.Approach: We propose a novel differentiable decoding pipeline consisting of learnable filters and a pre-determined feature extraction module. Specifically, we introduce filters parameterized by generalized Gaussian functions that offer a smooth derivative for stable end-to-end model training and allow for learning interpretable features. For the feature module, we use signal magnitude and functional connectivity estimates.Main results.We demonstrate the utility of our model on a new EEG dataset of unprecedented size (i.e. 721 subjects), where we identify consistent trends of music perception and related individual differences. Furthermore, we train and apply our model in two additional datasets, specifically for emotion recognition on SEED and workload classification on simultaneous task EEG workload. The discovered features align well with previous neuroscience studies and offer new insights, such as marked differences in the functional connectivity profile between left and right temporal areas during music listening. This agrees with the specialisation of the temporal lobes regarding music perception proposed in the literature.Significance. The proposed method offers strong interpretability of learned features while reaching similar levels of accuracy achieved by black box deep learning models. This improved trustworthiness may promote the use of deep learning models in real world applications. The model code is available athttps://github.com/SMLudwig/EEGminer/.

Distributed representations of prediction error signals across the cortical hierarchy are synergistic

A relevant question concerning inter-areal communication in the cortex is whether these interactions are synergistic. Synergy refers to the complementary effect of multiple brain signals conveying more information than the sum of each isolated signal. Redundancy, on the other hand, refers to the common information shared between brain signals. Here, we dissociated cortical interactions encoding complementary information (synergy) from those sharing common information (redundancy) during prediction error (PE) processing. We analyzed auditory and frontal electrocorticography (ECoG) signals in five common awake marmosets performing two distinct auditory oddball tasks and investigated to what extent event-related potentials (ERP) and broadband (BB) dynamics encoded synergistic and redundant information about PE processing. The information conveyed by ERPs and BB signals was synergistic even at lower stages of the hierarchy in the auditory cortex and between auditory and frontal regions. Using a brain-constrained neural network, we simulated the synergy and redundancy observed in the experimental results and demonstrated that the emergence of synergy between auditory and frontal regions requires the presence of strong, long-distance, feedback, and feedforward connections. These results indicate that distributed representations of PE signals across the cortical hierarchy can be highly synergistic.

Sex differences during development in cortical temporal processing and event related potentials in wild-type and fragile X syndrome model mice

BACKGROUND

Autism spectrum disorder (ASD) is currently diagnosed in approximately 1 in 44 children in the United States, based on a wide array of symptoms, including sensory dysfunction and abnormal language development. Boys are diagnosed ~ 3.8 times more frequently than girls. Auditory temporal processing is crucial for speech recognition and language development. Abnormal development of temporal processing may account for ASD language impairments. Sex differences in the development of temporal processing may underlie the differences in language outcomes in male and female children with ASD. To understand mechanisms of potential sex differences in temporal processing requires a preclinical model. However, there are no studies that have addressed sex differences in temporal processing across development in any animal model of ASD.

METHODS

To fill this major gap, we compared the development of auditory temporal processing in male and female wildtype (WT) and Fmr1 knock-out (KO) mice, a model of Fragile X Syndrome (FXS), a leading genetic cause of ASD-associated behaviors. Using epidural screw electrodes, we recorded auditory event related potentials (ERP) and auditory temporal processing with a gap-in-noise auditory steady state response (ASSR) paradigm at young (postnatal (p)21 and p30) and adult (p60) ages from both auditory and frontal cortices of awake, freely moving mice.

RESULTS

The results show that ERP amplitudes were enhanced in both sexes of Fmr1 KO mice across development compared to WT counterparts, with greater enhancement in adult female than adult male KO mice. Gap-ASSR deficits were seen in the frontal, but not auditory, cortex in early development (p21) in female KO mice. Unlike male KO mice, female KO mice show WT-like temporal processing at p30. There were no temporal processing deficits in the adult mice of both sexes.

CONCLUSIONS

These results show a sex difference in the developmental trajectories of temporal processing and hypersensitive responses in Fmr1 KO mice. Male KO mice show slower maturation of temporal processing than females. Female KO mice show stronger hypersensitive responses than males later in development. The differences in maturation rates of temporal processing and hypersensitive responses during various critical periods of development may lead to sex differences in language function, arousal and anxiety in FXS.

Auditory neuroscience: Sounds make the face move

Animals including humans often react to sounds by involuntarily moving their face and body. A new study shows that facial movements provide a simple and reliable readout of a mouse's hearing ability that is more sensitive than traditional measurements.

Exploring the Differences Between an Immature and a Mature Human Auditory System Through Auditory Late Responses in Quiet and in Noise

Children are disadvantaged compared to adults when they perceive speech in a noisy environment. Noise reduces their ability to extract and understand auditory information. Auditory-Evoked Late Responses (ALRs) offer insight into how the auditory system can process information in noise. This study investigated how noise, signal-to-noise ratio (SNR), and stimulus type affect ALRs in children and adults. Fifteen participants from each group with normal hearing were studied under various conditions. The findings revealed that both groups experienced delayed latencies and reduced amplitudes in noise but that children had fewer identifiable waves than adults. Babble noise had a significant impact on both groups, limiting the analysis to one condition: the /da/ stimulus at +10 dB SNR for the P1 wave. P1 amplitude was greater in quiet for children compared to adults, with no stimulus effect. Children generally exhibited longer latencies. N1 latency was longer in noise, with larger amplitudes in white noise compared to quiet for both groups. P2 latency was shorter with the verbal stimulus in quiet, with larger amplitudes in children than adults. N2 latency was shorter in quiet, with no amplitude differences between the groups. Overall, noise prolonged latencies and reduced amplitudes. Different noise types had varying impacts, with the eight-talker babble noise causing more disruption. Children's auditory system responded similarly to adults but may be more susceptible to noise. This research emphasizes the need to understand noise's impact on children's auditory development, given their exposure to noisy environments, requiring further exploration of noise parameters in children.

Auditory change detection and visual selective attention: association between MMN and N2pc

While the auditory and visual systems each provide distinct information to our brain, they also work together to process and prioritize input to address ever-changing conditions. Previous studies highlighted the trade-off between auditory change detection and visual selective attention; however, the relationship between them is still unclear. Here, we recorded electroencephalography signals from 106 healthy adults in three experiments. Our findings revealed a positive correlation at the population level between the amplitudes of event-related potential indices associated with auditory change detection (mismatch negativity) and visual selective attention (posterior contralateral N2) when elicited in separate tasks. This correlation persisted even when participants performed a visual task while disregarding simultaneous auditory stimuli. Interestingly, as visual attention demand increased, participants whose posterior contralateral N2 amplitude increased the most exhibited the largest reduction in mismatch negativity, suggesting a within-subject trade-off between the two processes. Taken together, our results suggest an intimate relationship and potential shared mechanism between auditory change detection and visual selective attention. We liken this to a total capacity limit that varies between individuals, which could drive correlated individual differences in auditory change detection and visual selective attention, and also within-subject competition between the two, with task-based modulation of visual attention causing within-participant decrease in auditory change detection sensitivity.

Disrupted interoceptive awareness by auditory distractor: Difficulty inferring the internal bodily states?

Recent studies have associated interoceptive awareness, the perception of internal bodily sensations, with a predictive mechanism of perception across all sensory modalities. According to the framework, volitional attention plays a pivotal role in interoceptive awareness by prioritizing interoceptive sensations over exteroceptive ones. Consequently, it is hypothesized that the presence of irrelevant stimuli would disrupt the attentional modulation and interoceptive awareness, which remains untested. In this study, we investigated if interoceptive awareness is diminished by unrelated auditory distractors to validate the proposed perceptual framework. A total of 30 healthy human volunteers performed the heartbeat counting task both with and without auditory distractors. Additionally, we measured participant's psychophysiological traits related to interoception, including the high-frequency component of heart rate variability (HF-HRV) and trait interoceptive sensibility. The results showed that interoceptive accuracy, confidence, and heartbeat intensity decreased in the presence of distractor sound. Moreover, individuals with higher HF-HRV and a greater tendency to worry about bodily states experienced a more pronounced distractor effect on interoceptive awareness. These results provide support for the perceptual mechanism of interoceptive awareness in terms of the predictive process, highlighting the impact of relative precision across interoceptive and exteroceptive signals on perceptual experiences.

The neuro-oscillatory profiles of static and dynamic music-induced visual imagery

Visual imagery, i.e., seeing in the absence of the corresponding retinal input, has been linked to visual and motor processing areas of the brain. Music listening provides an ideal vehicle for exploring the neural correlates of visual imagery because it has been shown to reliably induce a broad variety of content, ranging from abstract shapes to dynamic scenes. Forty-two participants listened with closed eyes to twenty-four excerpts of music, while a 15-channel EEG was recorded, and, after each excerpt, rated the extent to which they experienced static and dynamic visual imagery. Our results show both static and dynamic imagery to be associated with posterior alpha suppression (especially in lower alpha) early in the onset of music listening, while static imagery was associated with an additional alpha enhancement later in the listening experience. With regard to the beta band, our results demonstrate beta enhancement to static imagery, but first beta suppression before enhancement in response to dynamic imagery. We also observed a positive association, early in the listening experience, between gamma power and dynamic imagery ratings that was not present for static imagery ratings. Finally, we offer evidence that musical training may selectively drive effects found with respect to static and dynamic imagery and alpha, beta, and gamma band oscillations. Taken together, our results show the promise of using music listening as an effective stimulus for examining the neural correlates of visual imagery and its contents. Our study also highlights the relevance of future work seeking to study the temporal dynamics of music-induced visual imagery.

Differences in audiovisual temporal processing in autistic adults are specific to simultaneity judgments

Research has shown that children on the autism spectrum and adults with high levels of autistic traits are less sensitive to audiovisual asynchrony compared to their neurotypical peers. However, this evidence has been limited to simultaneity judgments (SJ) which require participants to consider the timing of two cues together. Given evidence of partly divergent perceptual and neural mechanisms involved in making temporal order judgments (TOJ) and SJ, and given that SJ require a more global type of processing which may be impaired in autistic individuals, here we ask whether the observed differences in audiovisual temporal processing are task and stimulus specific. We examined the ability to detect audiovisual asynchrony in a group of 26 autistic adult males and a group of age and IQ-matched neurotypical males. Participants were presented with beep-flash, point-light drumming, and face-voice displays with varying degrees of asynchrony and asked to make SJ and TOJ. The results indicated that autistic participants were less able to detect audiovisual asynchrony compared to the control group, but this effect was specific to SJ and more complex social stimuli (e.g., face-voice) with stronger semantic correspondence between the cues, requiring a more global type of processing. This indicates that audiovisual temporal processing is not generally different in autistic individuals and that a similar level of performance could be achieved by using a more local type of processing, thus informing multisensory integration theory as well as multisensory training aimed to aid perceptual abilities in this population.

The development of audiovisual speech perception in Mandarin-speaking children: Evidence from the McGurk paradigm

The developmental trajectory of audiovisual speech perception in Mandarin-speaking children remains understudied. This cross-sectional study in Mandarin-speaking 3- to 4-year-old, 5- to 6-year-old, 7- to 8-year-old children, and adults from Xiamen, China (n = 87, 44 males) investigated this issue using the McGurk paradigm with three levels of auditory noise. For the identification of congruent stimuli, 3- to 4-year-olds underperformed older groups whose performances were comparable. For the perception of the incongruent stimuli, a developmental shift was observed as 3- to 4-year-olds made significantly more audio-dominant but fewer audiovisual-integrated responses to incongruent stimuli than older groups. With increasing auditory noise, the difference between children and adults widened in identifying congruent stimuli but narrowed in perceiving incongruent ones. The findings regarding noise effects agree with the statistically optimal hypothesis.

Effects of the cardiac cycle on auditory processing: A preregistered study on mismatch negativity

The systolic and diastolic phases of the cardiac cycle are known to affect perception and cognition differently. Higher order processing tends to be facilitated at systole, whereas sensory processing of external stimuli tends to be impaired at systole compared to diastole. The current study aims to examine whether the cardiac cycle affects auditory deviance detection, as reflected in the mismatch negativity (MMN) of the event-related brain potential (ERP). We recorded the intensity deviance response to deviant tones (70 dB) presented among standard tones (60 or 80 dB, depending on blocks) and calculated the MMN by subtracting standard ERP waveforms from deviant ERP waveforms. We also assessed intensity-dependent N1 and P2 amplitude changes by subtracting ERPs elicited by soft standard tones (60 dB) from ERPs elicited by loud standard tones (80 dB). These subtraction methods were used to eliminate phase-locked cardiac-related electric artifacts that overlap auditory ERPs. The endogenous MMN was expected to be larger at systole, reflecting the facilitation of memory-based auditory deviance detection, whereas the exogenous N1 and P2 would be smaller at systole, reflecting impaired exteroceptive sensory processing. However, after the elimination of cardiac-related artifacts, there were no significant differences between systole and diastole in any ERP components. The intensity-dependent N1 and P2 amplitude changes were not obvious in either cardiac phase, probably because of the short interstimulus intervals. The lack of a cardiac phase effect on MMN amplitude suggests that preattentive auditory processing may not be affected by bodily signals from the heart.

The impact of acute asymmetric hearing loss on multisensory integration

Humans have the remarkable ability to integrate information from different senses, which greatly facilitates the detection, localization and identification of events in the environment. About 466 million people worldwide suffer from hearing loss. Yet, the impact of hearing loss on how the senses work together is rarely investigated. Here, we investigate how a common sensory impairment, asymmetric conductive hearing loss (AHL), alters the way our senses interact by examining human orienting behaviour with normal hearing (NH) and acute AHL. This type of hearing loss disrupts auditory localization. We hypothesized that this creates a conflict between auditory and visual spatial estimates and alters how auditory and visual inputs are integrated to facilitate multisensory spatial perception. We analysed the spatial and temporal properties of saccades to auditory, visual and audiovisual stimuli before and after plugging the right ear of participants. Both spatial and temporal aspects of multisensory integration were affected by AHL. Compared with NH, AHL caused participants to make slow, inaccurate and unprecise saccades towards auditory targets. Surprisingly, increased weight on visual input resulted in accurate audiovisual localization with AHL. This came at a cost: saccade latencies for audiovisual targets increased significantly. The larger the auditory localization errors, the less participants were able to benefit from audiovisual integration in terms of saccade latency. Our results indicate that observers immediately change sensory weights to effectively deal with acute AHL and preserve audiovisual accuracy in a way that cannot be fully explained by statistical models of optimal cue integration.

Unsupervised Characterization of Prediction Error Markers in Unisensory and Multisensory Streams Reveal the Spatiotemporal Hierarchy of Cortical Information Processing

Elicited upon violation of regularity in stimulus presentation, mismatch negativity (MMN) reflects the brain's ability to perform automatic comparisons between consecutive stimuli and provides an electrophysiological index of sensory error detection whereas P300 is associated with cognitive processes such as updating of the working memory. To date, there has been extensive research on the roles of MMN and P300 individually, because of their potential to be used as clinical markers of consciousness and attention, respectively. Here, we intend to explore with an unsupervised and rigorous source estimation approach, the underlying cortical generators of MMN and P300, in the context of prediction error propagation along the hierarchies of brain information processing in healthy human participants. The existing methods of characterizing the two ERPs involve only approximate estimations of their amplitudes and latencies based on specific sensors of interest. Our objective is twofold: first, we introduce a novel data-driven unsupervised approach to compute latencies and amplitude of ERP components accurately on an individual-subject basis and reconfirm earlier findings. Second, we demonstrate that in multisensory environments, MMN generators seem to reflect a significant overlap of "modality-specific" and "modality-independent" information processing while P300 generators mark a shift toward completely "modality-independent" processing. Advancing earlier understanding that multisensory contexts speed up early sensory processing, our study reveals that temporal facilitation extends to even the later components of prediction error processing, using EEG experiments. Such knowledge can be of value to clinical research for characterizing the key developmental stages of lifespan aging, schizophrenia, and depression.

Atypical brain responses to 40-Hz click trains in girls with Rett syndrome: Auditory steady-state response and sustained wave

AIM

The current study aimed to infer neurophysiological mechanisms of auditory processing in children with Rett syndrome (RTT)-rare neurodevelopmental disorders caused by MECP2 mutations. We examined two brain responses elicited by 40-Hz click trains: auditory steady-state response (ASSR), which reflects fine temporal analysis of auditory input, and sustained wave (SW), which is associated with integral processing of the auditory signal.

METHODS

We recorded electroencephalogram findings in 43 patients with RTT (aged 2.92-17.1 years) and 43 typically developing children of the same age during 40-Hz click train auditory stimulation, which lasted for 500 ms and was presented with interstimulus intervals of 500 to 800 ms. Mixed-model ancova with age as a covariate was used to compare amplitude of ASSR and SW between groups, taking into account the temporal dynamics and topography of the responses.

RESULTS

Amplitude of SW was atypically small in children with RTT starting from early childhood, with the difference from typically developing children decreasing with age. ASSR showed a different pattern of developmental changes: the between-group difference was negligible in early childhood but increased with age as ASSR increased in the typically developing group, but not in those with RTT. Moreover, ASSR was associated with expressive speech development in patients, so that children who could use words had more pronounced ASSR.

CONCLUSION

ASSR and SW show promise as noninvasive electrophysiological biomarkers of auditory processing that have clinical relevance and can shed light onto the link between genetic impairment and the RTT phenotype.

Variance aftereffect within and between sensory modalities for visual and auditory domains

We can grasp various features of the outside world using summary statistics efficiently. Among these statistics, variance is an index of information homogeneity or reliability. Previous research has shown that visual variance information in the context of spatial integration is encoded directly as a unique feature, and currently perceived variance can be distorted by that of the preceding stimuli. In this study, we focused on variance perception in temporal integration. We investigated whether any variance aftereffects occurred in visual size and auditory pitch. Furthermore, to examine the mechanism of cross-modal variance perception, we also investigated whether variance aftereffects occur between different modalities. Four experimental conditions (a combination of sensory modalities of adaptor and test: visual-to-visual, visual-to-auditory, auditory-to-auditory, and auditory-to-visual) were conducted. Participants observed a sequence of visual or auditory stimuli perturbed in size or pitch with certain variance and performed a variance classification task before and after the variance adaptation phase. We found that in visual size, within modality adaptation to small or large variance, resulted in a variance aftereffect, indicating that variance judgments are biased in the direction away from that of the adapting stimulus. In auditory pitch, within modality adaptation to small variance caused variance aftereffect. For cross-modal combinations, adaptation to small variance in visual size resulted in variance aftereffect. However, the effect was weak, and variance aftereffect did not occur in other conditions. These findings indicate that the variance information of sequentially presented stimuli is encoded independently in visual and auditory domains.

Is there a bilingual advantage in auditory attention among children? A systematic review and meta-analysis of standardized auditory attention tests

A wealth of research has investigated the associations between bilingualism and cognition, especially in regards to executive function. Some developmental studies reveal different cognitive profiles between monolinguals and bilinguals in visual or audio-visual attention tasks, which might stem from their attention allocation differences. Yet, whether such distinction exists in the auditory domain alone is unknown. In this study, we compared differences in auditory attention, measured by standardized tests, between monolingual and bilingual children. A comprehensive literature search was conducted in three electronic databases: OVID Medline, OVID PsycInfo, and EBSCO CINAHL. Twenty studies using standardized tests to assess auditory attention in monolingual and bilingual participants aged less than 18 years were identified. We assessed the quality of these studies using a scoring tool for evaluating primary research. For statistical analysis, we pooled the effect size in a random-effects meta-analytic model, where between-study heterogeneity was quantified using the I2 statistic. No substantial publication bias was observed based on the funnel plot. Further, meta-regression modelling suggests that test measure (accuracy vs. response times) significantly affected the studies' effect sizes whereas other factors (e.g., participant age, stimulus type) did not. Specifically, studies reporting accuracy observed marginally greater accuracy in bilinguals (g = 0.10), whereas those reporting response times indicated faster latency in monolinguals (g = -0.34). There was little difference between monolingual and bilingual children's performance on standardized auditory attention tests. We also found that studies tend to include a wide variety of bilingual children but report limited language background information of the participants. This, unfortunately, limits the potential theoretical contributions of the reviewed studies. Recommendations to improve the quality of future research are discussed.

Virtual dynamic interaction games reveal impaired multisensory integration in women with migraine

OBJECTIVE

In this cross-sectional observational study, we aimed to investigate sensory profiles and multisensory integration processes in women with migraine using virtual dynamic interaction systems.

BACKGROUND

Compared to studies on unimodal sensory processing, fewer studies show that multisensory integration differs in patients with migraine. Multisensory integration of visual, auditory, verbal, and haptic modalities has not been evaluated in migraine.

METHODS

A 12-min virtual dynamic interaction game consisting of four parts was played by the participants. During the game, the participants were exposed to either visual stimuli only or multisensory stimuli in which auditory, verbal, and haptic stimuli were added to the visual stimuli. A total of 78 women participants (28 with migraine without aura and 50 healthy controls) were enrolled in this prospective exploratory study. Patients with migraine and healthy participants who met the inclusion criteria were randomized separately into visual and multisensory groups: Migraine multisensory (14 adults), migraine visual (14 adults), healthy multisensory (25 adults), and healthy visual (25 adults). The Sensory Profile Questionnaire was utilized to assess the participants' sensory profiles. The game scores and survey results were analyzed.

RESULTS

In visual stimulus, the gaming performance scores of patients with migraine without aura were similar to the healthy controls, at a median (interquartile range [IQR]) of 81.8 (79.5-85.8) and 80.9 (77.1-84.2) (p = 0.149). Error rate of visual stimulus in patients with migraine without aura were comparable to healthy controls, at a median (IQR) of 0.11 (0.08-0.13) and 0.12 (0.10-0.14), respectively (p = 0,166). In multisensory stimulation, average gaming score was lower in patients with migraine without aura compared to healthy individuals (median [IQR] 82.2 [78.8-86.3] vs. 78.6 [74.0-82.4], p = 0.028). In women with migraine, exposure to new sensory modality upon visual stimuli in the fourth, seventh, and tenth rounds (median [IQR] 78.1 [74.1-82.0], 79.7 [77.2-82.5], 76.5 [70.2-82.1]) exhibited lower game scores compared to visual stimuli only (median [IQR] 82.3 [77.9-87.8], 84.2 [79.7-85.6], 80.8 [79.0-85.7], p = 0.044, p = 0.049, p = 0.016). According to the Sensory Profile Questionnaire results, sensory sensitivity, and sensory avoidance scores of patients with migraine (median [IQR] score 45.5 [41.0-54.7] and 47.0 [41.5-51.7]) were significantly higher than healthy participants (median [IQR] score 39.0 [34.0-44.2] and 40.0 [34.0-48.0], p < 0.001, p = 0.001).

CONCLUSION

The virtual dynamic game approach showed for the first time that the gaming performance of patients with migraine without aura was negatively affected by the addition of auditory, verbal, and haptic stimuli onto visual stimuli. Multisensory integration of sensory modalities including haptic stimuli is disturbed even in the interictal period in women with migraine. Virtual games can be employed to assess the impact of sensory problems in the course of the disease. Also, sensory training could be a potential therapy target to improve multisensory processing in migraine.

Young adults and multisensory time perception: Visual and auditory pathways in comparison

The brain continuously encodes information about time, but how sensorial channels interact to achieve a stable representation of such ubiquitous information still needs to be determined. According to recent research, children show a potential interference in multisensory conditions, leading to a trade-off between two senses (sight and audition) when considering time-perception tasks. This study aimed to examine how healthy young adults behave when performing a time-perception task. In Experiment 1, we tested the effects of temporary sensory deprivation on both visual and auditory senses in a group of young adults. In Experiment 2, we compared the temporal performances of young adults in the auditory modality with those of two samples of children (sighted and sighted but blindfolded) selected from a previous study. Statistically significant results emerged when comparing the two pathways: young adults overestimated and showed a higher sensitivity to time in the auditory modality compared to the visual modality. Restricting visual and auditory input did not affect their time sensitivity. Moreover, children were more accurate at estimating time than young adults after a transient visual deprivation. This implies that as we mature, sensory deprivation does not constitute a benefit to time perception, and supports the hypothesis of a calibration process between senses with age. However, more research is needed to determine how this calibration process affects the developmental trajectories of time perception.

Audiovisual integration of rhythm in musicians and dancers

Music training is associated with better beat processing in the auditory modality. However, it is unknown how rhythmic training that emphasizes visual rhythms, such as dance training, might affect beat processing, nor whether training effects in general are modality specific. Here we examined how music and dance training interacted with modality during audiovisual integration and synchronization to auditory and visual isochronous sequences. In two experiments, musicians, dancers, and controls completed an audiovisual integration task and an audiovisual target-distractor synchronization task using dynamic visual stimuli (a bouncing figure). The groups performed similarly on the audiovisual integration tasks (Experiments 1 and 2). However, in the finger-tapping synchronization task (Experiment 1), musicians were more influenced by auditory distractors when synchronizing to visual sequences, while dancers were more influenced by visual distractors when synchronizing to auditory sequences. When participants synchronized with whole-body movements instead of finger-tapping (Experiment 2), all groups were more influenced by the visual distractor than the auditory distractor. Taken together, this study highlights how training is associated with audiovisual processing, and how different types of visual rhythmic stimuli and different movements alter beat perception and production outcome measures. Implications for the modality appropriateness hypothesis are discussed.

Modeling the Impact of Head-Body Rotations on Audio-Visual Spatial Perception for Virtual Reality Applications

Humans perceive the world by integrating multimodal sensory feedback, including visual and auditory stimuli, which holds true in virtual reality (VR) environments. Proper synchronization of these stimuli is crucial for perceiving a coherent and immersive VR experience. In this work, we focus on the interplay between audio and vision during localization tasks involving natural head-body rotations. We explore the impact of audio-visual offsets and rotation velocities on users' directional localization acuity for various viewing modes. Using psychometric functions, we model perceptual disparities between visual and auditory cues and determine offset detection thresholds. Our findings reveal that target localization accuracy is affected by perceptual audio-visual disparities during head-body rotations, but remains consistent in the absence of stimuli-head relative motion. We then showcase the effectiveness of our approach in predicting and enhancing users' localization accuracy within realistic VR gaming applications. To provide additional support for our findings, we implement a natural VR game wherein we apply a compensatory audio-visual offset derived from our measured psychometric functions. As a result, we demonstrate a substantial improvement of up to 40% in participants' target localization accuracy. We additionally provide guidelines for content creation to ensure coherent and seamless VR experiences.

Dissecting Mismatch Negativity: Early and Late Subcomponents for Detecting Deviants in Local and Global Sequence Regularities

Mismatch negativity (MMN) is commonly recognized as a neural signal of prediction error evoked by deviants from the expected patterns of sensory input. Studies show that MMN diminishes when sequence patterns become more predictable over a longer timescale. This implies that MMN is composed of multiple subcomponents, each responding to different levels of temporal regularities. To probe the hypothesized subcomponents in MMN, we record human electroencephalography during an auditory local-global oddball paradigm where the tone-to-tone transition probability (local regularity) and the overall sequence probability (global regularity) are manipulated to control temporal predictabilities at two hierarchical levels. We find that the size of MMN is correlated with both probabilities and the spatiotemporal structure of MMN can be decomposed into two distinct subcomponents. Both subcomponents appear as negative waveforms, with one peaking early in the central-frontal area and the other late in a more frontal area. With a quantitative predictive coding model, we map the early and late subcomponents to the prediction errors that are tied to local and global regularities, respectively. Our study highlights the hierarchical complexity of MMN and offers an experimental and analytical platform for developing a multitiered neural marker applicable in clinical settings.

The effects of music and auditory stimulation on autonomic arousal, cognition and attention: A systematic review

According to the arousal-mood hypothesis, changes in arousal and mood when exposed to auditory stimulation underlie the detrimental effects or improvements in cognitive performance. Findings supporting or against this hypothesis are, however, often based on subjective ratings of arousal rather than autonomic/physiological indices of arousal. To assess the arousal-mood hypothesis, we carried out a systematic review of the literature on 31 studies investigating cardiac, electrodermal, and pupillometry measures when exposed to different types of auditory stimulation (music, ambient noise, white noise, and binaural beats) in relation to cognitive performance. Our review suggests that the effects of music, noise, or binaural beats on cardiac, electrodermal, and pupillometry measures in relation to cognitive performance are either mixed or insufficient to draw conclusions. Importantly, the evidence for or against the arousal-mood hypothesis is at best indirect because autonomic arousal and cognitive performance are often considered separately. Future research is needed to directly evaluate the effects of auditory stimulation on autonomic arousal and cognitive performance holistically.

Theta and alpha oscillations may underlie improved attention and working memory in musically trained children

INTRODUCTION

Attention and working memory are key cognitive functions that allow us to select and maintain information in our mind for a short time, being essential for our daily life and, in particular, for learning and academic performance. It has been shown that musical training can improve working memory performance, but it is still unclear if and how the neural mechanisms of working memory and particularly attention are implicated in this process. In this work, we aimed to identify the oscillatory signature of bimodal attention and working memory that contributes to improved working memory in musically trained children.

MATERIALS AND METHODS

We recruited children with and without musical training and asked them to complete a bimodal (auditory/visual) attention and working memory task, whereas their brain activity was measured using electroencephalography. Behavioral, time-frequency, and source reconstruction analyses were made.

RESULTS

Results showed that, overall, musically trained children performed better on the task than children without musical training. When comparing musically trained children with children without musical training, we found modulations in the alpha band pre-stimuli onset and the beginning of stimuli onset in the frontal and parietal regions. These correlated with correct responses to the attended modality. Moreover, during the end phase of stimuli presentation, we found modulations correlating with correct responses independent of attention condition in the theta and alpha bands, in the left frontal and right parietal regions.

CONCLUSIONS

These results suggest that musically trained children have improved neuronal mechanisms for both attention allocation and memory encoding. Our results can be important for developing interventions for people with attention and working memory difficulties.

Topography and Ensemble Activity in the Auditory Cortex of a Mouse Model of Fragile X Syndrome

Autism spectrum disorder (ASD) is often associated with social communication impairments and specific sound processing deficits, for example, problems in following speech in noisy environments. To investigate underlying neuronal processing defects located in the auditory cortex (AC), we performed two-photon Ca2+ imaging in FMR1 (fragile X messenger ribonucleoprotein 1) knock-out (KO) mice, a model for fragile X syndrome (FXS), the most common cause of hereditary ASD in humans. For primary AC (A1) and the anterior auditory field (AAF), topographic frequency representation was less ordered compared with control animals. We additionally analyzed ensemble AC activity in response to various sounds and found subfield-specific differences. In A1, ensemble correlations were lower in general, while in secondary AC (A2), correlations were higher in response to complex sounds, but not to pure tones. Furthermore, sound specificity of ensemble activity was decreased in AAF. Repeating these experiments 1 week later revealed no major differences regarding representational drift. Nevertheless, we found subfield- and genotype-specific changes in ensemble correlation values between the two times points, hinting at alterations in network stability in FMR1 KO mice. These detailed insights into AC network activity and topography in FMR1 KO mice add to the understanding of auditory processing defects in FXS.

Effects of absolute pitch on brain activation and functional connectivity during hearing-in-noise perception

Hearing-in-noise (HIN) ability is crucial in speech and music communication. Recent evidence suggests that absolute pitch (AP), the ability to identify isolated musical notes, is associated with HIN benefits. A theoretical account postulates a link between AP ability and neural network indices of segregation. However, how AP ability modulates the brain activation and functional connectivity underlying HIN perception remains unclear. Here we used functional magnetic resonance imaging to contrast brain responses among a sample (n = 45) comprising 15 AP musicians, 15 non-AP musicians, and 15 non-musicians in perceiving Mandarin speech and melody targets under varying signal-to-noise ratios (SNRs: No-Noise, 0, -9 dB). Results reveal that AP musicians exhibited increased activation in auditory and superior frontal regions across both HIN domains (music and speech), irrespective of noise levels. Notably, substantially higher sensorimotor activation was found in AP musicians when the target was music compared to speech. Furthermore, we examined AP effects on neural connectivity using psychophysiological interaction analysis with the auditory cortex as the seed region. AP musicians showed decreased functional connectivity with the sensorimotor and middle frontal gyrus compared to non-AP musicians. Crucially, AP differentially affected connectivity with parietal and frontal brain regions depending on the HIN domain being music or speech. These findings suggest that AP plays a critical role in HIN perception, manifested by increased activation and functional independence between auditory and sensorimotor regions for perceiving music and speech streams.

Alpha9alpha10 knockout mice show altered physiological and behavioral responses to signals in masking noise

Medial olivocochlear (MOC) efferents modulate outer hair cell motility through specialized nicotinic acetylcholine receptors to support encoding of signals in noise. Transgenic mice lacking the alpha9 subunits of these receptors (α9KOs) have normal hearing in quiet and noise, but lack classic cochlear suppression effects and show abnormal temporal, spectral, and spatial processing. Mice deficient for both the alpha9 and alpha10 receptor subunits (α9α10KOs) may exhibit more severe MOC-related phenotypes. Like α9KOs, α9α10KOs have normal auditory brainstem response (ABR) thresholds and weak MOC reflexes. Here, we further characterized auditory function in α9α10KO mice. Wild-type (WT) and α9α10KO mice had similar ABR thresholds and acoustic startle response amplitudes in quiet and noise, and similar frequency and intensity difference sensitivity. α9α10KO mice had larger ABR Wave I amplitudes than WTs in quiet and noise. Other ABR metrics of hearing-in-noise function yielded conflicting findings regarding α9α10KO susceptibility to masking effects. α9α10KO mice also had larger startle amplitudes in tone backgrounds than WTs. Overall, α9α10KO mice had grossly normal auditory function in quiet and noise, although their larger ABR amplitudes and hyperreactive startles suggest some auditory processing abnormalities. These findings contribute to the growing literature showing mixed effects of MOC dysfunction on hearing.

Population coding of time-varying sounds in the nonlemniscal inferior colliculus

The inferior colliculus (IC) of the midbrain is important for complex sound processing, such as discriminating conspecific vocalizations and human speech. The IC's nonlemniscal, dorsal "shell" region is likely important for this process, as neurons in these layers project to higher-order thalamic nuclei that subsequently funnel acoustic signals to the amygdala and nonprimary auditory cortices, forebrain circuits important for vocalization coding in a variety of mammals, including humans. However, the extent to which shell IC neurons transmit acoustic features necessary to discern vocalizations is less clear, owing to the technical difficulty of recording from neurons in the IC's superficial layers via traditional approaches. Here, we use two-photon Ca2+ imaging in mice of either sex to test how shell IC neuron populations encode the rate and depth of amplitude modulation, important sound cues for speech perception. Most shell IC neurons were broadly tuned, with a low neurometric discrimination of amplitude modulation rate; only a subset was highly selective to specific modulation rates. Nevertheless, neural network classifier trained on fluorescence data from shell IC neuron populations accurately classified amplitude modulation rate, and decoding accuracy was only marginally reduced when highly tuned neurons were omitted from training data. Rather, classifier accuracy increased monotonically with the modulation depth of the training data, such that classifiers trained on full-depth modulated sounds had median decoding errors of ∼0.2 octaves. Thus, shell IC neurons may transmit time-varying signals via a population code, with perhaps limited reliance on the discriminative capacity of any individual neuron.NEW & NOTEWORTHY The IC's shell layers originate a "nonlemniscal" pathway important for perceiving vocalization sounds. However, prior studies suggest that individual shell IC neurons are broadly tuned and have high response thresholds, implying a limited reliability of efferent signals. Using Ca2+ imaging, we show that amplitude modulation is accurately represented in the population activity of shell IC neurons. Thus, downstream targets can read out sounds' temporal envelopes from distributed rate codes transmitted by populations of broadly tuned neurons.