Effects of background noise on inter-trial phase coherence and auditory N1-P2 responses to speech stimuli

Functional benefits of continuous vs. categorical listening strategies on the neural encoding and perception of noise-degraded speech

Acoustic information in speech changes continuously, yet listeners form discrete perceptual categories to ease the demands of perception. Being a more continuous/gradient as opposed to a more discrete/categorical listener may be further advantageous for understanding speech in noise by increasing perceptual flexibility and resolving ambiguity. The degree to which a listener's responses to a continuum of speech sounds are categorical versus continuous can be quantified using visual analog scaling (VAS) during speech labeling tasks. Here, we recorded event-related brain potentials (ERPs) to vowels along an acoustic-phonetic continuum (/u/ to /a/) while listeners categorized phonemes in both clean and noise conditions. Behavior was assessed using standard two alternative forced choice (2AFC) and VAS paradigms to evaluate categorization under task structures that promote discrete vs. continuous hearing, respectively. Behaviorally, identification curves were steeper under 2AFC vs. VAS categorization but were relatively immune to noise, suggesting robust access to abstract, phonetic categories even under signal degradation. Behavioral slopes were correlated with listeners' QuickSIN scores; shallower slopes corresponded with better speech in noise performance, suggesting a perceptual advantage to noise degraded speech comprehension conferred by a more gradient listening strategy. At the neural level, P2 amplitudes and latencies of the ERPs were modulated by task and noise; VAS responses were larger and showed greater noise-related latency delays than 2AFC responses. More gradient responders had smaller shifts in ERP latency with noise, suggesting their neural encoding of speech was more resilient to noise degradation. Interestingly, source-resolved ERPs showed that more gradient listening was also correlated with stronger neural responses in left superior temporal gyrus. Our results demonstrate that listening strategy modulates the categorical organization of speech and behavioral success, with more continuous/gradient listening being advantageous to sentential speech in noise perception.

The temporal mismatch across listening sides affects cortical auditory evoked responses in normal hearing listeners and cochlear implant users with contralateral acoustic hearing

Combining a cochlear implant with contralateral acoustic hearing typically enhances speech understanding, although this improvement varies among CI users and can lead to an interference effect. This variability may be associated with the effectiveness of the integration between electric and acoustic stimulation, which might be affected by the temporal mismatch between the two listening sides. Finding methods to compensate for the temporal mismatch might contribute to the optimal adjustment of bimodal devices and to improve hearing in CI users with contralateral acoustic hearing. The current study investigates cortical auditory evoked potentials (CAEPs) in normal hearing listeners (NH) and CI users with contralateral acoustic hearing. In NH, the amplitude of the N1 peak and the maximum phase locking value (PLV) were analyzed under monaural, binaural, and binaural temporally mismatched conditions. In CI users, CAEPs were measured when listening with CI only (CIS_only), acoustically only (AS_only) and with both sides together (CIS+AS). When listening with CIS+AS, various interaural delays were introduced between the electric and acoustic stimuli. In NH listeners, interaural temporal mismatch resulted in decreased N1 amplitude and PLV. Moreover, PLV is suggested as a more sensitive measure to investigate the integration of information between the two listening sides. CI users showed varied N1 latencies between the AS_only and CIS_only listening conditions, with increased N1 amplitude when the temporal mismatch was compensated. A tendency towards increased PLV was also observed, however, to a lesser extent than in NH listeners, suggesting a limited integration between electric and acoustic stimulation. This work highlights the potential of CAEPs measurement to investigate cortical processing of the information between two listening sides in NH and bimodal CI users.

Contribution of inter-trial phase coherence at theta, alpha, and beta frequencies in auditory change detection

Introduction

Auditory change detection is a pre-attentive cortical auditory processing ability. Many neurological and psychological disorders can lead to defects in this process. Some studies have shown that phase synchronization may be related to auditory discrimination. However, the specific contributions of phase synchronization at different frequencies remain unclear.

Methods

We analyzed the electroencephalogram (EEG) data of 29 healthy adults using an oddball paradigm consisting of a standard stimulus and five deviant stimuli with varying frequency modulation patterns, including midpoint frequency transitions and linear frequency modulation. We then compared the peak amplitude and latency of inter-trial phase coherence (ITC) at the theta(θ), alpha(α), and beta(β) frequencies, as well as the N1 component, and their relationships with stimulus changes. At the same time, the characteristics of inter-trial phase coherence in response to the pure tone stimulation and chirp sound with a fine time-frequency structure were also assessed.

Result

When the stimulus frequency did not change relative to the standard stimulus, the peak latency of phase coherence at β and α frequencies was consistent with that of the N1 component. The inter-trial phase coherence at β frequency (β-ITC)served as a faster indicator for detecting frequency transition when the stimulus frequency was changed relative to the standard stimulus. β-ITC demonstrates temporal stability when detecting pure sinusoidal tones and their frequency changes, and is less susceptible to interference from other neural activities. The phase coherence at θ frequency could integrate the frequency and temporal characteristics of deviant into a single representation, which can be compared with the memory trace formed by the standard stimulus, thus effectively identifying auditory changes. Pure sinusoidal tone stimulation could induce higher inter-trial phase coherence in a smaller time window, but chirp sounds with a fine time-frequency structure required longer latencies to achieve phase coherence.

Conclusion

Phase coherence at theta, alpha, and beta frequencies are all involved in auditory change detection, but play different roles in this automatic process. Complex time-frequency modulated stimuli require longer processing time for effective change detection.

Detecting Emotional Prosody in Real Words: Electrophysiological Evidence From a Modified Multifeature Oddball Paradigm

PURPOSE

Emotional voice conveys important social cues that demand listeners' attention and timely processing. This event-related potential study investigated the feasibility of a multifeature oddball paradigm to examine adult listeners' neural responses to detecting emotional prosody changes in nonrepeating naturally spoken words.

METHOD

Thirty-three adult listeners completed the experiment by passively listening to the words in neutral and three alternating emotions while watching a silent movie. Previous research documented preattentive change-detection electrophysiological responses (e.g., mismatch negativity [MMN], P3a) to emotions carried by fixed syllables or words. Given that the MMN and P3a have also been shown to reflect extraction of abstract regularities over repetitive acoustic patterns, this study employed a multifeature oddball paradigm to compare listeners' MMN and P3a to emotional prosody change from neutral to angry, happy, and sad emotions delivered with hundreds of nonrepeating words in a single recording session.

RESULTS

Both MMN and P3a were successfully elicited by the emotional prosodic change over the varying linguistic context. Angry prosody elicited the strongest MMN compared with happy and sad prosodies. Happy prosody elicited the strongest P3a in the centro-frontal electrodes, and angry prosody elicited the smallest P3a.

CONCLUSIONS

The results demonstrated that listeners were able to extract the acoustic patterns for each emotional prosody category over constantly changing spoken words. The findings confirm the feasibility of the multifeature oddball paradigm in investigating emotional speech processing beyond simple acoustic change detection, which may potentially be applied to pediatric and clinical populations.

Slow neural oscillations explain temporal fluctuations in distractibility

Human environments comprise various sources of distraction, which often occur unexpectedly in time. The proneness to distraction (i.e., distractibility) is posited to be independent of attentional sampling of targets, but its temporal dynamics and neurobiological basis are largely unknown. Brain oscillations in the theta band (3 - 8Hz) have been associated with fluctuating neural excitability, which is hypothesised here to explain rhythmic modulation of distractibility. In a pitch discrimination task (N = 30) with unexpected auditory distractors, we show that distractor-evoked neural responses in the electroencephalogram and perceptual susceptibility to distraction were co-modulated and cycled approximately 3 - 5 times per second. Pre-distractor neural phase in left inferior frontal and insular cortex regions explained fluctuating distractibility. Thus, human distractibility is not constant but fluctuates on a subsecond timescale. Furthermore, slow neural oscillations subserve the behavioural consequences of a hitherto largely unexplained but ever-increasing phenomenon in modern environments - distraction by unexpected sound.

The effect of topic familiarity and volatility of auditory scene on selective auditory attention

Selective auditory attention has been shown to modulate the cortical representation of speech. This effect has been well documented in acoustically more challenging environments. However, the influence of top-down factors, in particular topic familiarity, on this process remains unclear, despite evidence that semantic information can promote speech-in-noise perception. Apart from individual features forming a static listening condition, dynamic and irregular changes of auditory scenes-volatile listening environments-have been less studied. To address these gaps, we explored the influence of topic familiarity and volatile listening on the selective auditory attention process during dichotic listening using electroencephalography. When stories with unfamiliar topics were presented, participants' comprehension was severely degraded. However, their cortical activity selectively tracked the speech of the target story well. This implies that topic familiarity hardly influences the speech tracking neural index, possibly when the bottom-up information is sufficient. However, when the listening environment was volatile and the listeners had to re-engage in new speech whenever auditory scenes altered, the neural correlates of the attended speech were degraded. In particular, the cortical response to the attended speech and the spatial asymmetry of the response to the left and right attention were significantly attenuated around 100-200 ms after the speech onset. These findings suggest that volatile listening environments could adversely affect the modulation effect of selective attention, possibly by hampering proper attention due to increased perceptual load.

Neurocognitive Dynamics of Prosodic Salience over Semantics during Explicit and Implicit Processing of Basic Emotions in Spoken Words

How language mediates emotional perception and experience is poorly understood. The present event-related potential (ERP) study examined the explicit and implicit processing of emotional speech to differentiate the relative influences of communication channel, emotion category and task type in the prosodic salience effect. Thirty participants (15 women) were presented with spoken words denoting happiness, sadness and neutrality in either the prosodic or semantic channel. They were asked to judge the emotional content (explicit task) and speakers' gender (implicit task) of the stimuli. Results indicated that emotional prosody (relative to semantics) triggered larger N100, P200 and N400 amplitudes with greater delta, theta and alpha inter-trial phase coherence (ITPC) and event-related spectral perturbation (ERSP) values in the corresponding early time windows, and continued to produce larger LPC amplitudes and faster responses during late stages of higher-order cognitive processing. The relative salience of prosodic and semantics was modulated by emotion and task, though such modulatory effects varied across different processing stages. The prosodic salience effect was reduced for sadness processing and in the implicit task during early auditory processing and decision-making but reduced for happiness processing in the explicit task during conscious emotion processing. Additionally, across-trial synchronization of delta, theta and alpha bands predicted the ERP components with higher ITPC and ERSP values significantly associated with stronger N100, P200, N400 and LPC enhancement. These findings reveal the neurocognitive dynamics of emotional speech processing with prosodic salience tied to stage-dependent emotion- and task-specific effects, which can reveal insights into understanding language and emotion processing from cross-linguistic/cultural and clinical perspectives.

Dynamic auditory contributions to error detection revealed in the discrimination of Same and Different syllable pairs

During speech production auditory regions operate in concert with the anterior dorsal stream to facilitate online error detection. As the dorsal stream also is known to activate in speech perception, the purpose of the current study was to probe the role of auditory regions in error detection during auditory discrimination tasks as stimuli are encoded and maintained in working memory. A priori assumptions are that sensory mismatch (i.e., error) occurs during the discrimination of Different (mismatched) but not Same (matched) syllable pairs. Independent component analysis was applied to raw EEG data recorded from 42 participants to identify bilateral auditory alpha rhythms, which were decomposed across time and frequency to reveal robust patterns of event related synchronization (ERS; inhibition) and desynchronization (ERD; processing) over the time course of discrimination events. Results were characterized by bilateral peri-stimulus alpha ERD transitioning to alpha ERS in the late trial epoch, with ERD interpreted as evidence of working memory encoding via Analysis by Synthesis and ERS considered evidence of speech-induced-suppression arising during covert articulatory rehearsal to facilitate working memory maintenance. The transition from ERD to ERS occurred later in the left hemisphere in Different trials than in Same trials, with ERD and ERS temporally overlapping during the early post-stimulus window. Results were interpreted to suggest that the sensory mismatch (i.e., error) arising from the comparison of the first and second syllable elicits further processing in the left hemisphere to support working memory encoding and maintenance. Results are consistent with auditory contributions to error detection during both encoding and maintenance stages of working memory, with encoding stage error detection associated with stimulus concordance and maintenance stage error detection associated with task-specific retention demands.

Age-related hearing loss is associated with alterations in temporal envelope processing in different neural generators along the auditory pathway

People with age-related hearing loss suffer from speech understanding difficulties, even after correcting for differences in hearing audibility. These problems are not only attributed to deficits in audibility but are also associated with changes in central temporal processing. The goal of this study is to obtain an understanding of potential alterations in temporal envelope processing for middle-aged and older persons with and without hearing impairment. The time series of activity of subcortical and cortical neural generators was reconstructed using a minimum-norm imaging technique. This novel technique allows for reconstructing a wide range of neural generators with minimal prior assumptions regarding the number and location of the generators. The results indicated that the response strength and phase coherence of middle-aged participants with hearing impairment (HI) were larger than for normal-hearing (NH) ones. In contrast, for the older participants, a significantly smaller response strength and phase coherence were observed in the participants with HI than the NH ones for most modulation frequencies. Hemispheric asymmetry in the response strength was also altered in middle-aged and older participants with hearing impairment and showed asymmetry toward the right hemisphere. Our brain source analyses show that age-related hearing loss is accompanied by changes in the temporal envelope processing, although the nature of these changes varies with age.

EEG and fMRI coupling and decoupling based on joint independent component analysis (jICA)

BACKGROUND

Meaningful integration of functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) requires knowing whether these measurements reflect the activity of the same neural sources, i.e., estimating the degree of coupling and decoupling between the neuroimaging modalities.

NEW METHOD

This paper proposes a method to quantify the coupling and decoupling of fMRI and EEG signals based on the mixing matrix produced by joint independent component analysis (jICA). The method is termed fMRI/EEG-jICA.

RESULTS

fMRI and EEG acquired during a syllable detection task with variable syllable presentation rates (0.25-3 Hz) were separated with jICA into two spatiotemporally distinct components, a primary component that increased nonlinearly in amplitude with syllable presentation rate, putatively reflecting an obligatory auditory response, and a secondary component that declined nonlinearly with syllable presentation rate, putatively reflecting an auditory attention orienting response. The two EEG subcomponents were of similar amplitude, but the secondary fMRI subcomponent was ten folds smaller than the primary one.

COMPARISON TO EXISTING METHOD

FMRI multiple regression analysis yielded a map more consistent with the primary than secondary fMRI subcomponent of jICA, as determined by a greater area under the curve (0.5 versus 0.38) in a sensitivity and specificity analysis of spatial overlap.

CONCLUSION

fMRI/EEG-jICA revealed spatiotemporally distinct brain networks with greater sensitivity than fMRI multiple regression analysis, demonstrating how this method can be used for leveraging EEG signals to inform the detection and functional characterization of fMRI signals. fMRI/EEG-jICA may be useful for studying neurovascular coupling at a macro-level, e.g., in neurovascular disorders.

Atypical processing in neural source analysis of speech envelope modulations in adolescents with dyslexia

Different studies have suggested that language and developmental disorders such as dyslexia are associated with a disturbance of auditory entrainment and of the functional hemispheric asymmetries during speech processing. These disorders typically result from an issue in the phonological component of language that causes problems to represent and manipulate the phonological structure of words at the syllable and/or phoneme level. We used Auditory Steady-State Responses (ASSRs) in EEG recordings to investigate the brain activation and hemisphere asymmetry of theta, alpha, beta and low-gamma range oscillations in typical readers and readers with dyslexia. The aim was to analyse whether the group differences found in previous electrode level studies were caused by a different source activation pattern or conversely was an effect that could be found on the active brain sources. We could not find differences in the brain locations of the main active brain sources. However, we observed differences in the extracted waveforms. The group average of the first DSS component of all signal-to-noise ratios of ASSR at source level was higher than the group averages at the electrode level. These analyses included a lower alpha synchronisation in adolescents with dyslexia and the possibility of compensatory mechanisms in theta, beta and low-gamma frequency bands. The main brain auditory sources were located in cortical regions around the auditory cortex. Thus, the differences observed in auditory EEG experiments would, according to our findings, have their origin in the intrinsic oscillatory mechanisms of the brain cortical sources related to speech perception.

Binaural Background Noise Enhances Neuromagnetic Responses from Auditory Cortex

The presence of binaural low-level background noise has been shown to enhance the transient evoked N1 response at about 100 ms after sound onset. This increase in N1 amplitude is thought to reflect noise-mediated efferent feedback facilitation from the auditory cortex to lower auditory centers. To test this hypothesis, we recorded auditory-evoked fields using magnetoencephalography while participants were presented with binaural harmonic complex tones embedded in binaural or monaural background noise at signal-to-noise ratios of 25 dB (low noise) or 5 dB (higher noise). Half of the stimuli contained a gap in the middle of the sound. The source activities were measured in bilateral auditory cortices. The onset and gap N1 response increased with low binaural noise, but high binaural and low monaural noise did not affect the N1 amplitudes. P1 and P2 onset and gap responses were consistently attenuated by background noise, and noise level and binaural/monaural presentation showed distinct effects. Moreover, the evoked gamma synchronization was also reduced by background noise, and it showed a lateralized reduction for monaural noise. The effects of noise on the N1 amplitude follow a bell-shaped characteristic that could reflect an optimal representation of acoustic information for transient events embedded in noise.

Impaired interaural correlation processing in people with schizophrenia

Detection of transient changes in interaural correlation is based on the temporal precision of the central representations of acoustic signals. Whether schizophrenia impairs the temporal precision in the interaural correlation process is not clear. In both participants with schizophrenia and matched healthy-control participants, this study examined the detection of a break in interaural correlation (BIC, a change in interaural correlation from 1 to 0 and back to 1), including the longest interaural delay at which a BIC was just audible, representing the temporal extent of the primitive auditory memory (PAM). Moreover, BIC-induced electroencephalograms (EEGs) and the relationships between the early binaural psychoacoustic processing and higher cognitive functions, which were assessed by the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), were examined. The results showed that compared to healthy controls, participants with schizophrenia exhibited poorer BIC detection, PAM and RBANS score. Both the BIC-detection accuracy and the PAM extent were correlated with the RBANS score. Moreover, participants with schizophrenia showed weaker BIC-induced N1-P2 amplitude which was correlated with both theta-band power and inter-trial phase coherence. These results suggested that schizophrenia impairs the temporal precision of the central representations of acoustic signals, affecting both interaural correlation processing and higher-order cognitions.

Neurophysiological improvements in speech-in-noise task after short-term choir training in older adults

Perceiving speech in noise (SIN) is important for health and well-being and decreases with age. Musicians show improved speech-in-noise abilities and reduced age-related auditory decline, yet it is unclear whether short term music engagement has similar effects. In this randomized control trial we used a pre-post design to investigate whether a 12-week music intervention in adults aged 50-65 without prior music training and with subjective hearing loss improves well-being, speech-in-noise abilities, and auditory encoding and voluntary attention as indexed by auditory evoked potentials (AEPs) in a syllable-in-noise task, and later AEPs in an oddball task. Age and gender-matched adults were randomized to a choir or control group. Choir participants sang in a 2-hr ensemble with 1-hr home vocal training weekly; controls listened to a 3-hr playlist weekly, attended concerts, and socialized online with fellow participants. From pre- to post-intervention, no differences between groups were observed on quantitative measures of well-being or behavioral speech-in-noise abilities. In the choir group, but not the control group, changes in the N1 component were observed for the syllable-in-noise task, with increased N1 amplitude in the passive condition and decreased N1 latency in the active condition. During the oddball task, larger N1 amplitudes to the frequent standard stimuli were also observed in the choir but not control group from pre to post intervention. Findings have implications for the potential role of music training to improve sound encoding in individuals who are in the vulnerable age range and at risk of auditory decline.

Neural coding of formant-exaggerated speech and nonspeech in children with and without autism spectrum disorders

The presence of vowel exaggeration in infant-directed speech (IDS) may adapt to the age-appropriate demands in speech and language acquisition. Previous studies have provided behavioral evidence of atypical auditory processing towards IDS in children with autism spectrum disorders (ASD), while the underlying neurophysiological mechanisms remain unknown. This event-related potential (ERP) study investigated the neural coding of formant-exaggerated speech and nonspeech in 24 4- to 11-year-old children with ASD and 24 typically-developing (TD) peers. The EEG data were recorded using an alternating block design, in which each stimulus type (exaggerated/non-exaggerated sound) was presented with equal probability. ERP waveform analysis revealed an enhanced P1 for vowel formant exaggeration in the TD group but not in the ASD group. This speech-specific atypical processing in ASD was not found for the nonspeech stimuli which showed similar P1 enhancement in both ASD and TD groups. Moreover, the time-frequency analysis indicated that children with ASD showed differences in neural synchronization in the delta-theta bands for processing acoustic formant changes embedded in nonspeech. Collectively, the results add substantiating neurophysiological evidence (i.e., a lack of neural enhancement effect of vowel exaggeration) for atypical auditory processing of IDS in children with ASD, which may exert a negative effect on phonetic encoding and language learning. LAY SUMMARY: Atypical responses to motherese might act as a potential early marker of risk for children with ASD. This study investigated the neural responses to such socially relevant stimuli in the ASD brain, and the results suggested a lack of neural enhancement responding to the motherese even in individuals without intellectual disability.

Effects of spatial separation with better- ear listening on N1-P2 complex

OBJECTIVE

The purpose of this study was to determine better- ear listening effect on spatial separation with the N1-P2 complex.

METHODS

Twenty individuals with normal hearing participated in this study. The speech stimulus /ba/ was presented in front of the participant (0°). Continuous Speech Noise (5 dB signal-to-noise ratio) was presented either in front of the participant (0°), left-side (-90°), or right-side (+90°). N1- P2 complex has been recorded in quiet and three noisy conditions.

RESULTS

There was a remarkable effect of noise direction on N1, P2 latencies. When the noise was separated from the stimulus, N1 and P2 latency increased in terms of when noise was co-located with the stimulus. There was no statistically significant difference in N1-P2 amplitudes between the stimulus-only and co-located condition. N1-P2 amplitude was increased when the noise came from the sides, according to the stimulus-only and co-located conditions.

CONCLUSION

These findings demonstrate that the latency shifts on N1-P2 complex explain cortical mechanisms of spatial separation in better-ear listening.

Neural Generators Underlying Temporal Envelope Processing Show Altered Responses and Hemispheric Asymmetry Across Age

Speech understanding problems are highly prevalent in the aging population, even when hearing sensitivity is clinically normal. These difficulties are attributed to changes in central temporal processing with age and can potentially be captured by age-related changes in neural generators. The aim of this study is to investigate age-related changes in a wide range of neural generators during temporal processing in middle-aged and older persons with normal audiometric thresholds. A minimum-norm imaging technique is employed to reconstruct cortical and subcortical neural generators of temporal processing for different acoustic modulations. The results indicate that for relatively slow modulations (<50 Hz), the response strength of neural sources is higher in older adults than in younger ones, while the phase-locking does not change. For faster modulations (80 Hz), both the response strength and the phase-locking of neural sources are reduced in older adults compared to younger ones. These age-related changes in temporal envelope processing of slow and fast acoustic modulations are possibly due to loss of functional inhibition, which is accompanied by aging. Both cortical (primary and non-primary) and subcortical neural generators demonstrate similar age-related changes in response strength and phase-locking. Hemispheric asymmetry is also altered in older adults compared to younger ones. Alterations depend on the modulation frequency and side of stimulation. The current findings at source level could have important implications for the understanding of age-related changes in auditory temporal processing and for developing advanced rehabilitation strategies to address speech understanding difficulties in the aging population.

Decoding Hearing-Related Changes in Older Adults' Spatiotemporal Neural Processing of Speech Using Machine Learning

Speech perception in noisy environments depends on complex interactions between sensory and cognitive systems. In older adults, such interactions may be affected, especially in those individuals who have more severe age-related hearing loss. Using a data-driven approach, we assessed the temporal (when in time) and spatial (where in the brain) characteristics of cortical speech-evoked responses that distinguish older adults with or without mild hearing loss. We performed source analyses to estimate cortical surface signals from the EEG recordings during a phoneme discrimination task conducted under clear and noise-degraded conditions. We computed source-level ERPs (i.e., mean activation within each ROI) from each of the 68 ROIs of the Desikan-Killiany (DK) atlas, averaged over a randomly chosen 100 trials without replacement to form feature vectors. We adopted a multivariate feature selection method called stability selection and control to choose features that are consistent over a range of model parameters. We use parameter optimized support vector machine (SVM) as a classifiers to investigate the time course and brain regions that segregate groups and speech clarity. For clear speech perception, whole-brain data revealed a classification accuracy of 81.50% [area under the curve (AUC) 80.73%; F1-score 82.00%], distinguishing groups within ∼60 ms after speech onset (i.e., as early as the P1 wave). We observed lower accuracy of 78.12% [AUC 77.64%; F1-score 78.00%] and delayed classification performance when speech was embedded in noise, with group segregation at 80 ms. Separate analysis using left (LH) and right hemisphere (RH) regions showed that LH speech activity was better at distinguishing hearing groups than activity measured in the RH. Moreover, stability selection analysis identified 12 brain regions (among 1428 total spatiotemporal features from 68 regions) where source activity segregated groups with >80% accuracy (clear speech); whereas 16 regions were critical for noise-degraded speech to achieve a comparable level of group segregation (78.7% accuracy). Our results identify critical time-courses and brain regions that distinguish mild hearing loss from normal hearing in older adults and confirm a larger number of active areas, particularly in RH, when processing noise-degraded speech information.

Investigating Influences of Medial Olivocochlear Efferent System on Central Auditory Processing and Listening in Noise: A Behavioral and Event-Related Potential Study

This electrophysiological study investigated the role of the medial olivocochlear (MOC) efferents in listening in noise. Both ears of eleven normal-hearing adult participants were tested. The physiological tests consisted of transient-evoked otoacoustic emission (TEOAE) inhibition and the measurement of cortical event-related potentials (ERPs). The mismatch negativity (MMN) and P300 responses were obtained in passive and active listening tasks, respectively. Behavioral responses for the word recognition in noise test were also analyzed. Consistent with previous findings, the TEOAE data showed significant inhibition in the presence of contralateral acoustic stimulation. However, performance in the word recognition in noise test was comparable for the two conditions (i.e., without contralateral stimulation and with contralateral stimulation). Peak latencies and peak amplitudes of MMN and P300 did not show changes with contralateral stimulation. Behavioral performance was also maintained in the P300 task. Together, the results show that the peripheral auditory efferent effects captured via otoacoustic emission (OAE) inhibition might not necessarily be reflected in measures of central cortical processing and behavioral performance. As the MOC effects may not play a role in all listening situations in adults, the functional significance of the cochlear effects of the medial olivocochlear efferents and the optimal conditions conducive to corresponding effects in behavioral and cortical responses remain to be elucidated.

Audiovisual Modulation in Music Perception for Musicians and Non-musicians

In audiovisual music perception, visual information from a musical instrument being played is available prior to the onset of the corresponding musical sound and consequently allows a perceiver to form a prediction about the upcoming audio music. This prediction in audiovisual music perception, compared to auditory music perception, leads to lower N1 and P2 amplitudes and latencies. Although previous research suggests that audiovisual experience, such as previous musical experience may enhance this prediction, a remaining question is to what extent musical experience modifies N1 and P2 amplitudes and latencies. Furthermore, corresponding event-related phase modulations quantified as inter-trial phase coherence (ITPC) have not previously been reported for audiovisual music perception. In the current study, audio video recordings of a keyboard key being played were presented to musicians and non-musicians in audio only (AO), video only (VO), and audiovisual (AV) conditions. With predictive movements from playing the keyboard isolated from AV music perception (AV-VO), the current findings demonstrated that, compared to the AO condition, both groups had a similar decrease in N1 amplitude and latency, and P2 amplitude, along with correspondingly lower ITPC values in the delta, theta, and alpha frequency bands. However, while musicians showed lower ITPC values in the beta-band in AV-VO compared to the AO, non-musicians did not show this pattern. Findings indicate that AV perception may be broadly correlated with auditory perception, and differences between musicians and non-musicians further indicate musical experience to be a specific factor influencing AV perception. Predicting an upcoming sound in AV music perception may involve visual predictory processes, as well as beta-band oscillations, which may be influenced by years of musical training. This study highlights possible interconnectivity in AV perception as well as potential modulation with experience.

Cortical mechanisms of talker normalization in fluent sentences

Adjusting to the vocal characteristics of a new talker is important for speech recognition. Previous research has indicated that adjusting to talker differences is an active cognitive process that depends on attention and working memory (WM). These studies have not examined how talker variability affects perception and neural responses in fluent speech. Here we use source analysis from high-density EEG to show that perceiving fluent speech in which the talker changes recruits early involvement of parietal and temporal cortical areas, suggesting functional involvement of WM and attention in talker normalization. We extend these findings to acoustic source change in general by examining understanding environmental sounds in spoken sentence context. Though there may be differences in cortical recruitment to processing demands for non-speech sounds versus a changing talker, the underlying mechanisms are similar, supporting the view that shared cognitive-general mechanisms assist both talker normalization and speech-to-nonspeech transitions.

Musical Expertise Affects Audiovisual Speech Perception: Findings From Event-Related Potentials and Inter-trial Phase Coherence

In audiovisual speech perception, visual information from a talker's face during mouth articulation is available before the onset of the corresponding audio speech, and thereby allows the perceiver to use visual information to predict the upcoming audio. This prediction from phonetically congruent visual information modulates audiovisual speech perception and leads to a decrease in N1 and P2 amplitudes and latencies compared to the perception of audio speech alone. Whether audiovisual experience, such as with musical training, influences this prediction is unclear, but if so, may explain some of the variations observed in previous research. The current study addresses whether audiovisual speech perception is affected by musical training, first assessing N1 and P2 event-related potentials (ERPs) and in addition, inter-trial phase coherence (ITPC). Musicians and non-musicians are presented the syllable, /ba/ in audio only (AO), video only (VO), and audiovisual (AV) conditions. With the predictory effect of mouth movement isolated from the AV speech (AV-VO), results showed that, compared to audio speech, both groups have a lower N1 latency and P2 amplitude and latency. Moreover, they also showed lower ITPCs in the delta, theta, and beta bands in audiovisual speech perception. However, musicians showed significant suppression of N1 amplitude and desynchronization in the alpha band in audiovisual speech, not present for non-musicians. Collectively, the current findings indicate that early sensory processing can be modified by musical experience, which in turn can explain some of the variations in previous AV speech perception research.

Effects of Sound-Pressure Change on the 40 Hz Auditory Steady-State Response and Change-Related Cerebral Response

The auditory steady-state response (ASSR) elicited by a periodic sound stimulus is a neural oscillation recorded by magnetoencephalography (MEG), which is phase-locked to the repeated sound stimuli. This ASSR phase alternates after an abrupt change in the feature of a periodic sound stimulus and returns to its steady-state value. An abrupt change also elicits a MEG component peaking at approximately 100-180 ms (called "Change-N1m"). We investigated whether both the ASSR phase deviation and Change-N1m were affected by the magnitude of change in sound pressure. The ASSR and Change-N1m to 40 Hz click-trains (1000 ms duration, 70 dB), with and without an abrupt change (± 5, ± 10, or ± 15 dB) were recorded in ten healthy subjects. We used the source strength waveforms obtained by a two-dipole model for measurement of the ASSR phase deviation and Change-N1m values (peak amplitude and latency). As the magnitude of change increased, Change-N1m increased in amplitude and decreased in latency. Similarly, ASSR phase deviation depended on the magnitude of sound-pressure change. Thus, we suspect that both Change-N1m and the ASSR phase deviation reflect the sensitivity of the brain's neural change-detection system.

Tone language experience-dependent advantage in pitch representation in brainstem and auditory cortex is maintained under reverberation

Long-term language and music experience enhances neural representation of temporal attributes of pitch in the brainstem and auditory cortex in favorable listening conditions. Herein we examine whether brainstem and cortical pitch mechanisms-shaped by long-term language experience-maintain this advantage in the presence of reverberation-induced degradation in pitch representation. Brainstem frequency following responses (FFR) and cortical pitch responses (CPR) were recorded concurrently from Chinese and English-speaking natives, using a Mandarin word exhibiting a high rising pitch (/yi²/). Stimuli were presented diotically in quiet (Dry), and in the presence of Slight, Mild, and Moderate reverberation conditions. Regardless of language group, the amplitude of both brainstem FFR (F0) and cortical CPR (NaPb) responses decreased with increases in reverberation. Response amplitude for Chinese, however, was larger than English in all reverberant conditions. The Chinese group also exhibited a robust rightward asymmetry at temporal electrode sites (T8 > T7) across stimulus conditions. Regardless of language group, direct comparison of brainstem and cortical responses revealed similar magnitude of change in response amplitude with increasing reverberation. These findings suggest that experience-dependent brainstem and cortical pitch mechanisms provide an enhanced and stable neural representation of pitch-relevant information that is maintained even in the presence of reverberation. Relatively greater degradative effects of reverberation on brainstem (FFR) compared to cortical (NaPb) responses suggest relatively stronger top-down influences on CPRs.

Contributions of non-primary cortical sources to auditory temporal processing

Temporal processing is essential for speech perception and directional hearing. However, the number and locations of cortical sources involved in auditory temporal processing are still a matter of debate. Using source reconstruction of human EEG responses, we show that, in addition to primary sources in the auditory cortices, sources outside the auditory cortex, designated as non-primary sources, are involved in auditory temporal processing. Non-primary sources within the left and right motor areas, the superior parietal lobe and the right occipital lobe were activated by amplitude-modulated stimuli, and were involved in the functional network. The robustness of these findings was checked for different stimulation conditions. The non-primary sources showed weaker phase-locking and lower activity than primary sources. These findings suggest that the non-primary sources belong to the non-primary auditory pathway. This pathway and non-primary sources detected in motor area explain how, in temporal prediction of upcoming stimuli and motor theory of speech perception, the motor area receives auditory inputs.

Brainstem-cortical functional connectivity for speech is differentially challenged by noise and reverberation

Everyday speech perception is challenged by external acoustic interferences that hinder verbal communication. Here, we directly compared how different levels of the auditory system (brainstem vs. cortex) code speech and how their neural representations are affected by two acoustic stressors: noise and reverberation. We recorded multichannel (64 ch) brainstem frequency-following responses (FFRs) and cortical event-related potentials (ERPs) simultaneously in normal hearing individuals to speech sounds presented in mild and moderate levels of noise and reverb. We matched signal-to-noise and direct-to-reverberant ratios to equate the severity between classes of interference. Electrode recordings were parsed into source waveforms to assess the relative contribution of region-specific brain areas [i.e., brainstem (BS), primary auditory cortex (A1), inferior frontal gyrus (IFG)]. Results showed that reverberation was less detrimental to (and in some cases facilitated) the neural encoding of speech compared to additive noise. Inter-regional correlations revealed associations between BS and A1 responses, suggesting subcortical speech representations influence higher auditory-cortical areas. Functional connectivity analyses further showed that directed signaling toward A1 in both feedforward cortico-collicular (BS→A1) and feedback cortico-cortical (IFG→A1) pathways were strong predictors of degraded speech perception and differentiated "good" vs. "poor" perceivers. Our findings demonstrate a functional interplay within the brain's speech network that depends on the form and severity of acoustic interference. We infer that in addition to the quality of neural representations within individual brain regions, listeners' success at the "cocktail party" is modulated based on how information is transferred among subcortical and cortical hubs of the auditory-linguistic network.

Role of inter-trial phase coherence in atypical auditory evoked potentials to speech and nonspeech stimuli in children with autism

OBJECTIVE

This autism study investigated how inter-trial phase coherence (ITPC) drives abnormalities in auditory evoked potential (AEP) responses for speech and nonspeech stimuli.

METHODS

Auditory P1-N2 responses and ITPCs in the theta band (4-7 Hz) for pure tones and words were assessed with EEG data from 15 school-age children with autism and 16 age-matched typically developing (TD) controls.

RESULTS

The autism group showed enhanced P1 and reduced N2 for both speech and nonspeech stimuli in comparison with the TD group. Group differences were also found with enhanced theta ITPC for P1 followed by ITPC reduction for N2 in the autism group. The ITPC values were significant predictors of P1 and N2 amplitudes in both groups.

CONCLUSIONS

Abnormal trial-to-trial phase synchrony plays an important role in AEP atypicalities in children with autism. ITPC-driven enhancement as well as attenuation in different AEP components may coexist, depending on the stage of information processing.

SIGNIFICANCE

It is necessary to examine the time course of auditory evoked potentials and the corresponding inter-trial coherence of neural oscillatory activities to better understand hyper- and hypo- sensitive responses in autism, which has important implications for sensory based treatment.

Relationship between speech-evoked neural responses and perception of speech in noise in older adults

Speech-in-noise (SPIN) perception involves neural encoding of temporal acoustic cues. Cues include temporal fine structure (TFS) and envelopes that modulate at syllable (Slow-rate ENV) and fundamental frequency (F₀-rate ENV) rates. Here the relationship between speech-evoked neural responses to these cues and SPIN perception was investigated in older adults. Theta-band phase-locking values (PLVs) that reflect cortical sensitivity to Slow-rate ENV and peripheral/brainstem frequency-following responses phase-locked to F₀-rate ENV (FFR_{ENV_F0}) and TFS (FFR_TFS) were measured from scalp-electroencephalography responses to a repeated speech syllable in steady-state speech-shaped noise (SpN) and 16-speaker babble noise (BbN). The results showed that (1) SPIN performance and PLVs were significantly higher under SpN than BbN, implying differential cortical encoding may serve as the neural mechanism of SPIN performance that varies as a function of noise types; (2) PLVs and FFR_TFS at resolved harmonics were significantly related to good SPIN performance, supporting the importance of phase-locked neural encoding of Slow-rate ENV and TFS of resolved harmonics during SPIN perception; (3) FFR_{ENV_F0} was not associated to SPIN performance until audiometric threshold was controlled for, indicating that hearing loss should be carefully controlled when studying the role of neural encoding of F₀-rate ENV. Implications are drawn with respect to fitting auditory prostheses.

Characterizing the roles of alpha and theta oscillations in multisensory attention

Cortical alpha oscillations (8-13Hz) appear to play a role in suppressing distractions when just one sensory modality is being attended, but do they also contribute when attention is distributed over multiple sensory modalities? For an answer, we examined cortical oscillations in human subjects who were dividing attention between auditory and visual sequences. In Experiment 1, subjects performed an oddball task with auditory, visual, or simultaneous audiovisual sequences in separate blocks, while the electroencephalogram was recorded using high-density scalp electrodes. Alpha oscillations were present continuously over posterior regions while subjects were attending to auditory sequences. This supports the idea that the brain suppresses processing of visual input in order to advantage auditory processing. During a divided-attention audiovisual condition, an oddball (a rare, unusual stimulus) occurred in either the auditory or the visual domain, requiring that attention be divided between the two modalities. Fronto-central theta band (4-7Hz) activity was strongest in this audiovisual condition, when subjects monitored auditory and visual sequences simultaneously. Theta oscillations have been associated with both attention and with short-term memory. Experiment 2 sought to distinguish these possible roles of fronto-central theta activity during multisensory divided attention. Using a modified version of the oddball task from Experiment 1, Experiment 2 showed that differences in theta power among conditions were independent of short-term memory load. Ruling out theta's association with short-term memory, we conclude that fronto-central theta activity is likely a marker of multisensory divided attention.

Application of Linear Mixed-Effects Models in Human Neuroscience Research: A Comparison with Pearson Correlation in Two Auditory Electrophysiology Studies

Neurophysiological studies are often designed to examine relationships between measures from different testing conditions, time points, or analysis techniques within the same group of participants. Appropriate statistical techniques that can take into account repeated measures and multivariate predictor variables are integral and essential to successful data analysis and interpretation. This work implements and compares conventional Pearson correlations and linear mixed-effects (LME) regression models using data from two recently published auditory electrophysiology studies. For the specific research questions in both studies, the Pearson correlation test is inappropriate for determining strengths between the behavioral responses for speech-in-noise recognition and the multiple neurophysiological measures as the neural responses across listening conditions were simply treated as independent measures. In contrast, the LME models allow a systematic approach to incorporate both fixed-effect and random-effect terms to deal with the categorical grouping factor of listening conditions, between-subject baseline differences in the multiple measures, and the correlational structure among the predictor variables. Together, the comparative data demonstrate the advantages as well as the necessity to apply mixed-effects models to properly account for the built-in relationships among the multiple predictor variables, which has important implications for proper statistical modeling and interpretation of human behavior in terms of neural correlates and biomarkers.

Speech-specific categorical perception deficit in autism: An Event-Related Potential study of lexical tone processing in Mandarin-speaking children

Recent studies reveal that tonal language speakers with autism have enhanced neural sensitivity to pitch changes in nonspeech stimuli but not to lexical tone contrasts in their native language. The present ERP study investigated whether the distinct pitch processing pattern for speech and nonspeech stimuli in autism was due to a speech-specific deficit in categorical perception of lexical tones. A passive oddball paradigm was adopted to examine two groups (16 in the autism group and 15 in the control group) of Chinese children’s Mismatch Responses (MMRs) to equivalent pitch deviations representing within-category and between-category differences in speech and nonspeech contexts. To further examine group-level differences in the MMRs to categorical perception of speech/nonspeech stimuli or lack thereof, neural oscillatory activities at the single trial level were further calculated with the inter-trial phase coherence (ITPC) measure for the theta and beta frequency bands. The MMR and ITPC data from the children with autism showed evidence for lack of categorical perception in the lexical tone condition. In view of the important role of lexical tones in acquiring a tonal language, the results point to the necessity of early intervention for the individuals with autism who show such a speech-specific categorical perception deficit.

Preliminary Investigation of the Passively Evoked N400 as a Tool for Estimating Speech-in-Noise Thresholds

PURPOSE

Speech-in-noise testing relies on a number of factors beyond the auditory system, such as cognitive function, compliance, and motor function. It may be possible to avoid these limitations by using electroencephalography. The present study explored this possibility using the N400.

METHOD

Eleven adults with typical hearing heard high-constraint sentences with congruent and incongruent terminal words in the presence of speech-shaped noise. Participants ignored all auditory stimulation and watched a video. The signal-to-noise ratio (SNR) was varied around each participant's behavioral threshold during electroencephalography recording. Speech was also heard in quiet.

RESULTS

The amplitude of the N400 effect exhibited a nonlinear relationship with SNR. In the presence of background noise, amplitude decreased from high (+4 dB) to low (+1 dB) SNR but increased dramatically at threshold before decreasing again at subthreshold SNR (-2 dB).

CONCLUSIONS

The SNR of speech in noise modulates the amplitude of the N400 effect to semantic anomalies in a nonlinear fashion. These results are the first to demonstrate modulation of the passively evoked N400 by SNR in speech-shaped noise and represent a first step toward the end goal of developing an N400-based physiological metric for speech-in-noise testing.

Perceptual Temporal Asymmetry Associated with Distinct ON and OFF Responses to Time-Varying Sounds with Rising versus Falling Intensity: A Magnetoencephalography Study

This magnetoencephalography (MEG) study investigated evoked ON and OFF responses to ramped and damped sounds in normal-hearing human adults. Two pairs of stimuli that differed in spectral complexity were used in a passive listening task; each pair contained identical acoustical properties except for the intensity envelope. Behavioral duration judgment was conducted in separate sessions, which replicated the perceptual bias in favour of the ramped sounds and the effect of spectral complexity on perceived duration asymmetry. MEG results showed similar cortical sites for the ON and OFF responses. There was a dominant ON response with stronger phase-locking factor (PLF) in the alpha (8–14 Hz) and theta (4–8 Hz) bands for the damped sounds. In contrast, the OFF response for sounds with rising intensity was associated with stronger PLF in the gamma band (30–70 Hz). Exploratory correlation analysis showed that the OFF response in the left auditory cortex was a good predictor of the perceived temporal asymmetry for the spectrally simpler pair. The results indicate distinct asymmetry in ON and OFF responses and neural oscillation patterns associated with the dynamic intensity changes, which provides important preliminary data for future studies to examine how the auditory system develops such an asymmetry as a function of age and learning experience and whether the absence of asymmetry or abnormal ON and OFF responses can be taken as a biomarker for certain neurological conditions associated with auditory processing deficits.

Neural indices of phonemic discrimination and sentence-level speech intelligibility in quiet and noise: A mismatch negativity study

Successful speech communication requires the extraction of important acoustic cues from irrelevant background noise. In order to better understand this process, this study examined the effects of background noise on mismatch negativity (MMN) latency, amplitude, and spectral power measures as well as behavioral speech intelligibility tasks. Auditory event-related potentials (AERPs) were obtained from 15 normal-hearing participants to determine whether pre-attentive MMN measures recorded in response to a consonant (from /ba/ to /bu/) and vowel change (from /ba/ to /da/) in a double-oddball paradigm can predict sentence-level speech perception. The results showed that background noise increased MMN latencies and decreased MMN amplitudes with a reduction in the theta frequency band power. Differential noise-induced effects were observed for the pre-attentive processing of consonant and vowel changes due to different degrees of signal degradation by noise. Linear mixed-effects models further revealed significant correlations between the MMN measures and speech intelligibility scores across conditions and stimuli. These results confirm the utility of MMN as an objective neural marker for understanding noise-induced variations as well as individual differences in speech perception, which has important implications for potential clinical applications.

Cortical processing of phonetic and emotional information in speech: A cross-modal priming study

The current study employed behavioral and electrophysiological measures to investigate the timing, localization, and neural oscillation characteristics of cortical activities associated with phonetic and emotional information processing of speech. The experimental design used a cross-modal priming paradigm in which the normal adult participants were presented a visual prime followed by an auditory target. Primes were facial expressions that systematically varied in emotional content (happy or angry) and mouth shape (corresponding to /a/ or /i/ vowels). Targets were spoken words that varied by emotional prosody (happy or angry) and vowel (/a/ or /i/). In both the phonetic and prosodic conditions, participants were asked to judge congruency status of the visual prime and the auditory target. Behavioral results showed a congruency effect for both percent correct and reaction time. Two ERP responses, the N400 and late positive response (LPR), were identified in both conditions. Source localization and inter-trial phase coherence of the N400 and LPR components further revealed different cortical contributions and neural oscillation patterns for selective processing of phonetic and emotional information in speech. The results provide corroborating evidence for the necessity of differentiating brain mechanisms underlying the representation and processing of co-existing linguistic and paralinguistic information in spoken language, which has important implications for theoretical models of speech recognition as well as clinical studies on the neural bases of language and social communication deficits.