Adaptation to Noise in Human Speech Recognition Depends on Noise-Level Statistics and Fast Dynamic-Range Compression

How does the human brain process noisy speech in real life? Insights from the second-person neuroscience perspective

Comprehending speech with the existence of background noise is of great importance for human life. In the past decades, a large number of psychological, cognitive and neuroscientific research has explored the neurocognitive mechanisms of speech-in-noise comprehension. However, as limited by the low ecological validity of the speech stimuli and the experimental paradigm, as well as the inadequate attention on the high-order linguistic and extralinguistic processes, there remains much unknown about how the brain processes noisy speech in real-life scenarios. A recently emerging approach, i.e., the second-person neuroscience approach, provides a novel conceptual framework. It measures both of the speaker's and the listener's neural activities, and estimates the speaker-listener neural coupling with regarding of the speaker's production-related neural activity as a standardized reference. The second-person approach not only promotes the use of naturalistic speech but also allows for free communication between speaker and listener as in a close-to-life context. In this review, we first briefly review the previous discoveries about how the brain processes speech in noise; then, we introduce the principles and advantages of the second-person neuroscience approach and discuss its implications to unravel the linguistic and extralinguistic processes during speech-in-noise comprehension; finally, we conclude by proposing some critical issues and calls for more research interests in the second-person approach, which would further extend the present knowledge about how people comprehend speech in noise.

Adaptation in auditory processing

Adaptation is an essential feature of auditory neurons, which reduces their responses to unchanging and recurring sounds and allows their response properties to be matched to the constantly changing statistics of sounds that reach the ears. As a consequence, processing in the auditory system highlights novel or unpredictable sounds and produces an efficient representation of the vast range of sounds that animals can perceive by continually adjusting the sensitivity and, to a lesser extent, the tuning properties of neurons to the most commonly encountered stimulus values. Together with attentional modulation, adaptation to sound statistics also helps to generate neural representations of sound that are tolerant to background noise and therefore plays a vital role in auditory scene analysis. In this review, we consider the diverse forms of adaptation that are found in the auditory system in terms of the processing levels at which they arise, the underlying neural mechanisms, and their impact on neural coding and perception. We also ask what the dynamics of adaptation, which can occur over multiple timescales, reveal about the statistical properties of the environment. Finally, we examine how adaptation to sound statistics is influenced by learning and experience and changes as a result of aging and hearing loss.

Assessing Medial Olivocochlear Reflex Strengths via Auditory Brainstem Response: Measurement and Variability in Normal-Hearing Individuals

PURPOSE

Optimal measurement settings to measure the medial olivocochlear reflex (MOCR) in humans have not yet been defined. The purpose of this study was to advance the representation of the MOCR in auditory brainstem response (ABR) as an addition to the current diagnostic portfolio.

PARTICIPANTS AND METHOD

Twelve female and 14 male normal-hearing adults participated in the study. Potential effects of a contralateral acoustic stimulus (CAS) on amplitude changes were investigated by recording ABR waveform profiles on the left side at click intensities of 50/60/70 dB nHL with and without CAS (60 dB SPL). Secondly, to detect potential chronological order influences, measurement settings were rearranged on the right side and measurements were repeated. Additionally, ABR thresholds were recorded with and without a CAS in 10 patients.

RESULTS

When the effect of contralateral suppression was analyzed on the basis of amplitude changes, there was a change under administration of the CAS signal that was statistically significant. Interestingly, the order of recordings affected the degree of amplitude change. In three out of 10 patients, reproducible suppression effects on ABR thresholds were detectable upon CAS presentation.

CONCLUSIONS

To our knowledge, this is the largest study dealing with the recording of the MOCR elicited by a contralateral noise via ABR in normal-hearing individuals. Effects of MOCR are measurable via amplitude changes upon CAS administration. Chronological orders influence the impact of this effect on amplitude changes. Optimal measurement settings have not yet been defined. However, experiments such as this study may help to further improve measurements, and thus advance the representation of the MOC reflex in ABR as an addition to the current diagnostic portfolio.

Semantic Cues Modulate Children’s and Adults’ Processing of Audio-Visual Face Mask Speech

During the COVID-19 pandemic, questions have been raised about the impact of face masks on communication in classroom settings. However, it is unclear to what extent visual obstruction of the speaker’s mouth or changes to the acoustic signal lead to speech processing difficulties, and whether these effects can be mitigated by semantic predictability, i.e., the availability of contextual information. The present study investigated the acoustic and visual effects of face masks on speech intelligibility and processing speed under varying semantic predictability. Twenty-six children (aged 8-12) and twenty-six adults performed an internet-based cued shadowing task, in which they had to repeat aloud the last word of sentences presented in audio-visual format. The results showed that children and adults made more mistakes and responded more slowly when listening to face mask speech compared to speech produced without a face mask. Adults were only significantly affected by face mask speech when both the acoustic and the visual signal were degraded. While acoustic mask effects were similar for children, removal of visual speech cues through the face mask affected children to a lesser degree. However, high semantic predictability reduced audio-visual mask effects, leading to full compensation of the acoustically degraded mask speech in the adult group. Even though children did not fully compensate for face mask speech with high semantic predictability, overall, they still profited from semantic cues in all conditions. Therefore, in classroom settings, strategies that increase contextual information such as building on students’ prior knowledge, using keywords, and providing visual aids, are likely to help overcome any adverse face mask effects.

Adaptation to noise in normal and impaired hearing

Many aspects of hearing function are negatively affected by background noise. Listeners, however, have some ability to adapt to background noise. For instance, the detection of pure tones and the recognition of isolated words embedded in noise can improve gradually as tones and words are delayed a few hundred milliseconds in the noise. While some evidence suggests that adaptation to noise could be mediated by the medial olivocochlear reflex, adaptation can occur for people who do not have a functional reflex. Since adaptation can facilitate hearing in noise, and hearing in noise is often harder for hearing-impaired than for normal-hearing listeners, it is conceivable that adaptation is impaired with hearing loss. It remains unclear, however, if and to what extent this is the case, or whether impaired adaptation contributes to the greater difficulties experienced by hearing-impaired listeners understanding speech in noise. Here, we review adaptation to noise, the mechanisms potentially contributing to this adaptation, and factors that might reduce the ability to adapt to background noise, including cochlear hearing loss, cochlear synaptopathy, aging, and noise exposure. The review highlights few knowns and many unknowns about adaptation to noise, and thus paves the way for further research on this topic.

Understanding degraded speech leads to perceptual gating of a brainstem reflex in human listeners

The ability to navigate "cocktail party" situations by focusing on sounds of interest over irrelevant, background sounds is often considered in terms of cortical mechanisms. However, subcortical circuits such as the pathway underlying the medial olivocochlear (MOC) reflex modulate the activity of the inner ear itself, supporting the extraction of salient features from auditory scene prior to any cortical processing. To understand the contribution of auditory subcortical nuclei and the cochlea in complex listening tasks, we made physiological recordings along the auditory pathway while listeners engaged in detecting non(sense) words in lists of words. Both naturally spoken and intrinsically noisy, vocoded speech-filtering that mimics processing by a cochlear implant (CI)-significantly activated the MOC reflex, but this was not the case for speech in background noise, which more engaged midbrain and cortical resources. A model of the initial stages of auditory processing reproduced specific effects of each form of speech degradation, providing a rationale for goal-directed gating of the MOC reflex based on enhancing the representation of the energy envelope of the acoustic waveform. Our data reveal the coexistence of 2 strategies in the auditory system that may facilitate speech understanding in situations where the signal is either intrinsically degraded or masked by extrinsic acoustic energy. Whereas intrinsically degraded streams recruit the MOC reflex to improve representation of speech cues peripherally, extrinsically masked streams rely more on higher auditory centres to denoise signals.

Effect of Noise on Sleep and Autonomic Activity in Children according to Source

BACKGROUND

Noise has been a major issue in children due to its effect on emotional symptoms and sleep problems. The present study investigated the effect of noise on sleep and autonomic activity in children according to three variables, namely, road traffic noise, aircraft noise, and noise sensitivity (NS).

METHODS

The study is epidemiological in nature. A total of 474 subjects of four elementary schools were evaluated in conjunction with the 2014 noise map data gathered from two metropolitan regions, Seoul and Ulsan, in South Korea. Each subject completed the children's sleep habits questionnaire, and heart rate variability (HRV) was recorded. Analysis of covariance was performed using monthly income as a covariate.

RESULTS

Aircraft noise was associated with night waking (P = 0.047), whereas no significant association was observed between sleep and road traffic noise. After stratifying by NS, no significant results were obtained for road traffic noise. However, high levels of aircraft noise in the high-NS group were associated with high degrees of bedtime resistance (P = 0.049). In terms of HRV, no significant difference was observed in the results between groups.

CONCLUSION

Road traffic noise did not significantly alter sleep or autonomic activity in children, whereas aircraft noise exerted a negative effect on sleep. The findings suggest that the impact of noise on sleep in children varies depending on noise sources.

Controlled (re)evaluation of the relationship between speech perception in noise and contralateral suppression of otoacoustic emissions

In people with normal hearing (NH), speech perception in noise (SPIN) improves when the speech signal is presented not gated with noise but after a delay. The medial olivocochlear reflex (MOCR) was thought to be involved in the neural dynamic range adaptation (NDRA) responsible for this adaptive SPIN; however, some of the recent studies do not support this hypothesis and suggest that adaptive SPIN involves the NDRA to noise-level statistics, irrespective of MOCR activation. A plausible reason for this discrepancy could be the variations and limitations of the experimental designs used in different studies. Using a relatively controlled and comprehensive study design, this study attempts to verify whether a delay between the delivery of speech and the noise improves the SPIN and whether MOCR mediates such effects. The SPIN was estimated by measuring speech reception thresholds (SRT) in noise under simultaneous-onset and delayed-onset (noise precedes speech onset by 300 ms) conditions. The SPIN in both ears was independently examined for ipsilateral, contralateral, and bilateral noise in women with normal hearing (N = 18; age range, 18-25 years). Contralateral suppression of transient-evoked otoacoustic emissions (CSOAEs) was used to estimate the MOCR based cochlear gain reduction. Under all test conditions, SPIN was improved in delayed-onset than in simultaneous-onset conditions, and the mean improvement in the SRT ranged from 0.7±1.7 to 1.8±1.8 dB. No significant correlation was obtained between CSOAEs and the mean temporal improvement in SRT, suggesting that MOCR may not be a predominant mechanism for the temporal improvement in SPIN.

On the importance of interaural noise coherence and the medial olivocochlear reflex for binaural unmasking in free-field listening

For speech in competition with a noise source in the free field, normal-hearing (NH) listeners recognize speech better when listening binaurally than when listening monaurally with the ear that has the better acoustic signal-to-noise ratio (SNR). This benefit from listening binaurally is known as binaural unmasking and indicates that the brain combines information from the two ears to improve intelligibility. Here, we address three questions pertaining to binaural unmasking for NH listeners. First, we investigate if binaural unmasking results from combining the speech and/or the noise from the two ears. In a simulated acoustic free field with speech and noise sources at 0° and 270°azimuth, respectively, we found comparable unmasking regardless of whether the speech was present or absent in the ear with the worse SNR. This indicates that binaural unmasking probably involves combining only the noise at the two ears. Second, we investigate if having binaurally coherent location cues for the noise signal is sufficient for binaural unmasking to occur. We found no unmasking when location cues were coherent but noise signals were generated incoherent or were processed unilaterally through a hearing aid with linear, minimal amplification. This indicates that binaural unmasking requires interaurally coherent noise signals, source location cues, and processing. Third, we investigate if the hypothesized antimasking benefits of the medial olivocochlear reflex (MOCR) contribute to binaural unmasking. We found comparable unmasking regardless of whether speech tokens (words) were sufficiently delayed from the noise onset to fully activate the MOCR or not. Moreover, unmasking was absent when the noise was binaurally incoherent whereas the physiological antimasking effects of the MOCR are similar for coherent and incoherent noises. This indicates that the MOCR is unlikely involved in binaural unmasking.

Effects of noise precursors on the detection of amplitude and frequency modulation for tones in noise

Recent studies on amplitude modulation (AM) detection for tones in noise reported that AM-detection thresholds improve when the AM stimulus is preceded by a noise precursor. The physiological mechanisms underlying this AM unmasking are unknown. One possibility is that adaptation to the level of the noise precursor facilitates AM encoding by causing a shift in neural rate-level functions to optimize level encoding around the precursor level. The aims of this study were to investigate whether such a dynamic-range adaptation is a plausible mechanism for the AM unmasking and whether frequency modulation (FM), thought to be encoded via AM, also exhibits the unmasking effect. Detection thresholds for AM and FM of tones in noise were measured with and without a fixed-level precursor. Listeners showing the unmasking effect were then tested with the precursor level roved over a wide range to modulate the effect of adaptation to the precursor level on the detection of the subsequent AM. It was found that FM detection benefits from a precursor and the magnitude of FM unmasking correlates with that of AM unmasking. Moreover, consistent with dynamic-range adaptation, the unmasking magnitude weakens as the level difference between the precursor and simultaneous masker of the tone increases.

Listening in complex acoustic scenes

Being able to pick out particular sounds, such as speech, against a background of other sounds represents one of the key tasks performed by the auditory system. Understanding how this happens is important because speech recognition in noise is particularly challenging for older listeners and for people with hearing impairments. Central to this ability is the capacity of neurons to adapt to the statistics of sounds reaching the ears, which helps to generate noise-tolerant representations of sounds in the brain. In more complex auditory scenes, such as a cocktail party — where the background noise comprises other voices, sound features associated with each source have to be grouped together and segregated from those belonging to other sources. This depends on precise temporal coding and modulation of cortical response properties when attending to a particular speaker in a multi-talker environment. Furthermore, the neural processing underlying auditory scene analysis is shaped by experience over multiple timescales.