Constructing noise-invariant representations of sound in the auditory pathway

Cortical encoding of hierarchical linguistic information when syllabic rhythms are obscured by echoes

In speech perception, low-frequency cortical activity tracks hierarchical linguistic units (e.g., syllables, phrases, and sentences) on top of acoustic features (e.g., speech envelope). Since the fluctuation of speech envelope typically corresponds to the syllabic boundaries, one common interpretation is that the acoustic envelope underlies the extraction of discrete syllables from continuous speech for subsequent linguistic processing. However, it remains unclear whether and how cortical activity encodes linguistic information when the speech envelope does not provide acoustic correlates of syllables. To address the issue, we introduced a frequency-tagging speech stream where the syllabic rhythm was obscured by echoic envelopes and investigated neural encoding of hierarchical linguistic information using electroencephalography (EEG). When listeners attended to the echoic speech, cortical activity showed reliable tracking of syllable, phrase, and sentence levels, among which the higher-level linguistic units elicited more robust neural responses. When attention was diverted from the echoic speech, reliable neural tracking of the syllable level was also observed in contrast to deteriorated neural tracking of the phrase and sentence levels. Further analyses revealed that the envelope aligned with the syllabic rhythm could be recovered from the echoic speech through a neural adaptation model, and the reconstructed envelope yielded higher predictive power for the neural tracking responses than either the original echoic envelope or anechoic envelope. Taken together, these results suggest that neural adaptation and attentional modulation jointly contribute to neural encoding of linguistic information in distorted speech where the syllabic rhythm is obscured by echoes.

Distinct roles of SNR, speech Intelligibility, and attentional effort on neural speech tracking in noise

Robust neural encoding of speech in noise is influenced by several factors, including signal-to-noise ratio (SNR), speech intelligibility (SI), and attentional effort (AE). Yet, the interaction and distinct role of these factors remain unclear. In this study, fourteen native English speakers performed selective speech listening tasks at various SNR levels while EEG responses were recorded. Attentional performance was assessed using a repeated word detection task, and attentional effort was inferred from subjects' gaze velocity. Results indicate that both SNR and SI enhance neural tracking of target speech, with distinct effects influenced by the previously overlooked role of attentional effort. Specifically, at high levels of SI, increasing SNR leads to reduced attentional effort, which in turn decreases neural speech tracking. Our findings highlight the importance of differentiating the roles of SNR, SI, and AE in neural speech processing and advance our understanding of how noisy speech is processed in the auditory pathway.

Interference of mid-level sound statistics underlie human speech recognition sensitivity in natural noise

Recognizing speech in noise, such as in a busy restaurant, is an essential cognitive skill where the task difficulty varies across environments and noise levels. Although there is growing evidence that the auditory system relies on statistical representations for perceiving 1-5 and coding4,6-9 natural sounds, it's less clear how statistical cues and neural representations contribute to segregating speech in natural auditory scenes. We demonstrate that human listeners rely on mid-level statistics to segregate and recognize speech in environmental noise. Using natural backgrounds and variants with perturbed spectro-temporal statistics, we show that speech recognition accuracy at a fixed noise level varies extensively across natural backgrounds (0% to 100%). Furthermore, for each background the unique interference created by summary statistics can mask or unmask speech, thus hindering or improving speech recognition. To identify the neural coding strategy and statistical cues that influence accuracy, we developed generalized perceptual regression, a framework that links summary statistics from a neural model to word recognition accuracy. Whereas a peripheral cochlear model accounts for only 60% of perceptual variance, summary statistics from a mid-level auditory midbrain model accurately predicts single trial sensory judgments, accounting for more than 90% of the perceptual variance. Furthermore, perceptual weights from the regression framework identify which statistics and tuned neural filters are influential and how they impact recognition. Thus, perception of speech in natural backgrounds relies on a mid-level auditory representation involving interference of multiple summary statistics that impact recognition beneficially or detrimentally across natural background sounds.

Higher-order interactions between hippocampal CA1 neurons are disrupted in amnestic mice

Across systems, higher-order interactions between components govern emergent dynamics. Here we tested whether contextual threat memory retrieval in mice relies on higher-order interactions between dorsal CA1 hippocampal neurons requiring learning-induced dendritic spine plasticity. We compared population-level Ca2+ transients as wild-type mice (with intact learning-induced spine plasticity and memory) and amnestic mice (TgCRND8 mice with high levels of amyloid-β and deficits in learning-induced spine plasticity and memory) were tested for memory. Using machine-learning classifiers with different capacities to use input data with complex interactions, our findings indicate complex neuronal interactions in the memory representation of wild-type, but not amnestic, mice. Moreover, a peptide that partially restored learning-induced spine plasticity also restored the statistical complexity of the memory representation and memory behavior in Tg mice. These findings provide a previously missing bridge between levels of analysis in memory research, linking receptors, spines, higher-order neuronal dynamics and behavior.

Sensory and Perceptual Decisional Processes Underlying the Perception of Reverberant Auditory Environments

Reverberation, a ubiquitous feature of real-world acoustic environments, exhibits statistical regularities that human listeners leverage to self-orient, facilitate auditory perception, and understand their environment. Despite the extensive research on sound source representation in the auditory system, it remains unclear how the brain represents real-world reverberant environments. Here, we characterized the neural response to reverberation of varying realism by applying multivariate pattern analysis to electroencephalographic (EEG) brain signals. Human listeners (12 males and 8 females) heard speech samples convolved with real-world and synthetic reverberant impulse responses and judged whether the speech samples were in a "real" or "fake" environment, focusing on the reverberant background rather than the properties of speech itself. Participants distinguished real from synthetic reverberation with ∼75% accuracy; EEG decoding reveals a multistage decoding time course, with dissociable components early in the stimulus presentation and later in the perioffset stage. The early component predominantly occurred in temporal electrode clusters, while the later component was prominent in centroparietal clusters. These findings suggest distinct neural stages in perceiving natural acoustic environments, likely reflecting sensory encoding and higher-level perceptual decision-making processes. Overall, our findings provide evidence that reverberation, rather than being largely suppressed as a noise-like signal, carries relevant environmental information and gains representation along the auditory system. This understanding also offers various applications; it provides insights for including reverberation as a cue to aid navigation for blind and visually impaired people. It also helps to enhance realism perception in immersive virtual reality settings, gaming, music, and film production.

Noises on-How the Brain Deals with Acoustic Noise

What is noise? When does a sound form part of the acoustic background and when might it come to our attention as part of the foreground? Our brain seems to filter out irrelevant sounds in a seemingly effortless process, but how this is achieved remains opaque and, to date, unparalleled by any algorithm. In this review, we discuss how noise can be both background and foreground, depending on what a listener/brain is trying to achieve. We do so by addressing questions concerning the brain's potential bias to interpret certain sounds as part of the background, the extent to which the interpretation of sounds depends on the context in which they are heard, as well as their ethological relevance, task-dependence, and a listener's overall mental state. We explore these questions with specific regard to the implicit, or statistical, learning of sounds and the role of feedback loops between cortical and subcortical auditory structures.

Reliability and stability of tactile perception in the whisker somatosensory system

Rodents rely on their whiskers as vital sensory tools for tactile perception, enabling them to distinguish textures and shapes. Ensuring the reliability and constancy of tactile perception under varying stimulus conditions remains a fascinating and fundamental inquiry. This study explores the impact of stimulus configurations, including whisker movement velocity and object spatial proximity, on texture discrimination and stability in rats. To address this issue, we employed three distinct approaches for our investigation. Stimulus configurations notably affected tactile inputs, altering whisker vibration's kinetic and kinematic aspects with consistent effects across various textures. Through a texture discrimination task, rats exhibited consistent discrimination performance irrespective of changes in stimulus configuration. However, alterations in stimulus configuration significantly affected the rats' ability to maintain stability in texture perception. Additionally, we investigated the influence of stimulus configurations on cortical neuronal responses by manipulating them experimentally. Notably, cortical neurons demonstrated substantial and intricate changes in firing rates without compromising the ability to discriminate between textures. Nevertheless, these changes resulted in a reduction in texture neuronal response stability. Stimulating multiple whiskers led to improved neuronal texture discrimination and maintained coding stability. These findings emphasize the importance of considering numerous factors and their interactions when studying the impact of stimulus configuration on neuronal responses and behavior.

Dissociable Roles of the Auditory Midbrain and Cortex in Processing the Statistical Features of Natural Sound Textures

Sound texture perception takes advantage of a hierarchy of time-averaged statistical features of acoustic stimuli, but much remains unclear about how these statistical features are processed along the auditory pathway. Here, we compared the neural representation of sound textures in the inferior colliculus (IC) and auditory cortex (AC) of anesthetized female rats. We recorded responses to texture morph stimuli that gradually add statistical features of increasingly higher complexity. For each texture, several different exemplars were synthesized using different random seeds. An analysis of transient and ongoing multiunit responses showed that the IC units were sensitive to every type of statistical feature, albeit to a varying extent. In contrast, only a small proportion of AC units were overtly sensitive to any statistical features. Differences in texture types explained more of the variance of IC neural responses than did differences in exemplars, indicating a degree of "texture type tuning" in the IC, but the same was, perhaps surprisingly, not the case for AC responses. We also evaluated the accuracy of texture type classification from single-trial population activity and found that IC responses became more informative as more summary statistics were included in the texture morphs, while for AC population responses, classification performance remained consistently very low. These results argue against the idea that AC neurons encode sound type via an overt sensitivity in neural firing rate to fine-grain spectral and temporal statistical features.

Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions

Models that predict brain responses to stimuli provide one measure of understanding of a sensory system and have many potential applications in science and engineering. Deep artificial neural networks have emerged as the leading such predictive models of the visual system but are less explored in audition. Prior work provided examples of audio-trained neural networks that produced good predictions of auditory cortical fMRI responses and exhibited correspondence between model stages and brain regions, but left it unclear whether these results generalize to other neural network models and, thus, how to further improve models in this domain. We evaluated model-brain correspondence for publicly available audio neural network models along with in-house models trained on 4 different tasks. Most tested models outpredicted standard spectromporal filter-bank models of auditory cortex and exhibited systematic model-brain correspondence: Middle stages best predicted primary auditory cortex, while deep stages best predicted non-primary cortex. However, some state-of-the-art models produced substantially worse brain predictions. Models trained to recognize speech in background noise produced better brain predictions than models trained to recognize speech in quiet, potentially because hearing in noise imposes constraints on biological auditory representations. The training task influenced the prediction quality for specific cortical tuning properties, with best overall predictions resulting from models trained on multiple tasks. The results generally support the promise of deep neural networks as models of audition, though they also indicate that current models do not explain auditory cortical responses in their entirety.

A sparse code for natural sound context in auditory cortex

Accurate sound perception can require integrating information over hundreds of milliseconds or even seconds. Spectro-temporal models of sound coding by single neurons in auditory cortex indicate that the majority of sound-evoked activity can be attributed to stimuli with a few tens of milliseconds. It remains uncertain how the auditory system integrates information about sensory context on a longer timescale. Here we characterized long-lasting contextual effects in auditory cortex (AC) using a diverse set of natural sound stimuli. We measured context effects as the difference in a neuron's response to a single probe sound following two different context sounds. Many AC neurons showed context effects lasting longer than the temporal window of a traditional spectro-temporal receptive field. The duration and magnitude of context effects varied substantially across neurons and stimuli. This diversity of context effects formed a sparse code across the neural population that encoded a wider range of contexts than any constituent neuron. Encoding model analysis indicates that context effects can be explained by activity in the local neural population, suggesting that recurrent local circuits support a long-lasting representation of sensory context in auditory cortex.

The neurobiology of vocal communication in marmosets

An increasingly popular animal model for studying the neural basis of social behavior, cognition, and communication is the common marmoset (Callithrix jacchus). Interest in this New World primate across neuroscience is now being driven by their proclivity for prosociality across their repertoire, high volubility, and rapid development, as well as their amenability to naturalistic testing paradigms and freely moving neural recording and imaging technologies. The complement of these characteristics set marmosets up to be a powerful model of the primate social brain in the years to come. Here, we focus on vocal communication because it is the area that has both made the most progress and illustrates the prodigious potential of this species. We review the current state of the field with a focus on the various brain areas and networks involved in vocal perception and production, comparing the findings from marmosets to other animals, including humans.

Dynamics of cortical contrast adaptation predict perception of signals in noise

Neurons throughout the sensory pathway adapt their responses depending on the statistical structure of the sensory environment. Contrast gain control is a form of adaptation in the auditory cortex, but it is unclear whether the dynamics of gain control reflect efficient adaptation, and whether they shape behavioral perception. Here, we trained mice to detect a target presented in background noise shortly after a change in the contrast of the background. The observed changes in cortical gain and behavioral detection followed the dynamics of a normative model of efficient contrast gain control; specifically, target detection and sensitivity improved slowly in low contrast, but degraded rapidly in high contrast. Auditory cortex was required for this task, and cortical responses were not only similarly affected by contrast but predicted variability in behavioral performance. Combined, our results demonstrate that dynamic gain adaptation supports efficient coding in auditory cortex and predicts the perception of sounds in noise.

Effect of Reverberation on Neural Responses to Natural Speech in Rabbit Auditory Midbrain: No Evidence for a Neural Dereverberation Mechanism

Reverberation is ubiquitous in everyday acoustic environments. It degrades both binaural cues and the envelope modulations of sounds and thus can impair speech perception. Still, both humans and animals can accurately perceive reverberant stimuli in most everyday settings. Previous neurophysiological and perceptual studies have suggested the existence of neural mechanisms that partially compensate for the effects of reverberation. However, these studies were limited by their use of either highly simplified stimuli or rudimentary reverberation simulations. To further characterize how reverberant stimuli are processed by the auditory system, we recorded single-unit (SU) and multiunit (MU) activity from the inferior colliculus (IC) of unanesthetized rabbits in response to natural speech utterances presented with no reverberation ("dry") and in various degrees of simulated reverberation (direct-to-reverberant energy ratios (DRRs) ranging from 9.4 to -8.2 dB). Linear stimulus reconstruction techniques (Mesgarani et al., 2009) were used to quantify the amount of speech information available in the responses of neural ensembles. We found that high-quality spectrogram reconstructions could be obtained for dry speech and in moderate reverberation from ensembles of 25 units. However, spectrogram reconstruction quality deteriorated in severe reverberation for both MUs and SUs such that the neural degradation paralleled the degradation in the stimulus spectrogram. Furthermore, spectrograms reconstructed from responses to reverberant stimuli resembled spectrograms of reverberant speech better than spectrograms of dry speech. Overall, the results provide no evidence for a dereverberation mechanism in neural responses from the rabbit IC when studied with linear reconstruction techniques.

Adaptation in auditory processing

Adaptation is an essential feature of auditory neurons, which reduces their responses to unchanging and recurring sounds and allows their response properties to be matched to the constantly changing statistics of sounds that reach the ears. As a consequence, processing in the auditory system highlights novel or unpredictable sounds and produces an efficient representation of the vast range of sounds that animals can perceive by continually adjusting the sensitivity and, to a lesser extent, the tuning properties of neurons to the most commonly encountered stimulus values. Together with attentional modulation, adaptation to sound statistics also helps to generate neural representations of sound that are tolerant to background noise and therefore plays a vital role in auditory scene analysis. In this review, we consider the diverse forms of adaptation that are found in the auditory system in terms of the processing levels at which they arise, the underlying neural mechanisms, and their impact on neural coding and perception. We also ask what the dynamics of adaptation, which can occur over multiple timescales, reveal about the statistical properties of the environment. Finally, we examine how adaptation to sound statistics is influenced by learning and experience and changes as a result of aging and hearing loss.

Deep neural networks effectively model neural adaptation to changing background noise and suggest nonlinear noise filtering methods in auditory cortex

The human auditory system displays a robust capacity to adapt to sudden changes in background noise, allowing for continuous speech comprehension despite changes in background environments. However, despite comprehensive studies characterizing this ability, the computations that underly this process are not well understood. The first step towards understanding a complex system is to propose a suitable model, but the classical and easily interpreted model for the auditory system, the spectro-temporal receptive field (STRF), cannot match the nonlinear neural dynamics involved in noise adaptation. Here, we utilize a deep neural network (DNN) to model neural adaptation to noise, illustrating its effectiveness at reproducing the complex dynamics at the levels of both individual electrodes and the cortical population. By closely inspecting the model's STRF-like computations over time, we find that the model alters both the gain and shape of its receptive field when adapting to a sudden noise change. We show that the DNN model's gain changes allow it to perform adaptive gain control, while the spectro-temporal change creates noise filtering by altering the inhibitory region of the model's receptive field. Further, we find that models of electrodes in nonprimary auditory cortex also exhibit noise filtering changes in their excitatory regions, suggesting differences in noise filtering mechanisms along the cortical hierarchy. These findings demonstrate the capability of deep neural networks to model complex neural adaptation and offer new hypotheses about the computations the auditory cortex performs to enable noise-robust speech perception in real-world, dynamic environments.

Reduction in sound discrimination in noise is related to envelope similarity and not to a decrease in envelope tracking abilities

Humans and animals constantly face challenging acoustic environments, such as various background noises, that impair the detection, discrimination and identification of behaviourally relevant sounds. Here, we disentangled the role of temporal envelope tracking in the reduction in neuronal and behavioural discrimination between communication sounds in situations of acoustic degradations. By collecting neuronal activity from six different levels of the auditory system, from the auditory nerve up to the secondary auditory cortex, in anaesthetized guinea-pigs, we found that tracking of slow changes of the temporal envelope is a general functional property of auditory neurons for encoding communication sounds in quiet conditions and in adverse, challenging conditions. Results from a go/no-go sound discrimination task in mice support the idea that the loss of distinct slow envelope cues in noisy conditions impacted the discrimination performance. Together, these results suggest that envelope tracking is potentially a universal mechanism operating in the central auditory system, which allows the detection of any between-stimulus difference in the slow envelope and thus copes with degraded conditions. KEY POINTS: In quiet conditions, envelope tracking in the low amplitude modulation range (<20 Hz) is correlated with the neuronal discrimination between communication sounds as quantified by mutual information from the cochlear nucleus up to the auditory cortex. At each level of the auditory system, auditory neurons retain their abilities to track the communication sound envelopes in situations of acoustic degradation, such as vocoding and the addition of masking noises up to a signal-to-noise ratio of -10 dB. In noisy conditions, the increase in between-stimulus envelope similarity explains the reduction in both behavioural and neuronal discrimination in the auditory system. Envelope tracking can be viewed as a universal mechanism that allows neural and behavioural discrimination as long as the temporal envelope of communication sounds displays some differences.

Accounting for Bias in the Estimation of r 2 between Two Sets of Noisy Neural Responses

The Pearson correlation coefficient squared, r 2, is an important tool used in the analysis of neural data to quantify the similarity between neural tuning curves. Yet this metric is biased by trial-to-trial variability; as trial-to-trial variability increases, measured correlation decreases. Major lines of research are confounded by this bias, including those involving the study of invariance of neural tuning across conditions and the analysis of the similarity of tuning across neurons. To address this, we extend an estimator, [Formula: see text], that was recently developed for estimating model-to-neuron correlation, in which a noisy signal is compared with a noise-free prediction, to the case of neuron-to-neuron correlation, in which two noisy signals are compared with each other. We compare the performance of our novel estimator to a prior method developed by Spearman, commonly used in other fields but widely overlooked in neuroscience, and find that our method has less bias. We then apply our estimator to demonstrate how it avoids drastic confounds introduced by trial-to-trial variability using data collected in two prior studies (macaque, both sexes) that examined two different forms of invariance in the neural encoding of visual inputs-translation invariance and fill-outline invariance. Our results quantify for the first time the gradual falloff with spatial offset of translation-invariant shape selectivity within visual cortical neuronal receptive fields and offer a principled method to compare invariance in noisy biological systems to that in noise-free models.SIGNIFICANCE STATEMENT Quantifying the similarity between two sets of averaged neural responses is fundamental to the analysis of neural data. A ubiquitous metric of similarity, the correlation coefficient, is attenuated by trial-to-trial variability that arises from many irrelevant factors. Spearman recognized this problem and proposed corrected methods that have been extended over a century. We show this method has large asymptotic biases that can be overcome using a novel estimator. Despite the frequent use of the correlation coefficient in neuroscience, consensus on how to address this fundamental statistical issue has not been reached. We provide an accurate estimator of the correlation coefficient and apply it to gain insight into visual invariance.

Decreased Modulation of Population Correlations in Auditory Cortex Is Associated with Decreased Auditory Detection Performance in Old Mice

Age-related hearing loss (presbycusis) affects one-third of the world's population. One hallmark of presbycusis is difficulty hearing in noisy environments. Presbycusis can be separated into two components: the aging ear and the aging brain. To date, the role of the aging brain in presbycusis is not well understood. Activity in the primary auditory cortex (A1) during a behavioral task is because of a combination of responses representing the acoustic stimuli, attentional gain, and behavioral choice. Disruptions in any of these aspects can lead to decreased auditory processing. To investigate how these distinct components are disrupted in aging, we performed in vivo 2-photon Ca2+ imaging in both male and female mice (Thy1-GCaMP6s × CBA/CaJ mice) that retain peripheral hearing into old age. We imaged A1 neurons of young adult (2-6 months) and old mice (16-24 months) during a tone detection task in broadband noise. While young mice performed well, old mice performed worse at low signal-to-noise ratios. Calcium imaging showed that old animals have increased prestimulus activity, reduced attentional gain, and increased noise correlations. Increased correlations in old animals exist regardless of cell tuning and behavioral outcome, and these correlated networks exist over a much larger portion of cortical space. Neural decoding techniques suggest that this prestimulus activity is predictive of old animals making early responses. Together, our results suggest a model in which old animals have higher and more correlated prestimulus activity and cannot fully suppress this activity, leading to the decreased representation of targets among distracting stimuli.SIGNIFICANCE STATEMENT Aging inhibits the ability to hear clearly in noisy environments. We show that the aging auditory cortex is unable to fully suppress its responses to background noise. During an auditory behavior, fewer neurons were suppressed in the old relative to young animals, which leads to higher prestimulus activity and more false alarms. We show that this excess activity additionally leads to increased correlations between neurons, reducing the amount of relevant stimulus information in the auditory cortex. Future work identifying the lost circuits that are responsible for proper background suppression could provide new targets for therapeutic strategies to preserve auditory processing ability into old age.

Relative changes in the cochlear summating potentials to paired-clicks predict speech-in-noise perception and subjective hearing acuity

Objective assays of human cochlear synaptopathy (CS) have been challenging to develop. It is suspected that relative summating potential (SP) changes are different in listeners with CS. In this proof-of-concept study, young, normal-hearing adults were recruited and assigned to a low/high-risk group for having CS based on their extended audiograms (9-16 kHz). SPs to paired-clicks with varying inter-click intervals isolated non-refractory receptor components of cochlear activity. Abrupt increases in SPs to paired- vs single-clicks were observed in high-risk listeners. Critically, exaggerated SPs predicted speech-in-noise and subjective hearing abilities, suggesting relative SP changes to rapid clicks might help identify putative synaptopathic listeners.

Best frequencies and temporal delays are similar across the low-frequency regions of the guinea pig cochlea

The cochlea maps tones with different frequencies to distinct anatomical locations. For instance, a faint 5000-hertz tone produces brisk responses at a place approximately 8 millimeters into the 18-millimeter-long guinea pig cochlea, but little response elsewhere. This place code pervades the auditory pathways, where neurons have "best frequencies" determined by their connections to the sensory cells in the hearing organ. However, frequency selectivity in cochlear regions encoding low-frequency sounds has not been systematically studied. Here, we show that low-frequency hearing works according to a unique principle that does not involve a place code. Instead, sound-evoked responses and temporal delays are similar across the low-frequency regions of the cochlea. These findings are a break from theories considered proven for 100 years and have broad implications for understanding information processing in the brainstem and cortex and for optimizing the stimulus delivery in auditory implants.

Cortical adaptation to sound reverberation

In almost every natural environment, sounds are reflected by nearby objects, producing many delayed and distorted copies of the original sound, known as reverberation. Our brains usually cope well with reverberation, allowing us to recognize sound sources regardless of their environments. In contrast, reverberation can cause severe difficulties for speech recognition algorithms and hearing-impaired people. The present study examines how the auditory system copes with reverberation. We trained a linear model to recover a rich set of natural, anechoic sounds from their simulated reverberant counterparts. The model neurons achieved this by extending the inhibitory component of their receptive filters for more reverberant spaces, and did so in a frequency-dependent manner. These predicted effects were observed in the responses of auditory cortical neurons of ferrets in the same simulated reverberant environments. Together, these results suggest that auditory cortical neurons adapt to reverberation by adjusting their filtering properties in a manner consistent with dereverberation.

Improved Speech Hearing in Noise with Invasive Electrical Brain Stimulation

Speech perception in noise is a challenging everyday task with which many listeners have difficulty. Here, we report a case in which electrical brain stimulation of implanted intracranial electrodes in the left planum temporale (PT) of a neurosurgical patient significantly and reliably improved subjective quality (up to 50%) and objective intelligibility (up to 97%) of speech in noise perception. Stimulation resulted in a selective enhancement of speech sounds compared with the background noises. The receptive fields of the PT sites whose stimulation improved speech perception were tuned to spectrally broad and rapidly changing sounds. Corticocortical evoked potential analysis revealed that the PT sites were located between the sites in Heschl's gyrus and the superior temporal gyrus. Moreover, the discriminability of speech from nonspeech sounds increased in population neural responses from Heschl's gyrus to the PT to the superior temporal gyrus sites. These findings causally implicate the PT in background noise suppression and may point to a novel potential neuroprosthetic solution to assist in the challenging task of speech perception in noise.SIGNIFICANCE STATEMENT Speech perception in noise remains a challenging task for many individuals. Here, we present a case in which the electrical brain stimulation of intracranially implanted electrodes in the planum temporale of a neurosurgical patient significantly improved both the subjective quality (up to 50%) and objective intelligibility (up to 97%) of speech perception in noise. Stimulation resulted in a selective enhancement of speech sounds compared with the background noises. Our local and network-level functional analyses placed the planum temporale sites in between the sites in the primary auditory areas in Heschl's gyrus and nonprimary auditory areas in the superior temporal gyrus. These findings causally implicate planum temporale in acoustic scene analysis and suggest potential neuroprosthetic applications to assist hearing in noise.

Harmonicity aids hearing in noise

Hearing in noise is a core problem in audition, and a challenge for hearing-impaired listeners, yet the underlying mechanisms are poorly understood. We explored whether harmonic frequency relations, a signature property of many communication sounds, aid hearing in noise for normal hearing listeners. We measured detection thresholds in noise for tones and speech synthesized to have harmonic or inharmonic spectra. Harmonic signals were consistently easier to detect than otherwise identical inharmonic signals. Harmonicity also improved discrimination of sounds in noise. The largest benefits were observed for two-note up-down "pitch" discrimination and melodic contour discrimination, both of which could be performed equally well with harmonic and inharmonic tones in quiet, but which showed large harmonic advantages in noise. The results show that harmonicity facilitates hearing in noise, plausibly by providing a noise-robust pitch cue that aids detection and discrimination.

Contributions of Distinct Auditory Cortical Inhibitory Neuron Types to the Detection of Sounds in Background Noise

The ability to separate background noise from relevant acoustic signals is essential for appropriate sound-driven behavior in natural environments. Examples of this separation are apparent in the auditory system, where neural responses to behaviorally relevant stimuli become increasingly noise invariant along the ascending auditory pathway. However, the mechanisms that underlie this reduction in responses to background noise are not well understood. To address this gap in knowledge, we first evaluated the effects of auditory cortical inactivation on mice of both sexes trained to perform a simple auditory signal-in-noise detection task and found that outputs from the auditory cortex are important for the detection of auditory stimuli in noisy environments. Next, we evaluated the contributions of the two most common cortical inhibitory cell types, parvalbumin-expressing (PV+) and somatostatin-expressing (SOM+) interneurons, to the perception of masked auditory stimuli. We found that inactivation of either PV+ or SOM+ cells resulted in a reduction in the ability of mice to determine the presence of auditory stimuli masked by noise. These results indicate that a disruption of auditory cortical network dynamics by either of these two types of inhibitory cells is sufficient to impair the ability to separate acoustic signals from noise.

Adaptation to noise in normal and impaired hearing

Many aspects of hearing function are negatively affected by background noise. Listeners, however, have some ability to adapt to background noise. For instance, the detection of pure tones and the recognition of isolated words embedded in noise can improve gradually as tones and words are delayed a few hundred milliseconds in the noise. While some evidence suggests that adaptation to noise could be mediated by the medial olivocochlear reflex, adaptation can occur for people who do not have a functional reflex. Since adaptation can facilitate hearing in noise, and hearing in noise is often harder for hearing-impaired than for normal-hearing listeners, it is conceivable that adaptation is impaired with hearing loss. It remains unclear, however, if and to what extent this is the case, or whether impaired adaptation contributes to the greater difficulties experienced by hearing-impaired listeners understanding speech in noise. Here, we review adaptation to noise, the mechanisms potentially contributing to this adaptation, and factors that might reduce the ability to adapt to background noise, including cochlear hearing loss, cochlear synaptopathy, aging, and noise exposure. The review highlights few knowns and many unknowns about adaptation to noise, and thus paves the way for further research on this topic.

Hearing in Complex Environments: Auditory Gain Control, Attention, and Hearing Loss

Listening in noisy or complex sound environments is difficult for individuals with normal hearing and can be a debilitating impairment for those with hearing loss. Extracting meaningful information from a complex acoustic environment requires the ability to accurately encode specific sound features under highly variable listening conditions and segregate distinct sound streams from multiple overlapping sources. The auditory system employs a variety of mechanisms to achieve this auditory scene analysis. First, neurons across levels of the auditory system exhibit compensatory adaptations to their gain and dynamic range in response to prevailing sound stimulus statistics in the environment. These adaptations allow for robust representations of sound features that are to a large degree invariant to the level of background noise. Second, listeners can selectively attend to a desired sound target in an environment with multiple sound sources. This selective auditory attention is another form of sensory gain control, enhancing the representation of an attended sound source while suppressing responses to unattended sounds. This review will examine both “bottom-up” gain alterations in response to changes in environmental sound statistics as well as “top-down” mechanisms that allow for selective extraction of specific sound features in a complex auditory scene. Finally, we will discuss how hearing loss interacts with these gain control mechanisms, and the adaptive and/or maladaptive perceptual consequences of this plasticity.

[Effects of background noise on auditory response characteristics of primary auditory cortex neurons in awake mice]

OBJECTIVE

To study the effects of different continuous background noises on auditory response characteristics of primary auditory cortex (A1) neurons in awake mice.

METHODS

We performed in vivo cell-attached recordings in layer 4 neurons of the A1 of awake mice to investigate how continuous background noises of different levels affected the intensity tuning, frequency tuning and time characteristics of individual A1 neurons. According to the intensity tuning characteristics and types of stimulation, 44 neurons were devided into 4 groups: monotonic-intensity group (20 monotonic neurons), nonmonotonic-intensity group (6 nonmonotonic neurons), monotonic-frequency group (25 monotonic neurons) and monotonic-latency group (15 monotonic neurons).

RESULTS

The A1 neurons only had transient spike response within 10 to 40 ms after the onset of continuous wild-band noise stimulation. The noise intensity had no significant effects on the background firing rates of the A1 neurons (P>0.05). The increase of background noise resulted in a significant linear elevation of the intensity threshold of monotonic and nonmonotonic neurons for tone-evoked response (R²>0.90, P < 0.05). No significant difference was observed in the slopes of threshold changes between monotonic and nonmonotonic neurons (P>0.05). The best intensity of nonmonotonic neurons increased along with the intensity of the background noise, and the variation of the best intensity was positively correlated with the change of the threshold of the same neuron (r=0.944, P < 0.001). The frequency response bandwidth and the firing rate of the A1 neurons decreased as the noise intensity increased (P < 0.001), but the best frequency almost remained unchanged (P < 0.001). The increase of background noise intensity resulted in an increased first spike latency of the neurons to the same tone stimulus (P < 0.05) without affecting the time accuracy of the first action potential (P>0.05).

CONCLUSION

The acoustic response threshold of the A1 neurons increases linearly with the increase of background noise intensity. An increased background noise leads to compressed frequency band-width, a decreased firing rate and a prolonged spike latency, but the frequency selectivity and the time accuracy of auditory response to the same noise remain stable.

Altered Response Dynamics and Increased Population Correlation to Tonal Stimuli Embedded in Noise in Aging Auditory Cortex

Age-related hearing loss (presbycusis) is a chronic health condition that affects one-third of the world population. One hallmark of presbycusis is a difficulty hearing in noisy environments. Presbycusis can be separated into two components: alterations of peripheral mechanotransduction of sound in the cochlea and central alterations of auditory processing areas of the brain. Although the effects of the aging cochlea in hearing loss have been well studied, the role of the aging brain in hearing loss is less well understood. Therefore, to examine how age-related central processing changes affect hearing in noisy environments, we used a mouse model (Thy1-GCaMP6s X CBA) that has excellent peripheral hearing in old age. We used in vivo two-photon Ca2+ imaging to measure the responses of neuronal populations in auditory cortex (ACtx) of adult (2-6 months, nine male, six female, 4180 neurons) and aging mice (15-17 months, six male, three female, 1055 neurons) while listening to tones in noisy backgrounds. We found that ACtx neurons in aging mice showed larger responses to tones and have less suppressed responses consistent with reduced inhibition. Aging neurons also showed less sensitivity to temporal changes. Population analysis showed that neurons in aging mice showed higher pairwise activity correlations and showed a reduced diversity in responses to sound stimuli. Using neural decoding techniques, we show a loss of information in neuronal populations in the aging brain. Thus, aging not only affects the responses of single neurons but also affects how these neurons jointly represent stimuli.SIGNIFICANCE STATEMENT Aging results in hearing deficits particularly under challenging listening conditions. We show that auditory cortex contains distinct subpopulations of excitatory neurons that preferentially encode different stimulus features and that aging selectively reduces certain subpopulations. We also show that aging increases correlated activity between neurons and thereby reduces the response diversity in auditory cortex. The loss of population response diversity leads to a decrease of stimulus information and deficits in sound encoding, especially in noisy backgrounds. Future work determining the identities of circuits affected by aging could provide new targets for therapeutic strategies.

Sensitivity of neural responses in the inferior colliculus to statistical features of sound textures

Previous psychophysical studies have identified a hierarchy of time-averaged statistics which determine the identity of natural sound textures. However, it is unclear whether the neurons in the inferior colliculus (IC) are sensitive to each of these statistical features in the natural sound textures. We used 13 representative sound textures spanning the space of 3 statistics extracted from over 200 natural textures. The synthetic textures were generated by incorporating the statistical features in a step-by-step manner, in which a particular statistical feature was changed while the other statistical features remain unchanged. The extracellular activity in response to the synthetic texture stimuli was recorded in the IC of anesthetized rats. Analysis of the transient and sustained multiunit activity after each transition of statistical feature showed that the IC units were sensitive to the changes of all types of statistics, although to a varying extent. For example, we found that more neurons were sensitive to the changes in variance than that in the modulation correlations. Our results suggest that the sensitivity of the statistical features in the subcortical levels contributes to the identification and discrimination of natural sound textures.

Understanding degraded speech leads to perceptual gating of a brainstem reflex in human listeners

The ability to navigate "cocktail party" situations by focusing on sounds of interest over irrelevant, background sounds is often considered in terms of cortical mechanisms. However, subcortical circuits such as the pathway underlying the medial olivocochlear (MOC) reflex modulate the activity of the inner ear itself, supporting the extraction of salient features from auditory scene prior to any cortical processing. To understand the contribution of auditory subcortical nuclei and the cochlea in complex listening tasks, we made physiological recordings along the auditory pathway while listeners engaged in detecting non(sense) words in lists of words. Both naturally spoken and intrinsically noisy, vocoded speech-filtering that mimics processing by a cochlear implant (CI)-significantly activated the MOC reflex, but this was not the case for speech in background noise, which more engaged midbrain and cortical resources. A model of the initial stages of auditory processing reproduced specific effects of each form of speech degradation, providing a rationale for goal-directed gating of the MOC reflex based on enhancing the representation of the energy envelope of the acoustic waveform. Our data reveal the coexistence of 2 strategies in the auditory system that may facilitate speech understanding in situations where the signal is either intrinsically degraded or masked by extrinsic acoustic energy. Whereas intrinsically degraded streams recruit the MOC reflex to improve representation of speech cues peripherally, extrinsically masked streams rely more on higher auditory centres to denoise signals.

When and How Does the Auditory Cortex Influence Subcortical Auditory Structures? New Insights About the Roles of Descending Cortical Projections

For decades, the corticofugal descending projections have been anatomically well described but their functional role remains a puzzling question. In this review, we will first describe the contributions of neuronal networks in representing communication sounds in various types of degraded acoustic conditions from the cochlear nucleus to the primary and secondary auditory cortex. In such situations, the discrimination abilities of collicular and thalamic neurons are clearly better than those of cortical neurons although the latter remain very little affected by degraded acoustic conditions. Second, we will report the functional effects resulting from activating or inactivating corticofugal projections on functional properties of subcortical neurons. In general, modest effects have been observed in anesthetized and in awake, passively listening, animals. In contrast, in behavioral tasks including challenging conditions, behavioral performance was severely reduced by removing or transiently silencing the corticofugal descending projections. This suggests that the discriminative abilities of subcortical neurons may be sufficient in many acoustic situations. It is only in particularly challenging situations, either due to the task difficulties and/or to the degraded acoustic conditions that the corticofugal descending connections bring additional abilities. Here, we propose that it is both the top-down influences from the prefrontal cortex, and those from the neuromodulatory systems, which allow the cortical descending projections to impact behavioral performance in reshaping the functional circuitry of subcortical structures. We aim at proposing potential scenarios to explain how, and under which circumstances, these projections impact on subcortical processing and on behavioral responses.

Natural Statistics as Inference Principles of Auditory Tuning in Biological and Artificial Midbrain Networks

Bats provide a powerful mammalian model to explore the neural representation of complex sounds, as they rely on hearing to survive in their environment. The inferior colliculus (IC) is a central hub of the auditory system that receives converging projections from the ascending pathway and descending inputs from auditory cortex. In this work, we build an artificial neural network to replicate auditory characteristics in IC neurons of the big brown bat. We first test the hypothesis that spectro-temporal tuning of IC neurons is optimized to represent the natural statistics of conspecific vocalizations. We estimate spectro-temporal receptive fields (STRFs) of IC neurons and compare tuning characteristics to statistics of bat calls. The results indicate that the FM tuning of IC neurons is matched with the statistics. Then, we investigate this hypothesis on the network optimized to represent natural sound statistics and to compare its output with biological responses. We also estimate biomimetic STRFs from the artificial network and correlate their characteristics to those of biological neurons. Tuning properties of both biological and artificial neurons reveal strong agreement along both spectral and temporal dimensions, and suggest the presence of nonlinearity, sparsity, and complexity constraints that underlie the neural representation in the auditory midbrain. Additionally, the artificial neurons replicate IC neural activities in discrimination of social calls, and provide simulated results for a noise robust discrimination. In this way, the biomimetic network allows us to infer the neural mechanisms by which the bat’s IC processes natural sounds used to construct the auditory scene.

Causal inference in environmental sound recognition

Sound is caused by physical events in the world. Do humans infer these causes when recognizing sound sources? We tested whether the recognition of common environmental sounds depends on the inference of a basic physical variable - the source intensity (i.e., the power that produces a sound). A source's intensity can be inferred from the intensity it produces at the ear and its distance, which is normally conveyed by reverberation. Listeners could thus use intensity at the ear and reverberation to constrain recognition by inferring the underlying source intensity. Alternatively, listeners might separate these acoustic cues from their representation of a sound's identity in the interest of invariant recognition. We compared these two hypotheses by measuring recognition accuracy for sounds with typically low or high source intensity (e.g., pepper grinders vs. trucks) that were presented across a range of intensities at the ear or with reverberation cues to distance. The recognition of low-intensity sources (e.g., pepper grinders) was impaired by high presentation intensities or reverberation that conveyed distance, either of which imply high source intensity. Neither effect occurred for high-intensity sources. The results suggest that listeners implicitly use the intensity at the ear along with distance cues to infer a source's power and constrain its identity. The recognition of real-world sounds thus appears to depend upon the inference of their physical generative parameters, even generative parameters whose cues might otherwise be separated from the representation of a sound's identity.

State Transitions During Discrimination Learning in the Gerbil Auditory Cortex Analyzed by Network Causality Metrics

This work studies the evolution of cortical networks during the transition from escape strategy to avoidance strategy in auditory discrimination learning in Mongolian gerbils trained by the well-established two-way active avoidance learning paradigm. The animals were implanted with electrode arrays centered on the surface of the primary auditory cortex and electrocorticogram (ECoG) recordings were made during performance of an auditory Go/NoGo discrimination task. Our experiments confirm previous results on a sudden behavioral change from the initial naïve state to an avoidance strategy as learning progresses. We employed two causality metrics using Granger Causality (GC) and New Causality (NC) to quantify changes in the causality flow between ECoG channels as the animals switched to avoidance strategy. We found that the number of channel pairs with inverse causal interaction significantly increased after the animal acquired successful discrimination, which indicates structural changes in the cortical networks as a result of learning. A suitable graph-theoretical model is developed to interpret the findings in terms of cortical networks evolving during cognitive state transitions. Structural changes lead to changes in the dynamics of neural populations, which are described as phase transitions in the network graph model with small-world connections. Overall, our findings underscore the importance of functional reorganization in sensory cortical areas as a possible neural contributor to behavioral changes.

Sustained Activation of PV+ Interneurons in Core Auditory Cortex Enables Robust Divisive Gain Control for Complex and Naturalistic Stimuli

Sensory cortices must flexibly adapt their operations to internal states and external requirements. Sustained modulation of activity levels in different inhibitory interneuron populations may provide network-level mechanisms for adjustment of sensory cortical processing on behaviorally relevant timescales. However, understanding of the computational roles of inhibitory interneuron modulation has mostly been restricted to effects at short timescales, through the use of phasic optogenetic activation and transient stimuli. Here, we investigated how modulation of inhibitory interneurons affects cortical computation on longer timescales, by using sustained, network-wide optogenetic activation of parvalbumin-positive interneurons (the largest class of cortical inhibitory interneurons) to study modulation of auditory cortical responses to prolonged and naturalistic as well as transient stimuli. We found highly conserved spectral and temporal tuning in auditory cortical neurons, despite a profound reduction in overall network activity. This reduction was predominantly divisive, and consistent across simple, complex, and naturalistic stimuli. A recurrent network model with power-law input-output functions replicated our results. We conclude that modulation of parvalbumin-positive interneurons on timescales typical of sustained neuromodulation may provide a means for robust divisive gain control conserving stimulus representations.

Auditory Cortical Plasticity Dependent on Environmental Noise Statistics

During critical periods, neural circuits develop to form receptive fields that adapt to the sensory environment and enable optimal performance of relevant tasks. We hypothesized that early exposure to background noise can improve signal-in-noise processing, and the resulting receptive field plasticity in the primary auditory cortex can reveal functional principles guiding that important task. We raised rat pups in different spectro-temporal noise statistics during their auditory critical period. As adults, they showed enhanced behavioral performance in detecting vocalizations in noise. Concomitantly, encoding of vocalizations in noise in the primary auditory cortex improves with noise-rearing. Significantly, spectro-temporal modulation plasticity shifts cortical preferences away from the exposed noise statistics, thus reducing noise interference with the foreground sound representation. Auditory cortical plasticity shapes receptive field preferences to optimally extract foreground information in noisy environments during noise-rearing. Early noise exposure induces cortical circuits to implement efficient coding in the joint spectral and temporal modulation domain.

After rearing rats in moderately loud spectro-temporally modulated background noise, Homma et al. investigated signal-in-noise processing in the primary auditory cortex. Noise-rearing improved vocalization-in-noise performance in both behavioral testing and neural decoding. Cortical plasticity shifted neuronal spectro-temporal modulation preferences away from the exposed noise statistics.

Robustness to Noise in the Auditory System: A Distributed and Predictable Property

Background noise strongly penalizes auditory perception of speech in humans or vocalizations in animals. Despite this, auditory neurons are still able to detect communications sounds against considerable levels of background noise. We collected neuronal recordings in cochlear nucleus (CN), inferior colliculus (IC), auditory thalamus, and primary and secondary auditory cortex in response to vocalizations presented either against a stationary or a chorus noise in anesthetized guinea pigs at three signal-to-noise ratios (SNRs; −10, 0, and 10 dB). We provide evidence that, at each level of the auditory system, five behaviors in noise exist within a continuum, from neurons with high-fidelity representations of the signal, mostly found in IC and thalamus, to neurons with high-fidelity representations of the noise, mostly found in CN for the stationary noise and in similar proportions in each structure for the chorus noise. The two cortical areas displayed fewer robust responses than the IC and thalamus. Furthermore, between 21% and 72% of the neurons (depending on the structure) switch categories from one background noise to another, even if the initial assignment of these neurons to a category was confirmed by a severe bootstrap procedure. Importantly, supervised learning pointed out that assigning a recording to one of the five categories can be predicted up to a maximum of 70% based on both the response to signal alone and noise alone.

Cochlear neural degeneration disrupts hearing in background noise by increasing auditory cortex internal noise

Correlational evidence in humans suggests that selective difficulties hearing in noisy, social settings may reflect premature auditory nerve degeneration. Here, we induced primary cochlear neural degeneration (CND) in adult mice and found direct behavioral evidence for selective detection deficits in background noise. To identify central determinants for this perceptual disorder, we tracked daily changes in ensembles of layer 2/3 auditory cortex parvalbumin-expressing inhibitory neurons and excitatory pyramidal neurons with chronic two-photon calcium imaging. CND induced distinct forms of plasticity in cortical excitatory and inhibitory neurons that culminated in net hyperactivity, increased neural gain, and reduced adaptation to background noise. Ensemble activity measured while mice detected targets in noise could accurately decode whether individual behavioral trials were hits or misses. After CND, random surges of hypercorrelated cortical activity occurring just before target onset reliably predicted impending detection failures, revealing a source of internal cortical noise underlying perceptual difficulties in external noise.

Ambient noise exposure induces long-term adaptations in adult brainstem neurons

To counterbalance long-term environmental changes, neuronal circuits adapt the processing of sensory information. In the auditory system, ongoing background noise drives long-lasting adaptive mechanism in binaural coincidence detector neurons in the superior olive. However, the compensatory cellular mechanisms of the binaural neurons in the medial superior olive (MSO) to long-term background changes are unexplored. Here we investigated the cellular properties of MSO neurons during long-lasting adaptations induced by moderate omnidirectional noise exposure. After noise exposure, the input resistance of MSO neurons of mature Mongolian gerbils was reduced, likely due to an upregulation of hyperpolarisation-activated cation and low voltage-activated potassium currents. Functionally, the long-lasting adaptations increased the action potential current threshold and facilitated high frequency output generation. Noise exposure accelerated the occurrence of spontaneous postsynaptic currents. Together, our data suggest that cellular adaptations in coincidence detector neurons of the MSO to continuous noise exposure likely increase the sensitivity to differences in sound pressure levels.

The effect of input noises on the activity of auditory neurons using GLM-based metrics

CONTEXT

The auditory system is extremely efficient in extracting auditory information in the presence of background noise. However, people with auditory implants have a hard time understanding speech in noisy conditions. Understanding the mechanisms of perception in noise could lead to better stimulation or preprocessing strategies for such implants.

OBJECTIVE

The neural mechanisms related to the processing of background noise, especially in the inferior colliculus (IC) where the auditory midbrain implant is located, are still not well understood. We thus wish to investigate if there is a difference in the activity of neurons in the IC when presenting noisy vocalizations with different types of noise (stationary vs. non-stationary), input signal-to-noise ratios (SNR) and signal levels.

APPROACH

We developed novel metrics based on a generalized linear model (GLM) to investigate the effect of a given input noise on neural activity. We used these metrics to analyze neural data recorded from the IC in ketamine-anesthetized female Hartley guinea pigs while presenting noisy vocalizations.

MAIN RESULTS

We found that non-stationary noise clearly contributes to the multi-unit neural activity in the IC by causing excitation, regardless of the SNR, input level or vocalization type. However, when presenting white or natural stationary noises, a great diversity of responses was observed for the different conditions, where the multi-unit activity of some sites was affected by the presence of noise and the activity of others was not.

SIGNIFICANCE

The GLM-based metrics allowed the identification of a clear distinction between the effect of white or natural stationary noises and that of non-stationary noise on the multi-unit activity in the IC. This had not been observed before and indicates that the so-called noise invariance in the IC is dependent on the input noisy conditions. This could suggest different preprocessing or stimulation approaches for auditory midbrain implants depending on the noisy conditions.

A Survey on Probabilistic Models in Human Perception and Machines

Extracting information from noisy signals is of fundamental importance for both biological and artificial perceptual systems. To provide tractable solutions to this challenge, the fields of human perception and machine signal processing (SP) have developed powerful computational models, including Bayesian probabilistic models. However, little true integration between these fields exists in their applications of the probabilistic models for solving analogous problems, such as noise reduction, signal enhancement, and source separation. In this mini review, we briefly introduce and compare selective applications of probabilistic models in machine SP and human psychophysics. We focus on audio and audio-visual processing, using examples of speech enhancement, automatic speech recognition, audio-visual cue integration, source separation, and causal inference to illustrate the basic principles of the probabilistic approach. Our goal is to identify commonalities between probabilistic models addressing brain processes and those aiming at building intelligent machines. These commonalities could constitute the closest points for interdisciplinary convergence.

The second harmonic neurons in auditory midbrain of Hipposideros pratti are more tolerant to background white noise

Although acoustic communication is inevitably influenced by noise, behaviorally relevant sounds are perceived reliably. The noise-tolerant and -invariant responses of auditory neurons are thought to be the underlying mechanism. So, it is reasonable to speculate that neurons with best frequency tuned to behaviorally relevant sounds will play important role in noise-tolerant perception. Echolocating bats live in groups and emit multiple harmonic signals and analyze the returning echoes to extract information about the target features, making them prone to deal with noise in their natural habitat. The echolocation signal of Hipposideros pratti usually contains 3-4 harmonics (H₁H₄), the second harmonic has the highest amplitude and is thought to play an essential role during echolocation behavior. Therefore, it is reasonable to propose that neurons tuned to the H₂, named the H₂ neurons, can be more noise-tolerant to background noise. Taking advantage of bat's stereotypical echolocation signal and single-cell recording, our present study showed that the minimal threshold increases (12.2 dB) of H₂ neurons in the auditory midbrain were comparable to increase in bat's call intensity (14.2 dB) observed in 70 dB SPL white noise condition, indicating that the H₂ neurons could work as background noise monitor. The H₂ neurons had higher minimal thresholds and sharper frequency tuning, which enabled them to be more tolerant to background noise. Furthermore, the H₂ neurons had consistent best amplitude spikes and sharper intensity tuning in background white noise condition than in silence. Taken together, these results suggest that the H₂ neurons might account for noise-tolerant perception of behaviorally relevant sounds.

Task Engagement Improves Neural Discriminability in the Auditory Midbrain of the Marmoset Monkey

While task-dependent changes have been demonstrated in auditory cortex for a number of behavioral paradigms and mammalian species, less is known about how behavioral state can influence neural coding in the midbrain areas that provide auditory information to cortex. We measured single-unit activity in the inferior colliculus (IC) of common marmosets of both sexes while they performed a tone-in-noise detection task and during passive presentation of identical task stimuli. In contrast to our previous study in the ferret IC, task engagement had little effect on sound-evoked activity in central (lemniscal) IC of the marmoset. However, activity was significantly modulated in noncentral fields, where responses were selectively enhanced for the target tone relative to the distractor noise. This led to an increase in neural discriminability between target and distractors. The results confirm that task engagement can modulate sound coding in the auditory midbrain, and support a hypothesis that subcortical pathways can mediate highly trained auditory behaviors.SIGNIFICANCE STATEMENT While the cerebral cortex is widely viewed as playing an essential role in the learning and performance of complex auditory behaviors, relatively little attention has been paid to the role of brainstem and midbrain areas that process sound information before it reaches cortex. This study demonstrates that the auditory midbrain is also modulated during behavior. These modulations amplify task-relevant sensory information, a process that is traditionally attributed to cortex.

Complementary Effects of Adaptation and Gain Control on Sound Encoding in Primary Auditory Cortex

An important step toward understanding how the brain represents complex natural sounds is to develop accurate models of auditory coding by single neurons. A commonly used model is the linear-nonlinear spectro-temporal receptive field (STRF; LN model). The LN model accounts for many features of auditory tuning, but it cannot account for long-lasting effects of sensory context on sound-evoked activity. Two mechanisms that may support these contextual effects are short-term plasticity (STP) and contrast-dependent gain control (GC), which have inspired expanded versions of the LN model. Both models improve performance over the LN model, but they have never been compared directly. Thus, it is unclear whether they account for distinct processes or describe one phenomenon in different ways. To address this question, we recorded activity of neurons in primary auditory cortex (A1) of awake ferrets during presentation of natural sounds. We then fit models incorporating one nonlinear mechanism (GC or STP) or both (GC+STP) using this single dataset, and measured the correlation between the models' predictions and the recorded neural activity. Both the STP and GC models performed significantly better than the LN model, but the GC+STP model outperformed both individual models. We also quantified the equivalence of STP and GC model predictions and found only modest similarity. Consistent results were observed for a dataset collected in clean and noisy acoustic contexts. These results establish general methods for evaluating the equivalence of arbitrarily complex encoding models and suggest that the STP and GC models describe complementary processes in the auditory system.

Simple transformations capture auditory input to cortex

Sounds are processed by the ear and central auditory pathway. These processing steps are biologically complex, and many aspects of the transformation from sound waveforms to cortical response remain unclear. To understand this transformation, we combined models of the auditory periphery with various encoding models to predict auditory cortical responses to natural sounds. The cochlear models ranged from detailed biophysical simulations of the cochlea and auditory nerve to simple spectrogram-like approximations of the information processing in these structures. For three different stimulus sets, we tested the capacity of these models to predict the time course of single-unit neural responses recorded in ferret primary auditory cortex. We found that simple models based on a log-spaced spectrogram with approximately logarithmic compression perform similarly to the best-performing biophysically detailed models of the auditory periphery, and more consistently well over diverse natural and synthetic sounds. Furthermore, we demonstrated that including approximations of the three categories of auditory nerve fiber in these simple models can substantially improve prediction, particularly when combined with a network encoding model. Our findings imply that the properties of the auditory periphery and central pathway may together result in a simpler than expected functional transformation from ear to cortex. Thus, much of the detailed biological complexity seen in the auditory periphery does not appear to be important for understanding the cortical representation of sound.

Foreground stimuli and task engagement enhance neuronal adaptation to background noise in the inferior colliculus of macaques

Auditory neuronal responses are modified by background noise. Inferior colliculus (IC) neuronal responses adapt to the most frequent sound level within an acoustic scene (adaptation to stimulus statistics), a mechanism that may preserve neuronal and behavioral thresholds for signal detection. However, it is still unclear whether the presence of foreground stimuli and/or task involvement can modify neuronal adaptation. To investigate how task engagement interacts with this mechanism, we compared the response of IC neurons to background noise, which caused adaptation to stimulus statistics, while macaque monkeys performed a masked tone detection task (task-driven condition) with responses recorded when the same background noise was presented alone (passive listening condition). In the task-dependent condition, monkeys performed a Go/No-Go task while 50-ms tones were embedded within an adaptation-inducing continuous background noise whose levels changed every 50 ms and were drawn from a probability distribution. The adaptation to noise stimulus statistics in IC neuronal responses was significantly enhanced in the task-driven condition compared with the passive listening condition, showing that foreground stimuli and/or task-engagement can modify IC neuronal responses. Additionally, the response of IC neurons to noise was significantly affected by the preceding sensory information (history effect) regardless of task involvement. These studies show that dynamic range adaptation in IC preserves behavioral and neurometric thresholds irrespective of noise type and a dependence of neuronal activity on task-related factors at subcortical levels of processing.NEW & NOTEWORTHY Auditory neuronal responses are influenced by maskers and distractors. However, it is still unclear whether the neuronal sensitivity to the masker stimulus is influenced by task-dependent factors. Our study represents one of the first attempts to investigate how task involvement influences the neural representation of background sounds in the subcortical, midbrain auditory neurons of behaving animals.

Neural speech restoration at the cocktail party: Auditory cortex recovers masked speech of both attended and ignored speakers

Humans are remarkably skilled at listening to one speaker out of an acoustic mixture of several speech sources. Two speakers are easily segregated, even without binaural cues, but the neural mechanisms underlying this ability are not well understood. One possibility is that early cortical processing performs a spectrotemporal decomposition of the acoustic mixture, allowing the attended speech to be reconstructed via optimally weighted recombinations that discount spectrotemporal regions where sources heavily overlap. Using human magnetoencephalography (MEG) responses to a 2-talker mixture, we show evidence for an alternative possibility, in which early, active segregation occurs even for strongly spectrotemporally overlapping regions. Early (approximately 70-millisecond) responses to nonoverlapping spectrotemporal features are seen for both talkers. When competing talkers’ spectrotemporal features mask each other, the individual representations persist, but they occur with an approximately 20-millisecond delay. This suggests that the auditory cortex recovers acoustic features that are masked in the mixture, even if they occurred in the ignored speech. The existence of such noise-robust cortical representations, of features present in attended as well as ignored speech, suggests an active cortical stream segregation process, which could explain a range of behavioral effects of ignored background speech.

How do humans focus on one speaker when several are talking? MEG responses to a continuous two-talker mixture suggest that, even though listeners attend only to one of the talkers, their auditory cortex tracks acoustic features from both speakers. This occurs even when those features are locally masked by the other speaker.

White Noise Background Improves Tone Discrimination by Suppressing Cortical Tuning Curves

The brain faces the difficult task of maintaining a stable representation of key features of the outside world in noisy sensory surroundings. How does the sensory representation change with noise, and how does the brain make sense of it? We investigated the effect of background white noise (WN) on tuning properties of neurons in mouse A1 and its impact on discrimination performance in a go/no-go task. We find that WN suppresses the activity of A1 neurons, which surprisingly increases the discriminability of tones spectrally close to each other. To confirm the involvement of A1, we optogenetically excited parvalbumin-positive (PV⁺) neurons in A1, which have similar effects as WN on both tuning properties and frequency discrimination. A population model suggests that the suppression of A1 tuning curves increases frequency selectivity and thereby improves discrimination. Our findings demonstrate that the cortical representation of pure tones adapts during noise to improve sensory acuity.

Listening in complex acoustic scenes

Being able to pick out particular sounds, such as speech, against a background of other sounds represents one of the key tasks performed by the auditory system. Understanding how this happens is important because speech recognition in noise is particularly challenging for older listeners and for people with hearing impairments. Central to this ability is the capacity of neurons to adapt to the statistics of sounds reaching the ears, which helps to generate noise-tolerant representations of sounds in the brain. In more complex auditory scenes, such as a cocktail party — where the background noise comprises other voices, sound features associated with each source have to be grouped together and segregated from those belonging to other sources. This depends on precise temporal coding and modulation of cortical response properties when attending to a particular speaker in a multi-talker environment. Furthermore, the neural processing underlying auditory scene analysis is shaped by experience over multiple timescales.