Rapid Task-Related Plasticity of Spectrotemporal Receptive Fields in the Auditory Midbrain

Task-specific invariant representation in auditory cortex

Categorical sensory representations are critical for many behaviors, including speech perception. In the auditory system, categorical information is thought to arise hierarchically, becoming increasingly prominent in higher-order cortical regions. The neural mechanisms that support this robust and flexible computation remain poorly understood. Here, we studied sound representations in the ferret primary and non-primary auditory cortex while animals engaged in a challenging sound discrimination task. Population-level decoding of simultaneously recorded single neurons revealed that task engagement caused categorical sound representations to emerge in non-primary auditory cortex. In primary auditory cortex, task engagement caused a general enhancement of sound decoding that was not specific to task-relevant categories. These findings are consistent with mixed selectivity models of neural disentanglement, in which early sensory regions build an overcomplete representation of the world and allow neurons in downstream brain regions to flexibly and selectively read out behaviorally relevant, categorical information.

Mixed Representations of Sound and Action in the Auditory Midbrain

Linking sensory input and its consequences is a fundamental brain operation. During behavior, the neural activity of neocortical and limbic systems often reflects dynamic combinations of sensory and task-dependent variables, and these "mixed representations" are suggested to be important for perception, learning, and plasticity. However, the extent to which such integrative computations might occur outside of the forebrain is less clear. Here, we conduct cellular-resolution two-photon Ca2+ imaging in the superficial "shell" layers of the inferior colliculus (IC), as head-fixed mice of either sex perform a reward-based psychometric auditory task. We find that the activity of individual shell IC neurons jointly reflects auditory cues, mice's actions, and behavioral trial outcomes, such that trajectories of neural population activity diverge depending on mice's behavioral choice. Consequently, simple classifier models trained on shell IC neuron activity can predict trial-by-trial outcomes, even when training data are restricted to neural activity occurring prior to mice's instrumental actions. Thus, in behaving mice, auditory midbrain neurons transmit a population code that reflects a joint representation of sound, actions, and task-dependent variables.

Neural correlates of flexible sound perception in the auditory midbrain and thalamus

Hearing is an active process in which listeners must detect and identify sounds, segregate and discriminate stimulus features, and extract their behavioral relevance. Adaptive changes in sound detection can emerge rapidly, during sudden shifts in acoustic or environmental context, or more slowly as a result of practice. Although we know that context- and learning-dependent changes in the spectral and temporal sensitivity of auditory cortical neurons support many aspects of flexible listening, the contribution of subcortical auditory regions to this process is less understood. Here, we recorded single- and multi-unit activity from the central nucleus of the inferior colliculus (ICC) and the ventral subdivision of the medial geniculate nucleus (MGV) of Mongolian gerbils under two different behavioral contexts: as animals performed an amplitude modulation (AM) detection task and as they were passively exposed to AM sounds. Using a signal detection framework to estimate neurometric sensitivity, we found that neural thresholds in both regions improved during task performance, and this improvement was driven by changes in firing rate rather than phase locking. We also found that ICC and MGV neurometric thresholds improved and correlated with behavioral performance as animals learn to detect small AM depths during a multi-day perceptual training paradigm. Finally, we reveal that in the MGV, but not the ICC, context-dependent enhancements in AM sensitivity grow stronger during perceptual training, mirroring prior observations in the auditory cortex. Together, our results suggest that the auditory midbrain and thalamus contribute to flexible sound processing and perception over rapid and slow timescales.

Organization of orbitofrontal-auditory pathways in the Mongolian gerbil

Sound perception is highly malleable, rapidly adjusting to the acoustic environment and behavioral demands. This flexibility is the result of ongoing changes in auditory cortical activity driven by fluctuations in attention, arousal, or prior expectations. Recent work suggests that the orbitofrontal cortex (OFC) may mediate some of these rapid changes, but the anatomical connections between the OFC and the auditory system are not well characterized. Here, we used virally mediated fluorescent tracers to map the projection from OFC to the auditory midbrain, thalamus, and cortex in a classic animal model for auditory research, the Mongolian gerbil (Meriones unguiculatus). We observed no connectivity between the OFC and the auditory midbrain, and an extremely sparse connection between the dorsolateral OFC and higher order auditory thalamic regions. In contrast, we observed a robust connection between the ventral and medial subdivisions of the OFC and the auditory cortex, with a clear bias for secondary auditory cortical regions. OFC axon terminals were found in all auditory cortical lamina but were significantly more concentrated in the infragranular layers. Tissue-clearing and lightsheet microscopy further revealed that auditory cortical-projecting OFC neurons send extensive axon collaterals throughout the brain, targeting both sensory and non-sensory regions involved in learning, decision-making, and memory. These findings provide a more detailed map of orbitofrontal-auditory connections and shed light on the possible role of the OFC in supporting auditory cognition.

Mixed representations of sound and action in the auditory midbrain

Linking sensory input and its consequences is a fundamental brain operation. Accordingly, neural activity of neo-cortical and limbic systems often reflects dynamic combinations of sensory and behaviorally relevant variables, and these "mixed representations" are suggested to be important for perception, learning, and plasticity. However, the extent to which such integrative computations might occur in brain regions upstream of the forebrain is less clear. Here, we conduct cellular-resolution 2-photon Ca2+ imaging in the superficial "shell" layers of the inferior colliculus (IC), as head-fixed mice of either sex perform a reward-based psychometric auditory task. We find that the activity of individual shell IC neurons jointly reflects auditory cues and mice's actions, such that trajectories of neural population activity diverge depending on mice's behavioral choice. Consequently, simple classifier models trained on shell IC neuron activity can predict trial-by-trial outcomes, even when training data are restricted to neural activity occurring prior to mice's instrumental actions. Thus in behaving animals, auditory midbrain neurons transmit a population code that reflects a joint representation of sound and action.

Hierarchical differences in the encoding of sound and choice in the subcortical auditory system

Detection of sounds is a fundamental function of the auditory system. Although studies of auditory cortex have gained substantial insight into detection performance using behaving animals, previous subcortical studies have mostly taken place under anesthesia, in passively listening animals, or have not measured performance at threshold. These limitations preclude direct comparisons between neuronal responses and behavior. To address this, we simultaneously measured auditory detection performance and single-unit activity in the inferior colliculus (IC) and cochlear nucleus (CN) in macaques. The spontaneous activity and response variability of CN neurons were higher than those observed for IC neurons. Signal detection theoretic methods revealed that the magnitude of responses of IC neurons provided more reliable estimates of psychometric threshold and slope compared with the responses of single CN neurons. However, pooling small populations of CN neurons provided reliable estimates of psychometric threshold and slope, suggesting sufficient information in CN population activity. Trial-by-trial correlations between spike count and behavioral response emerged 50-75 ms after sound onset for most IC neurons, but for few neurons in the CN. These results highlight hierarchical differences between neurometric-psychometric correlations in CN and IC and have important implications for how subcortical information could be decoded.NEW & NOTEWORTHY The cerebral cortex is widely recognized to play a role in sensory processing and decision-making. Accounts of the neural basis of auditory perception and its dysfunction are based on this idea. However, significantly less attention has been paid to midbrain and brainstem structures in this regard. Here, we find that subcortical auditory neurons represent stimulus information sufficient for detection and predict behavioral choice on a trial-by-trial basis.

Targeted dimensionality reduction enables reliable estimation of neural population coding accuracy from trial-limited data

Rapidly developing technology for large scale neural recordings has allowed researchers to measure the activity of hundreds to thousands of neurons at single cell resolution in vivo. Neural decoding analyses are a widely used tool used for investigating what information is represented in this complex, high-dimensional neural population activity. Most population decoding methods assume that correlated activity between neurons has been estimated accurately. In practice, this requires large amounts of data, both across observations and across neurons. Unfortunately, most experiments are fundamentally constrained by practical variables that limit the number of times the neural population can be observed under a single stimulus and/or behavior condition. Therefore, new analytical tools are required to study neural population coding while taking into account these limitations. Here, we present a simple and interpretable method for dimensionality reduction that allows neural decoding metrics to be calculated reliably, even when experimental trial numbers are limited. We illustrate the method using simulations and compare its performance to standard approaches for dimensionality reduction and decoding by applying it to single-unit electrophysiological data collected from auditory cortex.

Visual modulation of firing and spectrotemporal receptive fields in mouse auditory cortex

Recent studies have established significant anatomical and functional connections between visual areas and primary auditory cortex (A1), which may be important for cognitive processes such as communication and spatial perception. These studies have raised two important questions: First, which cell populations in A1 respond to visual input and/or are influenced by visual context? Second, which aspects of sound encoding are affected by visual context? To address these questions, we recorded single-unit activity across cortical layers in awake mice during exposure to auditory and visual stimuli. Neurons responsive to visual stimuli were most prevalent in the deep cortical layers and included both excitatory and inhibitory cells. The overwhelming majority of these neurons also responded to sound, indicating unimodal visual neurons are rare in A1. Other neurons for which sound-evoked responses were modulated by visual context were similarly excitatory or inhibitory but more evenly distributed across cortical layers. These modulatory influences almost exclusively affected sustained sound-evoked firing rate (FR) responses or spectrotemporal receptive fields (STRFs); transient FR changes at stimulus onset were rarely modified by visual context. Neuron populations with visually modulated STRFs and sustained FR responses were mostly non-overlapping, suggesting spectrotemporal feature selectivity and overall excitability may be differentially sensitive to visual context. The effects of visual modulation were heterogeneous, increasing and decreasing STRF gain in roughly equal proportions of neurons. Our results indicate visual influences are surprisingly common and diversely expressed throughout layers and cell types in A1, affecting nearly one in five neurons overall.

Rapid Enhancement of Subcortical Neural Responses to Sine-Wave Speech

The efferent auditory nervous system may be a potent force in shaping how the brain responds to behaviorally significant sounds. Previous human experiments using the frequency following response (FFR) have shown efferent-induced modulation of subcortical auditory function online and over short- and long-term time scales; however, a contemporary understanding of FFR generation presents new questions about whether previous effects were constrained solely to the auditory subcortex. The present experiment used sine-wave speech (SWS), an acoustically-sparse stimulus in which dynamic pure tones represent speech formant contours, to evoke FFR_SWS. Due to the higher stimulus frequencies used in SWS, this approach biased neural responses toward brainstem generators and allowed for three stimuli (/bɔ/, /bu/, and /bo/) to be used to evoke FFR_SWSbefore and after listeners in a training group were made aware that they were hearing a degraded speech stimulus. All SWS stimuli were rapidly perceived as speech when presented with a SWS carrier phrase, and average token identification reached ceiling performance during a perceptual training phase. Compared to a control group which remained naïve throughout the experiment, training group FFR_SWS amplitudes were enhanced post-training for each stimulus. Further, linear support vector machine classification of training group FFR_SWS significantly improved post-training compared to the control group, indicating that training-induced neural enhancements were sufficient to bolster machine learning classification accuracy. These results suggest that the efferent auditory system may rapidly modulate auditory brainstem representation of sounds depending on their context and perception as non-speech or speech.

Dimensionally Specific Attention Capture in Birds Performing Auditory Streaming Task

Previous studies in budgerigars (Melopsittacus undulatus) have indicated that they experience attention capture in a qualitatively similar way to humans. Here, we apply a similar objective auditory streaming paradigm, using modified budgerigar vocalizations instead of ABAB-… patterned pure tones, in the sound sequences. The birds were trained to respond to deviants in the target stream while ignoring the distractors in the background stream. The background distractor could vary among five different categories and two different sequential positions, while the target deviants could randomly appear at five different sequential positions and vary among two different categories. We found that unpredictable background distractors could deteriorate birds' sensitivity to the target deviants. Compared to conditions where the background distractor appeared right before the target deviant, the attention capture effect decayed in conditions when the background distractor appeared earlier. In contrast to results from the same paradigm using pure tones, the results here are evidence for a faster recovery from attention capture using modified vocalization segments. We found that the temporally modulated background distractor captured birds' attention more and deteriorated birds' performance more than other categories of background distractor, as the temporally modulated target deviant enabled the birds to focus their attention toward the temporal modulation dimension. However, different from humans, birds have lower tolerances for suppressing the distractors from the same feature dimensions as the targets, which is evidenced by higher false alarm rates for the temporally modulated distractor than other distractors from different feature dimensions.

Task-induced modulations of neuronal activity along the auditory pathway

Sensory processing varies depending on behavioral context. Here, we ask how task engagement modulates neurons in the auditory system. We train mice in a simple tone-detection task and compare their neuronal activity during passive hearing and active listening. Electrophysiological extracellular recordings in the inferior colliculus, medial geniculate body, primary auditory cortex, and anterior auditory field reveal widespread modulations across all regions and cortical layers and in both putative regular- and fast-spiking cortical neurons. Clustering analysis unveils ten distinct modulation patterns that can either enhance or suppress neuronal activity. Task engagement changes the tone-onset response in most neurons. Such modulations first emerge in subcortical areas, ruling out cortical feedback as the only mechanism underlying subcortical modulations. Half the neurons additionally display late modulations associated with licking, arousal, or reward. Our results reveal the presence of functionally distinct subclasses of neurons, differentially sensitive to specific task-related variables but anatomically distributed along the auditory pathway.

Listening in the Moment: How Bilingualism Interacts With Task Demands to Shape Active Listening

While there is evidence for bilingual enhancements of inhibitory control and auditory processing, two processes that are fundamental to daily communication, it is not known how bilinguals utilize these cognitive and sensory enhancements during real-world listening. To test our hypothesis that bilinguals engage their enhanced cognitive and sensory processing in real-world listening situations, bilinguals and monolinguals performed a selective attention task involving competing talkers, a common demand of everyday listening, and then later passively listened to the same competing sentences. During the active and passive listening periods, evoked responses to the competing talkers were collected to understand how online auditory processing facilitates active listening and if this processing differs between bilinguals and monolinguals. Additionally, participants were tested on a separate measure of inhibitory control to see if inhibitory control abilities related with performance on the selective attention task. We found that although monolinguals and bilinguals performed similarly on the selective attention task, the groups differed in the neural and cognitive processes engaged to perform this task, compared to when they were passively listening to the talkers. Specifically, during active listening monolinguals had enhanced cortical phase consistency while bilinguals demonstrated enhanced subcortical phase consistency in the response to the pitch contours of the sentences, particularly during passive listening. Moreover, bilinguals’ performance on the inhibitory control test related with performance on the selective attention test, a relationship that was not seen for monolinguals. These results are consistent with the hypothesis that bilinguals utilize inhibitory control and enhanced subcortical auditory processing in everyday listening situations to engage with sound in ways that are different than monolinguals.

Three psychophysical metrics of auditory temporal integration in macaques

The relationship between sound duration and detection threshold has long been thought to reflect temporal integration. Reports of species differences in this relationship are equivocal: some meta-analyses report no species differences, whereas others report substantial differences, particularly between humans and their close phylogenetic relatives, macaques. This renders translational work in macaques problematic. To reevaluate this difference, tone detection performance was measured in macaques using a go/no-go reaction time (RT) task at various tone durations and in the presence of broadband noise (BBN). Detection thresholds, RTs, and the dynamic range (DR) of the psychometric function decreased as the tone duration increased. The threshold by duration trends suggest macaques integrate at a similar rate to humans. The RT trends also resemble human data and are the first reported in animals. Whereas the BBN did not affect how the threshold or RT changed with the duration, it substantially reduced the DR at short durations. A probabilistic Poisson model replicated the effects of duration on threshold and DR and required integration from multiple simulated auditory nerve fibers to explain the performance at shorter durations. These data suggest that, contrary to previous studies, macaques are uniquely well-suited to model human temporal integration and form the baseline for future neurophysiological studies.

Interaural time difference tuning in the rat inferior colliculus is predictive of behavioral sensitivity

While a large body of literature has examined the encoding of binaural spatial cues in the auditory midbrain, studies that ask how quantitative measures of spatial tuning in midbrain neurons compare with an animal's psychoacoustic performance remain rare. Researchers have tried to explain deficits in spatial hearing in certain patient groups, such as binaural cochlear implant users, in terms of declines in apparent reductions in spatial tuning of midbrain neurons of animal models. However, the quality of spatial tuning can be quantified in many different ways, and in the absence of evidence that a given neural tuning measure correlates with psychoacoustic performance, the interpretation of such finding remains very tentative. Here, we characterize ITD tuning in the rat inferior colliculus (IC) to acoustic pulse train stimuli with varying envelopes and at varying rates, and explore whether quality of tuning correlates behavioral performance. We quantified both mutual information (MI) and neural d' as measures of ITD sensitivity. Neural d' values paralleled behavioral ones, declining with increasing click rates or when envelopes changed from rectangular to Hanning windows, and they correlated much better with behavioral performance than MI. Meanwhile, MI values were larger in an older, more experienced cohort of animals than in naive animals, but neural d' did not differ between cohorts. However, the results obtained with neural d' and MI were highly correlated when ITD values were coded simply as left or right ear leading, rather than specific ITD values. Thus, neural measures of lateralization ability (e.g. d' or left/right MI) appear to be highly predictive of psychoacoustic performance in a two-alternative forced choice task.

Implementation of Manual and Automated Water Regulation for Rats (Rattus norvegicus) and Ferrets (Mustela putorius)

Water regulation is a procedure that allows animals to consume water volumes equivalent to ad libitum access, but access is limited to specific time intervals (that is, water is not available outside of the designated access periods). Despite the relatively common use of water regulation in research, the implementation method is rarely detailed, stating only that water was available in the animal's home cage at specific times. For planned toxicologic assessments, we placed rats (n = 510) and ferrets (n = 16) on water regulation using both automated and manual methods. In testing our systems, we defined "successful implementation" as maintenance of appropriate weight gain and health status. An automated system that controlled water access to an entire rat rack was successful for most rats, but several rats failed to consume enough water even after 2 wk of experience. Manual methods of water regulation were successful in rats by either moving the cage to prevent access to the drinking valve or by placing/removing water bottles. An automated system that controlled water access from water bottles was implemented for ferrets and was maintained for up to 30 wk. Retrospective comparison of body weights to standard growth curves for both species showed that all animals grew normally despite water regulation. Differences in the systems and some species considerations provide insights into the key elements necessary for successful water regulation in rats and ferrets.

When and How Does the Auditory Cortex Influence Subcortical Auditory Structures? New Insights About the Roles of Descending Cortical Projections

For decades, the corticofugal descending projections have been anatomically well described but their functional role remains a puzzling question. In this review, we will first describe the contributions of neuronal networks in representing communication sounds in various types of degraded acoustic conditions from the cochlear nucleus to the primary and secondary auditory cortex. In such situations, the discrimination abilities of collicular and thalamic neurons are clearly better than those of cortical neurons although the latter remain very little affected by degraded acoustic conditions. Second, we will report the functional effects resulting from activating or inactivating corticofugal projections on functional properties of subcortical neurons. In general, modest effects have been observed in anesthetized and in awake, passively listening, animals. In contrast, in behavioral tasks including challenging conditions, behavioral performance was severely reduced by removing or transiently silencing the corticofugal descending projections. This suggests that the discriminative abilities of subcortical neurons may be sufficient in many acoustic situations. It is only in particularly challenging situations, either due to the task difficulties and/or to the degraded acoustic conditions that the corticofugal descending connections bring additional abilities. Here, we propose that it is both the top-down influences from the prefrontal cortex, and those from the neuromodulatory systems, which allow the cortical descending projections to impact behavioral performance in reshaping the functional circuitry of subcortical structures. We aim at proposing potential scenarios to explain how, and under which circumstances, these projections impact on subcortical processing and on behavioral responses.

Attention reinforces human corticofugal system to aid speech perception in noise

Perceiving speech-in-noise (SIN) demands precise neural coding between brainstem and cortical levels of the hearing system. Attentional processes can then select and prioritize task-relevant cues over competing background noise for successful speech perception. In animal models, brainstem-cortical interplay is achieved via descending corticofugal projections from cortex that shape midbrain responses to behaviorally-relevant sounds. Attentional engagement of corticofugal feedback may assist SIN understanding but has never been confirmed and remains highly controversial in humans. To resolve these issues, we recorded source-level, anatomically constrained brainstem frequency-following responses (FFRs) and cortical event-related potentials (ERPs) to speech via high-density EEG while listeners performed rapid SIN identification tasks. We varied attention with active vs. passive listening scenarios whereas task difficulty was manipulated with additive noise interference. Active listening (but not arousal-control tasks) exaggerated both ERPs and FFRs, confirming attentional gain extends to lower subcortical levels of speech processing. We used functional connectivity to measure the directed strength of coupling between levels and characterize "bottom-up" vs. "top-down" (corticofugal) signaling within the auditory brainstem-cortical pathway. While attention strengthened connectivity bidirectionally, corticofugal transmission disengaged under passive (but not active) SIN listening. Our findings (i) show attention enhances the brain's transcription of speech even prior to cortex and (ii) establish a direct role of the human corticofugal feedback system as an aid to cocktail party speech perception.

Robustness to Noise in the Auditory System: A Distributed and Predictable Property

Background noise strongly penalizes auditory perception of speech in humans or vocalizations in animals. Despite this, auditory neurons are still able to detect communications sounds against considerable levels of background noise. We collected neuronal recordings in cochlear nucleus (CN), inferior colliculus (IC), auditory thalamus, and primary and secondary auditory cortex in response to vocalizations presented either against a stationary or a chorus noise in anesthetized guinea pigs at three signal-to-noise ratios (SNRs; −10, 0, and 10 dB). We provide evidence that, at each level of the auditory system, five behaviors in noise exist within a continuum, from neurons with high-fidelity representations of the signal, mostly found in IC and thalamus, to neurons with high-fidelity representations of the noise, mostly found in CN for the stationary noise and in similar proportions in each structure for the chorus noise. The two cortical areas displayed fewer robust responses than the IC and thalamus. Furthermore, between 21% and 72% of the neurons (depending on the structure) switch categories from one background noise to another, even if the initial assignment of these neurons to a category was confirmed by a severe bootstrap procedure. Importantly, supervised learning pointed out that assigning a recording to one of the five categories can be predicted up to a maximum of 70% based on both the response to signal alone and noise alone.

Dissociation of task engagement and arousal effects in auditory cortex and midbrain

Both generalized arousal and engagement in a specific task influence sensory neural processing. To isolate effects of these state variables in the auditory system, we recorded single-unit activity from primary auditory cortex (A1) and inferior colliculus (IC) of ferrets during a tone detection task, while monitoring arousal via changes in pupil size. We used a generalized linear model to assess the influence of task engagement and pupil size on sound-evoked activity. In both areas, these two variables affected independent neural populations. Pupil size effects were more prominent in IC, while pupil and task engagement effects were equally likely in A1. Task engagement was correlated with larger pupil; thus, some apparent effects of task engagement should in fact be attributed to fluctuations in pupil size. These results indicate a hierarchy of auditory processing, where generalized arousal enhances activity in midbrain, and effects specific to task engagement become more prominent in cortex.

Auditory attentional filter in the absence of masking noise

Signals containing attended frequencies are facilitated while those with unexpected frequencies are suppressed by an auditory filtering process. The neurocognitive mechanism underlying the auditory attentional filter is, however, poorly understood. The olivocochlear bundle (OCB), a brainstem neural circuit that is part of the efferent system, has been suggested to be partly responsible for the filtering via its noise-dependent antimasking effect. The current study examined the role of the OCB in attentional filtering, particularly the validity of the antimasking hypothesis, by comparing attentional filters measured in quiet and in the presence of background noise in a group of normal-hearing listeners. Filters obtained in both conditions were comparable, suggesting that the presence of background noise is not crucial for attentional filter generation. In addition, comparison of frequency-specific changes of the cue-evoked enhancement component of filters in quiet and noise also did not reveal any major contribution of background noise to the cue effect. These findings argue against the involvement of an antimasking effect in the attentional process. Instead of the antimasking effect mediated via medial olivocochlear fibers, results from current and earlier studies can be explained by frequency-specific modulation of afferent spontaneous activity by lateral olivocochlear fibers. It is proposed that the activity of these lateral fibers could be driven by top-down cortical control via a noise-independent mechanism. SIGNIFICANCE: The neural basis for auditory attentional filter remains a fundamental but poorly understood area in auditory neuroscience. The efferent olivocochlear pathway that projects from the brainstem back to the cochlea has been suggested to mediate the attentional effect via its noise-dependent antimasking effect. The current study demonstrates that the filter generation is mostly independent of the background noise, and therefore is unlikely to be mediated by the olivocochlear brainstem reflex. It is proposed that the entire cortico-olivocochlear system might instead be used to alter the hearing sensitivity during focus attention via frequency-specific modulation of afferent spontaneous activity.

Task Engagement Improves Neural Discriminability in the Auditory Midbrain of the Marmoset Monkey

While task-dependent changes have been demonstrated in auditory cortex for a number of behavioral paradigms and mammalian species, less is known about how behavioral state can influence neural coding in the midbrain areas that provide auditory information to cortex. We measured single-unit activity in the inferior colliculus (IC) of common marmosets of both sexes while they performed a tone-in-noise detection task and during passive presentation of identical task stimuli. In contrast to our previous study in the ferret IC, task engagement had little effect on sound-evoked activity in central (lemniscal) IC of the marmoset. However, activity was significantly modulated in noncentral fields, where responses were selectively enhanced for the target tone relative to the distractor noise. This led to an increase in neural discriminability between target and distractors. The results confirm that task engagement can modulate sound coding in the auditory midbrain, and support a hypothesis that subcortical pathways can mediate highly trained auditory behaviors.SIGNIFICANCE STATEMENT While the cerebral cortex is widely viewed as playing an essential role in the learning and performance of complex auditory behaviors, relatively little attention has been paid to the role of brainstem and midbrain areas that process sound information before it reaches cortex. This study demonstrates that the auditory midbrain is also modulated during behavior. These modulations amplify task-relevant sensory information, a process that is traditionally attributed to cortex.

Cholinergic Projections From the Pedunculopontine Tegmental Nucleus Contact Excitatory and Inhibitory Neurons in the Inferior Colliculus

The inferior colliculus processes nearly all ascending auditory information. Most collicular cells respond to sound, and for a majority of these cells, the responses can be modulated by acetylcholine (ACh). The cholinergic effects are varied and, for the most part, the underlying mechanisms are unknown. The major source of cholinergic input to the inferior colliculus is the pedunculopontine tegmental nucleus (PPT), part of the pontomesencephalic tegmentum known for projections to the thalamus and roles in arousal and the sleep-wake cycle. Characterization of PPT inputs to the inferior colliculus has been complicated by the mixed neurotransmitter population within the PPT. Using selective viral-tract tracing techniques in a ChAT-Cre Long Evans rat, the present study characterizes the distribution and targets of cholinergic projections from PPT to the inferior colliculus. Following the deposit of viral vector in one PPT, cholinergic axons studded with boutons were present bilaterally in the inferior colliculus, with the greater density of axons and boutons ipsilateral to the injection site. On both sides, cholinergic axons were present throughout the inferior colliculus, distributing boutons to the central nucleus, lateral cortex, and dorsal cortex. In each inferior colliculus (IC) subdivision, the cholinergic PPT axons appear to contact both GABAergic and glutamatergic neurons. These findings suggest cholinergic projections from the PPT have a widespread influence over the IC, likely affecting many aspects of midbrain auditory processing. Moreover, the effects are likely to be mediated by direct cholinergic actions on both excitatory and inhibitory circuits in the inferior colliculus.

Prior Knowledge Guides Speech Segregation in Human Auditory Cortex

Segregating concurrent sound streams is a computationally challenging task that requires integrating bottom-up acoustic cues (e.g. pitch) and top-down prior knowledge about sound streams. In a multi-talker environment, the brain can segregate different speakers in about 100 ms in auditory cortex. Here, we used magnetoencephalographic (MEG) recordings to investigate the temporal and spatial signature of how the brain utilizes prior knowledge to segregate 2 speech streams from the same speaker, which can hardly be separated based on bottom-up acoustic cues. In a primed condition, the participants know the target speech stream in advance while in an unprimed condition no such prior knowledge is available. Neural encoding of each speech stream is characterized by the MEG responses tracking the speech envelope. We demonstrate that an effect in bilateral superior temporal gyrus and superior temporal sulcus is much stronger in the primed condition than in the unprimed condition. Priming effects are observed at about 100 ms latency and last more than 600 ms. Interestingly, prior knowledge about the target stream facilitates speech segregation by mainly suppressing the neural tracking of the non-target speech stream. In sum, prior knowledge leads to reliable speech segregation in auditory cortex, even in the absence of reliable bottom-up speech segregation cue.

Streaming of Repeated Noise in Primary and Secondary Fields of Auditory Cortex

Statistical regularities in natural sounds facilitate the perceptual segregation of auditory sources, or streams. Repetition is one cue that drives stream segregation in humans, but the neural basis of this perceptual phenomenon remains unknown. We demonstrated a similar perceptual ability in animals by training ferrets of both sexes to detect a stream of repeating noise samples (foreground) embedded in a stream of random samples (background). During passive listening, we recorded neural activity in primary auditory cortex (A1) and secondary auditory cortex (posterior ectosylvian gyrus, PEG). We used two context-dependent encoding models to test for evidence of streaming of the repeating stimulus. The first was based on average evoked activity per noise sample and the second on the spectro-temporal receptive field. Both approaches tested whether differences in neural responses to repeating versus random stimuli were better modeled by scaling the response to both streams equally (global gain) or by separately scaling the response to the foreground versus background stream (stream-specific gain). Consistent with previous observations of adaptation, we found an overall reduction in global gain when the stimulus began to repeat. However, when we measured stream-specific changes in gain, responses to the foreground were enhanced relative to the background. This enhancement was stronger in PEG than A1. In A1, enhancement was strongest in units with low sparseness (i.e., broad sensory tuning) and with tuning selective for the repeated sample. Enhancement of responses to the foreground relative to the background provides evidence for stream segregation that emerges in A1 and is refined in PEG.SIGNIFICANCE STATEMENT To interact with the world successfully, the brain must parse behaviorally important information from a complex sensory environment. Complex mixtures of sounds often arrive at the ears simultaneously or in close succession, yet they are effortlessly segregated into distinct perceptual sources. This process breaks down in hearing-impaired individuals and speech recognition devices. By identifying the underlying neural mechanisms that facilitate perceptual segregation, we can develop strategies for ameliorating hearing loss and improving speech recognition technology in the presence of background noise. Here, we present evidence to support a hierarchical process, present in primary auditory cortex and refined in secondary auditory cortex, in which sound repetition facilitates segregation.

Neural circuits underlying auditory contrast gain control and their perceptual implications

Neural adaptation enables sensory information to be represented optimally in the brain despite large fluctuations over time in the statistics of the environment. Auditory contrast gain control represents an important example, which is thought to arise primarily from cortical processing. Here we show that neurons in the auditory thalamus and midbrain of mice show robust contrast gain control, and that this is implemented independently of cortical activity. Although neurons at each level exhibit contrast gain control to similar degrees, adaptation time constants become longer at later stages of the processing hierarchy, resulting in progressively more stable representations. We also show that auditory discrimination thresholds in human listeners compensate for changes in contrast, and that the strength of this perceptual adaptation can be predicted from physiological measurements. Contrast adaptation is therefore a robust property of both the subcortical and cortical auditory system and accounts for the short-term adaptability of perceptual judgments.

Hierarchical Encoding of Attended Auditory Objects in Multi-talker Speech Perception

Humans can easily focus on one speaker in a multi-talker acoustic environment, but how different areas of the human auditory cortex (AC) represent the acoustic components of mixed speech is unknown. We obtained invasive recordings from the primary and nonprimary AC in neurosurgical patients as they listened to multi-talker speech. We found that neural sites in the primary AC responded to individual speakers in the mixture and were relatively unchanged by attention. In contrast, neural sites in the nonprimary AC were less discerning of individual speakers but selectively represented the attended speaker. Moreover, the encoding of the attended speaker in the nonprimary AC was invariant to the degree of acoustic overlap with the unattended speaker. Finally, this emergent representation of attended speech in the nonprimary AC was linearly predictable from the primary AC responses. Our results reveal the neural computations underlying the hierarchical formation of auditory objects in human AC during multi-talker speech perception.

Auditory cortical generators of the Frequency Following Response are modulated by intermodal attention

The efferent auditory system suggests that brainstem auditory regions could also be sensitive to top-down processes. In electrophysiology, the Frequency Following Response (FFR) to speech stimuli has been used extensively to study brainstem areas. Despite seemingly straight-forward in addressing the issue of attentional modulations of brainstem regions by means of the FFR, the existing results are inconsistent. Moreover, the notion that the FFR exclusively represents subcortical generators has been challenged. We aimed to gain a more differentiated perspective on how the generators of the FFR are modulated by either attending to the visual or auditory input while neural activity was recorded using magnetoencephalography (MEG). In a first step our results confirm the strong contribution of also cortical regions to the FFR. Interestingly, of all regions exhibiting a measurable FFR response, only the right primary auditory cortex was significantly affected by intermodal attention. By showing a clear cortical contribution to the attentional FFR effect, our work significantly extends previous reports that focus on surface level recordings only. It underlines the importance of making a greater effort to disentangle the different contributing sources of the FFR and serves as a clear precaution of simplistically interpreting the FFR as brainstem response.

Afferent-efferent connectivity between auditory brainstem and cortex accounts for poorer speech-in-noise comprehension in older adults

Speech-in-noise (SIN) comprehension deficits in older adults have been linked to changes in both subcortical and cortical auditory evoked responses. However, older adults' difficulty understanding SIN may also be related to an imbalance in signal transmission (i.e., functional connectivity) between brainstem and auditory cortices. By modeling high-density scalp recordings of speech-evoked responses with sources in brainstem (BS) and bilateral primary auditory cortices (PAC), we show that beyond attenuating neural activity, hearing loss in older adults compromises the transmission of speech information between subcortical and early cortical hubs of the speech network. We found that the strength of afferent BS→PAC neural signaling (but not the reverse efferent flow; PAC→BS) varied with mild declines in hearing acuity and this "bottom-up" functional connectivity robustly predicted older adults' performance in a SIN identification task. Connectivity was also a better predictor of SIN processing than unitary subcortical or cortical responses alone. Our neuroimaging findings suggest that in older adults (i) mild hearing loss differentially reduces neural output at several stages of auditory processing (PAC > BS), (ii) subcortical-cortical connectivity is more sensitive to peripheral hearing loss than top-down (cortical-subcortical) control, and (iii) reduced functional connectivity in afferent auditory pathways plays a significant role in SIN comprehension problems.

State-dependent encoding of sound and behavioral meaning in a tertiary region of the ferret auditory cortex

In higher sensory cortices, there is a gradual transformation from sensation to perception and action. In the auditory system, this transformation is revealed by responses in the rostral ventral posterior auditory field (VPr), a tertiary area in the ferret auditory cortex, which shows long-term learning in trained compared to naïve animals, arising from selectively enhanced responses to behaviorally relevant target stimuli. This enhanced representation is further amplified during active performance of spectral or temporal auditory discrimination tasks. VPr also shows sustained short-term memory activity after target stimulus offset, correlated with task response timing and action. These task-related changes in auditory filter properties enable VPr neurons to quickly and nimbly switch between different responses to the same acoustic stimuli, reflecting either spectrotemporal properties, timing, or behavioral meaning of the sound. Furthermore, they demonstrate an interaction between the dynamics of short-term attention and long-term learning, as incoming sound is selectively attended, recognized, and translated into action.

Brainstem correlates of concurrent speech identification in adverse listening conditions

When two voices compete, listeners can segregate and identify concurrent speech sounds using pitch (fundamental frequency, F0) and timbre (harmonic) cues. Speech perception is also hindered by the signal-to-noise ratio (SNR). How clear and degraded concurrent speech sounds are represented at early, pre-attentive stages of the auditory system is not well understood. To this end, we measured scalp-recorded frequency-following responses (FFR) from the EEG while human listeners heard two concurrently presented, steady-state (time-invariant) vowels whose F0 differed by zero or four semitones (ST) presented diotically in either clean (no noise) or noise-degraded (+5dB SNR) conditions. Listeners also performed a speeded double vowel identification task in which they were required to identify both vowels correctly. Behavioral results showed that speech identification accuracy increased with F0 differences between vowels, and this perceptual F0 benefit was larger for clean compared to noise degraded (+5dB SNR) stimuli. Neurophysiological data demonstrated more robust FFR F0 amplitudes for single compared to double vowels and considerably weaker responses in noise. F0 amplitudes showed speech-on-speech masking effects, along with a non-linear constructive interference at 0ST, and suppression effects at 4ST. Correlations showed that FFR F0 amplitudes failed to predict listeners' identification accuracy. In contrast, FFR F1 amplitudes were associated with faster reaction times, although this correlation was limited to noise conditions. The limited number of brain-behavior associations suggests subcortical activity mainly reflects exogenous processing rather than perceptual correlates of concurrent speech perception. Collectively, our results demonstrate that FFRs reflect pre-attentive coding of concurrent auditory stimuli that only weakly predict the success of identifying concurrent speech.

Puncta of Neuronal Nitric Oxide Synthase (nNOS) Mediate NMDA Receptor Signaling in the Auditory Midbrain

Nitric oxide (NO) is a neurotransmitter synthesized in the brain by neuronal nitric oxide synthase (nNOS). Using immunohistochemistry and confocal imaging in the inferior colliculus (IC, auditory midbrain) of the guinea pig (Cavia porcellus, male and female), we show that nNOS occurs in two distinct cellular distributions. We confirm that, in the cortices of the IC, a subset of neurons show cytoplasmic labeling for nNOS, whereas in the central nucleus (ICc), such neurons are not present. However, we demonstrate that all neurons in the ICc do in fact express nNOS in the form of discrete puncta found at the cell membrane. Our multi-labeling studies reveal that nNOS puncta form multiprotein complexes with NMDA receptors, soluble guanylyl cyclase (sGC), and PSD95. These complexes are found apposed to glutamatergic terminals, which is indicative of synaptic function. Interestingly, these glutamatergic terminals express both vesicular glutamate transporters 1 and 2 denoting a specific source of brainstem inputs. With in vivo electrophysiological recordings of multiunit activity in the ICc, we found that local application of NMDA enhances sound-driven activity in a concentration-dependent and reversible fashion. This response is abolished by blockade of nNOS or sGC, indicating that the NMDA effect is mediated solely via the NO and cGMP signaling pathway. This discovery of a ubiquitous, but highly localized, expression of nNOS throughout the ICc and demonstration of the dramatic influence of the NMDA activated NO pathway on sound-driven neuronal activity imply a key role for NO signaling in auditory processing.SIGNIFICANCE STATEMENT We show that neuronal nitric oxide synthase (nNOS), the enzyme that synthesizes nitric oxide (NO), occurs as puncta in apparently all neurons in the central nucleus of the inferior colliculus (ICc) in the auditory midbrain. Punctate nNOS appears at glutamatergic synapses in a complex with glutamate NMDA receptors (NMDA-Rs), soluble guanylyl cyclase (sGC, the NO receptor), and PSD95 (a protein that anchors receptors and enzymes at the postsynaptic density). We show that NMDA-R modulation of sound-driven activity in the ICc is solely mediated by activation of nNOS and sGC. The presence of nNOS throughout this sensory nucleus argues for a major role of NO in hearing. Furthermore, this punctate form of nNOS expression may exist and have gone unnoticed in other brain regions.

Inherent auditory skills rather than formal music training shape the neural encoding of speech

Musical training is associated with a myriad of neuroplastic changes in the brain, including more robust and efficient neural processing of clean and degraded speech signals at brainstem and cortical levels. These assumptions stem largely from cross-sectional studies between musicians and nonmusicians which cannot address whether training itself is sufficient to induce physiological changes or whether preexisting superiority in auditory function before training predisposes individuals to pursue musical interests and appear to have similar neuroplastic benefits as musicians. Here, we recorded neuroelectric brain activity to clear and noise-degraded speech sounds in individuals without formal music training but who differed in their receptive musical perceptual abilities as assessed objectively via the Profile of Music Perception Skills. We found that listeners with naturally more adept listening skills ("musical sleepers") had enhanced frequency-following responses to speech that were also more resilient to the detrimental effects of noise, consistent with the increased fidelity of speech encoding and speech-in-noise benefits observed previously in highly trained musicians. Further comparisons between these musical sleepers and actual trained musicians suggested that experience provides an additional boost to the neural encoding and perception of speech. Collectively, our findings suggest that the auditory neuroplasticity of music engagement likely involves a layering of both preexisting (nature) and experience-driven (nurture) factors in complex sound processing. In the absence of formal training, individuals with intrinsically proficient auditory systems can exhibit musician-like auditory function that can be further shaped in an experience-dependent manner.

Focal Suppression of Distractor Sounds by Selective Attention in Auditory Cortex

Auditory selective attention is required for parsing crowded acoustic environments, but cortical systems mediating the influence of behavioral state on auditory perception are not well characterized. Previous neurophysiological studies suggest that attention produces a general enhancement of neural responses to important target sounds versus irrelevant distractors. However, behavioral studies suggest that in the presence of masking noise, attention provides a focal suppression of distractors that compete with targets. Here, we compared effects of attention on cortical responses to masking versus non-masking distractors, controlling for effects of listening effort and general task engagement. We recorded single-unit activity from primary auditory cortex (A1) of ferrets during behavior and found that selective attention decreased responses to distractors masking targets in the same spectral band, compared with spectrally distinct distractors. This suppression enhanced neural target detection thresholds, suggesting that limited attention resources serve to focally suppress responses to distractors that interfere with target detection. Changing effort by manipulating target salience consistently modulated spontaneous but not evoked activity. Task engagement and changing effort tended to affect the same neurons, while attention affected an independent population, suggesting that distinct feedback circuits mediate effects of attention and effort in A1.

Auditory midbrain coding of statistical learning that results from discontinuous sensory stimulation

Detecting regular patterns in the environment, a process known as statistical learning, is essential for survival. Neuronal adaptation is a key mechanism in the detection of patterns that are continuously repeated across short (seconds to minutes) temporal windows. Here, we found in mice that a subcortical structure in the auditory midbrain was sensitive to patterns that were repeated discontinuously, in a temporally sparse manner, across windows of minutes to hours. Using a combination of behavioral, electrophysiological, and molecular approaches, we found changes in neuronal response gain that varied in mechanism with the degree of sound predictability and resulted in changes in frequency coding. Analysis of population activity (structural tuning) revealed an increase in frequency classification accuracy in the context of increased overlap in responses across frequencies. The increase in accuracy and overlap was paralleled at the behavioral level in an increase in generalization in the absence of diminished discrimination. Gain modulation was accompanied by changes in gene and protein expression, indicative of long-term plasticity. Physiological changes were largely independent of corticofugal feedback, and no changes were seen in upstream cochlear nucleus responses, suggesting a key role of the auditory midbrain in sensory gating. Subsequent behavior demonstrated learning of predictable and random patterns and their importance in auditory conditioning. Using longer timescales than previously explored, the combined data show that the auditory midbrain codes statistical learning of temporally sparse patterns, a process that is critical for the detection of relevant stimuli in the constant soundscape that the animal navigates through.

Some things are learned simply because they are there and not because they are relevant at that moment in time. This is particularly true of surrounding sounds, which we process automatically and continuously, detecting their repetitive patterns or singularities. Learning about rewards and punishment is typically attributed to cortical structures in the brain and known to occur over long time windows. Learning of surrounding regularities, on the other hand, is attributed to subcortical structures and has been shown to occur in seconds. The brain can, however, also detect the regularity in sounds that are discontinuously repeated across intervals of minutes and hours. For example, we learn to identify people by the sound of their steps through an unconscious process involving repeated but isolated exposures to the coappearance of sound and person. Here, we show that a subcortical structure, the auditory midbrain, can code such temporally spread regularities. Neurons in the auditory midbrain changed their response pattern in mice that heard a fixed tone whenever they went into one room in the environment they lived in. Learning of temporally spread sound patterns can, therefore, occur in subcortical structures.

Developmental deprivation-induced perceptual and cortical processing deficits in awake-behaving animals

Sensory deprivation during development induces lifelong changes to central nervous system function that are associated with perceptual impairments. However, the relationship between neural and behavioral deficits is uncertain due to a lack of simultaneous measurements during task performance. Therefore, we telemetrically recorded from auditory cortex neurons in gerbils reared with developmental conductive hearing loss as they performed an auditory task in which rapid fluctuations in amplitude are detected. These data were compared to a measure of auditory brainstem temporal processing from each animal. We found that developmental HL diminished behavioral performance, but did not alter brainstem temporal processing. However, the simultaneous assessment of neural and behavioral processing revealed that perceptual deficits were associated with a degraded cortical population code that could be explained by greater trial-to-trial response variability. Our findings suggest that the perceptual limitations that attend early hearing loss are best explained by an encoding deficit in auditory cortex.

Subcortical pathways: Towards a better understanding of auditory disorders

Hearing loss is a significant problem that affects at least 15% of the population. This percentage, however, is likely significantly higher because of a variety of auditory disorders that are not identifiable through traditional tests of peripheral hearing ability. In these disorders, individuals have difficulty understanding speech, particularly in noisy environments, even though the sounds are loud enough to hear. The underlying mechanisms leading to such deficits are not well understood. To enable the development of suitable treatments to alleviate or prevent such disorders, the affected processing pathways must be identified. Historically, mechanisms underlying speech processing have been thought to be a property of the auditory cortex and thus the study of auditory disorders has largely focused on cortical impairments and/or cognitive processes. As we review here, however, there is strong evidence to suggest that, in fact, deficits in subcortical pathways play a significant role in auditory disorders. In this review, we highlight the role of the auditory brainstem and midbrain in processing complex sounds and discuss how deficits in these regions may contribute to auditory dysfunction. We discuss current research with animal models of human hearing and then consider human studies that implicate impairments in subcortical processing that may contribute to auditory disorders.

Tracing the Trajectory of Sensory Plasticity across Different Stages of Speech Learning in Adulthood

Although challenging, adults can learn non-native phonetic contrasts with extensive training [1, 2], indicative of perceptual learning beyond an early sensitivity period [3, 4]. Training can alter low-level sensory encoding of newly acquired speech sound patterns [5]; however, the time-course, behavioral relevance, and long-term retention of such sensory plasticity is unclear. Some theories argue that sensory plasticity underlying signal enhancement is immediate and critical to perceptual learning [6, 7]. Others, like the reverse hierarchy theory (RHT), posit a slower time-course for sensory plasticity [8]. RHT proposes that higher-level categorical representations guide immediate, novice learning, while lower-level sensory changes do not emerge until expert stages of learning [9]. We trained 20 English-speaking adults to categorize a non-native phonetic contrast (Mandarin lexical tones) using a criterion-dependent sound-to-category training paradigm. Sensory and perceptual indices were assayed across operationally defined learning phases (novice, experienced, over-trained, and 8-week retention) by measuring the frequency-following response, a neurophonic potential that reflects fidelity of sensory encoding, and the perceptual identification of a tone continuum. Our results demonstrate that while robust changes in sensory encoding and perceptual identification of Mandarin tones emerged with training and were retained, such changes followed different timescales. Sensory changes were evidenced and related to behavioral performance only when participants were over-trained. In contrast, changes in perceptual identification reflecting improvement in categorical percept emerged relatively earlier. Individual differences in perceptual identification, and not sensory encoding, related to faster learning. Our findings support the RHT-sensory plasticity accompanies, rather than drives, expert levels of non-native speech learning.

How neuroscience can inform the study of individual differences in cognitive abilities

Theories of human mental abilities should be consistent with what is known in neuroscience. Currently, tests of human mental abilities are modeled by cognitive constructs such as attention, working memory, and speed of information processing. These constructs are in turn related to a single general ability. However, brains are very complex systems and whether most of the variability between the operations of different brains can be ascribed to a single factor is questionable. Research in neuroscience suggests that psychological processes such as perception, attention, decision, and executive control are emergent properties of interacting distributed networks. The modules that make up these networks use similar computational processes that involve multiple forms of neural plasticity, each having different time constants. Accordingly, these networks might best be characterized in terms of the information they process rather than in terms of abstract psychological processes such as working memory and executive control.

Incorporating behavioral and sensory context into spectro-temporal models of auditory encoding

For several decades, auditory neuroscientists have used spectro-temporal encoding models to understand how neurons in the auditory system represent sound. Derived from early applications of systems identification tools to the auditory periphery, the spectro-temporal receptive field (STRF) and more sophisticated variants have emerged as an efficient means of characterizing representation throughout the auditory system. Most of these encoding models describe neurons as static sensory filters. However, auditory neural coding is not static. Sensory context, reflecting the acoustic environment, and behavioral context, reflecting the internal state of the listener, can both influence sound-evoked activity, particularly in central auditory areas. This review explores recent efforts to integrate context into spectro-temporal encoding models. It begins with a brief tutorial on the basics of estimating and interpreting STRFs. Then it describes three recent studies that have characterized contextual effects on STRFs, emerging over a range of timescales, from many minutes to tens of milliseconds. An important theme of this work is not simply that context influences auditory coding, but also that contextual effects span a large continuum of internal states. The added complexity of these context-dependent models introduces new experimental and theoretical challenges that must be addressed in order to be used effectively. Several new methodological advances promise to address these limitations and allow the development of more comprehensive context-dependent models in the future.

Plasticity in the auditory system

Over the last 30 years a wide range of manipulations of auditory input and experience have been shown to result in plasticity in auditory cortical and subcortical structures. The time course of plasticity ranges from very rapid stimulus-specific adaptation to longer-term changes associated with, for example, partial hearing loss or perceptual learning. Evidence for plasticity as a consequence of these and a range of other manipulations of auditory input and/or its significance is reviewed, with an emphasis on plasticity in adults and in the auditory cortex. The nature of the changes in auditory cortex associated with attention, memory and perceptual learning depend critically on task structure, reward contingencies, and learning strategy. Most forms of auditory system plasticity are adaptive, in that they serve to optimize auditory performance, prompting attempts to harness this plasticity for therapeutic purposes. However, plasticity associated with cochlear trauma and partial hearing loss appears to be maladaptive, and has been linked to tinnitus. Three important forms of human learning-related auditory system plasticity are those associated with language development, musical training, and improvement in performance with a cochlear implant. Almost all forms of plasticity involve changes in synaptic excitatory - inhibitory balance within existing patterns of connectivity. An attractive model applicable to a number of forms of learning-related plasticity is dynamic multiplexing by individual neurons, such that learning involving a particular stimulus attribute reflects a particular subset of the diverse inputs to a given neuron being gated by top-down influences. The plasticity evidence indicates that auditory cortex is a component of complex distributed networks that integrate the representation of auditory stimuli with attention, decision and reward processes.

Encoding and Decoding Models in Cognitive Electrophysiology

Cognitive neuroscience has seen rapid growth in the size and complexity of data recorded from the human brain as well as in the computational tools available to analyze this data. This data explosion has resulted in an increased use of multivariate, model-based methods for asking neuroscience questions, allowing scientists to investigate multiple hypotheses with a single dataset, to use complex, time-varying stimuli, and to study the human brain under more naturalistic conditions. These tools come in the form of "Encoding" models, in which stimulus features are used to model brain activity, and "Decoding" models, in which neural features are used to generated a stimulus output. Here we review the current state of encoding and decoding models in cognitive electrophysiology and provide a practical guide toward conducting experiments and analyses in this emerging field. Our examples focus on using linear models in the study of human language and audition. We show how to calculate auditory receptive fields from natural sounds as well as how to decode neural recordings to predict speech. The paper aims to be a useful tutorial to these approaches, and a practical introduction to using machine learning and applied statistics to build models of neural activity. The data analytic approaches we discuss may also be applied to other sensory modalities, motor systems, and cognitive systems, and we cover some examples in these areas. In addition, a collection of Jupyter notebooks is publicly available as a complement to the material covered in this paper, providing code examples and tutorials for predictive modeling in python. The aim is to provide a practical understanding of predictive modeling of human brain data and to propose best-practices in conducting these analyses.

Spectro-temporal cues enhance modulation sensitivity in cochlear implant users

Although speech understanding is highly variable amongst cochlear implants (CIs) subjects, the remarkably high speech recognition performance of many CI users is unexpected and not well understood. Numerous factors, including neural health and degradation of the spectral information in the speech signal of CIs, likely contribute to speech understanding. We studied the ability to use spectro-temporal modulations, which may be critical for speech understanding and discrimination, and hypothesize that CI users adopt a different perceptual strategy than normal-hearing (NH) individuals, whereby they rely more heavily on joint spectro-temporal cues to enhance detection of auditory cues. Modulation detection sensitivity was studied in CI users and NH subjects using broadband "ripple" stimuli that were modulated spectrally, temporally, or jointly, i.e., spectro-temporally. The spectro-temporal modulation transfer functions of CI users and NH subjects was decomposed into spectral and temporal dimensions and compared to those subjects' spectral-only and temporal-only modulation transfer functions. In CI users, the joint spectro-temporal sensitivity was better than that predicted by spectral-only and temporal-only sensitivity, indicating a heightened spectro-temporal sensitivity. Such an enhancement through the combined integration of spectral and temporal cues was not observed in NH subjects. The unique use of spectro-temporal cues by CI patients can yield benefits for use of cues that are important for speech understanding. This finding has implications for developing sound processing strategies that may rely on joint spectro-temporal modulations to improve speech comprehension of CI users, and the findings of this study may be valuable for developing clinical assessment tools to optimize CI processor performance.

Perceptual Plasticity for Auditory Object Recognition

In our auditory environment, we rarely experience the exact acoustic waveform twice. This is especially true for communicative signals that have meaning for listeners. In speech and music, the acoustic signal changes as a function of the talker (or instrument), speaking (or playing) rate, and room acoustics, to name a few factors. Yet, despite this acoustic variability, we are able to recognize a sentence or melody as the same across various kinds of acoustic inputs and determine meaning based on listening goals, expectations, context, and experience. The recognition process relates acoustic signals to prior experience despite variability in signal-relevant and signal-irrelevant acoustic properties, some of which could be considered as "noise" in service of a recognition goal. However, some acoustic variability, if systematic, is lawful and can be exploited by listeners to aid in recognition. Perceivable changes in systematic variability can herald a need for listeners to reorganize perception and reorient their attention to more immediately signal-relevant cues. This view is not incorporated currently in many extant theories of auditory perception, which traditionally reduce psychological or neural representations of perceptual objects and the processes that act on them to static entities. While this reduction is likely done for the sake of empirical tractability, such a reduction may seriously distort the perceptual process to be modeled. We argue that perceptual representations, as well as the processes underlying perception, are dynamically determined by an interaction between the uncertainty of the auditory signal and constraints of context. This suggests that the process of auditory recognition is highly context-dependent in that the identity of a given auditory object may be intrinsically tied to its preceding context. To argue for the flexible neural and psychological updating of sound-to-meaning mappings across speech and music, we draw upon examples of perceptual categories that are thought to be highly stable. This framework suggests that the process of auditory recognition cannot be divorced from the short-term context in which an auditory object is presented. Implications for auditory category acquisition and extant models of auditory perception, both cognitive and neural, are discussed.

Perceptually Salient Regions of the Modulation Power Spectrum for Musical Instrument Identification

The ability of a listener to recognize sound sources, and in particular musical instruments from the sounds they produce, raises the question of determining the acoustical information used to achieve such a task. It is now well known that the shapes of the temporal and spectral envelopes are crucial to the recognition of a musical instrument. More recently, Modulation Power Spectra (MPS) have been shown to be a representation that potentially explains the perception of musical instrument sounds. Nevertheless, the question of which specific regions of this representation characterize a musical instrument is still open. An identification task was applied to two subsets of musical instruments: tuba, trombone, cello, saxophone, and clarinet on the one hand, and marimba, vibraphone, guitar, harp, and viola pizzicato on the other. The sounds were processed with filtered spectrotemporal modulations with 2D Gaussian windows. The most relevant regions of this representation for instrument identification were determined for each instrument and reveal the regions essential for their identification. The method used here is based on a “molecular approach,” the so-called bubbles method. Globally, the instruments were correctly identified and the lower values of spectrotemporal modulations are the most important regions of the MPS for recognizing instruments. Interestingly, instruments that were confused with each other led to non-overlapping regions and were confused when they were filtered in the most salient region of the other instrument. These results suggest that musical instrument timbres are characterized by specific spectrotemporal modulations, information which could contribute to music information retrieval tasks such as automatic source recognition.

Effect of informational content of noise on speech representation in the aging midbrain and cortex

The ability to understand speech is significantly degraded by aging, particularly in noisy environments. One way that older adults cope with this hearing difficulty is through the use of contextual cues. Several behavioral studies have shown that older adults are better at following a conversation when the target speech signal has high contextual content or when the background distractor is not meaningful. Specifically, older adults gain significant benefit in focusing on and understanding speech if the background is spoken by a talker in a language that is not comprehensible to them (i.e., a foreign language). To understand better the neural mechanisms underlying this benefit in older adults, we investigated aging effects on midbrain and cortical encoding of speech when in the presence of a single competing talker speaking in a language that is meaningful or meaningless to the listener (i.e., English vs. Dutch). Our results suggest that neural processing is strongly affected by the informational content of noise. Specifically, older listeners' cortical responses to the attended speech signal are less deteriorated when the competing speech signal is an incomprehensible language rather than when it is their native language. Conversely, temporal processing in the midbrain is affected by different backgrounds only during rapid changes in speech and only in younger listeners. Additionally, we found that cognitive decline is associated with an increase in cortical envelope tracking, suggesting an age-related over (or inefficient) use of cognitive resources that may explain their difficulty in processing speech targets while trying to ignore interfering noise.

Enhanced brainstem and cortical encoding of sound during synchronized movement

Movement to a steady beat has been widely studied as a model of alignment of motor outputs on sensory inputs. However, how the encoding of sensory inputs is shaped during synchronized movements along the sensory pathway remains unknown. To investigate this, we simultaneously recorded brainstem and cortical electro-encephalographic activity while participants listened to periodic amplitude-modulated tones. Participants listened either without moving or while tapping in sync on every second beat. Cortical responses were identified at the envelope modulation rate (beat frequency), whereas brainstem responses were identified at the partials frequencies of the chord and at their modulation by the beat frequency (sidebands). During sensorimotor synchronization, cortical responses at beat frequency were larger than during passive listening. Importantly, brainstem responses were also enhanced, with a selective amplification of the sidebands, in particular at the lower-pitched tone of the chord, and no significant correlation with electromyographic measures at tapping frequency. These findings provide first evidence for an online gain in the cortical and subcortical encoding of sounds during synchronized movement, selective to behavior-relevant sound features. Moreover, the frequency-tagging method to isolate concurrent brainstem and cortical activities even during actual movements appears promising to reveal coordinated processes along the human auditory pathway.