Passive stimulation and behavioral training differentially transform temporal processing in the inferior colliculus and primary auditory cortex

Mixed representations of sound and action in the auditory midbrain

Linking sensory input and its consequences is a fundamental brain operation. Accordingly, neural activity of neo-cortical and limbic systems often reflects dynamic combinations of sensory and behaviorally relevant variables, and these "mixed representations" are suggested to be important for perception, learning, and plasticity. However, the extent to which such integrative computations might occur in brain regions upstream of the forebrain is less clear. Here, we conduct cellular-resolution 2-photon Ca2+ imaging in the superficial "shell" layers of the inferior colliculus (IC), as head-fixed mice of either sex perform a reward-based psychometric auditory task. We find that the activity of individual shell IC neurons jointly reflects auditory cues and mice's actions, such that trajectories of neural population activity diverge depending on mice's behavioral choice. Consequently, simple classifier models trained on shell IC neuron activity can predict trial-by-trial outcomes, even when training data are restricted to neural activity occurring prior to mice's instrumental actions. Thus in behaving animals, auditory midbrain neurons transmit a population code that reflects a joint representation of sound and action.

Electrophysiological and Psychophysical Measures of Temporal Pitch Sensitivity in Normal-hearing Listeners

To obtain combined behavioural and electrophysiological measures of pitch perception, we presented harmonic complexes, bandpass filtered to contain only high-numbered harmonics, to normal-hearing listeners. These stimuli resemble bandlimited pulse trains and convey pitch using a purely temporal code. A core set of conditions consisted of six stimuli with baseline pulse rates of 94, 188 and 280 pps, filtered into a HIGH (3365-4755 Hz) or VHIGH (7800-10,800 Hz) region, alternating with a 36% higher pulse rate. Brainstem and cortical processing were measured using the frequency following response (FFR) and auditory change complex (ACC), respectively. Behavioural rate change difference limens (DLs) were measured by requiring participants to discriminate between a stimulus that changed rate twice (up-down or down-up) during its 750-ms presentation from a constant-rate pulse train. FFRs revealed robust brainstem phase locking whose amplitude decreased with increasing rate. Moderate-sized but reliable ACCs were obtained in response to changes in purely temporal pitch and, like the psychophysical DLs, did not depend consistently on the direction of rate change or on the pulse rate for baseline rates between 94 and 280 pps. ACCs were larger and DLs lower for stimuli in the HIGH than in the VHGH region. We argue that the ACC may be a useful surrogate for behavioural measures of rate discrimination, both for normal-hearing listeners and for cochlear-implant users. We also showed that rate DLs increased markedly when the baseline rate was reduced to 48 pps, and compared the behavioural and electrophysiological findings to recent cat data obtained with similar stimuli and methods.

Brainstem speech encoding is dynamically shaped online by fluctuations in cortical α state

Experimental evidence in animals demonstrates cortical neurons innervate subcortex bilaterally to tune brainstem auditory coding. Yet, the role of the descending (corticofugal) auditory system in modulating earlier sound processing in humans during speech perception remains unclear. Here, we measured EEG activity as listeners performed speech identification tasks in different noise backgrounds designed to tax perceptual and attentional processing. We hypothesized brainstem speech coding might be tied to attention and arousal states (indexed by cortical α power) that actively modulate the interplay of brainstem-cortical signal processing. When speech-evoked brainstem frequency-following responses (FFRs) were categorized according to cortical α states, we found low α FFRs in noise were weaker, correlated positively with behavioral response times, and were more "decodable" via neural classifiers. Our data provide new evidence for online corticofugal interplay in humans and establish that brainstem sensory representations are continuously yoked to (i.e., modulated by) the ebb and flow of cortical states to dynamically update perceptual processing.

Cochlear Implant Research and Development in the Twenty-first Century: A Critical Update

Cochlear implants (CIs) are the world's most successful sensory prosthesis and have been the subject of intense research and development in recent decades. We critically review the progress in CI research, and its success in improving patient outcomes, from the turn of the century to the present day. The review focuses on the processing, stimulation, and audiological methods that have been used to try to improve speech perception by human CI listeners, and on fundamental new insights in the response of the auditory system to electrical stimulation. The introduction of directional microphones and of new noise reduction and pre-processing algorithms has produced robust and sometimes substantial improvements. Novel speech-processing algorithms, the use of current-focusing methods, and individualised (patient-by-patient) deactivation of subsets of electrodes have produced more modest improvements. We argue that incremental advances have and will continue to be made, that collectively these may substantially improve patient outcomes, but that the modest size of each individual advance will require greater attention to experimental design and power. We also briefly discuss the potential and limitations of promising technologies that are currently being developed in animal models, and suggest strategies for researchers to collectively maximise the potential of CIs to improve hearing in a wide range of listening situations.

Auditory memory for random time patterns in cochlear implant listeners

Learning about new sounds is essential for cochlear-implant and normal-hearing listeners alike, with the additional challenge for implant listeners that spectral resolution is severely degraded. Here, a task measuring the rapid learning of slow or fast stochastic temporal sequences [Kang, Agus, and Pressnitzer (2017). J. Acoust. Soc. Am. 142, 2219-2232] was performed by cochlear-implant (N = 10) and normal-hearing (N = 9) listeners, using electric or acoustic pulse sequences, respectively. Rapid perceptual learning was observed for both groups, with highly similar characteristics. Moreover, for cochlear-implant listeners, an additional condition tested ultra-fast electric pulse sequences that would be impossible to represent temporally when presented acoustically. This condition also demonstrated learning. Overall, the results suggest that cochlear-implant listeners have access to the neural plasticity mechanisms needed for the rapid perceptual learning of complex temporal sequences.

Attention reinforces human corticofugal system to aid speech perception in noise

Perceiving speech-in-noise (SIN) demands precise neural coding between brainstem and cortical levels of the hearing system. Attentional processes can then select and prioritize task-relevant cues over competing background noise for successful speech perception. In animal models, brainstem-cortical interplay is achieved via descending corticofugal projections from cortex that shape midbrain responses to behaviorally-relevant sounds. Attentional engagement of corticofugal feedback may assist SIN understanding but has never been confirmed and remains highly controversial in humans. To resolve these issues, we recorded source-level, anatomically constrained brainstem frequency-following responses (FFRs) and cortical event-related potentials (ERPs) to speech via high-density EEG while listeners performed rapid SIN identification tasks. We varied attention with active vs. passive listening scenarios whereas task difficulty was manipulated with additive noise interference. Active listening (but not arousal-control tasks) exaggerated both ERPs and FFRs, confirming attentional gain extends to lower subcortical levels of speech processing. We used functional connectivity to measure the directed strength of coupling between levels and characterize "bottom-up" vs. "top-down" (corticofugal) signaling within the auditory brainstem-cortical pathway. While attention strengthened connectivity bidirectionally, corticofugal transmission disengaged under passive (but not active) SIN listening. Our findings (i) show attention enhances the brain's transcription of speech even prior to cortex and (ii) establish a direct role of the human corticofugal feedback system as an aid to cocktail party speech perception.

The Effect of Phantom Stimulation and Pseudomonophasic Pulse Shapes on Pitch Perception by Cochlear Implant Listeners

It has been suggested that a specialized high-temporal-acuity brainstem pathway can be activated by stimulating more apically in the cochlea than is achieved by cochlear implants (CIs) when programmed with contemporary clinical settings. We performed multiple experiments to test the effect on pitch perception of phantom stimulation and asymmetric current pulses, both supposedly stimulating beyond the most apical electrode of a CI. The two stimulus types were generated using a bipolar electrode pair, composed of the most apical electrode of the array and a neighboring, more basal electrode. Experiment 1 used a pitch-ranking procedure where neural excitation was shifted apically or basally using so-called phantom stimulation. No benefit of apical phantom stimulation was found on the highest rate up to which pitch ranks increased (upper limit), nor on the slopes of the pitch-ranking function above 300 pulses per second (pps). Experiment 2 used the same procedure to study the effects of apical pseudomonophasic pulses, where the locus of excitation was manipulated by changing stimulus polarity. A benefit of apical stimulation was obtained for the slopes above 300 pps. Experiment 3 used an adaptive rate discrimination procedure and found a small but significant benefit of both types of apical stimulation. Overall, the results show some benefit for apical stimulation on temporal pitch processing at high pulse rates but reveal that the effect is smaller and more variable across listeners than suggested by previous research. The results also provide some indication that the benefit of apical stimulation may decline over time since implantation.

Auditory cortical plasticity in cochlear implant users

Cochlear implants are one of the most successful neuroprosthetic devices that have been developed to date. Profoundly deaf patients can achieve speech perception after complete loss of sensory input. Despite the improvements many patients experience, there is still a large degree of outcome variability. It has been proposed that central plasticity may be a major factor in the different levels of benefit that patients experience. However, the neural mechanisms of how plasticity impacts cochlear implant learning and the degree of plasticity's influence remain unknown. Here, we review the human and animal research on three of the main ways that central plasticity affects cochlear implant outcomes.

Afferent-efferent connectivity between auditory brainstem and cortex accounts for poorer speech-in-noise comprehension in older adults

Speech-in-noise (SIN) comprehension deficits in older adults have been linked to changes in both subcortical and cortical auditory evoked responses. However, older adults' difficulty understanding SIN may also be related to an imbalance in signal transmission (i.e., functional connectivity) between brainstem and auditory cortices. By modeling high-density scalp recordings of speech-evoked responses with sources in brainstem (BS) and bilateral primary auditory cortices (PAC), we show that beyond attenuating neural activity, hearing loss in older adults compromises the transmission of speech information between subcortical and early cortical hubs of the speech network. We found that the strength of afferent BS→PAC neural signaling (but not the reverse efferent flow; PAC→BS) varied with mild declines in hearing acuity and this "bottom-up" functional connectivity robustly predicted older adults' performance in a SIN identification task. Connectivity was also a better predictor of SIN processing than unitary subcortical or cortical responses alone. Our neuroimaging findings suggest that in older adults (i) mild hearing loss differentially reduces neural output at several stages of auditory processing (PAC > BS), (ii) subcortical-cortical connectivity is more sensitive to peripheral hearing loss than top-down (cortical-subcortical) control, and (iii) reduced functional connectivity in afferent auditory pathways plays a significant role in SIN comprehension problems.

Effect of Chronic Stimulation and Stimulus Level on Temporal Processing by Cochlear Implant Listeners

A series of experiments investigated potential changes in temporal processing during the months following activation of a cochlear implant (CI) and as a function of stimulus level. Experiment 1 tested patients on the day of implant activation and 2 and 6 months later. All stimuli were presented using direct stimulation of a single apical electrode. The dependent variables were rate discrimination ratios (RDRs) for pulse trains with rates centred on 120 pulses per second (pps), obtained using an adaptive procedure, and a measure of the upper limit of temporal pitch, obtained using a pitch-ranking procedure. All stimuli were presented at their most comfortable level (MCL). RDRs decreased from 1.23 to 1.16 and the upper limit increased from 357 to 485 pps from 0 to 2 months post-activation, with no overall change from 2 to 6 months. Because MCLs and hence the testing level increased across sessions, two further experiments investigated whether the performance changes observed across sessions could be due to level differences. Experiment 2 re-tested a subset of subjects at 9 months post-activation, using current levels similar to those used at 0 months. Although the stimuli sounded softer, some subjects showed lower RDRs and/or higher upper limits at this re-test. Experiment 3 measured RDRs and the upper limit for a separate group of subjects at levels equal to 60 %, 80 % and 100 % of the dynamic range. RDRs decreased with increasing level. The upper limit increased with increasing level for most subjects, with two notable exceptions. Implications of the results for temporal plasticity are discussed, along with possible influences of the effects of level and of across-session learning.

Brainstem correlates of concurrent speech identification in adverse listening conditions

When two voices compete, listeners can segregate and identify concurrent speech sounds using pitch (fundamental frequency, F0) and timbre (harmonic) cues. Speech perception is also hindered by the signal-to-noise ratio (SNR). How clear and degraded concurrent speech sounds are represented at early, pre-attentive stages of the auditory system is not well understood. To this end, we measured scalp-recorded frequency-following responses (FFR) from the EEG while human listeners heard two concurrently presented, steady-state (time-invariant) vowels whose F0 differed by zero or four semitones (ST) presented diotically in either clean (no noise) or noise-degraded (+5dB SNR) conditions. Listeners also performed a speeded double vowel identification task in which they were required to identify both vowels correctly. Behavioral results showed that speech identification accuracy increased with F0 differences between vowels, and this perceptual F0 benefit was larger for clean compared to noise degraded (+5dB SNR) stimuli. Neurophysiological data demonstrated more robust FFR F0 amplitudes for single compared to double vowels and considerably weaker responses in noise. F0 amplitudes showed speech-on-speech masking effects, along with a non-linear constructive interference at 0ST, and suppression effects at 4ST. Correlations showed that FFR F0 amplitudes failed to predict listeners' identification accuracy. In contrast, FFR F1 amplitudes were associated with faster reaction times, although this correlation was limited to noise conditions. The limited number of brain-behavior associations suggests subcortical activity mainly reflects exogenous processing rather than perceptual correlates of concurrent speech perception. Collectively, our results demonstrate that FFRs reflect pre-attentive coding of concurrent auditory stimuli that only weakly predict the success of identifying concurrent speech.

Neural ITD Sensitivity and Temporal Coding with Cochlear Implants in an Animal Model of Early-Onset Deafness

Users of cochlear implant (CI) face challenges in everyday situations such as understanding conversations in noise, even with CIs in both ears. These challenges are related to difficulties with tasks that require fine temporal processing such as discrimination of pulse rates or interaural time differences (ITD), a major cue for sound localization. The degradation in temporal processing and ITD sensitivity are especially acute in those who lost hearing in early childhood. Here, we characterized temporal coding and ITD sensitivity of single neurons in a novel animal model of early-onset deafness. Rabbits were deafened as neonates and deprived of auditory stimulation until they reached adult age when single-unit recordings from the auditory midbrain were made chronically using an unanesthetized preparation. The results are compared to measurements from adult-deafened rabbits with normal auditory development to understand the effect of early-onset deafness on neural temporal coding and ITD sensitivity. Neurons in the inferior colliculus (IC) of early-deafened rabbits were less likely to show sustained, excitatory responses to pulse train stimulation and more likely to show suppressive responses compared to neurons in adult-deaf animals. Fewer neurons showed synchronized responses to pulse trains at any rate in the early-deaf group. In addition, fewer neurons showed significant ITD sensitivity in their overall firing rate in the early-deaf group compared to adult-deaf animals. Neural ITD discrimination thresholds in the early-deaf group were poorer than thresholds in adult-deaf group, especially at high pulse rates. The overall degradation in neural ITD sensitivity is consistent with the difficulties encountered by human CI users with early-onset hearing loss. These results lay the groundwork for investigating whether the degradations in temporal coding and ITD sensitivity observed in early-deaf animals can be reversed by appropriate CI stimulation during development.

Evaluation of Possible Effects of a Potassium Channel Modulator on Temporal Processing by Cochlear Implant Listeners

Temporal processing by cochlear implant listeners is degraded and is affected by auditory deprivation. The fast-acting Kv3.1 potassium channel is important for sustained temporally accurate firing and is also susceptible to deprivation, the effects of which can be partially restored in animals by the molecule AUT00063. We report the results of a randomised placebo-controlled double-blind study on psychophysical tests of the effects of AUT00063 on temporal processing by CI listeners. The study measured the upper limit of temporal pitch, gap detection, and discrimination of low rates (centred on 120 pps) for monopolar pulse trains presented to an apical electrode. The upper limit was measured using the optimally efficient midpoint comparison (MPC) pitch-ranking procedure; thresholds were obtained for the other two measures using an adaptive procedure. Twelve CI users (MedEl and Cochlear) were tested before and after two periods of AUT00063 or placebo in a within-subject crossover study. No significant differences occurred between post-drug and post-placebo conditions. This absence of effect occurred despite high test-retest reliability for all three measures, obtained by comparing performance on the two baseline visits, and despite the demonstrated sensitivity of the measures to modest changes in temporal processing obtained in other studies from our laboratory. Hence, we have no evidence that AUT00063 improves temporal processing for the doses and patient population employed.

Plasticity in the auditory system

Over the last 30 years a wide range of manipulations of auditory input and experience have been shown to result in plasticity in auditory cortical and subcortical structures. The time course of plasticity ranges from very rapid stimulus-specific adaptation to longer-term changes associated with, for example, partial hearing loss or perceptual learning. Evidence for plasticity as a consequence of these and a range of other manipulations of auditory input and/or its significance is reviewed, with an emphasis on plasticity in adults and in the auditory cortex. The nature of the changes in auditory cortex associated with attention, memory and perceptual learning depend critically on task structure, reward contingencies, and learning strategy. Most forms of auditory system plasticity are adaptive, in that they serve to optimize auditory performance, prompting attempts to harness this plasticity for therapeutic purposes. However, plasticity associated with cochlear trauma and partial hearing loss appears to be maladaptive, and has been linked to tinnitus. Three important forms of human learning-related auditory system plasticity are those associated with language development, musical training, and improvement in performance with a cochlear implant. Almost all forms of plasticity involve changes in synaptic excitatory - inhibitory balance within existing patterns of connectivity. An attractive model applicable to a number of forms of learning-related plasticity is dynamic multiplexing by individual neurons, such that learning involving a particular stimulus attribute reflects a particular subset of the diverse inputs to a given neuron being gated by top-down influences. The plasticity evidence indicates that auditory cortex is a component of complex distributed networks that integrate the representation of auditory stimuli with attention, decision and reward processes.

Representations of Time-Varying Cochlear Implant Stimulation in Auditory Cortex of Awake Marmosets (Callithrix jacchus)

Electrical stimulation of the auditory periphery organ by cochlear implant (CI) generates highly synchronized inputs to the auditory system. It has long been thought such inputs would lead to highly synchronized neural firing along the ascending auditory pathway. However, neurophysiological studies with hearing animals have shown that the central auditory system progressively converts temporal representations of time-varying sounds to firing rate-based representations. It is not clear whether this coding principle also applies to highly synchronized CI inputs. Higher-frequency modulations in CI stimulation have been found to evoke largely transient responses with little sustained firing in previous studies of the primary auditory cortex (A1) in anesthetized animals. Here, we show that, in addition to neurons displaying synchronized firing to CI stimuli, a large population of A1 neurons in awake marmosets (Callithrix jacchus) responded to rapid time-varying CI stimulation with discharges that were not synchronized to CI stimuli, yet reflected changing repetition frequency by increased firing rate. Marmosets of both sexes were included in this study. By comparing directly each neuron's responses to time-varying acoustic and CI signals, we found that individual A1 neurons encode both modalities with similar firing patterns (stimulus-synchronized or nonsynchronized). These findings suggest that A1 neurons use the same basic coding schemes to represent time-varying acoustic or CI stimulation and provide new insights into mechanisms underlying how the brain processes natural sounds via a CI device.SIGNIFICANCE STATEMENT In modern cochlear implant (CI) processors, the temporal information in speech or environmental sounds is delivered through modulated electric pulse trains. How the auditory cortex represents temporally modulated CI stimulation across multiple time scales has remained largely unclear. In this study, we compared directly neuronal responses in primary auditory cortex (A1) to time-varying acoustic and CI signals in awake marmoset monkeys (Callithrix jacchus). We found that A1 neurons encode both modalities using similar coding schemes, but some important differences were identified. Our results provide insights into mechanisms underlying how the brain processes sounds via a CI device and suggest a candidate neural code underlying rate-pitch perception limitations often observed in CI users.

Positive impacts of early auditory training on cortical processing at an older age

Progressive negative behavioral changes in normal aging are paralleled by a complex series of physical and functional declines expressed in the cerebral cortex. In studies conducted in the auditory domain, these degrading physical and functional cortical changes have been shown to be broadly reversed by intensive progressive training that improves the spectral and temporal resolution of acoustic inputs and suppresses behavioral distractors. Here we found older rats that were intensively trained on an attentionally demanding modulation-rate recognition task in young adulthood substantially retained training-driven improvements in temporal rate discrimination abilities over a subsequent 18-mo epoch-that is, forward into their older age. In parallel, this young-adult auditory training enduringly enhanced temporal and spectral information processing in their primary auditory cortices (A1). Substantially greater numbers of parvalbumin- and somatostatin-labeled inhibitory neurons (closer to the numbers recorded in young vigorous adults) were recorded in the A1 and hippocampus in old trained versus untrained age-matched rats. These results show that a simple form of training in young adulthood in this rat model enduringly delays the otherwise expected deterioration of the physical status and functional operations of the auditory nervous system, with evident training impacts generalized to the hippocampus.