Implicit Memory for Complex Sounds in Higher Auditory Cortex of the Ferret

Rapid rebalancing of co-tuned ensemble activity in the auditory cortex

Sensory information is represented by small neuronal ensembles in sensory cortices. Neuronal activity shows high trial-by-trial variability in that repeated presentation of the same stimulus, e. g., multiple presentations of the same sound activate differing ensembles in the auditory cortex (AC). How the differing ensembles interact to selectively activate to process incoming sound inputs with reduced energy is unknown. Efficient processing of complex acoustic signals requires that these sparsely distributed neuronal ensembles actively interact in order to provide a constant percept. Here, we probe interactions within and across ensembles by combining in vivo 2-photon Ca2+ imaging and holographic optogenetic stimulation to study how increased activity of single cells level affects the cortical network. We stimulated a small number of neurons sharing the same frequency preference alongside the presentation of a target pure tone, further increasing their tone-evoked activity. We found that other non-stimulated co-tuned neurons decreased their tone-evoked activity while non co-tuned neurons were unaffected. This shows that co-tuned ensembles communicated and balanced their total activity across the network. The rebalanced activity due to external stimulation remained constant. These effects suggest that co-tuned ensembles in AC interact and rapidly rebalance their activity to maintain encoding homeostasis, and that the rebalanced network is persistent.

Temporal Coherence Shapes Cortical Responses to Speech Mixtures in a Ferret Cocktail Party

Segregation of complex sounds such as speech, music and animal vocalizations as they simultaneously emanate from multiple sources (referred to as the "cocktail party problem") is a remarkable ability that is common in humans and animals alike. The neural underpinnings of this process have been extensively studied behaviorally and physiologically in non-human animals primarily with simplified sounds (tones and noise sequences). In humans, segregation experiments utilizing more complex speech mixtures are common; but physiological experiments have relied on EEG/MEG/ECoG recordings that sample activity from thousands of neurons, often obscuring the detailed processes that give rise to the observed segregation. The present study combines the insights from animal single-unit physiology with segregation of speech-like mixtures. Ferrets were trained to attend to a female voice and detect a target word, both in presence or absence of a concurrent, equally salient male voice. Single neuron recordings were obtained from primary and secondary ferret auditory cortical fields, as well as frontal cortex. During task performance, representation of the female words became more enhanced relative to those of the (distractor) male in all cortical regions, especially in the higher auditory cortical field. Analysis of the temporal and spectral response characteristics during task performance reveals how speech segregation gradually emerges in the auditory cortex. A computational model evaluated on the same voice mixtures replicates and extends these results to different attentional targets (attention to female or male voices). These findings are consistent with the temporal coherence theory whereby attention to a target voice anchors neural activity in cortical networks hence binding together channels that are coherently temporally-modulated with the target, and ultimately forming a common auditory stream.

A convolutional neural network provides a generalizable model of natural sound coding by neural populations in auditory cortex

Convolutional neural networks (CNNs) can provide powerful and flexible models of neural sensory processing. However, the utility of CNNs in studying the auditory system has been limited by their requirement for large datasets and the complex response properties of single auditory neurons. To address these limitations, we developed a population encoding model: a CNN that simultaneously predicts activity of several hundred neurons recorded during presentation of a large set of natural sounds. This approach defines a shared spectro-temporal space and pools statistical power across neurons. Population models of varying architecture performed consistently and substantially better than traditional linear-nonlinear models on data from primary and non-primary auditory cortex. Moreover, population models were highly generalizable. The output layer of a model pre-trained on one population of neurons could be fit to data from novel single units, achieving performance equivalent to that of neurons in the original fit data. This ability to generalize suggests that population encoding models capture a complete representational space across neurons in an auditory cortical field.

Repetition plasticity in primary auditory cortex occurs across long timescales for spectrotemporally randomized pure-tones

Repetition plasticity is a ubiquitous property of sensory systems in which repetitive sensation causes either a decrease ("repetition suppression", i.e. "adaptation") or increase ("repetition enhancement", i.e. "facilitation") in the amplitude of neural responses. Timescales of repetition plasticity for sensory neurons typically span milliseconds to tens of seconds, with longer durations for cortical vs subcortical regions. Here, we used 2-photon (2P) imaging to study repetition plasticity in mouse primary auditory cortex (A1) layer 2/3 (L2/3) during the presentation of spectrotemporally randomized pure-tone frequencies. Our study revealed subpopulations of neurons with repetition plasticity for equiprobable frequencies spaced minutes apart over a 20-minute period. We found both repetition suppression and enhancement in individual neurons and on average across populations. Each neuron tended to show repetition plasticity for 1-2 pure-tone frequencies near the neuron's best frequency. Moreover, we found correlated changes in neural response amplitude and latency across stimulus repetitions. Together, our results highlight cortical specialization for pattern recognition over long timescales in complex acoustic sequences.

Auditory memory of complex sounds in sparsely distributed, highly correlated neurons in the auditory cortex

Listening in complex sound environments requires rapid segregation of different sound sources e.g., speakers from each other, speakers from other sounds, or different instruments in an orchestra, and also adjust auditory processing on the prevailing sound conditions. Thus, fast encoding of inputs and identifying and adapting to reoccurring sounds are necessary for efficient and agile sound perception. This adaptation process represents an early phase of developing implicit learning of sound statistics and thus represents a form of auditory memory. The auditory cortex (ACtx) is known to play a key role in this encoding process but the underlying circuits and if hierarchical processing exists are not known. To identify ACtx regions and cells involved in this process, we simultaneously imaged population of neurons in different ACtx subfields using in vivo 2-photon imaging in awake mice. We used an experimental stimulus paradigm adapted from human studies that triggers rapid and robust implicit learning to passively present complex sounds and imaged A1 Layer 4 (L4), A1 L2/3, and A2 L2/3. In this paradigm, a frozen spectro-temporally complex 'Target' sound would be randomly re-occurring within a stream of random other complex sounds. We find distinct groups of cells that are specifically responsive to complex acoustic sequences across all subregions indicating that even the initial thalamocortical input layers (A1 L4) respond to complex sounds. Cells in all imaged regions showed decreased response amplitude for reoccurring Target sounds indicating that a memory signature is present even in the thalamocortical input layers. On the population level we find increased synchronized activity across cells to the Target sound and that this synchronized activity was more consistent across cells regardless of the duration of frozen token within Target sounds in A2, compared to A1. These findings suggest that ACtx and its input layers play a role in auditory memory for complex sounds and suggest a hierarchical structure of processes for auditory memory.

Multiscale temporal integration organizes hierarchical computation in human auditory cortex

To derive meaning from sound, the brain must integrate information across many timescales. What computations underlie multiscale integration in human auditory cortex? Evidence suggests that auditory cortex analyses sound using both generic acoustic representations (for example, spectrotemporal modulation tuning) and category-specific computations, but the timescales over which these putatively distinct computations integrate remain unclear. To answer this question, we developed a general method to estimate sensory integration windows-the time window when stimuli alter the neural response-and applied our method to intracranial recordings from neurosurgical patients. We show that human auditory cortex integrates hierarchically across diverse timescales spanning from ~50 to 400 ms. Moreover, we find that neural populations with short and long integration windows exhibit distinct functional properties: short-integration electrodes (less than ~200 ms) show prominent spectrotemporal modulation selectivity, while long-integration electrodes (greater than ~200 ms) show prominent category selectivity. These findings reveal how multiscale integration organizes auditory computation in the human brain.

Neural Correlates of Auditory Pattern Learning in the Auditory Cortex

Learning of new auditory stimuli often requires repetitive exposure to the stimulus. Fast and implicit learning of sounds presented at random times enables efficient auditory perception. However, it is unclear how such sensory encoding is processed on a neural level. We investigated neural responses that are developed from a passive, repetitive exposure to a specific sound in the auditory cortex of anesthetized rats, using electrocorticography. We presented a series of random sequences that are generated afresh each time, except for a specific reference sequence that remains constant and re-appears at random times across trials. We compared induced activity amplitudes between reference and fresh sequences. Neural responses from both primary and non-primary auditory cortical regions showed significantly decreased induced activity amplitudes for reference sequences compared to fresh sequences, especially in the beta band. This is the first study showing that neural correlates of auditory pattern learning can be evoked even in anesthetized, passive listening animal models.

Prediction error signaling explains neuronal mismatch responses in the medial prefrontal cortex

The mismatch negativity (MMN) is a key biomarker of automatic deviance detection thought to emerge from 2 cortical sources. First, the auditory cortex (AC) encodes spectral regularities and reports frequency-specific deviances. Then, more abstract representations in the prefrontal cortex (PFC) allow to detect contextual changes of potential behavioral relevance. However, the precise location and time asynchronies between neuronal correlates underlying this frontotemporal network remain unclear and elusive. Our study presented auditory oddball paradigms along with "no-repetition" controls to record mismatch responses in neuronal spiking activity and local field potentials at the rat medial PFC. Whereas mismatch responses in the auditory system are mainly induced by stimulus-dependent effects, we found that auditory responsiveness in the PFC was driven by unpredictability, yielding context-dependent, comparatively delayed, more robust and longer-lasting mismatch responses mostly comprised of prediction error signaling activity. This characteristically different composition discarded that mismatch responses in the PFC could be simply inherited or amplified downstream from the auditory system. Conversely, it is more plausible for the PFC to exert top-down influences on the AC, since the PFC exhibited flexible and potent predictive processing, capable of suppressing redundant input more efficiently than the AC. Remarkably, the time course of the mismatch responses we observed in the spiking activity and local field potentials of the AC and the PFC combined coincided with the time course of the large-scale MMN-like signals reported in the rat brain, thereby linking the microscopic, mesoscopic, and macroscopic levels of automatic deviance detection.

Long-term implicit memory for sequential auditory patterns in humans

Memory, on multiple timescales, is critical to our ability to discover the structure of our surroundings, and efficiently interact with the environment. We combined behavioural manipulation and modelling to investigate the dynamics of memory formation for rarely reoccurring acoustic patterns. In a series of experiments, participants detected the emergence of regularly repeating patterns within rapid tone-pip sequences. Unbeknownst to them, a few patterns reoccurred every ~3 min. All sequences consisted of the same 20 frequencies and were distinguishable only by the order of tone-pips. Despite this, reoccurring patterns were associated with a rapidly growing detection-time advantage over novel patterns. This effect was implicit, robust to interference, and persisted for 7 weeks. The results implicate an interplay between short (a few seconds) and long-term (over many minutes) integration in memory formation and demonstrate the remarkable sensitivity of the human auditory system to sporadically reoccurring structure within the acoustic environment.

Spectral tuning of adaptation supports coding of sensory context in auditory cortex

Perception of vocalizations and other behaviorally relevant sounds requires integrating acoustic information over hundreds of milliseconds. Sound-evoked activity in auditory cortex typically has much shorter latency, but the acoustic context, i.e., sound history, can modulate sound evoked activity over longer periods. Contextual effects are attributed to modulatory phenomena, such as stimulus-specific adaption and contrast gain control. However, an encoding model that links context to natural sound processing has yet to be established. We tested whether a model in which spectrally tuned inputs undergo adaptation mimicking short-term synaptic plasticity (STP) can account for contextual effects during natural sound processing. Single-unit activity was recorded from primary auditory cortex of awake ferrets during presentation of noise with natural temporal dynamics and fully natural sounds. Encoding properties were characterized by a standard linear-nonlinear spectro-temporal receptive field (LN) model and variants that incorporated STP-like adaptation. In the adapting models, STP was applied either globally across all input spectral channels or locally to subsets of channels. For most neurons, models incorporating local STP predicted neural activity as well or better than LN and global STP models. The strength of nonlinear adaptation varied across neurons. Within neurons, adaptation was generally stronger for spectral channels with excitatory than inhibitory gain. Neurons showing improved STP model performance also tended to undergo stimulus-specific adaptation, suggesting a common mechanism for these phenomena. When STP models were compared between passive and active behavior conditions, response gain often changed, but average STP parameters were stable. Thus, spectrally and temporally heterogeneous adaptation, subserved by a mechanism with STP-like dynamics, may support representation of the complex spectro-temporal patterns that comprise natural sounds across wide-ranging sensory contexts.

Adaptive Efficient Coding of Correlated Acoustic Properties

Natural sounds such as vocalizations often have covarying acoustic attributes, resulting in redundancy in neural coding. The efficient coding hypothesis proposes that sensory systems are able to detect such covariation and adapt to reduce redundancy, leading to more efficient neural coding. Recent psychoacoustic studies have shown the auditory system can rapidly adapt to efficiently encode two covarying dimensions as a single dimension, following passive exposure to sounds in which temporal and spectral attributes covaried in a correlated fashion. However, these studies observed a cost to this adaptation, which was a loss of sensitivity to the orthogonal dimension. Here we explore the neural basis of this psychophysical phenomenon by recording single-unit responses from the primary auditory cortex in awake ferrets exposed passively to stimuli with two correlated attributes, similar in stimulus design to the psychoacoustic experiments in humans. We found: (1) the signal-to-noise ratio of spike-rate coding of cortical responses driven by sounds with correlated attributes remained unchanged along the exposure dimension, but was reduced along the orthogonal dimension; (2) performance of a decoder trained with spike data to discriminate stimuli along the orthogonal dimension was equally reduced; (3) correlations between neurons tuned to the two covarying attributes decreased after exposure; and (4) these exposure effects still occurred if sounds were correlated along two acoustic dimensions, but varied randomly along a third dimension. These neurophysiological results are consistent with the efficient coding hypothesis and may help deepen our understanding of how the auditory system encodes and represents acoustic regularities and covariance.SIGNIFICANCE STATEMENT The efficient coding (EC) hypothesis (Attneave, 1954; Barlow, 1961) proposes that the neural code in sensory systems efficiently encodes natural stimuli by minimizing the number of spikes to transmit a sensory signal. Results of recent psychoacoustic studies in humans are consistent with the EC hypothesis in that, following passive exposure to stimuli with correlated attributes, the auditory system rapidly adapts so as to more efficiently encode the two covarying dimensions as a single dimension. In the current neurophysiological experiments, using a similar stimulus design and the experimental paradigm to the psychoacoustic studies of Stilp et al. (2010) and Stilp and Kluender (2011, 2012, 2016), we recorded responses from single neurons in the auditory cortex of the awake ferret, showing adaptive efficient neural coding of two correlated acoustic attributes.

Rapid and long-lasting improvements in neural discrimination of acoustic signals with passive familiarization

Sensory representations in the adult brain must undergo dynamic changes to adapt to the complexity of the external world. This study investigated how passive exposure to novel sounds modifies neural representations to facilitate recognition and discrimination, using the zebra finch model organism. The neural responses in an auditory structure in the zebra finch brain, Caudal Medial Nidopallium (NCM), undergo a long-term form of adaptation with repeated stimulus presentation, providing an excellent substrate to probe the neural underpinnings of adaptive sensory representations. In Experiment 1, electrophysiological activity in NCM was recorded under passive listening conditions as novel natural vocalizations were familiarized through playback. Neural decoding of stimuli using the temporal profiles of both single-unit and multi-unit responses improved dramatically during the first few stimulus presentations. During subsequent encounters, these signals were recognized after hearing fewer initial acoustic features. Remarkably, the accuracy of neural decoding was higher when different stimuli were heard in separate blocks compared to when they were presented randomly in a shuffled sequence. NCM neurons with narrow spike waveforms generally yielded higher neural decoding accuracy than wide spike neurons, but the rate at which these accuracies improved with passive exposure was comparable between the two neuron types. Experiment 2 supported and extended these findings by showing that the rapid gains in neural decoding of novel vocalizations with passive familiarization were long-lasting, maintained for 20 hours after the initial encounter, in multi-unit responses. Taken together, these findings provide valuable insights into the mechanisms by which the nervous system dynamically modulates sensory representations to improve discrimination of novel complex signals over short and long timescales. Similar mechanisms may also be engaged during processing of human speech signals, and thus may have potential translational relevance for elucidating the neural basis of speech comprehension difficulties.