Nonlinear spectrotemporal interactions underlying selectivity for complex sounds in auditory cortex

Receptive-field nonlinearities in primary auditory cortex: a comparative perspective

Cortical processing of auditory information can be affected by interspecies differences as well as brain states. Here we compare multifeature spectro-temporal receptive fields (STRFs) and associated input/output functions or nonlinearities (NLs) of neurons in primary auditory cortex (AC) of four mammalian species. Single-unit recordings were performed in awake animals (female squirrel monkeys, female, and male mice) and anesthetized animals (female squirrel monkeys, rats, and cats). Neuronal responses were modeled as consisting of two STRFs and their associated NLs. The NLs for the STRF with the highest information content show a broad distribution between linear and quadratic forms. In awake animals, we find a higher percentage of quadratic-like NLs as opposed to more linear NLs in anesthetized animals. Moderate sex differences of the shape of NLs were observed between male and female unanesthetized mice. This indicates that the core AC possesses a rich variety of potential computations, particularly in awake animals, suggesting that multiple computational algorithms are at play to enable the auditory system's robust recognition of auditory events.

Case reopened: A temporal basis for harmonic pitch templates in the early auditory system?a)

A fundamental assumption of rate-place models of pitch is the existence of harmonic templates in the central nervous system (CNS). Shamma and Klein [(2000). J. Acoust. Soc. Am. 107, 2631-2644] hypothesized that these templates have a temporal basis. Coincidences in the temporal fine-structure of neural spike trains, even in response to nonharmonic, stochastic stimuli, would be sufficient for the development of harmonic templates. The physiological plausibility of this hypothesis is tested. Responses to pure tones, low-pass noise, and broadband noise from auditory nerve fibers and brainstem "high-sync" neurons are studied. Responses to tones simulate the output of fibers with infinitely sharp filters: for these responses, harmonic structure in a coincidence matrix comparing pairs of spike trains is indeed found. However, harmonic template structure is not observed in coincidences across responses to broadband noise, which are obtained from nerve fibers or neurons with enhanced synchronization. Using a computer model based on that of Shamma and Klein, it is shown that harmonic templates only emerge when consecutive processing steps (cochlear filtering, lateral inhibition, and temporal enhancement) are implemented in extreme, physiologically implausible form. It is concluded that current physiological knowledge does not support the hypothesis of Shamma and Klein (2000).

A sparse code for natural sound context in auditory cortex

Accurate sound perception can require integrating information over hundreds of milliseconds or even seconds. Spectro-temporal models of sound coding by single neurons in auditory cortex indicate that the majority of sound-evoked activity can be attributed to stimuli with a few tens of milliseconds. It remains uncertain how the auditory system integrates information about sensory context on a longer timescale. Here we characterized long-lasting contextual effects in auditory cortex (AC) using a diverse set of natural sound stimuli. We measured context effects as the difference in a neuron's response to a single probe sound following two different context sounds. Many AC neurons showed context effects lasting longer than the temporal window of a traditional spectro-temporal receptive field. The duration and magnitude of context effects varied substantially across neurons and stimuli. This diversity of context effects formed a sparse code across the neural population that encoded a wider range of contexts than any constituent neuron. Encoding model analysis indicates that context effects can be explained by activity in the local neural population, suggesting that recurrent local circuits support a long-lasting representation of sensory context in auditory cortex.

Distinct nonlinear spectrotemporal integration in primary and secondary auditory cortices

Animals sense sounds through hierarchical neural pathways that ultimately reach higher-order cortices to extract complex acoustic features, such as vocalizations. Elucidating how spectrotemporal integration varies along the hierarchy from primary to higher-order auditory cortices is a crucial step in understanding this elaborate sensory computation. Here we used two-photon calcium imaging and two-tone stimuli with various frequency-timing combinations to compare spectrotemporal integration between primary (A1) and secondary (A2) auditory cortices in mice. Individual neurons showed mixed supralinear and sublinear integration in a frequency-timing combination-specific manner, and we found unique integration patterns in these two areas. Temporally asymmetric spectrotemporal integration in A1 neurons suggested their roles in discriminating frequency-modulated sweep directions. In contrast, temporally symmetric and coincidence-preferring integration in A2 neurons made them ideal spectral integrators of concurrent multifrequency sounds. Moreover, the ensemble neural activity in A2 was sensitive to two-tone timings, and coincident two-tones evoked distinct ensemble activity patterns from the linear sum of component tones. Together, these results demonstrate distinct roles of A1 and A2 in encoding complex acoustic features, potentially suggesting parallel rather than sequential information extraction between these regions.

Temporal profiles of neuronal responses to repeated tone stimuli in the mouse primary auditory cortex

How the auditory system processes temporal information of sound has been investigated extensively using repeated stimuli. Recent studies on how the response of neurons in the primary auditory cortex (A1) changes with the progression of stimulus repetition, have reported response temporal profiles of two categories: "adaptation", i.e., gradual decrease, and "facilitation", i.e., gradual increase. To explore the existence of profiles of other categories and to examine the tone-frequency-dependence of the profile category in single neurons, here we studied the response of mouse A1 neurons to four or five tone-trains; each train comprised 10 identical tone pips, with 0.5-s inter-tone-intervals, and the four or five trains differed only in tone frequency. The response to each tone in a train was evaluated using the peak of the ON response, and how the peak response changed with the tone number in the train, i.e., the response temporal profile, was examined. We confirmed the existence of profiles of both "adaptation" and "facilitation" categories; "adaptation" could be further subcategorized into "slow adaptation" and "fast adaptation" profiles, with the latter being encountered more frequently. Moreover, two new categories of non-monotonic profiles were identified: an "adaptation with recovery" profile and a "facilitation followed by adaptation" profile. Examination of single neurons with trains of different tone frequencies revealed that some A1 neurons exhibited profiles of the same category to tone trains of different tone frequencies, whereas others exhibited profiles of different categories, depending on the tone frequency. These results demonstrate the variety in the response temporal profiles of mouse A1 neurons, which may benefit the encoding of individual tones in a train.

Quantitative models of auditory cortical processing

To generate insight from experimental data, it is critical to understand the inter-relationships between individual data points and place them in context within a structured framework. Quantitative modeling can provide the scaffolding for such an endeavor. Our main objective in this review is to provide a primer on the range of quantitative tools available to experimental auditory neuroscientists. Quantitative modeling is advantageous because it can provide a compact summary of observed data, make underlying assumptions explicit, and generate predictions for future experiments. Quantitative models may be developed to characterize or fit observed data, to test theories of how a task may be solved by neural circuits, to determine how observed biophysical details might contribute to measured activity patterns, or to predict how an experimental manipulation would affect neural activity. In complexity, quantitative models can range from those that are highly biophysically realistic and that include detailed simulations at the level of individual synapses, to those that use abstract and simplified neuron models to simulate entire networks. Here, we survey the landscape of recently developed models of auditory cortical processing, highlighting a small selection of models to demonstrate how they help generate insight into the mechanisms of auditory processing. We discuss examples ranging from models that use details of synaptic properties to explain the temporal pattern of cortical responses to those that use modern deep neural networks to gain insight into human fMRI data. We conclude by discussing a biologically realistic and interpretable model that our laboratory has developed to explore aspects of vocalization categorization in the auditory pathway.

Distinct nonlinear spectrotemporal integration in primary and secondary auditory cortices

Animals sense sounds through hierarchical neural pathways that ultimately reach higher-order cortices to extract complex acoustic features, such as vocalizations. Elucidating how spectrotemporal integration varies along the hierarchy from primary to higher-order auditory cortices is a crucial step in understanding this elaborate sensory computation. Here we used two-photon calcium imaging and two-tone stimuli with various frequency-timing combinations to compare spectrotemporal integration between primary (A1) and secondary (A2) auditory cortices in mice. Individual neurons showed mixed supralinear and sublinear integration in a frequency-timing combination-specific manner, and we found unique integration patterns in these two areas. Temporally asymmetric spectrotemporal integration in A1 neurons enabled their discrimination of frequency-modulated sweep directions. In contrast, temporally symmetric and coincidence-preferring integration in A2 neurons made them ideal spectral integrators of concurrent multifrequency sounds. Moreover, the ensemble neural activity in A2 was sensitive to two-tone timings, and coincident two-tones evoked distinct ensemble activity patterns from the linear sum of component tones. Together, these results demonstrate distinct roles of A1 and A2 in encoding complex acoustic features, potentially suggesting parallel rather than sequential information extraction between these regions.

A Redundant Cortical Code for Speech Envelope

Animal communication sounds exhibit complex temporal structure because of the amplitude fluctuations that comprise the sound envelope. In human speech, envelope modulations drive synchronized activity in auditory cortex (AC), which correlates strongly with comprehension (Giraud and Poeppel, 2012; Peelle and Davis, 2012; Haegens and Zion Golumbic, 2018). Studies of envelope coding in single neurons, performed in nonhuman animals, have focused on periodic amplitude modulation (AM) stimuli and use response metrics that are not easy to juxtapose with data from humans. In this study, we sought to bridge these fields. Specifically, we looked directly at the temporal relationship between stimulus envelope and spiking, and we assessed whether the apparent diversity across neurons' AM responses contributes to the population representation of speech-like sound envelopes. We gathered responses from single neurons to vocoded speech stimuli and compared them to sinusoidal AM responses in auditory cortex (AC) of alert, freely moving Mongolian gerbils of both sexes. While AC neurons displayed heterogeneous tuning to AM rate, their temporal dynamics were stereotyped. Preferred response phases accumulated near the onsets of sinusoidal AM periods for slower rates (<8 Hz), and an over-representation of amplitude edges was apparent in population responses to both sinusoidal AM and vocoded speech envelopes. Crucially, this encoding bias imparted a decoding benefit: a classifier could discriminate vocoded speech stimuli using summed population activity, while higher frequency modulations required a more sophisticated decoder that tracked spiking responses from individual cells. Together, our results imply that the envelope structure relevant to parsing an acoustic stream could be read-out from a distributed, redundant population code.SIGNIFICANCE STATEMENT Animal communication sounds have rich temporal structure and are often produced in extended sequences, including the syllabic structure of human speech. Although the auditory cortex (AC) is known to play a crucial role in representing speech syllables, the contribution of individual neurons remains uncertain. Here, we characterized the representations of both simple, amplitude-modulated sounds and complex, speech-like stimuli within a broad population of cortical neurons, and we found an overrepresentation of amplitude edges. Thus, a phasic, redundant code in auditory cortex can provide a mechanistic explanation for segmenting acoustic streams like human speech.

Selective attention sharpens population receptive fields in human auditory cortex

Selective attention enables the preferential processing of relevant stimulus aspects. Invasive animal studies have shown that attending a sound feature rapidly modifies neuronal tuning throughout the auditory cortex. Human neuroimaging studies have reported enhanced auditory cortical responses with selective attention. To date, it remains unclear how the results obtained with functional magnetic resonance imaging (fMRI) in humans relate to the electrophysiological findings in animal models. Here we aim to narrow the gap between animal and human research by combining a selective attention task similar in design to those used in animal electrophysiology with high spatial resolution ultra-high field fMRI at 7 Tesla. Specifically, human participants perform a detection task, whereas the probability of target occurrence varies with sound frequency. Contrary to previous fMRI studies, we show that selective attention resulted in population receptive field sharpening, and consequently reduced responses, at the attended sound frequencies. The difference between our results to those of previous fMRI studies supports the notion that the influence of selective attention on auditory cortex is diverse and may depend on context, stimulus, and task.

Enhanced stability of complex sound representations relative to simple sounds in the auditory cortex

Typical everyday sounds, such as those of speech or running water, are spectrotemporally complex. The ability to recognize complex sounds (CxS) and their associated meaning is presumed to rely on their stable neural representations across time. The auditory cortex is critical for processing of CxS, yet little is known of the degree of stability of auditory cortical representations of CxS across days. Previous studies have shown that the auditory cortex represents CxS identity with a substantial degree of invariance to basic sound attributes such as frequency. We therefore hypothesized that auditory cortical representations of CxS are more stable across days than those of sounds that lack spectrotemporal structure such as pure tones (PTs). To test this hypothesis, we recorded responses of identified L2/3 auditory cortical excitatory neurons to both PTs and CxS across days using two-photon calcium imaging in awake mice. Auditory cortical neurons showed significant daily changes of responses to both types of sounds, yet responses to CxS exhibited significantly lower rates of daily change than those of PTs. Furthermore, daily changes in response profiles to PTs tended to be more stimulus-specific, reflecting changes in sound selectivity, as compared to changes of CxS responses. Lastly, the enhanced stability of responses to CxS was evident across longer time intervals as well. Together, these results suggest that spectrotemporally CxS are more stably represented in the auditory cortex across time than PTs. These findings support a role of the auditory cortex in representing CxS identity across time.Significance statementThe ability to recognize everyday complex sounds such as those of speech or running water is presumed to rely on their stable neural representations. Yet, little is known of the degree of stability of single-neuron sound responses across days. As the auditory cortex is critical for complex sound perception, we hypothesized that the auditory cortical representations of complex sounds are relatively stable across days. To test this, we recorded sound responses of identified auditory cortical neurons across days in awake mice. We found that auditory cortical responses to complex sounds are significantly more stable across days as compared to those of simple pure tones. These findings support a role of the auditory cortex in representing complex sound identity across time.

Sequential transmission of task-relevant information in cortical neuronal networks

Cortical processing of task-relevant information enables recognition of behaviorally meaningful sensory events. It is unclear how task-related information is represented within cortical networks by the activity of individual neurons and their functional interactions. Here, we use two-photon imaging to record neuronal activity from the primary auditory cortex of mice during a pure-tone discrimination task. We find that a subset of neurons transiently encode sensory information used to inform behavioral choice. Using Granger causality analysis, we show that these neurons form functional networks in which information transmits sequentially. Network structures differ for target versus non-target tones, encode behavioral choice, and differ between correct versus incorrect behavioral choices. Correct behavioral choices are associated with shorter communication timescales, larger functional correlations, and greater information redundancy. In summary, specialized neurons in primary auditory cortex integrate task-related information and form functional networks whose structures encode both sensory input and behavioral choice.

Francis et al. find that, as mice perform an auditory discrimination task, cortical neurons form functional networks in which task-relevant information transmits sequentially between neurons. Network structures encode behavioral choice, and correct behavioral choices are associated with shorter communication timescales, larger functional correlations, and greater information redundancy between neurons.

Common principles in the lateralisation of auditory cortex structure and function for vocal communication in primates and rodents

This review summarises recent findings on the lateralisation of communicative sound processing in the auditory cortex (AC) of humans, non-human primates, and rodents. Functional imaging in humans has demonstrated a left hemispheric preference for some acoustic features of speech, but it is unclear to which degree this is caused by bottom-up acoustic feature selectivity or top-down modulation from language areas. Although non-human primates show a less pronounced functional lateralisation in AC, the properties of AC fields and behavioral asymmetries are qualitatively similar. Rodent studies demonstrate microstructural circuits that might underlie bottom-up acoustic feature selectivity in both hemispheres. Functionally, the left AC in the mouse appears to be specifically tuned to communication calls, whereas the right AC may have a more 'generalist' role. Rodents also show anatomical AC lateralisation, such as differences in size and connectivity. Several of these functional and anatomical characteristics are also lateralized in human AC. Thus, complex vocal communication processing shares common features among rodents and primates. We argue that a synthesis of results from humans, non-human primates, and rodents is necessary to identify the neural circuitry of vocal communication processing. However, data from different species and methods are often difficult to compare. Recent advances may enable better integration of methods across species. Efforts to standardise data formats and analysis tools would benefit comparative research and enable synergies between psychological and biological research in the area of vocal communication processing.

An Emergent Population Code in Primary Auditory Cortex Supports Selective Attention to Spectral and Temporal Sound Features

Textbook descriptions of primary sensory cortex (PSC) revolve around single neurons' representation of low-dimensional sensory features, such as visual object orientation in primary visual cortex (V1), location of somatic touch in primary somatosensory cortex (S1), and sound frequency in primary auditory cortex (A1). Typically, studies of PSC measure neurons' responses along few (one or two) stimulus and/or behavioral dimensions. However, real-world stimuli usually vary along many feature dimensions and behavioral demands change constantly. In order to illuminate how A1 supports flexible perception in rich acoustic environments, we recorded from A1 neurons while rhesus macaques (one male, one female) performed a feature-selective attention task. We presented sounds that varied along spectral and temporal feature dimensions (carrier bandwidth and temporal envelope, respectively). Within a block, subjects attended to one feature of the sound in a selective change detection task. We found that single neurons tend to be high-dimensional, in that they exhibit substantial mixed selectivity for both sound features, as well as task context. We found no overall enhancement of single-neuron coding of the attended feature, as attention could either diminish or enhance this coding. However, a population-level analysis reveals that ensembles of neurons exhibit enhanced encoding of attended sound features, and this population code tracks subjects' performance. Importantly, surrogate neural populations with intact single-neuron tuning but shuffled higher-order correlations among neurons fail to yield attention- related effects observed in the intact data. These results suggest that an emergent population code not measurable at the single-neuron level might constitute the functional unit of sensory representation in PSC.SIGNIFICANCE STATEMENT The ability to adapt to a dynamic sensory environment promotes a range of important natural behaviors. We recorded from single neurons in monkey primary auditory cortex (A1), while subjects attended to either the spectral or temporal features of complex sounds. Surprisingly, we found no average increase in responsiveness to, or encoding of, the attended feature across single neurons. However, when we pooled the activity of the sampled neurons via targeted dimensionality reduction (TDR), we found enhanced population-level representation of the attended feature and suppression of the distractor feature. This dissociation of the effects of attention at the level of single neurons versus the population highlights the synergistic nature of cortical sound encoding and enriches our understanding of sensory cortical function.

Neuronal selectivity to complex vocalization features emerges in the superficial layers of primary auditory cortex

Early in auditory processing, neural responses faithfully reflect acoustic input. At higher stages of auditory processing, however, neurons become selective for particular call types, eventually leading to specialized regions of cortex that preferentially process calls at the highest auditory processing stages. We previously proposed that an intermediate step in how nonselective responses are transformed into call-selective responses is the detection of informative call features. But how neural selectivity for informative call features emerges from nonselective inputs, whether feature selectivity gradually emerges over the processing hierarchy, and how stimulus information is represented in nonselective and feature-selective populations remain open question. In this study, using unanesthetized guinea pigs (GPs), a highly vocal and social rodent, as an animal model, we characterized the neural representation of calls in 3 auditory processing stages-the thalamus (ventral medial geniculate body (vMGB)), and thalamorecipient (L4) and superficial layers (L2/3) of primary auditory cortex (A1). We found that neurons in vMGB and A1 L4 did not exhibit call-selective responses and responded throughout the call durations. However, A1 L2/3 neurons showed high call selectivity with about a third of neurons responding to only 1 or 2 call types. These A1 L2/3 neurons only responded to restricted portions of calls suggesting that they were highly selective for call features. Receptive fields of these A1 L2/3 neurons showed complex spectrotemporal structures that could underlie their high call feature selectivity. Information theoretic analysis revealed that in A1 L4, stimulus information was distributed over the population and was spread out over the call durations. In contrast, in A1 L2/3, individual neurons showed brief bursts of high stimulus-specific information and conveyed high levels of information per spike. These data demonstrate that a transformation in the neural representation of calls occurs between A1 L4 and A1 L2/3, leading to the emergence of a feature-based representation of calls in A1 L2/3. Our data thus suggest that observed cortical specializations for call processing emerge in A1 and set the stage for further mechanistic studies.

Distinct patterns of activity in individual cortical neurons and local networks in primary somatosensory cortex of mice evoked by square-wave mechanical limb stimulation

Artificial forms of mechanical limb stimulation are used within multiple fields of study to determine the level of cortical excitability and to map the trajectory of neuronal recovery from cortical damage or disease. Square-wave mechanical or electrical stimuli are often used in these studies, but a characterization of sensory-evoked response properties to square-waves with distinct fundamental frequencies but overlapping harmonics has not been performed. To distinguish between somatic stimuli, the primary somatosensory cortex must be able to represent distinct stimuli with unique patterns of activity, even if they have overlapping features. Thus, mechanical square-wave stimulation was used in conjunction with regional and cellular imaging to examine regional and cellular response properties evoked by different frequencies of stimulation. Flavoprotein autofluorescence imaging was used to map the somatosensory cortex of anaesthetized C57BL/6 mice, and in vivo two-photon Ca2+ imaging was used to define patterns of neuronal activation during mechanical square-wave stimulation of the contralateral forelimb or hindlimb at various frequencies (3, 10, 100, 200, and 300 Hz). The data revealed that neurons within the limb associated somatosensory cortex responding to various frequencies of square-wave stimuli exhibit stimulus-specific patterns of activity. Subsets of neurons were found to have sensory-evoked activity that is either primarily responsive to single stimulus frequencies or broadly responsive to multiple frequencies of limb stimulation. High frequency stimuli were shown to elicit more population activity, with a greater percentage of the population responding and greater percentage of cells with high amplitude responses. Stimulus-evoked cell-cell correlations within these neuronal networks varied as a function of frequency of stimulation, such that each stimulus elicited a distinct pattern that was more consistent across multiple trials of the same stimulus compared to trials at different frequencies of stimulation. The variation in cortical response to different square-wave stimuli can thus be represented by the population pattern of supra-threshold Ca2+ transients, the magnitude and temporal properties of the evoked activity, and the structure of the stimulus-evoked correlation between neurons.

Recurrent network dynamics shape direction selectivity in primary auditory cortex

Detecting the direction of frequency modulation (FM) is essential for vocal communication in both animals and humans. Direction-selective firing of neurons in the primary auditory cortex (A1) has been classically attributed to temporal offsets between feedforward excitatory and inhibitory inputs. However, it remains unclear how cortical recurrent circuitry contributes to this computation. Here, we used two-photon calcium imaging and whole-cell recordings in awake mice to demonstrate that direction selectivity is not caused by temporal offsets between synaptic currents, but by an asymmetry in total synaptic charge between preferred and non-preferred directions. Inactivation of cortical somatostatin-expressing interneurons (SOM cells) reduced direction selectivity, revealing its cortical contribution. Our theoretical models showed that charge asymmetry arises due to broad spatial topography of SOM cell-mediated inhibition which regulates signal amplification in strongly recurrent circuitry. Together, our findings reveal a major contribution of recurrent network dynamics in shaping cortical tuning to behaviorally relevant complex sounds.

Complementary Effects of Adaptation and Gain Control on Sound Encoding in Primary Auditory Cortex

An important step toward understanding how the brain represents complex natural sounds is to develop accurate models of auditory coding by single neurons. A commonly used model is the linear-nonlinear spectro-temporal receptive field (STRF; LN model). The LN model accounts for many features of auditory tuning, but it cannot account for long-lasting effects of sensory context on sound-evoked activity. Two mechanisms that may support these contextual effects are short-term plasticity (STP) and contrast-dependent gain control (GC), which have inspired expanded versions of the LN model. Both models improve performance over the LN model, but they have never been compared directly. Thus, it is unclear whether they account for distinct processes or describe one phenomenon in different ways. To address this question, we recorded activity of neurons in primary auditory cortex (A1) of awake ferrets during presentation of natural sounds. We then fit models incorporating one nonlinear mechanism (GC or STP) or both (GC+STP) using this single dataset, and measured the correlation between the models' predictions and the recorded neural activity. Both the STP and GC models performed significantly better than the LN model, but the GC+STP model outperformed both individual models. We also quantified the equivalence of STP and GC model predictions and found only modest similarity. Consistent results were observed for a dataset collected in clean and noisy acoustic contexts. These results establish general methods for evaluating the equivalence of arbitrarily complex encoding models and suggest that the STP and GC models describe complementary processes in the auditory system.

Differential Short-Term Plasticity of PV and SST Neurons Accounts for Adaptation and Facilitation of Cortical Neurons to Auditory Tones

Cortical responses to sensory stimuli are strongly modulated by temporal context. One of the best studied examples of such modulation is sensory adaptation. We first show that in response to repeated tones pyramidal (Pyr) neurons in male mouse auditory cortex (A1) exhibit facilitating and stable responses, in addition to adapting responses. To examine the potential mechanisms underlying these distinct temporal profiles, we developed a reduced spiking model of sensory cortical circuits that incorporated the signature short-term synaptic plasticity (STP) profiles of the inhibitory parvalbumin (PV) and somatostatin (SST) interneurons. The model accounted for all three temporal response profiles as the result of dynamic changes in excitatory/inhibitory balance produced by STP, primarily through shifts in the relative latency of Pyr and inhibitory neurons. Transition between the three response profiles was possible by changing the strength of the inhibitory PV→Pyr and SST→Pyr synapses. The model predicted that a unit's latency would be related to its temporal profile. Consistent with this prediction, the latency of stable units was significantly shorter than that of adapting and facilitating units. Furthermore, because of the history-dependence of STP the model generated a paradoxical prediction: that inactivation of inhibitory neurons during one tone would decrease the response of A1 neurons to a subsequent tone. Indeed, we observed that optogenetic inactivation of PV neurons during one tone counterintuitively decreased the spiking of Pyr neurons to a subsequent tone 400 ms later. These results provide evidence that STP is critical to temporal context-dependent responses in the sensory cortex.SIGNIFICANCE STATEMENT Our perception of speech and music depends strongly on temporal context, i.e., the significance of a stimulus depends on the preceding stimuli. Complementary neural mechanisms are needed to sometimes ignore repetitive stimuli (e.g., the tic of a clock) or detect meaningful repetition (e.g., consecutive tones in Morse code). We modeled a neural circuit that accounts for diverse experimentally-observed response profiles in auditory cortex (A1) neurons, based on known forms of short-term synaptic plasticity (STP). Whether the simulated circuit reduced, maintained, or enhanced its response to repeated tones depended on the relative dominance of two different types of inhibitory cells. The model made novel predictions that were experimentally validated. Results define an important role for STP in temporal context-dependent perception.

Multivoxel codes for representing and integrating acoustic features in human cortex

Using fMRI and multivariate pattern analysis, we determined whether spectral and temporal acoustic features are represented by independent or integrated multivoxel codes in human cortex. Listeners heard band-pass noise varying in frequency (spectral) and amplitude-modulation (AM) rate (temporal) features. In the superior temporal plane, changes in multivoxel activity due to frequency were largely invariant with respect to AM rate (and vice versa), consistent with an independent representation. In contrast, in posterior parietal cortex, multivoxel representation was exclusively integrated and tuned to specific conjunctions of frequency and AM features (albeit weakly). Direct between-region comparisons show that whereas independent coding of frequency weakened with increasing levels of the hierarchy, such a progression for AM and integrated coding was less fine-grained and only evident in the higher hierarchical levels from non-core to parietal cortex (with AM coding weakening and integrated coding strengthening). Our findings support the notion that primary auditory cortex can represent spectral and temporal acoustic features in an independent fashion and suggest a role for parietal cortex in feature integration and the structuring of sensory input.

Navigating Through Time: A Spatial Navigation Perspective on How the Brain May Encode Time

Interval timing, which operates on timescales of seconds to minutes, is distributed across multiple brain regions and may use distinct circuit mechanisms as compared to millisecond timing and circadian rhythms. However, its study has proven difficult, as timing on this scale is deeply entangled with other behaviors. Several circuit and cellular mechanisms could generate sequential or ramping activity patterns that carry timing information. Here we propose that a productive approach is to draw parallels between interval timing and spatial navigation, where direct analogies can be made between the variables of interest and the mathematical operations necessitated. Along with designing experiments that isolate or disambiguate timing behavior from other variables, new techniques will facilitate studies that directly address the neural mechanisms that are responsible for interval timing.

Estimating and interpreting nonlinear receptive field of sensory neural responses with deep neural network models

Our understanding of nonlinear stimulus transformations by neural circuits is hindered by the lack of comprehensive yet interpretable computational modeling frameworks. Here, we propose a data-driven approach based on deep neural networks to directly model arbitrarily nonlinear stimulus-response mappings. Reformulating the exact function of a trained neural network as a collection of stimulus-dependent linear functions enables a locally linear receptive field interpretation of the neural network. Predicting the neural responses recorded invasively from the auditory cortex of neurosurgical patients as they listened to speech, this approach significantly improves the prediction accuracy of auditory cortical responses, particularly in nonprimary areas. Moreover, interpreting the functions learned by neural networks uncovered three distinct types of nonlinear transformations of speech that varied considerably from primary to nonprimary auditory regions. The ability of this framework to capture arbitrary stimulus-response mappings while maintaining model interpretability leads to a better understanding of cortical processing of sensory signals.

Complexity of frequency receptive fields predicts tonotopic variability across species

Primary cortical areas contain maps of sensory features, including sound frequency in primary auditory cortex (A1). Two-photon calcium imaging in mice has confirmed the presence of these global tonotopic maps, while uncovering an unexpected local variability in the stimulus preferences of individual neurons in A1 and other primary regions. Here we show that local heterogeneity of frequency preferences is not unique to rodents. Using two-photon calcium imaging in layers 2/3, we found that local variance in frequency preferences is equivalent in ferrets and mice. Neurons with multipeaked frequency tuning are less spatially organized than those tuned to a single frequency in both species. Furthermore, we show that microelectrode recordings may describe a smoother tonotopic arrangement due to a sampling bias towards neurons with simple frequency tuning. These results help explain previous inconsistencies in cortical topography across species and recording techniques.

The Neural Basis of Timing: Distributed Mechanisms for Diverse Functions

Timing is critical to most forms of learning, behavior, and sensory-motor processing. Converging evidence supports the notion that, precisely because of its importance across a wide range of brain functions, timing relies on intrinsic and general properties of neurons and neural circuits; that is, the brain uses its natural cellular and network dynamics to solve a diversity of temporal computations. Many circuits have been shown to encode elapsed time in dynamically changing patterns of neural activity-so-called population clocks. But temporal processing encompasses a wide range of different computations, and just as there are different circuits and mechanisms underlying computations about space, there are a multitude of circuits and mechanisms underlying the ability to tell time and generate temporal patterns.

Elementary motion sequence detectors in whisker somatosensory cortex

How somatosensory cortex (S1) encodes complex patterns of touch, as occur during tactile exploration, is poorly understood. In mouse whisker S1, temporally dense stimulation of local whisker pairs revealed that most neurons are not classical single-whisker feature detectors, but instead are strongly tuned to 2-whisker sequences involving the columnar whisker (CW) and one, specific surround whisker (SW), usually in SW-leading-CW order. Tuning was spatiotemporally precise and diverse across cells, generating a rate code for local motion vectors defined by SW-CW combinations. Spatially asymmetric, sublinear suppression for suboptimal combinations and near-linearity for preferred combinations sharpened combination tuning relative to linearly predicted tuning. This resembles computation of motion direction selectivity in vision. SW-tuned neurons, misplaced in the classical whisker map, had the strongest combination tuning. Thus, each S1 column contains a rate code for local motion sequences involving the CW, providing a basis for higher-order feature extraction.

Multisensory perceptual interactions between higher-order temporal frequency signals

Naturally occurring signals in audition and touch can be complex and marked by temporal variations in frequency and amplitude. Auditory frequency sweep processing has been studied extensively; however, much less is known about sweep processing in touch because studies have primarily focused on the perception of simple sinusoidal vibrations. Given the extensive interactions between audition and touch in the frequency processing of pure tone signals, we reasoned that these senses might also interact in the processing of higher-order frequency representations like sweeps. In a series of psychophysical experiments, we characterized the influence of auditory distractors on the ability of participants to discriminate tactile frequency sweeps. Auditory frequency sweeps systematically biased the tactile perception of sweep direction. Importantly, auditory cues exerted little influence on tactile sweep direction perception when the sounds and vibrations occupied different absolute frequency ranges or when the sounds consisted of intensity sweeps. Thus, audition and touch interact in frequency sweep perception in a frequency- and feature-specific manner. Our results demonstrate that audio-tactile interactions are not constrained to the processing of simple sinusoids. Because higher-order frequency representations may be synthesized from simpler representations, our findings imply that multisensory interactions in the temporal frequency domain span multiple hierarchical levels in sensory processing. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Binaural unmasking of the accuracy of envelope-signal representation in rat auditory cortex but not auditory midbrain

Accurate neural representations of acoustic signals under noisy conditions are critical for animals' survival. Detecting signal against background noise can be improved by binaural hearing particularly when an interaural-time-difference (ITD) disparity is introduced between the signal and the noise, a phenomenon known as binaural unmasking. Previous studies have mainly focused on the binaural unmasking effect on response magnitudes, and it is not clear whether binaural unmasking affects the accuracy of central representations of target acoustic signals and the relative contributions of different central auditory structures to this accuracy. Frequency following responses (FFRs), which are sustained phase-locked neural activities, can be used for measuring the accuracy of the representation of signals. Using intracranial recordings of local field potentials, this study aimed to assess whether the binaural unmasking effects include an improvement of the accuracy of neural representations of sound-envelope signals in the rat IC and/or auditory cortex (AC). The results showed that (1) when a narrow-band noise was presented binaurally, the stimulus-response (S-R) coherence of the FFRs to the envelope (FFR_envelope) of the narrow-band noise recorded in the IC was higher than that recorded in the AC. (2) Presenting a broad-band masking noise caused a larger reduction of the S-R coherence for FFR_envelope in the IC than that in the AC. (3) Introducing an ITD disparity between the narrow-band signal noise and the broad-band masking noise did not affect the IC S-R coherence, but enhanced both the AC S-R coherence and the coherence between the IC FFR_envelope and AC FFR_envelope. Thus, although the accuracy of representing envelope signals in the AC is lower than that in the IC, it can be binaurally unmasked, indicating a binaural-unmasking mechanism that is formed during the signal transmission from the IC to the AC.

Processing complexity increases in superficial layers of human primary auditory cortex

The layers of the neocortex each have a unique anatomical connectivity and functional role. Their exploration in the human brain, however, has been severely restricted by the limited spatial resolution of non-invasive measurement techniques. Here, we exploit the sensitivity and specificity of ultra-high field fMRI at 7 Tesla to investigate responses to natural sounds at deep, middle, and superficial cortical depths of the human auditory cortex. Specifically, we compare the performance of computational models that represent different hypotheses on sound processing inside and outside the primary auditory cortex (PAC). We observe that while BOLD responses in deep and middle PAC layers are equally well represented by a simple frequency model and a more complex spectrotemporal modulation model, responses in superficial PAC are better represented by the more complex model. This indicates an increase in processing complexity in superficial PAC, which remains present throughout cortical depths in the non-primary auditory cortex. These results suggest that a relevant transformation in sound processing takes place between the thalamo-recipient middle PAC layers and superficial PAC. This transformation may be a first computational step towards sound abstraction and perception, serving to form an increasingly more complex representation of the physical input.

Optimal features for auditory categorization

Humans and vocal animals use vocalizations to communicate with members of their species. A necessary function of auditory perception is to generalize across the high variability inherent in vocalization production and classify them into behaviorally distinct categories ('words' or 'call types'). Here, we demonstrate that detecting mid-level features in calls achieves production-invariant classification. Starting from randomly chosen marmoset call features, we use a greedy search algorithm to determine the most informative and least redundant features necessary for call classification. High classification performance is achieved using only 10-20 features per call type. Predictions of tuning properties of putative feature-selective neurons accurately match some observed auditory cortical responses. This feature-based approach also succeeds for call categorization in other species, and for other complex classification tasks such as caller identification. Our results suggest that high-level neural representations of sounds are based on task-dependent features optimized for specific computational goals.

Mice can learn phonetic categories

Speech is perceived as a series of relatively invariant phonemes despite extreme variability in the acoustic signal. To be perceived as nearly-identical phonemes, speech sounds that vary continuously over a range of acoustic parameters must be perceptually discretized by the auditory system. Such many-to-one mappings of undifferentiated sensory information to a finite number of discrete categories are ubiquitous in perception. Although many mechanistic models of phonetic perception have been proposed, they remain largely unconstrained by neurobiological data. Current human neurophysiological methods lack the necessary spatiotemporal resolution to provide it: speech is too fast, and the neural circuitry involved is too small. This study demonstrates that mice are capable of learning generalizable phonetic categories, and can thus serve as a model for phonetic perception. Mice learned to discriminate consonants and generalized consonant identity across novel vowel contexts and speakers, consistent with true category learning. A mouse model, given the powerful genetic and electrophysiological tools for probing neural circuits available for them, has the potential to powerfully augment a mechanistic understanding of phonetic perception.

Neural processes of vocal social perception: Dog-human comparative fMRI studies

In this review we focus on the exciting new opportunities in comparative neuroscience to study neural processes of vocal social perception by comparing dog and human neural activity using fMRI methods. The dog is a relatively new addition to this research area; however, it has a large potential to become a standard species in such investigations. Although there has been great interest in the emergence of human language abilities, in case of fMRI methods, most research to date focused on homologue comparisons within Primates. By belonging to a very different clade of mammalian evolution, dogs could give such research agendas a more general mammalian foundation. In addition, broadening the scope of investigations into vocal communication in general can also deepen our understanding of human vocal skills. Being selected for and living in an anthropogenic environment, research with dogs may also be informative about the way in which human non-linguistic and linguistic signals are represented in a mammalian brain without skills for language production.

Parallel pathways for sound processing and functional connectivity among layer 5 and 6 auditory corticofugal neurons

Cortical layers (L) 5 and 6 are populated by intermingled cell-types with distinct inputs and downstream targets. Here, we made optogenetically guided recordings from L5 corticofugal (CF) and L6 corticothalamic (CT) neurons in the auditory cortex of awake mice to discern differences in sensory processing and underlying patterns of functional connectivity. Whereas L5 CF neurons showed broad stimulus selectivity with sluggish response latencies and extended temporal non-linearities, L6 CTs exhibited sparse selectivity and rapid temporal processing. L5 CF spikes lagged behind neighboring units and imposed weak feedforward excitation within the local column. By contrast, L6 CT spikes drove robust and sustained activity, particularly in local fast-spiking interneurons. Our findings underscore a duality among sub-cortical projection neurons, where L5 CF units are canonical broadcast neurons that integrate sensory inputs for transmission to distributed downstream targets, while L6 CT neurons are positioned to regulate thalamocortical response gain and selectivity.

Response properties of single neurons in higher level auditory cortex of adult songbirds

The caudomedial nidopallium (NCM) is a higher level region of auditory cortex in songbirds that has been implicated in encoding learned vocalizations and mediating perception of complex sounds. We made cell-attached recordings in awake adult male zebra finches ( Taeniopygia guttata) to characterize responses of single NCM neurons to playback of tones and songs. Neurons fell into two broad classes: narrow fast-spiking cells and broad sparsely firing cells. Virtually all narrow-spiking cells responded to playback of pure tones, compared with approximately half of broad-spiking cells. In addition, narrow-spiking cells tended to have lower thresholds and faster, less variable spike onset latencies than did broad-spiking cells, as well as higher firing rates. Tonal responses of narrow-spiking cells also showed broader ranges for both frequency and amplitude compared with broad-spiking neurons and were more apt to have V-shaped tuning curves compared with broad-spiking neurons, which tended to have complex (discontinuous), columnar, or O-shaped frequency response areas. In response to playback of conspecific songs, narrow-spiking neurons showed high firing rates and low levels of selectivity whereas broad-spiking neurons responded sparsely and selectively. Broad-spiking neurons in which tones failed to evoke a response showed greater song selectivity compared with those with a clear tuning curve. These results are consistent with the idea that narrow-spiking neurons represent putative fast-spiking interneurons, which may provide a source of intrinsic inhibition that contributes to the more selective tuning in broad-spiking cells. NEW & NOTEWORTHY The response properties of neurons in higher level regions of auditory cortex in songbirds are of fundamental interest because processing in such regions is essential for vocal learning and plasticity and for auditory perception of complex sounds. Within a region of secondary auditory cortex, neurons with narrow spikes exhibited high firing rates to playback of both tones and multiple conspecific songs, whereas broad-spiking neurons responded sparsely and selectively to both tones and songs.

Neural responses to natural and model-matched stimuli reveal distinct computations in primary and nonprimary auditory cortex

A central goal of sensory neuroscience is to construct models that can explain neural responses to natural stimuli. As a consequence, sensory models are often tested by comparing neural responses to natural stimuli with model responses to those stimuli. One challenge is that distinct model features are often correlated across natural stimuli, and thus model features can predict neural responses even if they do not in fact drive them. Here, we propose a simple alternative for testing a sensory model: we synthesize a stimulus that yields the same model response as each of a set of natural stimuli, and test whether the natural and "model-matched" stimuli elicit the same neural responses. We used this approach to test whether a common model of auditory cortex-in which spectrogram-like peripheral input is processed by linear spectrotemporal filters-can explain fMRI responses in humans to natural sounds. Prior studies have that shown that this model has good predictive power throughout auditory cortex, but this finding could reflect feature correlations in natural stimuli. We observed that fMRI responses to natural and model-matched stimuli were nearly equivalent in primary auditory cortex (PAC) but that nonprimary regions, including those selective for music or speech, showed highly divergent responses to the two sound sets. This dissociation between primary and nonprimary regions was less clear from model predictions due to the influence of feature correlations across natural stimuli. Our results provide a signature of hierarchical organization in human auditory cortex, and suggest that nonprimary regions compute higher-order stimulus properties that are not well captured by traditional models. Our methodology enables stronger tests of sensory models and could be broadly applied in other domains.

Distributed representation of vocalization pitch in marmoset primary auditory cortex

The pitch of vocalizations is a key communication feature aiding recognition of individuals and separating sound sources in complex acoustic environments. The neural representation of the pitch of periodic sounds is well defined. However, many natural sounds, like complex vocalizations, contain rich, aperiodic or not strictly periodic frequency content and/or include high-frequency components, but still evoke a strong sense of pitch. Indeed, such sounds are the rule, not the exception but the cortical mechanisms for encoding pitch of such sounds are unknown. We investigated how neurons in the high-frequency representation of primary auditory cortex (A1) of marmosets encoded changes in pitch of four natural vocalizations, two centred around a dominant frequency similar to the neuron's best sensitivity and two around a much lower dominant frequency. Pitch was varied over a fine range that can be used by marmosets to differentiate individuals. The responses of most high-frequency A1 neurons were sensitive to pitch changes in all four vocalizations, with a smaller proportion of the neurons showing pitch-insensitive responses. Classically defined excitatory drive, from the neuron's monaural frequency response area, predicted responses to changes in vocalization pitch in <30% of neurons suggesting most pitch tuning observed is not simple frequency-level response. Moreover, 39% of A1 neurons showed call-invariant tuning of pitch. These results suggest that distributed activity across A1 can represent the pitch of natural sounds over a fine, functionally relevant range, and exhibits pitch tuning for vocalizations within and outside the classical neural tuning area.

Changes in Neuronal Representations of Consonants in the Ascending Auditory System and Their Role in Speech Recognition

A fundamental task of the ascending auditory system is to produce representations that facilitate the recognition of complex sounds. This is particularly challenging in the context of acoustic variability, such as that between different talkers producing the same phoneme. These representations are transformed as information is propagated throughout the ascending auditory system from the inner ear to the auditory cortex (AI). Investigating these transformations and their role in speech recognition is key to understanding hearing impairment and the development of future clinical interventions. Here, we obtained neural responses to an extensive set of natural vowel-consonant-vowel phoneme sequences, each produced by multiple talkers, in three stages of the auditory processing pathway. Auditory nerve (AN) representations were simulated using a model of the peripheral auditory system and extracellular neuronal activity was recorded in the inferior colliculus (IC) and primary auditory cortex (AI) of anaesthetized guinea pigs. A classifier was developed to examine the efficacy of these representations for recognizing the speech sounds. Individual neurons convey progressively less information from AN to AI. Nonetheless, at the population level, representations are sufficiently rich to facilitate recognition of consonants with a high degree of accuracy at all stages indicating a progression from a dense, redundant representation to a sparse, distributed one. We examined the timescale of the neural code for consonant recognition and found that optimal timescales increase throughout the ascending auditory system from a few milliseconds in the periphery to several tens of milliseconds in the cortex. Despite these longer timescales, we found little evidence to suggest that representations up to the level of AI become increasingly invariant to across-talker differences. Instead, our results support the idea that the role of the subcortical auditory system is one of dimensionality expansion, which could provide a basis for flexible classification of arbitrary speech sounds.

Describing complex cells in primary visual cortex: a comparison of context and multifilter LN models

Receptive field (RF) models are an important tool for deciphering neural responses to sensory stimuli. The two currently popular RF models are multifilter linear-nonlinear (LN) models and context models. Models are, however, never correct, and they rely on assumptions to keep them simple enough to be interpretable. As a consequence, different models describe different stimulus-response mappings, which may or may not be good approximations of real neural behavior. In the current study, we take up two tasks: 1) we introduce new ways to estimate context models with realistic nonlinearities, that is, with logistic and exponential functions, and 2) we evaluate context models and multifilter LN models in terms of how well they describe recorded data from complex cells in cat primary visual cortex. Our results, based on single-spike information and correlation coefficients, indicate that context models outperform corresponding multifilter LN models of equal complexity (measured in terms of number of parameters), with the best increase in performance being achieved by the novel context models. Consequently, our results suggest that the multifilter LN-model framework is suboptimal for describing the behavior of complex cells: the context-model framework is clearly superior while still providing interpretable quantizations of neural behavior. NEW & NOTEWORTHY We used data from complex cells in primary visual cortex to estimate a wide variety of receptive field models from two frameworks that have previously not been compared with each other. The models included traditionally used multifilter linear-nonlinear models and novel variants of context models. Using mutual information and correlation coefficients as performance measures, we showed that context models are superior for describing complex cells and that the novel context models performed the best.

Cortical Coding of Auditory Features

How the cerebral cortex encodes auditory features of biologically important sounds, including speech and music, is one of the most important questions in auditory neuroscience. The pursuit to understand related neural coding mechanisms in the mammalian auditory cortex can be traced back several decades to the early exploration of the cerebral cortex. Significant progress in this field has been made in the past two decades with new technical and conceptual advances. This article reviews the progress and challenges in this area of research.

Processing of temporally patterned sounds in the auditory cortex of Seba's short-tailed bat,Carollia perspicillata

This article presents a characterization of cortical responses to artificial and natural temporally patterned sounds in the bat species Carollia perspicillata, a species that produces vocalizations at rates above 50 Hz. Multi-unit activity was recorded in three different experiments. In the first experiment, amplitude-modulated (AM) pure tones were used as stimuli to drive auditory cortex (AC) units. AC units of both ketamine-anesthetized and awake bats could lock their spikes to every cycle of the stimulus modulation envelope, but only if the modulation frequency was below 22 Hz. In the second experiment, two identical communication syllables were presented at variable intervals. Suppressed responses to the lagging syllable were observed, unless the second syllable followed the first one with a delay of at least 80 ms (i.e., 12.5 Hz repetition rate). In the third experiment, natural distress vocalization sequences were used as stimuli to drive AC units. Distress sequences produced by C. perspicillata contain bouts of syllables repeated at intervals of ~60 ms (16 Hz). Within each bout, syllables are repeated at intervals as short as 14 ms (~71 Hz). Cortical units could follow the slow temporal modulation flow produced by the occurrence of multisyllabic bouts, but not the fast acoustic flow created by rapid syllable repetition within the bouts. Taken together, our results indicate that even in fast vocalizing animals, such as bats, cortical neurons can only track the temporal structure of acoustic streams modulated at frequencies lower than 22 Hz.

A Computational Account of the Role of Cochlear Nucleus and Inferior Colliculus in Stabilizing Auditory Nerve Firing for Auditory Category Learning

It is well known that auditory nerve (AN) fibers overcome bandwidth limitations through the volley principle, a form of multiplexing. What is less well known is that the volley principle introduces a degree of unpredictability into AN neural firing patterns that may be affecting even simple stimulus categorization learning. We use a physiologically grounded, unsupervised spiking neural network model of the auditory brain with spike time dependent plasticity learning to demonstrate that plastic auditory cortex is unable to learn even simple auditory object categories when exposed to the raw AN firing input without subcortical preprocessing. We then demonstrate the importance of nonplastic subcortical preprocessing within the cochlear nucleus and the inferior colliculus for stabilizing and denoising AN responses. Such preprocessing enables the plastic auditory cortex to learn efficient robust representations of the auditory object categories. The biological realism of our model makes it suitable for generating neurophysiologically testable hypotheses.

Reconstructing Tone Sequences from Functional Magnetic Resonance Imaging Blood-Oxygen Level Dependent Responses within Human Primary Auditory Cortex

Here we show that, using functional magnetic resonance imaging (fMRI) blood-oxygen level dependent (BOLD) responses in human primary auditory cortex, it is possible to reconstruct the sequence of tones that a person has been listening to over time. First, we characterized the tonotopic organization of each subject’s auditory cortex by measuring auditory responses to randomized pure tone stimuli and modeling the frequency tuning of each fMRI voxel as a Gaussian in log frequency space. Then, we tested our model by examining its ability to work in reverse. Auditory responses were re-collected in the same subjects, except this time they listened to sequences of frequencies taken from simple songs (e.g., “Somewhere Over the Rainbow”). By finding the frequency that minimized the difference between the model’s prediction of BOLD responses and actual BOLD responses, we were able to reconstruct tone sequences, with mean frequency estimation errors of half an octave or less, and little evidence of systematic biases.

Processing of Natural Echolocation Sequences in the Inferior Colliculus of Seba's Fruit Eating Bat, Carollia perspicillata

For the purpose of orientation, echolocating bats emit highly repetitive and spatially directed sonar calls. Echoes arising from call reflections are used to create an acoustic image of the environment. The inferior colliculus (IC) represents an important auditory stage for initial processing of echolocation signals. The present study addresses the following questions: (1) how does the temporal context of an echolocation sequence mimicking an approach flight of an animal affect neuronal processing of distance information to echo delays? (2) how does the IC process complex echolocation sequences containing echo information from multiple objects (multiobject sequence)? Here, we conducted neurophysiological recordings from the IC of ketamine-anaesthetized bats of the species Carollia perspicillata and compared the results from the IC with the ones from the auditory cortex (AC). Neuronal responses to an echolocation sequence was suppressed when compared to the responses to temporally isolated and randomized segments of the sequence. The neuronal suppression was weaker in the IC than in the AC. In contrast to the cortex, the time course of the acoustic events is reflected by IC activity. In the IC, suppression sharpens the neuronal tuning to specific call-echo elements and increases the signal-to-noise ratio in the units' responses. When presenting multiple-object sequences, despite collicular suppression, the neurons responded to each object-specific echo. The latter allows parallel processing of multiple echolocation streams at the IC level. Altogether, our data suggests that temporally-precise neuronal responses in the IC could allow fast and parallel processing of multiple acoustic streams.

Functional magnetic resonance imaging of auditory cortical fields in awake marmosets

The primate auditory cortex is organized into a network of anatomically and functionally distinct processing fields. Because of its tonotopic properties, the auditory core has been the main target of neurophysiological studies ranging from sensory encoding to perceptual decision-making. By comparison, the auditory belt has been less extensively studied, in part due to the fact that neurons in the belt areas prefer more complex stimuli and integrate over a wider frequency range than neurons in the core, which prefer pure tones of a single frequency. Complementary approaches, such as functional magnetic resonance imaging (fMRI), allow the anatomical identification of both the auditory core and belt and facilitate their functional characterization by rapidly testing a range of stimuli across multiple brain areas simultaneously that can be used to guide subsequent neural recordings. Bridging these technologies in primates will serve to further expand our understanding of primate audition. Here, we developed a novel preparation to test whether different areas of the auditory cortex could be identified using fMRI in common marmosets (Callithrix jacchus), a powerful model of the primate auditory system. We used two types of stimulation, band pass noise and pure tones, to parse apart the auditory core from surrounding secondary belt fields. In contrast to most auditory fMRI experiments in primates, we employed a continuous sampling paradigm to rapidly collect data with little deleterious effects. Here we found robust bilateral auditory cortex activation in two marmosets and unilateral activation in a third utilizing this preparation. Furthermore, we confirmed results previously reported in electrophysiology experiments, such as the tonotopic organization of the auditory core and regions activating preferentially to complex over simple stimuli. Overall, these data establish a key preparation for future research to investigate various functional properties of marmoset auditory cortex.

Harmonic template neurons in primate auditory cortex underlying complex sound processing

Harmonicity is a fundamental element of music, speech, and animal vocalizations. How the auditory system extracts harmonic structures embedded in complex sounds and uses them to form a coherent unitary entity is not fully understood. Despite the prevalence of sounds rich in harmonic structures in our everyday hearing environment, it has remained largely unknown what neural mechanisms are used by the primate auditory cortex to extract these biologically important acoustic structures. In this study, we discovered a unique class of harmonic template neurons in the core region of auditory cortex of a highly vocal New World primate, the common marmoset (Callithrix jacchus), across the entire hearing frequency range. Marmosets have a rich vocal repertoire and a similar hearing range to that of humans. Responses of these neurons show nonlinear facilitation to harmonic complex sounds over inharmonic sounds, selectivity for particular harmonic structures beyond two-tone combinations, and sensitivity to harmonic number and spectral regularity. Our findings suggest that the harmonic template neurons in auditory cortex may play an important role in processing sounds with harmonic structures, such as animal vocalizations, human speech, and music.

Multiscale mapping of frequency sweep rate in mouse auditory cortex

Functional organization is a key feature of the neocortex that often guides studies of sensory processing, development, and plasticity. Tonotopy, which arises from the transduction properties of the cochlea, is the most widely studied organizational feature in auditory cortex; however, in order to process complex sounds, cortical regions are likely specialized for higher order features. Here, motivated by the prevalence of frequency modulations in mouse ultrasonic vocalizations and aided by the use of a multiscale imaging approach, we uncover a functional organization across the extent of auditory cortex for the rate of frequency modulated (FM) sweeps. In particular, using two-photon Ca²⁺ imaging of layer 2/3 neurons, we identify a tone-insensitive region at the border of AI and AAF. This central sweep region behaves fundamentally differently from nearby neurons in AI and AII, responding preferentially to fast FM sweeps but not to tones or bandlimited noise. Together these findings define a second dimension of organization in the mouse auditory cortex for sweep rate complementary to that of tone frequency.

Selective Neuronal Activation by Cochlear Implant Stimulation in Auditory Cortex of Awake Primate

Despite the success of cochlear implants (CIs) in human populations, most users perform poorly in noisy environments and music and tonal language perception. How CI devices engage the brain at the single neuron level has remained largely unknown, in particular in the primate brain. By comparing neuronal responses with acoustic and CI stimulation in marmoset monkeys unilaterally implanted with a CI electrode array, we discovered that CI stimulation was surprisingly ineffective at activating many neurons in auditory cortex, particularly in the hemisphere ipsilateral to the CI. Further analyses revealed that the CI-nonresponsive neurons were narrowly tuned to frequency and sound level when probed with acoustic stimuli; such neurons likely play a role in perceptual behaviors requiring fine frequency and level discrimination, tasks that CI users find especially challenging. These findings suggest potential deficits in central auditory processing of CI stimulation and provide important insights into factors responsible for poor CI user performance in a wide range of perceptual tasks.

SIGNIFICANCE STATEMENT

The cochlear implant (CI) is the most successful neural prosthetic device to date and has restored hearing in hundreds of thousands of deaf individuals worldwide. However, despite its huge successes, CI users still face many perceptual limitations, and the brain mechanisms involved in hearing through CI devices remain poorly understood. By directly comparing single-neuron responses to acoustic and CI stimulation in auditory cortex of awake marmoset monkeys, we discovered that neurons unresponsive to CI stimulation were sharply tuned to frequency and sound level. Our results point out a major deficit in central auditory processing of CI stimulation and provide important insights into mechanisms underlying the poor CI user performance in a wide range of perceptual tasks.

Capturing contextual effects in spectro-temporal receptive fields

Spectro-temporal receptive fields (STRFs) are thought to provide descriptive images of the computations performed by neurons along the auditory pathway. However, their validity can be questioned because they rely on a set of assumptions that are probably not fulfilled by real neurons exhibiting contextual effects, that is, nonlinear interactions in the time or frequency dimension that cannot be described with a linear filter. We used a novel approach to investigate how a variety of contextual effects, due to facilitating nonlinear interactions and synaptic depression, affect different STRF models, and if these effects can be captured with a context field (CF). Contextual effects were incorporated in simulated networks of spiking neurons, allowing one to define the true STRFs of the neurons. This, in turn, made it possible to evaluate the performance of each STRF model by comparing the estimations with the true STRFs. We found that currently used STRF models are particularly poor at estimating inhibitory regions. Specifically, contextual effects make estimated STRFs dependent on stimulus density in a contrasting fashion: inhibitory regions are underestimated at lower densities while artificial inhibitory regions emerge at higher densities. The CF was found to provide a solution to this dilemma, but only when it is used together with a generalized linear model. Our results therefore highlight the limitations of the traditional STRF approach and provide useful recipes for how different STRF models and stimuli can be used to arrive at reliable quantifications of neural computations in the presence of contextual effects. The results therefore push the purpose of STRF analysis from simply finding an optimal stimulus toward describing context-dependent computations of neurons along the auditory pathway.

Interactions across Multiple Stimulus Dimensions in Primary Auditory Cortex

Although sensory cortex is thought to be important for the perception of complex objects, its specific role in representing complex stimuli remains unknown. Complex objects are rich in information along multiple stimulus dimensions. The position of cortex in the sensory hierarchy suggests that cortical neurons may integrate across these dimensions to form a more gestalt representation of auditory objects. Yet, studies of cortical neurons typically explore single or few dimensions due to the difficulty of determining optimal stimuli in a high dimensional stimulus space. Evolutionary algorithms (EAs) provide a potentially powerful approach for exploring multidimensional stimulus spaces based on real-time spike feedback, but two important issues arise in their application. First, it is unclear whether it is necessary to characterize cortical responses to multidimensional stimuli or whether it suffices to characterize cortical responses to a single dimension at a time. Second, quantitative methods for analyzing complex multidimensional data from an EA are lacking. Here, we apply a statistical method for nonlinear regression, the generalized additive model (GAM), to address these issues. The GAM quantitatively describes the dependence between neural response and all stimulus dimensions. We find that auditory cortical neurons in mice are sensitive to interactions across dimensions. These interactions are diverse across the population, indicating significant integration across stimulus dimensions in auditory cortex. This result strongly motivates using multidimensional stimuli in auditory cortex. Together, the EA and the GAM provide a novel quantitative paradigm for investigating neural coding of complex multidimensional stimuli in auditory and other sensory cortices.

Input-Specific Gain Modulation by Local Sensory Context Shapes Cortical and Thalamic Responses to Complex Sounds

Sensory neurons are customarily characterized by one or more linearly weighted receptive fields describing sensitivity in sensory space and time. We show that in auditory cortical and thalamic neurons, the weight of each receptive field element depends on the pattern of sound falling within a local neighborhood surrounding it in time and frequency. Accounting for this change in effective receptive field with spectrotemporal context improves predictions of both cortical and thalamic responses to stationary complex sounds. Although context dependence varies among neurons and across brain areas, there are strong shared qualitative characteristics. In a spectrotemporally rich soundscape, sound elements modulate neuronal responsiveness more effectively when they coincide with sounds at other frequencies, and less effectively when they are preceded by sounds at similar frequencies. This local-context-driven lability in the representation of complex sounds—a modulation of “input-specific gain” rather than “output gain”—may be a widespread motif in sensory processing.

•

Gain of neuronal responses to sound components varies with immediate acoustic context

•

“Contextual gain fields” can be estimated from neuronal responses to complex sounds

•

Coincident sound at different frequencies boosts gain in cortex and thalamus

•

Preceding sound at similar frequency reduces gain for longer in cortex than thalamus

Music in Research and Rehabilitation of Disorders of Consciousness: Psychological and Neurophysiological Foundations

According to a prevailing view, the visual system works by dissecting stimuli into primitives, whereas the auditory system processes simple and complex stimuli with their corresponding features in parallel. This makes musical stimulation particularly suitable for patients with disorders of consciousness (DoC), because the processing pathways related to complex stimulus features can be preserved even when those related to simple features are no longer available. An additional factor speaking in favor of musical stimulation in DoC is the low efficiency of visual stimulation due to prevalent maladies of vision or gaze fixation in DoC patients. Hearing disorders, in contrast, are much less frequent in DoC, which allows us to use auditory stimulation at various levels of complexity. The current paper overviews empirical data concerning the four main domains of brain functioning in DoC patients that musical stimulation can address: perception (e.g., pitch, timbre, and harmony), cognition (e.g., musical syntax and meaning), emotions, and motor functions. Music can approach basic levels of patients' self-consciousness, which may even exist when all higher-level cognitions are lost, whereas music induced emotions and rhythmic stimulation can affect the dopaminergic reward-system and activity in the motor system respectively, thus serving as a starting point for rehabilitation.