Neural mechanisms of auditory categorization: from across brain areas to within local microcircuits

Spatially clustered neurons encode vocalization categories in the bat midbrain

Rapid categorization of vocalizations enables adaptive behavior across species. While categorical perception is thought to arise in the neocortex, humans and other animals could benefit from functional organization of ethologically-relevant sounds at earlier stages in the auditory hierarchy. Here, we developed two-photon calcium imaging in the awake echolocating bat (Eptesicus fuscus) to study encoding of sound meaning in the Inferior Colliculus, which is as few as two synapses from the inner ear. Echolocating bats produce and interpret frequency sweep-based vocalizations for social communication and navigation. Auditory playback experiments demonstrated that individual neurons responded selectively to social or navigation calls, enabling robust population-level decoding across categories. Strikingly, category-selective neurons formed spatial clusters, independent of tonotopy within the IC. These findings support a revised view of categorical processing in which specified channels for ethologically-relevant sounds are spatially segregated early in the auditory hierarchy, enabling rapid subcortical organization of call meaning.

Sparse ensemble neural code for a complete vocal repertoire

The categorization of animal vocalizations into distinct behaviorally relevant groups for communication is an essential operation that must be performed by the auditory system. This auditory object recognition is a difficult task that requires selectivity to the group identifying acoustic features and invariance to renditions within each group. We find that small ensembles of auditory neurons in the forebrain of a social songbird can code the bird's entire vocal repertoire (∼10 call types). Ensemble neural discrimination is not, however, correlated with single unit selectivity, but instead with how well the joint single unit tunings to characteristic spectro-temporal modulations span the acoustic subspace optimized for the discrimination of call types. Thus, akin to face recognition in the visual system, call type recognition in the auditory system is based on a sparse code representing a small number of high-level features and not on highly selective grandmother neurons.

The mental representation of occupational stereotypes is driven as much by their affective as by their semantic content

BACKGROUND

Studies on person perception showed that stereotypes can be activated by presenting either characteristic traits of group members, or labels associated to these groups. However, it is not clear whether these pieces of semantic information activate negative and positive stereotypes directly, or via an indirect cognitive pathway leading through brain regions responsible for affective responses. Our main objective with this study was to disentangle the effects of semantic and affective contents. To this end, we intended to scrutinize whether the representation of occupational labels is independent of the emotions they evoke.

METHODS

Participants (N = 73, M = 27.0, SD = 9.1, 31 men 42 women,) were asked to complete two tasks presented online. In the first task they had to arrange 20 occupational labels-randomly chosen from a pool of 60 items-in a two-dimensional space, moving the mouse pointer along two undefined axes. In a second task the axes' names were defined a priori. Subjects were asked to arrange the labels according to valence, the extent to which the word evoked pleasant or unpleasant feelings, and arousal, the extent to which the word evoked excitement or calmness.

RESULTS

Based on the final coordinates of the labels, two cluster analyses were carried out separately in the two tasks. The two clusters were compared with Fisher's exact test, which revealed that the cluster structures overlap significantly.

CONCLUSIONS

The results suggest that the spontaneous categorization and the semantic representation of occupations rely largely on the affective state they evoke. We propose that affective content might have a primacy over detailed semantic information in many aspects of person perception, including social categorization.

The Brain and the New Foundations of Mathematics

Many concepts in mathematics are not fully defined, and their properties are implicit, which leads to paradoxes. New foundations of mathematics were formulated based on the concept of innate programs of behavior and thinking. The basic axiom of mathematics is proposed, according to which any mathematical object has a physical carrier. This carrier can store and process only a finite amount of information. As a result of the D-procedure (encoding of any mathematical objects and operations on them in the form of qubits), a mathematical object is digitized. As a consequence, the basis of mathematics is the interaction of brain qubits, which can only implement arithmetic operations on numbers. A proof in mathematics is an algorithm for finding the correct statement from a list of already-existing statements. Some mathematical paradoxes (e.g., Banach–Tarski and Russell) and Smale’s 18th problem are solved by means of the D-procedure. The axiom of choice is a consequence of the equivalence of physical states, the choice among which can be made randomly. The proposed mathematics is constructive in the sense that any mathematical object exists if it is physically realized. The consistency of mathematics is due to directed evolution, which results in effective structures. Computing with qubits is based on the nontrivial quantum effects of biologically important molecules in neurons and the brain.

Data-driven machine learning models for decoding speech categorization from evoked brain responses

Objective.Categorical perception (CP) of audio is critical to understand how the human brain perceives speech sounds despite widespread variability in acoustic properties. Here, we investigated the spatiotemporal characteristics of auditory neural activity that reflects CP for speech (i.e. differentiates phonetic prototypes from ambiguous speech sounds).Approach.We recorded 64-channel electroencephalograms as listeners rapidly classified vowel sounds along an acoustic-phonetic continuum. We used support vector machine classifiers and stability selection to determine when and where in the brain CP was best decoded across space and time via source-level analysis of the event-related potentials.Main results. We found that early (120 ms) whole-brain data decoded speech categories (i.e. prototypical vs. ambiguous tokens) with 95.16% accuracy (area under the curve 95.14%;F1-score 95.00%). Separate analyses on left hemisphere (LH) and right hemisphere (RH) responses showed that LH decoding was more accurate and earlier than RH (89.03% vs. 86.45% accuracy; 140 ms vs. 200 ms). Stability (feature) selection identified 13 regions of interest (ROIs) out of 68 brain regions [including auditory cortex, supramarginal gyrus, and inferior frontal gyrus (IFG)] that showed categorical representation during stimulus encoding (0-260 ms). In contrast, 15 ROIs (including fronto-parietal regions, IFG, motor cortex) were necessary to describe later decision stages (later 300-800 ms) of categorization but these areas were highly associated with the strength of listeners' categorical hearing (i.e. slope of behavioral identification functions).Significance.Our data-driven multivariate models demonstrate that abstract categories emerge surprisingly early (∼120 ms) in the time course of speech processing and are dominated by engagement of a relatively compact fronto-temporal-parietal brain network.

Data-driven machine learning models for decoding speech categorization from evoked brain responses

Objective.Categorical perception (CP) of audio is critical to understand how the human brain perceives speech sounds despite widespread variability in acoustic properties. Here, we investigated the spatiotemporal characteristics of auditory neural activity that reflects CP for speech (i.e. differentiates phonetic prototypes from ambiguous speech sounds).Approach.We recorded 64-channel electroencephalograms as listeners rapidly classified vowel sounds along an acoustic-phonetic continuum. We used support vector machine classifiers and stability selection to determine when and where in the brain CP was best decoded across space and time via source-level analysis of the event-related potentials.Main results. We found that early (120 ms) whole-brain data decoded speech categories (i.e. prototypical vs. ambiguous tokens) with 95.16% accuracy (area under the curve 95.14%;F1-score 95.00%). Separate analyses on left hemisphere (LH) and right hemisphere (RH) responses showed that LH decoding was more accurate and earlier than RH (89.03% vs. 86.45% accuracy; 140 ms vs. 200 ms). Stability (feature) selection identified 13 regions of interest (ROIs) out of 68 brain regions [including auditory cortex, supramarginal gyrus, and inferior frontal gyrus (IFG)] that showed categorical representation during stimulus encoding (0-260 ms). In contrast, 15 ROIs (including fronto-parietal regions, IFG, motor cortex) were necessary to describe later decision stages (later 300-800 ms) of categorization but these areas were highly associated with the strength of listeners' categorical hearing (i.e. slope of behavioral identification functions).Significance.Our data-driven multivariate models demonstrate that abstract categories emerge surprisingly early (∼120 ms) in the time course of speech processing and are dominated by engagement of a relatively compact fronto-temporal-parietal brain network.

Parent TRAP: Discriminating Infant Cries Requires a Higher-Order Auditory Association Area in Mice

A circuit understanding of how perception links to response requires integrating neural connectivity, activity, and behavior. In this issue of Neuron, Tasaka et al. (2020) target neurons activated by ultrasonic pup vocalizations and discover a functional synaptic network embedded through acoustically selective TeA neurons that help link the calls to a discriminative maternal behavioral response.

Computational and neurophysiological principles underlying auditory perceptual decisions

A fundamental scientific goal in auditory neuroscience is identifying what mechanisms allow the brain to transform an unlabeled mixture of auditory stimuli into distinct perceptual representations. This transformation is accomplished by a complex interaction of multiple neurocomputational processes, including Gestalt grouping mechanisms, categorization, attention, and perceptual decision-making. Despite a great deal of scientific energy devoted to understanding these principles of hearing, we still do not understand either how auditory perception arises from neural activity or the causal relationship between neural activity and auditory perception. Here, we review the contributions of cortical and subcortical regions to auditory perceptual decisions with an emphasis on those studies that simultaneously measure behavior and neural activity. We also put forth challenges to the field that must be faced if we are to further our understanding of the relationship between neural activity and auditory perception.

Mechanisms of competitive selection: A canonical neural circuit framework

Competitive selection, the transformation of multiple competing sensory inputs and internal states into a unitary choice, is a fundamental component of animal behavior. Selection behaviors have been studied under several intersecting umbrellas including decision-making, action selection, perceptual categorization, and attentional selection. Neural correlates of these behaviors and computational models have been investigated extensively. However, specific, identifiable neural circuit mechanisms underlying the implementation of selection remain elusive. Here, we employ a first principles approach to map competitive selection explicitly onto neural circuit elements. We decompose selection into six computational primitives, identify demands that their execution places on neural circuit design, and propose a canonical neural circuit framework. The resulting framework has several links to neural literature, indicating its biological feasibility, and has several common elements with prominent computational models, suggesting its generality. We propose that this framework can help catalyze experimental discovery of the neural circuit underpinnings of competitive selection.

Dynamics and Hierarchical Encoding of Non-compact Acoustic Categories in Auditory and Frontal Cortex

Categorical perception is a fundamental cognitive function enabling animals to flexibly assign sounds into behaviorally relevant categories. This study investigates the nature of acoustic category representations, their emergence in an ascending series of ferret auditory and frontal cortical fields, and the dynamics of this representation during passive listening to task-relevant stimuli and during active retrieval from memory while engaging in learned categorization tasks. Ferrets were trained on two auditory Go-NoGo categorization tasks to discriminate two non-compact sound categories (composed of tones or amplitude-modulated noise). Neuronal responses became progressively more categorical in higher cortical fields, especially during task performance. The dynamics of the categorical responses exhibited a cascading top-down modulation pattern that began earliest in the frontal cortex and subsequently flowed downstream to the secondary auditory cortex, followed by the primary auditory cortex. In a subpopulation of neurons, categorical responses persisted even during the passive listening condition, demonstrating memory for task categories and their enhanced categorical boundaries.

A roadmap to integrate astrocytes into Systems Neuroscience

Systems neuroscience is still mainly a neuronal field, despite the plethora of evidence supporting the fact that astrocytes modulate local neural circuits, networks, and complex behaviors. In this article, we sought to identify which types of studies are necessary to establish whether astrocytes, beyond their well-documented homeostatic and metabolic functions, perform computations implementing mathematical algorithms that sub-serve coding and higher-brain functions. First, we reviewed Systems-like studies that include astrocytes in order to identify computational operations that these cells may perform, using Ca2+ transients as their encoding language. The analysis suggests that astrocytes may carry out canonical computations in a time scale of subseconds to seconds in sensory processing, neuromodulation, brain state, memory formation, fear, and complex homeostatic reflexes. Next, we propose a list of actions to gain insight into the outstanding question of which variables are encoded by such computations. The application of statistical analyses based on machine learning, such as dimensionality reduction and decoding in the context of complex behaviors, combined with connectomics of astrocyte-neuronal circuits, is, in our view, fundamental undertakings. We also discuss technical and analytical approaches to study neuronal and astrocytic populations simultaneously, and the inclusion of astrocytes in advanced modeling of neural circuits, as well as in theories currently under exploration such as predictive coding and energy-efficient coding. Clarifying the relationship between astrocytic Ca2+ and brain coding may represent a leap forward toward novel approaches in the study of astrocytes in health and disease.

Invariant neural responses for sensory categories revealed by the time-varying information for communication calls

Although information theoretic approaches have been used extensively in the analysis of the neural code, they have yet to be used to describe how information is accumulated in time while sensory systems are categorizing dynamic sensory stimuli such as speech sounds or visual objects. Here, we present a novel method to estimate the cumulative information for stimuli or categories. We further define a time-varying categorical information index that, by comparing the information obtained for stimuli versus categories of these same stimuli, quantifies invariant neural representations. We use these methods to investigate the dynamic properties of avian cortical auditory neurons recorded in zebra finches that were listening to a large set of call stimuli sampled from the complete vocal repertoire of this species. We found that the time-varying rates carry 5 times more information than the mean firing rates even in the first 100 ms. We also found that cumulative information has slow time constants (100–600 ms) relative to the typical integration time of single neurons, reflecting the fact that the behaviorally informative features of auditory objects are time-varying sound patterns. When we correlated firing rates and information values, we found that average information correlates with average firing rate but that higher-rates found at the onset response yielded similar information values as the lower-rates found in the sustained response: the onset and sustained response of avian cortical auditory neurons provide similar levels of independent information about call identity and call-type. Finally, our information measures allowed us to rigorously define categorical neurons; these categorical neurons show a high degree of invariance for vocalizations within a call-type. Peak invariance is found around 150 ms after stimulus onset. Surprisingly, call-type invariant neurons were found in both primary and secondary avian auditory areas.

Just as the recognition of faces requires neural representations that are invariant to scale and rotation, the recognition of behaviorally relevant auditory objects, such as spoken words, requires neural representations that are invariant to the speaker uttering the word and to his or her location. Here, we used information theory to investigate the time course of the neural representation of bird communication calls and of behaviorally relevant categories of these same calls: the call-types of the bird’s repertoire. We found that neurons in both the primary and secondary avian auditory cortex exhibit invariant responses to call renditions within a call-type, suggestive of a potential role for extracting the meaning of these communication calls. We also found that time plays an important role: first, neural responses carry significantly more information when represented by temporal patterns calculated at the small time scale of 10 ms than when measured as average rates and, second, this information accumulates in a non-redundant fashion up to long integration times of 600 ms. This rich temporal neural representation is matched to the temporal richness found in the communication calls of this species.

Training Humans to Categorize Monkey Calls: Auditory Feature- and Category-Selective Neural Tuning Changes

Grouping auditory stimuli into common categories is essential for a variety of auditory tasks, including speech recognition. We trained human participants to categorize auditory stimuli from a large novel set of morphed monkey vocalizations. Using fMRI-rapid adaptation (fMRI-RA) and multi-voxel pattern analysis (MVPA) techniques, we gained evidence that categorization training results in two distinct sets of changes: sharpened tuning to monkey call features (without explicit category representation) in left auditory cortex and category selectivity for different types of calls in lateral prefrontal cortex. In addition, the sharpness of neural selectivity in left auditory cortex, as estimated with both fMRI-RA and MVPA, predicted the steepness of the categorical boundary, whereas categorical judgment correlated with release from adaptation in the left inferior frontal gyrus. These results support the theory that auditory category learning follows a two-stage model analogous to the visual domain, suggesting general principles of perceptual category learning in the human brain.

Neuronal correlates of auditory streaming in the auditory cortex of behaving monkeys

This study tested the hypothesis that spiking activity in the primary auditory cortex of monkeys is related to auditory stream formation. Evidence for this hypothesis was previously obtained in animals that were passively exposed to stimuli and in which differences in the streaming percept were confounded with differences between the stimuli. In this study, monkeys performed an operant task on sequences that were composed of light flashes and tones. The tones alternated between a high and a low frequency and could be perceived either as one auditory stream or two auditory streams. The flashes promoted either a one-stream percept or a two-stream percept. Comparison of different types of sequences revealed that the neuronal responses to the alternating tones were more similar when the flashes promoted auditory stream integration, and were more dissimilar when the flashes promoted auditory stream segregation. Thus our findings show that the spiking activity in the monkey primary auditory cortex is related to auditory stream formation.

Familiarity with social sounds alters c-Fos expression in auditory cortex and interacts with estradiol in locus coeruleus

When a social sound category initially gains behavioral significance to an animal, plasticity events presumably enhance the ability to recognize that sound category in the future. In the context of learning natural social stimuli, neuromodulators such as norepinephrine and estrogen have been associated with experience-dependent plasticity and processing of newly salient social cues, yet continued plasticity once stimuli are familiar could disrupt the stability of sensorineural representations. Here we employed a maternal mouse model of natural sensory cortical plasticity for infant vocalizations to ask whether the engagement of the noradrenergic locus coeruleus (LC) by the playback of pup-calls is affected by either prior experience with the sounds or estrogen availability, using a well-studied cellular activity and plasticity marker, the immediate early gene c-Fos. We counted call-induced c-Fos immunoreactive (cFos-IR) cells in both LC and physiologically validated fields within the auditory cortex (AC) of estradiol or blank-implanted virgin female mice with either 0 or 5-days prior experience caring for vocalizing pups. Estradiol and pup experience interacted both in the induction of c-Fos-IR in the LC, as well as in behavioral measures of locomotion during playback, consistent with the neuromodulatory center’s activity being an online reflection of both hormonal and experience-dependent influences on arousal. Throughout core AC, as well as in a high frequency sub-region of AC and in secondary AC, a main effect of pup experience was to reduce call-induced c-Fos-IR, irrespective of estradiol availability. This is consistent with the hypothesis that sound familiarity leads to less c-Fos-mediated plasticity, and less disrupted sensory representations of a meaningful call category. Taken together, our data support the view that any coupling between these sensory and neuromodulatory areas is situationally dependent, and their engagement depends differentially on both internal state factors like hormones and external state factors like prior experience.

Meaning in the avian auditory cortex: neural representation of communication calls

Understanding how the brain extracts the behavioral meaning carried by specific vocalization types that can be emitted by various vocalizers and in different conditions is a central question in auditory research. This semantic categorization is a fundamental process required for acoustic communication, and presupposes discriminative and invariance properties of the auditory system for conspecific vocalizations. Songbirds have been used extensively to study vocal learning, but the communicative function of all their vocalizations and their neural representation has yet to be examined. In this study, we first generated a library containing almost the entire zebra finch vocal repertoire, and organised communication calls along nine different categories according to their behavioral meaning. We then investigated the neural representations of these semantic categories in the primary and secondary auditory areas of six anesthetised zebra finches. To analyse how single units encode these call categories, we described neural responses in terms of their discrimination, selectivity and invariance properties. Quantitative measures for these neural properties were obtained with an optimal decoder using both spike counts and spike patterns. Information theoretic metrics show that almost half of the single units encode semantic information. Neurons achieve higher discrimination of these semantic categories by being more selective and more invariant. These results demonstrate that computations necessary for semantic categorization of meaningful vocalizations are already present in the auditory cortex, and emphasise the value of a neuro-ethological approach to understand vocal communication.

Role of cortical neurodynamics for understanding the neural basis of motivated behavior - lessons from auditory category learning

Rhythmic activity appears in the auditory cortex in both microscopic and macroscopic observables and is modulated by both bottom-up and top-down processes. How this activity serves both types of processes is largely unknown. Here we review studies that have recently improved our understanding of potential functional roles of large-scale global dynamic activity patterns in auditory cortex. The experimental paradigm of auditory category learning allowed critical testing of the hypothesis that global auditory cortical activity states are associated with endogenous cognitive states mediating the meaning associated with an acoustic stimulus rather than with activity states that merely represent the stimulus for further processing.