Temporally dynamic frequency tuning of population responses in monkey primary auditory cortex

Extensive representation of sensory deviance in the responses to auditory gaps in unanesthetized rats

Unexpected changes in incoming sensory streams are associated with large errors in predicting the deviant stimulus relative to a memory trace of past stimuli. Mismatch negativity (MMN) in human studies and the release from stimulus-specific adaptation (SSA) in animal models correlate with prediction errors and deviance detection.¹ In human studies, violation of expectations elicited by an unexpected stimulus omission resulted in an omission MMN.^2,3,4,5 These responses are evoked after the expected occurrence time of the omitted stimulus, implying that they reflect the violation of a temporal expectancy.⁶ Because they are often time locked to the end of the omitted stimulus,^4,6,7 they resemble off responses. Indeed, suppression of cortical activity after the termination of the gap disrupts gap detection, suggesting an essential role for offset responses.⁸ Here, we demonstrate that brief gaps in short noise bursts in the auditory cortex of unanesthetized rats frequently evoke offset responses. Importantly, we show that omission responses are elicited when these gaps are expected but are omitted. These omission responses, together with the release from SSA of both onset and offset responses to rare gaps, form a rich and varied representation of prediction-related signals in the auditory cortex of unanesthetized rats, extending substantially and refining the representations described previously in anesthetized rats.

Ratphones: An Affordable Tool for Highly Controlled Sound Presentation in Freely Moving Rats

Encoding and processing sensory information is key to understanding the environment and to guiding behavior accordingly. Characterizing the behavioral and neural correlates of these processes requires the experimenter to have a high degree of control over stimuli presentation. For auditory stimulation in animals with relatively large heads, this can be accomplished by using headphones. However, it has proven more challenging in smaller species, such as rats and mice, and has been only partially solved using closed-field speakers in anesthetized or head-restrained preparations. To overcome the limitations of such preparations and to deliver sound with high precision to freely moving animals, we have developed a set of miniature headphones for rats. The headphones consist of a small, skull-implantable base attached with magnets to a fully adjustable structure that holds the speakers and keeps them in the same position with respect to the ears.

More than the end: OFF response plasticity as a mnemonic signature of a sound’s behavioral salience

In studying how neural populations in sensory cortex code dynamically varying stimuli to guide behavior, the role of spiking after stimuli have ended has been underappreciated. This is despite growing evidence that such activity can be tuned, experience-and context-dependent and necessary for sensory decisions that play out on a slower timescale. Here we review recent studies, focusing on the auditory modality, demonstrating that this so-called OFF activity can have a more complex temporal structure than the purely phasic firing that has often been interpreted as just marking the end of stimuli. While diverse and still incompletely understood mechanisms are likely involved in generating phasic and tonic OFF firing, more studies point to the continuing post-stimulus activity serving a short-term, stimulus-specific mnemonic function that is enhanced when the stimuli are particularly salient. We summarize these results with a conceptual model highlighting how more neurons within the auditory cortical population fire for longer duration after a sound’s termination during an active behavior and can continue to do so even while passively listening to behaviorally salient stimuli. Overall, these studies increasingly suggest that tonic auditory cortical OFF activity holds an echoic memory of specific, salient sounds to guide behavioral decisions.

Altered Response Dynamics and Increased Population Correlation to Tonal Stimuli Embedded in Noise in Aging Auditory Cortex

Age-related hearing loss (presbycusis) is a chronic health condition that affects one-third of the world population. One hallmark of presbycusis is a difficulty hearing in noisy environments. Presbycusis can be separated into two components: alterations of peripheral mechanotransduction of sound in the cochlea and central alterations of auditory processing areas of the brain. Although the effects of the aging cochlea in hearing loss have been well studied, the role of the aging brain in hearing loss is less well understood. Therefore, to examine how age-related central processing changes affect hearing in noisy environments, we used a mouse model (Thy1-GCaMP6s X CBA) that has excellent peripheral hearing in old age. We used in vivo two-photon Ca2+ imaging to measure the responses of neuronal populations in auditory cortex (ACtx) of adult (2-6 months, nine male, six female, 4180 neurons) and aging mice (15-17 months, six male, three female, 1055 neurons) while listening to tones in noisy backgrounds. We found that ACtx neurons in aging mice showed larger responses to tones and have less suppressed responses consistent with reduced inhibition. Aging neurons also showed less sensitivity to temporal changes. Population analysis showed that neurons in aging mice showed higher pairwise activity correlations and showed a reduced diversity in responses to sound stimuli. Using neural decoding techniques, we show a loss of information in neuronal populations in the aging brain. Thus, aging not only affects the responses of single neurons but also affects how these neurons jointly represent stimuli.SIGNIFICANCE STATEMENT Aging results in hearing deficits particularly under challenging listening conditions. We show that auditory cortex contains distinct subpopulations of excitatory neurons that preferentially encode different stimulus features and that aging selectively reduces certain subpopulations. We also show that aging increases correlated activity between neurons and thereby reduces the response diversity in auditory cortex. The loss of population response diversity leads to a decrease of stimulus information and deficits in sound encoding, especially in noisy backgrounds. Future work determining the identities of circuits affected by aging could provide new targets for therapeutic strategies.

Encoding of acquired sound-sequence salience by auditory cortical offset responses

Behaviorally relevant sounds are often composed of distinct acoustic units organized into specific temporal sequences. The meaning of such sound sequences can therefore be fully recognized only when they have terminated. However, the neural mechanisms underlying the perception of sound sequences remain unclear. Here, we use two-photon calcium imaging in the auditory cortex of behaving mice to test the hypothesis that neural responses to termination of sound sequences ("Off-responses") encode their acoustic history and behavioral salience. We find that auditory cortical Off-responses encode preceding sound sequences and that learning to associate a sound sequence with a reward induces enhancement of Off-responses relative to responses during the sound sequence ("On-responses"). Furthermore, learning enhances network-level discriminability of sound sequences by Off-responses. Last, learning-induced plasticity of Off-responses but not On-responses lasts to the next day. These findings identify auditory cortical Off-responses as a key neural signature of acquired sound-sequence salience.

Phasic Off responses of auditory cortex underlie perception of sound duration

It has been proposed that sound information is separately streamed into onset and offset pathways for parallel processing. However, how offset responses contribute to auditory perception remains unclear. Here, loose-patch and whole-cell recordings in awake mouse primary auditory cortex (A1) reveal that a subset of pyramidal neurons exhibit a transient "Off" response, with its onset tightly time-locked to the sound termination and its frequency tuning similar to that of the transient "On" response. Both responses are characterized by excitation briefly followed by inhibition, with the latter mediated by parvalbumin (PV) inhibitory neurons. Optogenetically manipulating sound-evoked A1 responses at different temporal phases or artificially creating phantom sounds in A1 further reveals that the A1 phasic On and Off responses are critical for perceptual discrimination of sound duration. Our results suggest that perception of sound duration is dependent on precisely encoding its onset and offset timings by phasic On and Off responses.

Parallel Processing of Sound Dynamics across Mouse Auditory Cortex via Spatially Patterned Thalamic Inputs and Distinct Areal Intracortical Circuits

Natural sounds have rich spectrotemporal dynamics. Spectral information is spatially represented in the auditory cortex (ACX) via large-scale maps. However, the representation of temporal information, e.g., sound offset, is unclear. We perform multiscale imaging of neuronal and thalamic activity evoked by sound onset and offset in awake mouse ACX. ACX areas differed in onset responses (On-Rs) and offset responses (Off-Rs). Most excitatory L2/3 neurons show either On-Rs or Off-Rs, and ACX areas are characterized by differing fractions of On and Off-R neurons. Somatostatin and parvalbumin interneurons show distinct temporal dynamics, potentially amplifying Off-Rs. Functional network analysis shows that ACX areas contain distinct parallel onset and offset networks. Thalamic (MGB) terminals show either On-Rs or Off-Rs, indicating a thalamic origin of On and Off-R pathways. Thus, ACX areas spatially represent temporal features, and this representation is created by spatial convergence and co-activation of distinct MGB inputs and is refined by specific intracortical connectivity.

Experience-Dependent Coding of Time-Dependent Frequency Trajectories by Off Responses in Secondary Auditory Cortex

Time-dependent frequency trajectories are an inherent feature of many behaviorally relevant sounds, such as species-specific vocalizations. Dynamic frequency trajectories, even in short sounds, often convey meaningful information, which may be used to differentiate sound categories. However, it is not clear what and where neural responses in the auditory cortical pathway are critical for conveying information about behaviorally relevant frequency trajectories, and how these responses change with experience. Here, we uncover tuning to subtle variations in frequency trajectories in auditory cortex of female mice. We found that auditory cortical responses could be modulated by variations in a pure tone trajectory as small as 1/24th of an octave, comparable to what has been reported in primates. In particular, late spiking after the end of a sound stimulus was more often sensitive to the sound's subtle frequency variation compared with spiking during the sound. Such “Off” responses in the adult A2, but not those in core auditory cortex, were plastic in a way that may enhance the representation of a newly acquired, behaviorally relevant sound category. We illustrate this with the maternal mouse paradigm for natural vocalization learning. By using an ethologically inspired paradigm to drive auditory responses in higher-order neurons, our results demonstrate that mouse auditory cortex can track fine frequency changes, which allows A2 Off responses in particular to better respond to pitch trajectories that distinguish behaviorally relevant, natural sound categories.

SIGNIFICANCE STATEMENT A whistle's pitch conveys meaning to its listener, as when dogs learn that distinct pitch trajectories whistled by their owner differentiate specific commands. Many species use pitch trajectories in their own vocalizations to distinguish sound categories, such as in human languages, such as Mandarin. How and where auditory neural activity encodes these pitch trajectories as their meaning is learned but not well understood, especially for short-duration sounds. We studied this in mice, where infants use ultrasonic whistles to communicate to adults. We found that late neural firing after a sound ends can be tuned to how the pitch changes in time, and that this response in a secondary auditory cortical field changes with experience to acquire a pitch change's meaning.

A Cortico-Collicular Amplification Mechanism for Gap Detection

Auditory cortex (AC) is necessary for the detection of brief gaps in ongoing sounds, but not for the detection of longer gaps or other stimuli such as tones or noise. It remains unclear why this is so, and what is special about brief gaps in particular. Here, we used both optogenetic suppression and conventional lesions to show that the cortical dependence of brief gap detection hinges specifically on gap termination. We then identified a cortico-collicular gap detection circuit that amplifies cortical gap termination responses before projecting to inferior colliculus (IC) to impact behavior. We found that gaps evoked off-responses and on-responses in cortical neurons, which temporally overlapped for brief gaps, but not long gaps. This overlap specifically enhanced cortical responses to brief gaps, whereas IC neurons preferred longer gaps. Optogenetic suppression of AC reduced collicular responses specifically to brief gaps, indicating that under normal conditions, the enhanced cortical representation of brief gaps amplifies collicular gap responses. Together these mechanisms explain how and why AC contributes to the behavioral detection of brief gaps, which are critical cues for speech perception, perceptual grouping, and auditory scene analysis.

Age-related changes in sound onset and offset intensity coding in auditory cortical fields A1 and CL of rhesus macaques

Inhibition plays a key role in shaping sensory processing in the central auditory system and has been implicated in sculpting receptive field properties such as sound intensity coding and also in shaping temporal patterns of neuronal firing such as onset- or offset-evoked responses. There is substantial evidence supporting a decrease in inhibition throughout the ascending auditory pathway in geriatric animals. We therefore examined intensity coding of onset (ON) and offset (OFF) responses in auditory cortex of aged and young monkeys. A large proportion of cells in the primary auditory cortex (A1) and the caudolateral field (CL) displayed nonmonotonic rate-level functions for OFF responses in addition to nonmonotonic coding of ON responses. Aging differentially affected ON and OFF responses; the magnitude of effects was generally greater for ON responses. In addition to higher firing rates, neurons in old monkeys exhibited a significant increase in the proportion of monotonic rate-level functions and had higher best intensities than those in young monkeys. OFF responses in young monkeys displayed a range of intensity coding relationships with ON responses of the same cells, ranging from highly similar to highly dissimilar. Dissimilarity in ON/OFF coding was greater in CL and was reduced with aging, which was largely explained by a preferential decrease in the percentage of cells with nonmonotonic coding of ON and OFF responses. The changes we observed are consistent with previously demonstrated alterations in inhibition in the ascending auditory pathway of primates and could be involved in age-related deficits in the temporal processing of sounds.NEW & NOTEWORTHY Aging has a major impact on intensity coding of neurons in auditory cortex of rhesus macaques. Neural responses to sound onset and offset were affected to different extents, and their rate-level functions became more mutually similar, which could be accounted for by the loss of nonmonotonic intensity coding in geriatric monkeys. These findings were consistent with weakened inhibition in the central auditory system and could contribute to auditory processing deficits in elderly subjects.

Neural responses to natural and model-matched stimuli reveal distinct computations in primary and nonprimary auditory cortex

A central goal of sensory neuroscience is to construct models that can explain neural responses to natural stimuli. As a consequence, sensory models are often tested by comparing neural responses to natural stimuli with model responses to those stimuli. One challenge is that distinct model features are often correlated across natural stimuli, and thus model features can predict neural responses even if they do not in fact drive them. Here, we propose a simple alternative for testing a sensory model: we synthesize a stimulus that yields the same model response as each of a set of natural stimuli, and test whether the natural and "model-matched" stimuli elicit the same neural responses. We used this approach to test whether a common model of auditory cortex-in which spectrogram-like peripheral input is processed by linear spectrotemporal filters-can explain fMRI responses in humans to natural sounds. Prior studies have that shown that this model has good predictive power throughout auditory cortex, but this finding could reflect feature correlations in natural stimuli. We observed that fMRI responses to natural and model-matched stimuli were nearly equivalent in primary auditory cortex (PAC) but that nonprimary regions, including those selective for music or speech, showed highly divergent responses to the two sound sets. This dissociation between primary and nonprimary regions was less clear from model predictions due to the influence of feature correlations across natural stimuli. Our results provide a signature of hierarchical organization in human auditory cortex, and suggest that nonprimary regions compute higher-order stimulus properties that are not well captured by traditional models. Our methodology enables stronger tests of sensory models and could be broadly applied in other domains.

ON-OFF receptive fields in auditory cortex diverge during development and contribute to directional sweep selectivity

Neurons in the auditory cortex exhibit distinct frequency tuning to the onset and offset of sounds, but the cause and significance of ON and OFF receptive field (RF) organisation are not understood. Here we demonstrate that distinct ON and OFF frequency tuning is largely absent in immature mouse auditory cortex and is thus a consequence of cortical development. Simulations using a novel implementation of a standard Hebbian plasticity model show that the natural alternation of sound onset and offset is sufficient for the formation of non-overlapping adjacent ON and OFF RFs in cortical neurons. Our model predicts that ON/OFF RF arrangement contributes towards direction selectivity to frequency-modulated tone sweeps, which we confirm by neuronal recordings. These data reveal that a simple and universally accepted learning rule can explain the organisation of ON and OFF RFs and direction selectivity in the developing auditory cortex.

Auditory cortex neurons exhibit distinct frequency tuning to sound onset and offset. Here the authors demonstrate that during development ON-OFF receptive fields diverge to occupy adjacent frequency ranges that may underlie their direction selective responses to frequency modulated sweeps.

A Crucial Test of the Population Separation Model of Auditory Stream Segregation in Macaque Primary Auditory Cortex

An important aspect of auditory scene analysis is auditory stream segregation-the organization of sound sequences into perceptual streams reflecting different sound sources in the environment. Several models have been proposed to account for stream segregation. According to the "population separation" (PS) model, alternating ABAB tone sequences are perceived as a single stream or as two separate streams when "A" and "B" tones activate the same or distinct frequency-tuned neuronal populations in primary auditory cortex (A1), respectively. A crucial test of the PS model is whether it can account for the observation that A and B tones are generally perceived as a single stream when presented synchronously, rather than in an alternating pattern, even if they are widely separated in frequency. Here, we tested the PS model by recording neural responses to alternating (ALT) and synchronous (SYNC) tone sequences in A1 of male macaques. Consistent with predictions of the PS model, a greater effective tonotopic separation of A and B tone responses was observed under ALT than under SYNC conditions, thus paralleling the perceptual organization of the sequences. While other models of stream segregation, such as temporal coherence, are not excluded by the present findings, we conclude that PS is sufficient to account for the perceptual organization of ALT and SYNC sequences and thus remains a viable model of auditory stream segregation.SIGNIFICANCE STATEMENT According to the population separation (PS) model of auditory stream segregation, sounds that activate the same or separate neural populations in primary auditory cortex (A1) are perceived as one or two streams, respectively. It is unclear, however, whether the PS model can account for the perception of sounds as a single stream when they are presented synchronously. Here, we tested the PS model by recording neural responses to alternating (ALT) and synchronous (SYNC) tone sequences in macaque A1. A greater effective separation of tonotopic activity patterns was observed under ALT than under SYNC conditions, thus paralleling the perceptual organization of the sequences. Based on these findings, we conclude that PS remains a plausible neurophysiological model of auditory stream segregation.

Engaging and disengaging recurrent inhibition coincides with sensing and unsensing of a sensory stimulus

Even simple sensory stimuli evoke neural responses that are dynamic and complex. Are the temporally patterned neural activities important for controlling the behavioral output? Here, we investigated this issue. Our results reveal that in the insect antennal lobe, due to circuit interactions, distinct neural ensembles are activated during and immediately following the termination of every odorant. Such non-overlapping response patterns are not observed even when the stimulus intensity or identities were changed. In addition, we find that ON and OFF ensemble neural activities differ in their ability to recruit recurrent inhibition, entrain field-potential oscillations and more importantly in their relevance to behaviour (initiate versus reset conditioned responses). Notably, we find that a strikingly similar strategy is also used for encoding sound onsets and offsets in the marmoset auditory cortex. In sum, our results suggest a general approach where recurrent inhibition is associated with stimulus ‘recognition' and ‘derecognition'.

Sensory stimuli evoke temporally dynamic responses. Here the authors report that responses to odour onset and offset are orthogonally represented in the locust antennal lobe, differentially entrain oscillations, and propose a model in which they are necessary for initiation and termination of behaviour.

Spectral and spatial tuning of onset and offset response functions in auditory cortical fields A1 and CL of rhesus macaques

The mammalian auditory cortex is necessary for spectral and spatial processing of acoustic stimuli. Most physiological studies of single neurons in the auditory cortex have focused on the onset and sustained portions of evoked responses, but there have been far fewer studies on the relationship between onset and offset responses. In the current study, we compared spectral and spatial tuning of onset and offset responses of neurons in primary auditory cortex (A1) and the caudolateral (CL) belt area of awake macaque monkeys. Several different metrics were used to determine the relationship between onset and offset response profiles in both frequency and space domains. In the frequency domain, a substantial proportion of neurons in A1 and CL displayed highly dissimilar best stimuli for onset- and offset-evoked responses, although even for these neurons, there was usually a large overlap in the range of frequencies that elicited onset, and offset responses and distributions of tuning overlap metrics were mostly unimodal. In the spatial domain, the vast majority of neurons displayed very similar best locations for onset- and offset-evoked responses, along with unimodal distributions of all tuning overlap metrics considered. Finally, for both spectral and spatial tuning, a slightly larger fraction of neurons in A1 displayed nonoverlapping onset and offset response profiles, relative to CL, which supports hierarchical differences in the processing of sounds in the two areas. However, these differences are small compared with differences in proportions of simple cells (low overlap) and complex cells (high overlap) in primary and secondary visual areas.NEW & NOTEWORTHY In the current study, we examine the relationship between the tuning of neural responses evoked by the onset and offset of acoustic stimuli in the primary auditory cortex, as well as a higher-order auditory area-the caudolateral belt field-in awake rhesus macaques. In these areas, the relationship between onset and offset response profiles in frequency and space domains formed a continuum, ranging from highly overlapping to highly nonoverlapping.

Mind the Gap: Two Dissociable Mechanisms of Temporal Processing in the Auditory System

High temporal acuity of auditory processing underlies perception of speech and other rapidly varying sounds. A common measure of auditory temporal acuity in humans is the threshold for detection of brief gaps in noise. Gap-detection deficits, observed in developmental disorders, are considered evidence for "sluggish" auditory processing. Here we show, in a mouse model of gap-detection deficits, that auditory brain sensitivity to brief gaps in noise can be impaired even without a general loss of central auditory temporal acuity. Extracellular recordings in three different subdivisions of the auditory thalamus in anesthetized mice revealed a stimulus-specific, subdivision-specific deficit in thalamic sensitivity to brief gaps in noise in experimental animals relative to controls. Neural responses to brief gaps in noise were reduced, but responses to other rapidly changing stimuli unaffected, in lemniscal and nonlemniscal (but not polysensory) subdivisions of the medial geniculate body. Through experiments and modeling, we demonstrate that the observed deficits in thalamic sensitivity to brief gaps in noise arise from reduced neural population activity following noise offsets, but not onsets. These results reveal dissociable sound-onset-sensitive and sound-offset-sensitive channels underlying auditory temporal processing, and suggest that gap-detection deficits can arise from specific impairment of the sound-offset-sensitive channel.

SIGNIFICANCE STATEMENT

The experimental and modeling results reported here suggest a new hypothesis regarding the mechanisms of temporal processing in the auditory system. Using a mouse model of auditory temporal processing deficits, we demonstrate the existence of specific abnormalities in auditory thalamic activity following sound offsets, but not sound onsets. These results reveal dissociable sound-onset-sensitive and sound-offset-sensitive mechanisms underlying auditory processing of temporally varying sounds. Furthermore, the findings suggest that auditory temporal processing deficits, such as impairments in gap-in-noise detection, could arise from reduced brain sensitivity to sound offsets alone.

Neural Representation of Concurrent Vowels in Macaque Primary Auditory Cortex

Successful speech perception in real-world environments requires that the auditory system segregate competing voices that overlap in frequency and time into separate streams. Vowels are major constituents of speech and are comprised of frequencies (harmonics) that are integer multiples of a common fundamental frequency (F0). The pitch and identity of a vowel are determined by its F0 and spectral envelope (formant structure), respectively. When two spectrally overlapping vowels differing in F0 are presented concurrently, they can be readily perceived as two separate “auditory objects” with pitches at their respective F0s. A difference in pitch between two simultaneous vowels provides a powerful cue for their segregation, which in turn, facilitates their individual identification. The neural mechanisms underlying the segregation of concurrent vowels based on pitch differences are poorly understood. Here, we examine neural population responses in macaque primary auditory cortex (A1) to single and double concurrent vowels (/a/ and /i/) that differ in F0 such that they are heard as two separate auditory objects with distinct pitches. We find that neural population responses in A1 can resolve, via a rate-place code, lower harmonics of both single and double concurrent vowels. Furthermore, we show that the formant structures, and hence the identities, of single vowels can be reliably recovered from the neural representation of double concurrent vowels. We conclude that A1 contains sufficient spectral information to enable concurrent vowel segregation and identification by downstream cortical areas.

Diverse cortical codes for scene segmentation in primate auditory cortex

The temporal coherence of amplitude fluctuations is a critical cue for segmentation of complex auditory scenes. The auditory system must accurately demarcate the onsets and offsets of acoustic signals. We explored how and how well the timing of onsets and offsets of gated tones are encoded by auditory cortical neurons in awake rhesus macaques. Temporal features of this representation were isolated by presenting otherwise identical pure tones of differing durations. Cortical response patterns were diverse, including selective encoding of onset and offset transients, tonic firing, and sustained suppression. Spike train classification methods revealed that many neurons robustly encoded tone duration despite substantial diversity in the encoding process. Excellent discrimination performance was achieved by neurons whose responses were primarily phasic at tone offset and by those that responded robustly while the tone persisted. Although diverse cortical response patterns converged on effective duration discrimination, this diversity significantly constrained the utility of decoding models referenced to a spiking pattern averaged across all responses or averaged within the same response category. Using maximum likelihood-based decoding models, we demonstrated that the spike train recorded in a single trial could support direct estimation of stimulus onset and offset. Comparisons between different decoding models established the substantial contribution of bursts of activity at sound onset and offset to demarcating the temporal boundaries of gated tones. Our results indicate that relatively few neurons suffice to provide temporally precise estimates of such auditory "edges," particularly for models that assume and exploit the heterogeneity of neural responses in awake cortex.

Neural representation of concurrent harmonic sounds in monkey primary auditory cortex: implications for models of auditory scene analysis

The ability to attend to a particular sound in a noisy environment is an essential aspect of hearing. To accomplish this feat, the auditory system must segregate sounds that overlap in frequency and time. Many natural sounds, such as human voices, consist of harmonics of a common fundamental frequency (F0). Such harmonic complex tones (HCTs) evoke a pitch corresponding to their F0. A difference in pitch between simultaneous HCTs provides a powerful cue for their segregation. The neural mechanisms underlying concurrent sound segregation based on pitch differences are poorly understood. Here, we examined neural responses in monkey primary auditory cortex (A1) to two concurrent HCTs that differed in F0 such that they are heard as two separate "auditory objects" with distinct pitches. We found that A1 can resolve, via a rate-place code, the lower harmonics of both HCTs, a prerequisite for deriving their pitches and for their perceptual segregation. Onset asynchrony between the HCTs enhanced the neural representation of their harmonics, paralleling their improved perceptual segregation in humans. Pitches of the concurrent HCTs could also be temporally represented by neuronal phase-locking at their respective F0s. Furthermore, a model of A1 responses using harmonic templates could qualitatively reproduce psychophysical data on concurrent sound segregation in humans. Finally, we identified a possible intracortical homolog of the "object-related negativity" recorded noninvasively in humans, which correlates with the perceptual segregation of concurrent sounds. Findings indicate that A1 contains sufficient spectral and temporal information for segregating concurrent sounds based on differences in pitch.

Localization of click trains and speech by cats: the negative level effect

Although localization of sound in elevation is believed to depend on spectral cues, it has been shown with human listeners that the temporal features of sound can also greatly affect localization performance. Of particular interest is a phenomenon known as the negative level effect, which describes the deterioration of localization ability in elevation with increasing sound level and is observed only with impulsive or short-duration sound. The present study uses the gaze positions of domestic cats as measures of perceived locations of sound targets varying in azimuth and elevation. The effects of sound level on localization in terms of accuracy, precision, and response latency were tested for sound with different temporal features, such as a click train, a single click, a continuous sound that had the same frequency spectrum of the click train, and speech segments. In agreement with previous human studies, negative level effects were only observed with click-like stimuli and only in elevation. In fact, localization of speech sounds in elevation benefited significantly when the sound level increased. Our findings indicate that the temporal continuity of a sound can affect the frequency analysis performed by the auditory system, and the variation in the frequency spectrum contained in speech sound does not interfere much with the spectral coding for its location in elevation.

The function of offset neurons in auditory information processing

Offset neurons which respond to the termination of the sound stimulation may play important roles in auditory temporal information processing, sound signal recognition, and complex distinction. Two additional possible mechanisms were reviewed: neural inhibition and the intrinsic conductance property of offset neuron membranes. The underlying offset response was postulated to be located in the superior paraolivary nucleus of mice. The biological significance of the offset neurons was discussed as well.

Representation of speech in human auditory cortex: is it special?

Successful categorization of phonemes in speech requires that the brain analyze the acoustic signal along both spectral and temporal dimensions. Neural encoding of the stimulus amplitude envelope is critical for parsing the speech stream into syllabic units. Encoding of voice onset time (VOT) and place of articulation (POA), cues necessary for determining phonemic identity, occurs within shorter time frames. An unresolved question is whether the neural representation of speech is based on processing mechanisms that are unique to humans and shaped by learning and experience, or is based on rules governing general auditory processing that are also present in non-human animals. This question was examined by comparing the neural activity elicited by speech and other complex vocalizations in primary auditory cortex of macaques, who are limited vocal learners, with that in Heschl's gyrus, the putative location of primary auditory cortex in humans. Entrainment to the amplitude envelope is neither specific to humans nor to human speech. VOT is represented by responses time-locked to consonant release and voicing onset in both humans and monkeys. Temporal representation of VOT is observed both for isolated syllables and for syllables embedded in the more naturalistic context of running speech. The fundamental frequency of male speakers is represented by more rapid neural activity phase-locked to the glottal pulsation rate in both humans and monkeys. In both species, the differential representation of stop consonants varying in their POA can be predicted by the relationship between the frequency selectivity of neurons and the onset spectra of the speech sounds. These findings indicate that the neurophysiology of primary auditory cortex is similar in monkeys and humans despite their vastly different experience with human speech, and that Heschl's gyrus is engaged in general auditory, and not language-specific, processing. This article is part of a Special Issue entitled "Communication Sounds and the Brain: New Directions and Perspectives".

Neural representation of harmonic complex tones in primary auditory cortex of the awake monkey

Many natural sounds are periodic and consist of frequencies (harmonics) that are integer multiples of a common fundamental frequency (F0). Such harmonic complex tones (HCTs) evoke a pitch corresponding to their F0, which plays a key role in the perception of speech and music. "Pitch-selective" neurons have been identified in non-primary auditory cortex of marmoset monkeys. Noninvasive studies point to a putative "pitch center" located in a homologous cortical region in humans. It remains unclear whether there is sufficient spectral and temporal information available at the level of primary auditory cortex (A1) to enable reliable pitch extraction in non-primary auditory cortex. Here we evaluated multiunit responses to HCTs in A1 of awake macaques using a stimulus design employed in auditory nerve studies of pitch encoding. The F0 of the HCTs was varied in small increments, such that harmonics of the HCTs fell either on the peak or on the sides of the neuronal pure tone tuning functions. Resultant response-amplitude-versus-harmonic-number functions ("rate-place profiles") displayed a periodic pattern reflecting the neuronal representation of individual HCT harmonics. Consistent with psychoacoustic findings in humans, lower harmonics were better resolved in rate-place profiles than higher harmonics. Lower F0s were also temporally represented by neuronal phase-locking to the periodic waveform of the HCTs. Findings indicate that population responses in A1 contain sufficient spectral and temporal information for extracting the pitch of HCTs by neurons in downstream cortical areas that receive their input from A1.

Effects of temporal stimulus properties on the perception of across-frequency asynchrony

The role of temporal stimulus parameters in the perception of across-frequency synchrony and asynchrony was investigated using pairs of 500-ms tones consisting of a 250-Hz tone and a tone with a higher frequency of 1, 2, 4, or 6 kHz. Subjective judgments suggested veridical perception of across-frequency synchrony but with greater sensitivity to changes in asynchrony for pairs in which the lower-frequency tone was leading than for pairs in which it was lagging. Consistent with the subjective judgments, thresholds for the detection of asynchrony measured in a three-alternative forced-choice task were lower when the signal interval contained a pair with the low-frequency tone leading than a pair with a high-frequency tone leading. A similar asymmetry was observed for asynchrony discrimination when the standard asynchrony was relatively small (≤20 ms) but not for larger standard asynchronies. Independent manipulation of onset and offset ramp durations indicated a dominant role of onsets in the perception of across-frequency asynchrony. A physiologically inspired model, involving broadly tuned monaural coincidence detectors that receive inputs from frequency-selective onset detectors, was able to accurately reproduce the asymmetric distributions of synchrony judgments. The model provides testable predictions for future physiological investigations of responses to broadband stimuli with across-frequency delays.

Searching for the mismatch negativity in primary auditory cortex of the awake monkey: deviance detection or stimulus specific adaptation?

The mismatch negativity (MMN) is a preattentive component of the auditory event-related potential that is elicited by a change in a repetitive acoustic pattern. While MMN has been extensively used in human electrophysiological studies of auditory processing, the neural mechanisms and brain regions underlying its generation remain unclear. We investigate possible homologs of the MMN in macaque primary auditory cortex (A1) using a frequency oddball paradigm in which rare "deviant" tones are randomly interspersed among frequent "standard" tones. Standards and deviants had frequencies equal to the best frequency (BF) of the recorded neural population or to a frequency that evoked a response half the amplitude of the BF response. Early and later field potentials, current source density components, multiunit activity, and induced high-gamma band responses were larger when elicited by deviants than by standards of the same frequency. Laminar analysis indicated that differences between deviant and standard responses were more prominent in later activity, thus suggesting cortical amplification of initial responses driven by thalamocortical inputs. However, unlike the human MMN, larger deviant responses were characterized by the enhancement of "obligatory" responses rather than the introduction of new components. Furthermore, a control condition wherein deviants were interspersed among many tones of variable frequency replicated the larger responses to deviants under the oddball condition. Results suggest that differential responses under the oddball condition in macaque A1 reflect stimulus-specific adaptation rather than deviance detection per se. We conclude that neural mechanisms of deviance detection likely reside in cortical areas outside of A1.

Detection of appearing and disappearing objects in complex acoustic scenes

The ability to detect sudden changes in the environment is critical for survival. Hearing is hypothesized to play a major role in this process by serving as an "early warning device," rapidly directing attention to new events. Here, we investigate listeners' sensitivity to changes in complex acoustic scenes-what makes certain events "pop-out" and grab attention while others remain unnoticed? We use artificial "scenes" populated by multiple pure-tone components, each with a unique frequency and amplitude modulation rate. Importantly, these scenes lack semantic attributes, which may have confounded previous studies, thus allowing us to probe low-level processes involved in auditory change perception. Our results reveal a striking difference between "appear" and "disappear" events. Listeners are remarkably tuned to object appearance: change detection and identification performance are at ceiling; response times are short, with little effect of scene-size, suggesting a pop-out process. In contrast, listeners have difficulty detecting disappearing objects, even in small scenes: performance rapidly deteriorates with growing scene-size; response times are slow, and even when change is detected, the changed component is rarely successfully identified. We also measured change detection performance when a noise or silent gap was inserted at the time of change or when the scene was interrupted by a distractor that occurred at the time of change but did not mask any scene elements. Gaps adversely affected the processing of item appearance but not disappearance. However, distractors reduced both appearance and disappearance detection. Together, our results suggest a role for neural adaptation and sensitivity to transients in the process of auditory change detection, similar to what has been demonstrated for visual change detection. Importantly, listeners consistently performed better for item addition (relative to deletion) across all scene interruptions used, suggesting a robust perceptual representation of item appearance.

Activity related to perceptual judgment and action in primary auditory cortex

Recent evidence is reshaping the view of primary auditory cortex (A1) from a unisensory area to one more involved in dynamically integrating multisensory- and task-related information. We found A1 single- (SU) and multiple-unit (MU) activity correlated with macaques' choices in an amplitude modulation (AM) discrimination task. Animals were trained to discriminate AM noise from unmodulated noise by releasing a lever for AM noise and holding down the lever for unmodulated noise. Activity for identical stimuli was compared between trials where the animals reported AM and trials where they did not. We found 47.4% of MUs and 22.8% of SUs significantly increased firing shortly before the animal's behavioral response to report AM when compared to the equivalent time period on trials where AM was not reported. Activity was also linked to lever release in a different task context, suggesting A1 modulation by somatosensory, or efference copy, input. When spikes were counted only during the stimulus, 19.6% of MUs and 13.8% of SUs increased firing rate when animals reported AM compared to when they did not, suggesting an attentional effect, or that A1 activity can be used by higher decision areas, or that such areas provide feedback to A1. Activity associated with AM reporting was correlated with a unit's AM sensitivity, suggesting AM sensitive neurons' involvement in task performance. A1 neurons' phase locking to AM correlated more weakly (compared to firing rate) with the animals' report of AM, suggesting a preferential role for rate-codes in A1 for this AM discrimination task.

Neural mechanisms of rhythmic masking release in monkey primary auditory cortex: implications for models of auditory scene analysis

The ability to detect and track relevant acoustic signals embedded in a background of other sounds is crucial for hearing in complex acoustic environments. This ability is exemplified by a perceptual phenomenon known as "rhythmic masking release" (RMR). To demonstrate RMR, a sequence of tones forming a target rhythm is intermingled with physically identical "Distracter" sounds that perceptually mask the rhythm. The rhythm can be "released from masking" by adding "Flanker" tones in adjacent frequency channels that are synchronous with the Distracters. RMR represents a special case of auditory stream segregation, whereby the target rhythm is perceptually segregated from the background of Distracters when they are accompanied by the synchronous Flankers. The neural basis of RMR is unknown. Previous studies suggest the involvement of primary auditory cortex (A1) in the perceptual organization of sound patterns. Here, we recorded neural responses to RMR sequences in A1 of awake monkeys in order to identify neural correlates and potential mechanisms of RMR. We also tested whether two current models of stream segregation, when applied to these responses, could account for the perceptual organization of RMR sequences. Results suggest a key role for suppression of Distracter-evoked responses by the simultaneous Flankers in the perceptual restoration of the target rhythm in RMR. Furthermore, predictions of stream segregation models paralleled the psychoacoustics of RMR in humans. These findings reinforce the view that preattentive or "primitive" aspects of auditory scene analysis may be explained by relatively basic neural mechanisms at the cortical level.

Multiplexed and robust representations of sound features in auditory cortex

We can recognize the melody of a familiar song when it is played on different musical instruments. Similarly, an animal must be able to recognize a warning call whether the caller has a high-pitched female or a lower-pitched male voice, and whether they are sitting in a tree to the left or right. This type of perceptual invariance to "nuisance" parameters comes easily to listeners, but it is unknown whether or how such robust representations of sounds are formed at the level of sensory cortex. In this study, we investigate whether neurons in both core and belt areas of ferret auditory cortex can robustly represent the pitch, formant frequencies, or azimuthal location of artificial vowel sounds while the other two attributes vary. We found that the spike rates of the majority of cortical neurons that are driven by artificial vowels carry robust representations of these features, but the most informative temporal response windows differ from neuron to neuron and across five auditory cortical fields. Furthermore, individual neurons can represent multiple features of sounds unambiguously by independently modulating their spike rates within distinct time windows. Such multiplexing may be critical to identifying sounds that vary along more than one perceptual dimension. Finally, we observed that formant information is encoded in cortex earlier than pitch information, and we show that this time course matches ferrets' behavioral reaction time differences on a change detection task.

Binaural sensitivity changes between cortical on and off responses

Neurons exhibiting on and off responses with different frequency tuning have previously been described in the primary auditory cortex (A1) of anesthetized and awake animals, but it is unknown whether other tuning properties, including sensitivity to binaural localization cues, also differ between on and off responses. We measured the sensitivity of A1 neurons in anesthetized ferrets to 1) interaural level differences (ILDs), using unmodulated broadband noise with varying ILDs and average binaural levels, and 2) interaural time delays (ITDs), using sinusoidally amplitude-modulated broadband noise with varying envelope ITDs. We also assessed fine-structure ITD sensitivity and frequency tuning, using pure-tone stimuli. Neurons most commonly responded to stimulus onset only, but purely off responses and on-off responses were also recorded. Of the units exhibiting significant binaural sensitivity nearly one-quarter showed binaural sensitivity in both on and off responses, but in almost all (∼97%) of these units the binaural tuning of the on responses differed significantly from that seen in the off responses. Moreover, averaged, normalized ILD and ITD tuning curves calculated from all units showing significant sensitivity to binaural cues indicated that on and off responses displayed different sensitivity patterns across the population. A principal component analysis of ITD response functions suggested a continuous cortical distribution of binaural sensitivity, rather than discrete response classes. Rather than reflecting a release from inhibition without any functional significance, we propose that binaural off responses may be important to cortical encoding of sound-source location.

Neural correlates of auditory scene analysis based on inharmonicity in monkey primary auditory cortex

Segregation of concurrent sounds in complex acoustic environments is a fundamental feature of auditory scene analysis. A powerful cue used by the auditory system to segregate concurrent sounds, such as speakers' voices at a cocktail party, is inharmonicity. This can be demonstrated when a component of a harmonic complex tone is perceived as a separate tone "popping out" from the complex as a whole when it is sufficiently mistuned from its harmonic value. The neural bases of perceptual "pop out" of mistuned harmonics are unclear. We recorded multiunit activity from primary auditory cortex (A1) of behaving monkeys elicited by harmonic complex tones that were either "in tune" or that contained a mistuned third harmonic set at the best frequency of the neural populations. Responses to mistuned sounds were enhanced relative to responses to "in-tune" sounds, thus correlating with the enhanced perceptual salience of the mistuned component. Consistent with human psychophysics of "pop out," response enhancements increased with the degree of mistuning, were maximal for neural populations tuned to the frequency of the mistuned component, and were not observed under comparable stimulus conditions that do not elicit perceptual "pop out." Mistuning was also associated with changes in neuronal temporal response patterns phase locked to "beats" in the stimuli. Intracortical auditory evoked potentials paralleled noninvasive neurophysiological correlates of perceptual "pop out" in humans, further augmenting the translational relevance of the results. Findings suggest two complementary neural mechanisms for "pop out," based on the detection of local differences in activation level or coherence of temporal response patterns across A1.

Enhanced physiologic discriminability of stop consonants with prolonged formant transitions in awake monkeys based on the tonotopic organization of primary auditory cortex

Many children with specific language impairment (SLI) have difficulty in perceiving stop consonant-vowel syllables (e.g., /ba/, /ga/, /da/) with rapid formant transitions, but perform normally when formant transitions are extended in time. This influential observation has helped lead to the development of the auditory temporal processing hypothesis, which posits that SLI is causally related to the processing of rapidly changing sounds in aberrantly expanded windows of temporal integration. We tested a potential physiological basis for this observation by examining whether syllables varying in their consonant place of articulation (POA) with prolonged formant transitions would evoke better differentiated patterns of activation along the tonotopic axis of A1 in awake monkeys when compared to syllables with short formant transitions, especially for more prolonged windows of temporal integration. Amplitudes of multi-unit activity evoked by /ba/, /ga/, and /da/ were ranked according to predictions based on responses to tones centered at the spectral maxima of frication at syllable onset. Population responses representing consonant POA were predicted by the tone responses. Predictions were stronger for syllables with prolonged formant transitions, especially for longer windows of temporal integration. Relevance of findings to normal perception and that occurring in SLI are discussed.

Nonoverlapping sets of synapses drive on responses and off responses in auditory cortex

Neurons in visual, somatosensory, and auditory cortex can respond to the termination as well as the onset of a sensory stimulus. In auditory cortex, these off responses may underlie the ability of the auditory system to use sound offsets as cues for perceptual grouping. Off responses have been widely proposed to arise from postinhibitory rebound, but this hypothesis has never been directly tested. We used in vivo whole-cell recordings to measure the synaptic inhibition evoked by sound onset. We find that inhibition is invariably transient, indicating that off responses are not caused by postinhibitory rebound in auditory cortical neurons. Instead, on and off responses appear to be driven by distinct sets of synapses, because they have distinct frequency tuning and different excitatory-inhibitory balance. Furthermore, an on-on sequence causes complete forward suppression, whereas an off-on sequence causes no suppression at all. We conclude that on and off responses are driven by largely nonoverlapping sets of synaptic inputs.