Neural mechanisms for the abstraction and use of pitch information in auditory cortex

Functional network properties of the auditory cortex

The auditory system transforms auditory stimuli from the external environment into perceptual auditory objects. Recent studies have focused on the contribution of the auditory cortex to this transformation. Other studies have yielded important insights into the contributions of neural activity in the auditory cortex to cognition and decision-making. However, despite this important work, the relationship between auditory-cortex activity and behavior/perception has not been fully elucidated. Two of the more important gaps in our understanding are (1) the specific and differential contributions of different fields of the auditory cortex to auditory perception and behavior and (2) the way networks of auditory neurons impact and facilitate auditory information processing. Here, we focus on recent work from non-human-primate models of hearing and review work related to these gaps and put forth challenges to further our understanding of how single-unit activity and network activity in different cortical fields contribution to behavior and perception.

Auditory and tactile frequency representations are co-embedded in modality-defined cortical sensory systems

Sensory information is represented and elaborated in hierarchical cortical systems that are thought to be dedicated to individual sensory modalities. This traditional view of sensory cortex organization has been challenged by recent evidence of multimodal responses in primary and association sensory areas. Although it is indisputable that sensory areas respond to multiple modalities, it remains unclear whether these multimodal responses reflect selective information processing for particular stimulus features. Here, we used fMRI adaptation to identify brain regions that are sensitive to the temporal frequency information contained in auditory, tactile, and audiotactile stimulus sequences. A number of brain regions distributed over the parietal and temporal lobes exhibited frequency-selective temporal response modulation for both auditory and tactile stimulus events, as indexed by repetition suppression effects. A smaller set of regions responded to crossmodal adaptation sequences in a frequency-dependent manner. Despite an extensive overlap of multimodal frequency-selective responses across the parietal and temporal lobes, representational similarity analysis revealed a cortical "regional landscape" that clearly reflected distinct somatosensory and auditory processing systems that converged on modality-invariant areas. These structured relationships between brain regions were also evident in spontaneous signal fluctuation patterns measured at rest. Our results reveal that multimodal processing in human cortex can be feature-specific and that multimodal frequency representations are embedded in the intrinsically hierarchical organization of cortical sensory systems.

Auditory frequency discrimination in developmental dyslexia: A meta-analysis

Auditory frequency discrimination has been used as an index of sensory processing in developmental language disorders such as dyslexia, where group differences have often been interpreted as evidence for a basic deficit in auditory processing that underpins and constrains individual variability in the development of phonological skills. Here, we conducted a meta-analysis to evaluate the cumulative evidence for group differences in frequency discrimination and to explore the impact of some potential moderator variables that could contribute to variability in effect-size estimations across studies. Our analyses revealed mean effect sizes for group differences on frequency discrimination tasks on the order of three-quarters of a standard deviation, but in the presence of substantial inter-study variability in their magnitude. Moderator variable analyses indicated that factors related both to participant variability on behavioural and cognitive variables associated with the dyslexia phenotype, and to variability in the task design, contributed to differences in the magnitude of effect size across studies. The apparently complex pattern of results was compounded by the lack of concurrent, standardised metrics of cognitive and reading component skills across the constituent studies. Differences on sensory processing tasks are often reported in studies of developmental disorders, but these need to be more carefully interpreted in the context of non-sensory factors, which may explain significant inter- and intra-group variance in the dependent measure of interest.

Auditory Selectivity for Spectral Contrast in Cortical Neurons and Behavior

Vocal communication relies on the ability of listeners to identify, process, and respond to vocal sounds produced by others in complex environments. To accurately recognize these signals, animals' auditory systems must robustly represent acoustic features that distinguish vocal sounds from other environmental sounds. Vocalizations typically have spectral structure; power regularly fluctuates along the frequency axis, creating spectral contrast. Spectral contrast is closely related to harmonicity, which refers to spectral power peaks occurring at integer multiples of a fundamental frequency. Although both spectral contrast and harmonicity typify natural sounds, they may differ in salience for communication behavior and engage distinct neural mechanisms. Therefore, it is important to understand which of these properties of vocal sounds underlie the neural processing and perception of vocalizations.Here, we test the importance of vocalization-typical spectral features in behavioral recognition and neural processing of vocal sounds, using male zebra finches. We show that behavioral responses to natural and synthesized vocalizations rely on the presence of discrete frequency components, but not on harmonic ratios between frequencies. We identify a specific population of neurons in primary auditory cortex that are sensitive to the spectral resolution of vocal sounds. We find that behavioral and neural response selectivity is explained by sensitivity to spectral contrast rather than harmonicity. This selectivity emerges within the cortex; it is absent in the thalamorecipient region and present in the deep output region. Further, deep-region neurons that are contrast-sensitive show distinct temporal responses and selectivity for modulation density compared with unselective neurons.SIGNIFICANCE STATEMENT Auditory coding and perception are critical for vocal communication. Auditory neurons must encode acoustic features that distinguish vocalizations from other sounds in the environment and generate percepts that direct behavior. The acoustic features that drive neural and behavioral selectivity for vocal sounds are unknown, however. Here, we show that vocal response behavior scales with stimulus spectral contrast but not with harmonicity, in songbirds. We identify a distinct population of auditory cortex neurons in which response selectivity parallels behavioral selectivity. This neural response selectivity is explained by sensitivity to spectral contrast rather than to harmonicity. Our findings inform the understanding of how the auditory system encodes socially-relevant signals via detection of an acoustic feature that is ubiquitous in vocalizations.

Direct electrophysiological mapping of human pitch-related processing in auditory cortex

This work sought correlates of pitch perception, defined by neural activity above the lower limit of pitch (LLP), in auditory cortical neural ensembles, and examined their topographical distribution. Local field potentials (LFPs) were recorded in eight patients undergoing invasive recordings for pharmaco-resistant epilepsy. Stimuli consisted of bursts of broadband noise followed by regular interval noise (RIN). RIN was presented at rates below and above the LLP to distinguish responses related to the regularity of the stimulus and the presence of pitch itself. LFPs were recorded from human cortical homologues of auditory core, belt, and parabelt regions using multicontact depth electrodes implanted in Heschl's gyrus (HG) and Planum Temporale (PT), and subdural grid electrodes implanted over lateral superior temporal gyrus (STG). Evoked responses corresponding to the temporal regularity of the stimulus were assessed using autocorrelation of the evoked responses, and occurred for stimuli below and above the LLP. Induced responses throughout the high gamma range (60-200 Hz) were present for pitch values above the LLP, with onset latencies of approximately 70 ms. Mapping of the induced responses onto a common brain space demonstrated variability in the topographical distribution of high gamma responses across subjects. Induced responses were present throughout the length of HG and on PT, which is consistent with previous functional neuroimaging studies. Moreover, in each subject, a region within lateral STG showed robust induced responses at pitch-evoking stimulus rates. This work suggests a distributed representation of pitch processing in neural ensembles in human homologues of core and non-core auditory cortex.

The distribution and nature of responses to broadband sounds associated with pitch in the macaque auditory cortex

The organisation of pitch-perception mechanisms in the primate cortex is controversial, in that divergent results have been obtained, ranging from a single circumscribed 'pitch centre' to systems widely distributed across auditory cortex. Possible reasons for such discrepancies include different species, recording techniques, pitch stimuli, sampling of auditory fields, and the neural metrics recorded. In the present study, we sought to bridge some of these divisions by examining activity related to pitch in both neurons and neuronal ensembles within the auditory cortex of the rhesus macaque, a primate species with similar pitch perception and auditory cortical organisation to humans. We demonstrate similar responses, in primary and non-primary auditory cortex, to two different types of broadband pitch above the macaque lower limit in both neurons and local field potential (LFP) gamma oscillations. The majority of broadband pitch responses in neurons and LFP sites did not show equivalent tuning for sine tones.

Across-species differences in pitch perception are consistent with differences in cochlear filtering

Pitch perception is critical for recognizing speech, music and animal vocalizations, but its neurobiological basis remains unsettled, in part because of divergent results across species. We investigated whether species-specific differences exist in the cues used to perceive pitch and whether these can be accounted for by differences in the auditory periphery. Ferrets accurately generalized pitch discriminations to untrained stimuli whenever temporal envelope cues were robust in the probe sounds, but not when resolved harmonics were the main available cue. By contrast, human listeners exhibited the opposite pattern of results on an analogous task, consistent with previous studies. Simulated cochlear responses in the two species suggest that differences in the relative salience of the two pitch cues can be attributed to differences in cochlear filter bandwidths. The results support the view that cross-species variation in pitch perception reflects the constraints of estimating a sound’s fundamental frequency given species-specific cochlear tuning.

Modeling and MEG evidence of early consonance processing in auditory cortex

Pitch is a fundamental attribute of auditory perception. The interaction of concurrent pitches gives rise to a sensation that can be characterized by its degree of consonance or dissonance. In this work, we propose that human auditory cortex (AC) processes pitch and consonance through a common neural network mechanism operating at early cortical levels. First, we developed a new model of neural ensembles incorporating realistic neuronal and synaptic parameters to assess pitch processing mechanisms at early stages of AC. Next, we designed a magnetoencephalography (MEG) experiment to measure the neuromagnetic activity evoked by dyads with varying degrees of consonance or dissonance. MEG results show that dissonant dyads evoke a pitch onset response (POR) with a latency up to 36 ms longer than consonant dyads. Additionally, we used the model to predict the processing time of concurrent pitches; here, consonant pitch combinations were decoded faster than dissonant combinations, in line with the experimental observations. Specifically, we found a striking match between the predicted and the observed latency of the POR as elicited by the dyads. These novel results suggest that consonance processing starts early in human auditory cortex and may share the network mechanisms that are responsible for (single) pitch processing.

Humans with latent toxoplasmosis display altered reward modulation of cognitive control

Latent infection with Toxoplasma gondii has repeatedly been shown to be associated with behavioral changes that are commonly attributed to a presumed increase in dopaminergic signaling. Yet, virtually nothing is known about its effects on dopamine-driven reward processing. We therefore assessed behavior and event-related potentials in individuals with vs. without latent toxoplasmosis performing a rewarded control task. The data show that otherwise healthy young adults with latent toxoplasmosis show a greatly diminished response to monetary rewards as compared to their non-infected counterparts. While this selective effect eliminated a toxoplasmosis-induced speed advantage previously observed for non-rewarded behavior, Toxo-positive subjects could still be demonstrated to be superior to Toxo-negative subjects with respect to response accuracy. Event-related potential (ERP) and source localization analyses revealed that this advantage during rewarded behavior was based on increased allocation of processing resources reflected by larger visual late positive component (LPC) amplitudes and associated activity changes in the right temporo-parietal junction (BA40) and left auditory cortex (BA41). Taken together, individuals with latent toxoplasmosis show superior behavioral performance in challenging cognitive control situations but may at the same time have a reduced sensitivity towards motivational effects of rewards, which might be explained by the presumed increase in dopamine.

Mandarin Lexical Tone Acquisition in Cochlear Implant Users With Prelingual Deafness: A Review

PURPOSE

The purpose of this review article is to synthesize evidence from the fields of developmental linguistics and cochlear implant technology relevant to the production and perception of Mandarin lexical tone in cochlear implant users with prelingual deafness. The aim of this review was to identify potential factors that determine outcomes for tonal-language speaking cochlear implant users and possible directions for further research.

METHOD

A computerized database search of MEDLINE, CINAHL, Academic Search Premier, Web of Science, and Google Scholar was undertaken in June and July 2014. Search terms used were lexical tone AND tonal language, speech development AND/OR speech production AND/OR speech perception AND cochlear implants, and pitch perception AND cochlear implants, anywhere in the title or abstract.

CONCLUSION

Despite the demonstrated limitations of pitch perception in cochlear implant users, there is some evidence that typical production and perception of lexical tone is possible by cochlear implant users with prelingual deafness. Further studies are required to determine the factors that contribute to better outcomes to inform rehabilitation processes for cochlear implant users in tonal-language environments.

Theta oscillations accompanying concurrent auditory stream segregation

The ability to isolate a single sound source among concurrent sources is crucial for veridical auditory perception. The present study investigated the event-related oscillations evoked by complex tones, which could be perceived as a single sound and tonal complexes with cues promoting the perception of two concurrent sounds by inharmonicity, onset asynchrony, and/or perceived source location difference of the components tones. In separate task conditions, participants performed a visual change detection task (visual control), watched a silent movie (passive listening) or reported for each tone whether they perceived one or two concurrent sounds (active listening). In two time windows, the amplitude of theta oscillation was modulated by the presence vs. absence of the cues: 60-350ms/6-8Hz (early) and 350-450ms/4-8Hz (late). The early response appeared both in the passive and the active listening conditions; it did not closely match the task performance; and it had a fronto-central scalp distribution. The late response was only elicited in the active listening condition; it closely matched the task performance; and it had a centro-parietal scalp distribution. The neural processes reflected by these responses are probably involved in the processing of concurrent sound segregation cues, in sound categorization, and response preparation and monitoring. The current results are compatible with the notion that theta oscillations mediate some of the processes involved in concurrent sound segregation.

Marmosets: A Neuroscientific Model of Human Social Behavior

The common marmoset (Callithrix jacchus) has garnered interest recently as a powerful model for the future of neuroscience research. Much of this excitement has centered on the species' reproductive biology and compatibility with gene editing techniques, which together have provided a path for transgenic marmosets to contribute to the study of disease as well as basic brain mechanisms. In step with technical advances is the need to establish experimental paradigms that optimally tap into the marmosets' behavioral and cognitive capacities. While conditioned task performance of a marmoset can compare unfavorably with rhesus monkey performance on conventional testing paradigms, marmosets' social behavior and cognition are more similar to that of humans. For example, marmosets are among only a handful of primates that, like humans, routinely pair bond and care cooperatively for their young. They are also notably pro-social and exhibit social cognitive abilities, such as imitation, that are rare outside of the Apes. In this Primer, we describe key facets of marmoset natural social behavior and demonstrate that emerging behavioral paradigms are well suited to isolate components of marmoset cognition that are highly relevant to humans. These approaches generally embrace natural behavior, which has been rare in conventional primate testing, and thus allow for a new consideration of neural mechanisms underlying primate social cognition and signaling. We anticipate that through parallel technical and paradigmatic advances, marmosets will become an essential model of human social behavior, including its dysfunction in neuropsychiatric disorders.

Music in Research and Rehabilitation of Disorders of Consciousness: Psychological and Neurophysiological Foundations

According to a prevailing view, the visual system works by dissecting stimuli into primitives, whereas the auditory system processes simple and complex stimuli with their corresponding features in parallel. This makes musical stimulation particularly suitable for patients with disorders of consciousness (DoC), because the processing pathways related to complex stimulus features can be preserved even when those related to simple features are no longer available. An additional factor speaking in favor of musical stimulation in DoC is the low efficiency of visual stimulation due to prevalent maladies of vision or gaze fixation in DoC patients. Hearing disorders, in contrast, are much less frequent in DoC, which allows us to use auditory stimulation at various levels of complexity. The current paper overviews empirical data concerning the four main domains of brain functioning in DoC patients that musical stimulation can address: perception (e.g., pitch, timbre, and harmony), cognition (e.g., musical syntax and meaning), emotions, and motor functions. Music can approach basic levels of patients' self-consciousness, which may even exist when all higher-level cognitions are lost, whereas music induced emotions and rhythmic stimulation can affect the dopaminergic reward-system and activity in the motor system respectively, thus serving as a starting point for rehabilitation.

Auditory cortical processing in real-world listening: the auditory system going real

The auditory sense of humans transforms intrinsically senseless pressure waveforms into spectacularly rich perceptual phenomena: the music of Bach or the Beatles, the poetry of Li Bai or Omar Khayyam, or more prosaically the sense of the world filled with objects emitting sounds that is so important for those of us lucky enough to have hearing. Whereas the early representations of sounds in the auditory system are based on their physical structure, higher auditory centers are thought to represent sounds in terms of their perceptual attributes. In this symposium, we will illustrate the current research into this process, using four case studies. We will illustrate how the spectral and temporal properties of sounds are used to bind together, segregate, categorize, and interpret sound patterns on their way to acquire meaning, with important lessons to other sensory systems as well.

Optogenetic stimulation of the auditory pathway for research and future prosthetics

Sound is encoded by spiral ganglion neurons (SGNs) in the hearing organ, the cochlea, with great temporal, spectral and intensity resolution. When hearing fails, electric stimulation by implanted prostheses can partially restore hearing. Optical stimulation promises a fundamental advance of hearing restoration over electric prostheses since light can be conveniently focused and hence might dramatically improve frequency resolution of sound encoding. Combining optogenetic manipulation of neurons with innovative optical stimulation technology promises versatile spatiotemporal stimulation patterns in the auditory system. Therefore, using optical stimulation of SGNs also has great potential for auditory research. Here, I review recent progress in optogenetic stimulation of the auditory system and its potential for future application in research and hearing restoration.

Cortical pitch regions in humans respond primarily to resolved harmonics and are located in specific tonotopic regions of anterior auditory cortex

Pitch is a defining perceptual property of many real-world sounds, including music and speech. Classically, theories of pitch perception have differentiated between temporal and spectral cues. These cues are rendered distinct by the frequency resolution of the ear, such that some frequencies produce "resolved" peaks of excitation in the cochlea, whereas others are "unresolved," providing a pitch cue only via their temporal fluctuations. Despite longstanding interest, the neural structures that process pitch, and their relationship to these cues, have remained controversial. Here, using fMRI in humans, we report the following: (1) consistent with previous reports, all subjects exhibited pitch-sensitive cortical regions that responded substantially more to harmonic tones than frequency-matched noise; (2) the response of these regions was mainly driven by spectrally resolved harmonics, although they also exhibited a weak but consistent response to unresolved harmonics relative to noise; (3) the response of pitch-sensitive regions to a parametric manipulation of resolvability tracked psychophysical discrimination thresholds for the same stimuli; and (4) pitch-sensitive regions were localized to specific tonotopic regions of anterior auditory cortex, extending from a low-frequency region of primary auditory cortex into a more anterior and less frequency-selective region of nonprimary auditory cortex. These results demonstrate that cortical pitch responses are located in a stereotyped region of anterior auditory cortex and are predominantly driven by resolved frequency components in a way that mirrors behavior.

Adaptive auditory computations

The auditory system analyses acoustic signals, extracting their perceptual attributes, and exploiting them to navigate complex auditory environments. While many of the basic transformations that give rise to the early auditory representations are well studied and understood, little is known about the latter cognitive functions that bind, organize, and give meaning to them. They include the ability to attend to, segregate, and track one of many sound sources, to learn its identity, commit it to memory, robustly recognize it, and utilize it to make decisions. This review hints at the profound adaptive influences and contextual effects induced by cognitive functions during these behaviors, and the need for robust tractable mathematical models to understand them.

The harmonic organization of auditory cortex

A fundamental structure of sounds encountered in the natural environment is the harmonicity. Harmonicity is an essential component of music found in all cultures. It is also a unique feature of vocal communication sounds such as human speech and animal vocalizations. Harmonics in sounds are produced by a variety of acoustic generators and reflectors in the natural environment, including vocal apparatuses of humans and animal species as well as music instruments of many types. We live in an acoustic world full of harmonicity. Given the widespread existence of the harmonicity in many aspects of the hearing environment, it is natural to expect that it be reflected in the evolution and development of the auditory systems of both humans and animals, in particular the auditory cortex. Recent neuroimaging and neurophysiology experiments have identified regions of non-primary auditory cortex in humans and non-human primates that have selective responses to harmonic pitches. Accumulating evidence has also shown that neurons in many regions of the auditory cortex exhibit characteristic responses to harmonically related frequencies beyond the range of pitch. Together, these findings suggest that a fundamental organizational principle of auditory cortex is based on the harmonicity. Such an organization likely plays an important role in music processing by the brain. It may also form the basis of the preference for particular classes of music and voice sounds.

Neural representation of harmonic complex tones in primary auditory cortex of the awake monkey

Many natural sounds are periodic and consist of frequencies (harmonics) that are integer multiples of a common fundamental frequency (F0). Such harmonic complex tones (HCTs) evoke a pitch corresponding to their F0, which plays a key role in the perception of speech and music. "Pitch-selective" neurons have been identified in non-primary auditory cortex of marmoset monkeys. Noninvasive studies point to a putative "pitch center" located in a homologous cortical region in humans. It remains unclear whether there is sufficient spectral and temporal information available at the level of primary auditory cortex (A1) to enable reliable pitch extraction in non-primary auditory cortex. Here we evaluated multiunit responses to HCTs in A1 of awake macaques using a stimulus design employed in auditory nerve studies of pitch encoding. The F0 of the HCTs was varied in small increments, such that harmonics of the HCTs fell either on the peak or on the sides of the neuronal pure tone tuning functions. Resultant response-amplitude-versus-harmonic-number functions ("rate-place profiles") displayed a periodic pattern reflecting the neuronal representation of individual HCT harmonics. Consistent with psychoacoustic findings in humans, lower harmonics were better resolved in rate-place profiles than higher harmonics. Lower F0s were also temporally represented by neuronal phase-locking to the periodic waveform of the HCTs. Findings indicate that population responses in A1 contain sufficient spectral and temporal information for extracting the pitch of HCTs by neurons in downstream cortical areas that receive their input from A1.

The role of harmonic resolvability in pitch perception in a vocal nonhuman primate, the common marmoset (Callithrix jacchus)

Pitch is one of the most fundamental percepts in the auditory system and can be extracted using either spectral or temporal information in an acoustic signal. Although pitch perception has been extensively studied in human subjects, it is far less clear how nonhuman primates perceive pitch. We have addressed this question in a series of behavioral studies in which marmosets, a vocal nonhuman primate species, were trained to discriminate complex harmonic tones differing in either spectral (fundamental frequency [f0]) or temporal envelope (repetition rate) cues. We found that marmosets used temporal envelope information to discriminate pitch for acoustic stimuli with higher-order harmonics and lower f0 values and spectral information for acoustic stimuli with lower-order harmonics and higher f0 values. We further measured frequency resolution in marmosets using a psychophysical task in which pure tone thresholds were measured as a function of notched noise masker bandwidth. Results show that only the first four harmonics are resolved at low f0 values and up to 16 harmonics are resolved at higher f0 values. Resolvability in marmosets is different from that in humans, where the first five to nine harmonics are consistently resolved across most f0 values, and is likely the result of a smaller marmoset cochlea. In sum, these results show that marmosets use two mechanisms to extract pitch (harmonic templates [spectral] for resolved harmonics, and envelope extraction [temporal] for unresolved harmonics) and that species differences in stimulus resolvability need to be taken into account when investigating and comparing mechanisms of pitch perception across animals.

Dual-pitch processing mechanisms in primate auditory cortex

Pitch, our perception of how high or low a sound is on a musical scale, is a fundamental perceptual attribute of sounds and is important for both music and speech. After more than a century of research, the exact mechanisms used by the auditory system to extract pitch are still being debated. Theoretically, pitch can be computed using either spectral or temporal acoustic features of a sound. We have investigated how cues derived from the temporal envelope and spectrum of an acoustic signal are used for pitch extraction in the common marmoset (Callithrix jacchus), a vocal primate species, by measuring pitch discrimination behaviorally and examining pitch-selective neuronal responses in auditory cortex. We find that pitch is extracted by marmosets using temporal envelope cues for lower pitch sounds composed of higher-order harmonics, whereas spectral cues are used for higher pitch sounds with lower-order harmonics. Our data support dual-pitch processing mechanisms, originally proposed by psychophysicists based on human studies, whereby pitch is extracted using a combination of temporal envelope and spectral cues.

Revisiting place and temporal theories of pitch

The nature of pitch and its neural coding have been studied for over a century. A popular debate has revolved around the question of whether pitch is coded via "place" cues in the cochlea, or via timing cues in the auditory nerve. In the most recent incarnation of this debate, the role of temporal fine structure has been emphasized in conveying important pitch and speech information, particularly because the lack of temporal fine structure coding in cochlear implants might explain some of the difficulties faced by cochlear implant users in perceiving music and pitch contours in speech. In addition, some studies have postulated that hearing-impaired listeners may have a specific deficit related to processing temporal fine structure. This article reviews some of the recent literature surrounding the debate, and argues that much of the recent evidence suggesting the importance of temporal fine structure processing can also be accounted for using spectral (place) or temporal-envelope cues.

Pitch perception

Pitch is one of the primary auditory sensations and plays a defining role in music, speech, and auditory scene analysis. Although the main physical correlate of pitch is acoustic periodicity, or repetition rate, there are many interactions that complicate the relationship between the physical stimulus and the perception of pitch. In particular, the effects of other acoustic parameters on pitch judgments, and the complex interactions between perceptual organization and pitch, have uncovered interesting perceptual phenomena that should help to reveal the underlying neural mechanisms.