Cortical representations of pitch in monkeys and humans

Aftereffects of visuomanual prism adaptation in auditory modality: Review and perspectives

Visuomanual prism adaptation (PA), which consists of pointing to visual targets while wearing prisms that shift the visual field, is one of the oldest experimental paradigms used to investigate sensorimotor plasticity. Since the 2000's, a growing scientific interest emerged for the expansion of PA to cognitive functions in several sensory modalities. The present work focused on the aftereffects of PA within the auditory modality. Recent studies showed changes in mental representation of auditory frequencies and a shift of divided auditory attention following PA. Moreover, one study demonstrated benefits of PA in a patient suffering from tinnitus. According to these results, we tried to shed light on the following question: How could this be possible to modulate audition by inducing sensorimotor plasticity with glasses? Based on the literature, we suggest a bottom-up attentional mechanism involving cerebellar, parietal, and temporal structures to explain crossmodal aftereffects of PA. This review opens promising new avenues of research about aftereffects of PA in audition and its implication in the therapeutic field of auditory troubles.

Preliminary Evidence for Global Properties in Human Listeners During Natural Auditory Scene Perception

Theories of auditory and visual scene analysis suggest the perception of scenes relies on the identification and segregation of objects within it, resembling a detail-oriented processing style. However, a more global process may occur while analyzing scenes, which has been evidenced in the visual domain. It is our understanding that a similar line of research has not been explored in the auditory domain; therefore, we evaluated the contributions of high-level global and low-level acoustic information to auditory scene perception. An additional aim was to increase the field's ecological validity by using and making available a new collection of high-quality auditory scenes. Participants rated scenes on 8 global properties (e.g., open vs. enclosed) and an acoustic analysis evaluated which low-level features predicted the ratings. We submitted the acoustic measures and average ratings of the global properties to separate exploratory factor analyses (EFAs). The EFA of the acoustic measures revealed a seven-factor structure explaining 57% of the variance in the data, while the EFA of the global property measures revealed a two-factor structure explaining 64% of the variance in the data. Regression analyses revealed each global property was predicted by at least one acoustic variable (R2 = 0.33-0.87). These findings were extended using deep neural network models where we examined correlations between human ratings of global properties and deep embeddings of two computational models: an object-based model and a scene-based model. The results support that participants' ratings are more strongly explained by a global analysis of the scene setting, though the relationship between scene perception and auditory perception is multifaceted, with differing correlation patterns evident between the two models. Taken together, our results provide evidence for the ability to perceive auditory scenes from a global perspective. Some of the acoustic measures predicted ratings of global scene perception, suggesting representations of auditory objects may be transformed through many stages of processing in the ventral auditory stream, similar to what has been proposed in the ventral visual stream. These findings and the open availability of our scene collection will make future studies on perception, attention, and memory for natural auditory scenes possible.

Early social communication through music: State of the art and future perspectives

A growing body of research shows that the universal capacity for music perception and production emerges early in development. Possibly building on this predisposition, caregivers around the world often communicate with infants using songs or speech entailing song-like characteristics. This suggests that music might be one of the earliest developing and most accessible forms of interpersonal communication, providing a platform for studying early communicative behavior. However, little research has examined music in truly communicative contexts. The current work aims to facilitate the development of experimental approaches that rely on dynamic and naturalistic social interactions. We first review two longstanding lines of research that examine musical interactions by focusing either on the caregiver or the infant. These include defining the acoustic and non-acoustic features that characterize infant-directed (ID) music, as well as behavioral and neurophysiological research examining infants' processing of musical timing and pitch. Next, we review recent studies looking at early musical interactions holistically. This research focuses on how caregivers and infants interact using music to achieve co-regulation, mutual engagement, and increase affiliation and prosocial behavior. We conclude by discussing methodological, technological, and analytical advances that might empower a comprehensive study of musical communication in early childhood.

Functional network properties of the auditory cortex

The auditory system transforms auditory stimuli from the external environment into perceptual auditory objects. Recent studies have focused on the contribution of the auditory cortex to this transformation. Other studies have yielded important insights into the contributions of neural activity in the auditory cortex to cognition and decision-making. However, despite this important work, the relationship between auditory-cortex activity and behavior/perception has not been fully elucidated. Two of the more important gaps in our understanding are (1) the specific and differential contributions of different fields of the auditory cortex to auditory perception and behavior and (2) the way networks of auditory neurons impact and facilitate auditory information processing. Here, we focus on recent work from non-human-primate models of hearing and review work related to these gaps and put forth challenges to further our understanding of how single-unit activity and network activity in different cortical fields contribution to behavior and perception.

Distribution of multiunit pitch responses recorded intracranially from human auditory cortex

The perception of pitch is a fundamental percept, which is mediated by the auditory system, requiring the abstraction of stimulus properties related to the spectro-temporal structure of sound. Despite its importance, there is still debate as to the precise areas responsible for its encoding, which may be due to species differences or differences in the recording measures and choices of stimuli used in previous studies. Moreover, it was unknown whether the human brain contains pitch neurons and how distributed such neurons might be. Here, we present the first study to measure multiunit neural activity in response to pitch stimuli in the auditory cortex of intracranially implanted humans. The stimulus sets were regular-interval noise with a pitch strength that is related to the temporal regularity and a pitch value determined by the repetition rate and harmonic complexes. Specifically, we demonstrate reliable responses to these different pitch-inducing paradigms that are distributed throughout Heschl's gyrus, rather than being localized to a particular region, and this finding was evident regardless of the stimulus presented. These data provide a bridge across animal and human studies and aid our understanding of the processing of a critical percept associated with acoustic stimuli.

Innate frequency-discrimination hyperacuity in Williams-Beuren syndrome mice

Williams-Beuren syndrome (WBS) is a rare disorder caused by hemizygous microdeletion of ∼27 contiguous genes. Despite neurodevelopmental and cognitive deficits, individuals with WBS have spared or enhanced musical and auditory abilities, potentially offering an insight into the genetic basis of auditory perception. Here, we report that the mouse models of WBS have innately enhanced frequency-discrimination acuity and improved frequency coding in the auditory cortex (ACx). Chemogenetic rescue showed frequency-discrimination hyperacuity is caused by hyperexcitable interneurons in the ACx. Haploinsufficiency of one WBS gene, Gtf2ird1, replicated WBS phenotypes by downregulating the neuropeptide receptor VIPR1. VIPR1 is reduced in the ACx of individuals with WBS and in the cerebral organoids derived from human induced pluripotent stem cells with the WBS microdeletion. Vipr1 deletion or overexpression in ACx interneurons mimicked or reversed, respectively, the cellular and behavioral phenotypes of WBS mice. Thus, the Gtf2ird1-Vipr1 mechanism in ACx interneurons may underlie the superior auditory acuity in WBS.

MEG correlates of temporal regularity relevant to pitch perception in human auditory cortex

We recorded neural responses in human participants to three types of pitch-evoking regular stimuli at rates below and above the lower limit of pitch using magnetoencephalography (MEG). These bandpass filtered (1–4 kHz) stimuli were harmonic complex tones (HC), click trains (CT), and regular interval noise (RIN). Trials consisted of noise-regular-noise (NRN) or regular-noise-regular (RNR) segments in which the repetition rate (or fundamental frequency F0) was either above (250 Hz) or below (20 Hz) the lower limit of pitch. Neural activation was estimated and compared at the senor and source levels.

The pitch-relevant regular stimuli (F0 = 250 Hz) were all associated with marked evoked responses at around 140 ms after noise-to-regular transitions at both sensor and source levels. In particular, greater evoked responses to pitch-relevant stimuli than pitch-irrelevant stimuli (F0 = 20 Hz) were localized along the Heschl's sulcus around 140 ms. The regularity-onset responses for RIN were much weaker than for the other types of regular stimuli (HC, CT). This effect was localized over planum temporale, planum polare, and lateral Heschl's gyrus. Importantly, the effect of pitch did not interact with the stimulus type. That is, we did not find evidence to support different responses for different types of regular stimuli from the spatiotemporal cluster of the pitch effect (∼140 ms).

The current data demonstrate cortical sensitivity to temporal regularity relevant to pitch that is consistently present across different pitch-relevant stimuli in the Heschl's sulcus between Heschl's gyrus and planum temporale, both of which have been identified as a “pitch center” based on different modalities.

Common principles in the lateralisation of auditory cortex structure and function for vocal communication in primates and rodents

This review summarises recent findings on the lateralisation of communicative sound processing in the auditory cortex (AC) of humans, non-human primates, and rodents. Functional imaging in humans has demonstrated a left hemispheric preference for some acoustic features of speech, but it is unclear to which degree this is caused by bottom-up acoustic feature selectivity or top-down modulation from language areas. Although non-human primates show a less pronounced functional lateralisation in AC, the properties of AC fields and behavioral asymmetries are qualitatively similar. Rodent studies demonstrate microstructural circuits that might underlie bottom-up acoustic feature selectivity in both hemispheres. Functionally, the left AC in the mouse appears to be specifically tuned to communication calls, whereas the right AC may have a more 'generalist' role. Rodents also show anatomical AC lateralisation, such as differences in size and connectivity. Several of these functional and anatomical characteristics are also lateralized in human AC. Thus, complex vocal communication processing shares common features among rodents and primates. We argue that a synthesis of results from humans, non-human primates, and rodents is necessary to identify the neural circuitry of vocal communication processing. However, data from different species and methods are often difficult to compare. Recent advances may enable better integration of methods across species. Efforts to standardise data formats and analysis tools would benefit comparative research and enable synergies between psychological and biological research in the area of vocal communication processing.

Region-dependent Millisecond Time-scale Sensitivity in Spectrotemporal Integrations in Guinea Pig Primary Auditory Cortex

Spectrotemporal integration is a key function of our auditory system for discriminating spectrotemporally complex sounds, such as words. Response latency in the auditory cortex is known to change with the millisecond time-scale depending on acoustic parameters, such as sound frequency and intensity. The functional significance of the millisecond-range latency difference in the integration remains unclear. Actually, whether the auditory cortex has a sensitivity to the millisecond-range difference has not been systematically examined. Herein, we examined the sensitivity in the primary auditory cortex (A1) using voltage-sensitive dye imaging techniques in guinea pigs. Bandpass noise bursts in two different bands (band-noises), centered at 1 and 16 kHz, respectively, were used for the examination. Onset times of individual band-noises (spectral onset-times) were varied to virtually cancel or magnify the latency difference observed with the band-noises. Conventionally defined nonlinear effects in integration were analyzed at A1 with varying sound intensities (or response latencies) and/or spectral onset-times of the two band-noises. The nonlinear effect measured in the high-frequency region of the A1 linearly changed depending on the millisecond difference of the response onset-times, which were estimated from the spatially-local response latencies and spectral onset-times. In contrast, the low-frequency region of the A1 had no significant sensitivity to the millisecond difference. The millisecond-range latency difference may have functional significance in the spectrotemporal integration with the millisecond time-scale sensitivity at the high-frequency region of A1 but not at the low-frequency region.

Distinct neuron populations for simple and compound calls in the primary auditory cortex of awake marmosets

Marmosets are highly social non-human primates that live in families. They exhibit rich vocalization, but the neural basis underlying this complex vocal communication is largely unknown. Here we report the existence of specific neuron populations in marmoset A1 that respond selectively to distinct simple or compound calls made by conspecific marmosets. These neurons were spatially dispersed within A1 but distinct from those responsive to pure tones. Call-selective responses were markedly diminished when individual domains of the call were deleted or the domain sequence was altered, indicating the importance of the global rather than local spectral-temporal properties of the sound. Compound call-selective responses also disappeared when the sequence of the two simple-call components was reversed or their interval was extended beyond 1 s. Light anesthesia largely abolished call-selective responses. Our findings demonstrate extensive inhibitory and facilitatory interactions among call-evoked responses, and provide the basis for further study of circuit mechanisms underlying vocal communication in awake non-human primates.

Human cortical encoding of pitch in tonal and non-tonal languages

Languages can use a common repertoire of vocal sounds to signify distinct meanings. In tonal languages, such as Mandarin Chinese, pitch contours of syllables distinguish one word from another, whereas in non-tonal languages, such as English, pitch is used to convey intonation. The neural computations underlying language specialization in speech perception are unknown. Here, we use a cross-linguistic approach to address this. Native Mandarin- and English- speaking participants each listened to both Mandarin and English speech, while neural activity was directly recorded from the non-primary auditory cortex. Both groups show language-general coding of speaker-invariant pitch at the single electrode level. At the electrode population level, we find language-specific distribution of cortical tuning parameters in Mandarin speakers only, with enhanced sensitivity to Mandarin tone categories. Our results show that speech perception relies upon a shared cortical auditory feature processing mechanism, which may be tuned to the statistics of a given language.

Different languages rely on different vocal sounds to convey meaning. Here the authors show that language-general coding of pitch occurs in the non-primary auditory cortex for both tonal (Mandarin Chinese) and non-tonal (English) languages, with some language specificity on the population level.

Left hemispheric deficit in the sustained neuromagnetic response to periodic click trains in children with ASD

BACKGROUND

Deficits in perception and production of vocal pitch are often observed in people with autism spectrum disorder (ASD), but the neural basis of these deficits is unknown. In magnetoencephalogram (MEG), spectrally complex periodic sounds trigger two continuous neural responses-the auditory steady state response (ASSR) and the sustained field (SF). It has been shown that the SF in neurotypical individuals is associated with low-level analysis of pitch in the 'pitch processing center' of the Heschl's gyrus. Therefore, alternations in this auditory response may reflect atypical processing of vocal pitch. The SF, however, has never been studied in people with ASD.

METHODS

We used MEG and individual brain models to investigate the ASSR and SF evoked by monaural 40 Hz click trains in boys with ASD (N = 35) and neurotypical (NT) boys (N = 35) aged 7-12-years.

RESULTS

In agreement with the previous research in adults, the cortical sources of the SF in children were located in the left and right Heschl's gyri, anterolateral to those of the ASSR. In both groups, the SF and ASSR dominated in the right hemisphere and were higher in the hemisphere contralateral to the stimulated ear. The ASSR increased with age in both NT and ASD children and did not differ between the groups. The SF amplitude did not significantly change between the ages of 7 and 12 years. It was moderately attenuated in both hemispheres and was markedly delayed and displaced in the left hemisphere in boys with ASD. The SF delay in participants with ASD was present irrespective of their intelligence level and severity of autism symptoms.

LIMITATIONS

We did not test the language abilities of our participants. Therefore, the link between SF and processing of vocal pitch in children with ASD remains speculative.

CONCLUSION

Children with ASD demonstrate atypical processing of spectrally complex periodic sound at the level of the core auditory cortex of the left-hemisphere. The observed neural deficit may contribute to speech perception difficulties experienced by children with ASD, including their poor perception and production of linguistic prosody.

Early cortical processing of pitch height and the role of adaptation and musicality

Pitch is an important perceptual feature; however, it is poorly understood how its cortical correlates are shaped by absolute vs relative fundamental frequency (f₀), and by neural adaptation. In this study, we assessed transient and sustained auditory evoked fields (AEFs) at the onset, progression, and offset of short pitch height sequences, taking into account the listener's musicality. We show that neuromagnetic activity reflects absolute f₀ at pitch onset and offset, and relative f₀ at transitions within pitch sequences; further, sequences with fixed f₀ lead to larger response suppression than sequences with variable f₀ contour, and to enhanced offset activity. Musical listeners exhibit stronger f₀-related AEFs and larger differences between their responses to fixed vs variable sequences, both within sequences and at pitch offset. The results resemble prominent psychoacoustic phenomena in the perception of pitch contours; moreover, they suggest a strong influence of adaptive mechanisms on cortical pitch processing which, in turn, might be modulated by a listener's musical expertise.

Complex Acoustic Environments: Review, Framework, and Subjective Model

The concept of complex acoustic environments has appeared in several unrelated research areas within acoustics in different variations. Based on a review of the usage and evolution of this concept in the literature, a relevant framework was developed, which includes nine broad characteristics that are thought to drive the complexity of acoustic scenes. The framework was then used to study the most relevant characteristics for stimuli of realistic, everyday, acoustic scenes: multiple sources, source diversity, reverberation, and the listener’s task. The effect of these characteristics on perceived scene complexity was then evaluated in an exploratory study that reproduced the same stimuli with a three-dimensional loudspeaker array inside an anechoic chamber. Sixty-five subjects listened to the scenes and for each one had to rate 29 attributes, including complexity, both with and without target speech in the scenes. The data were analyzed using three-way principal component analysis with a (2 3 2) Tucker3 model in the dimensions of scales (or ratings), scenes, and subjects, explaining 42% of variation in the data. “Comfort” and “variability” were the dominant scale components, which span the perceived complexity. Interaction effects were observed, including the additional task of attending to target speech that shifted the complexity rating closer to the comfort scale. Also, speech contained in the background scenes introduced a second subject component, which suggests that some subjects are more distracted than others by background speech when listening to target speech. The results are interpreted in light of the proposed framework.

Musicians use speech-specific areas when processing tones: The key to their superior linguistic competence?

It is known that musicians compared to non-musicians have some superior speech and language competence, yet the mechanisms how musical training leads to this advantage are not well specified. This event-related fMRI study confirmed that musicians outperformed non-musicians in processing not only of musical tones but also syllables and identified a network differentiating musicians from non-musicians during processing of linguistic sounds. Within this network, the activation of bilateral superior temporal gyrus was shared with all subjects during processing of the acoustically well-matched musical and linguistic sounds, and with the activation distinguishing tones with a complex harmonic spectrum (bowed tone) from a simpler one (plucked tone). These results confirm that better speech processing in musicians relies on improved cross-domain spectral analysis. Activation of left posterior superior temporal sulcus (pSTS), premotor cortex, inferior frontal and fusiform gyrus (FG) also distinguishing musicians from non-musicians during syllable processing overlapped with the activation segregating linguistic from musical sounds in all subjects. Since these brain-regions were not involved during tone processing in non-musicians, they could code for functions which are specialized for speech. Musicians recruited pSTS and FG during tone processing, thus these speech-specialized brain-areas processed musical sounds in the presence of musical training. This study shows that the linguistic advantage of musicians is linked not only to improved cross-domain spectral analysis, but also to the functional adaptation of brain resources that are specialized for speech, but accessible to the domain of music in the presence of musical training.

Transient and sustained processing of musical consonance in auditory cortex and the effect of musicality

In recent years, electroencephalography and magnetoencephalography (MEG) have both been used to investigate the response in human auditory cortex to musical sounds that are perceived as consonant or dissonant. These studies have typically focused on the transient components of the physiological activity at sound onset, specifically, the N1 wave of the auditory evoked potential and the auditory evoked field, respectively. Unfortunately, the morphology of the N1 wave is confounded by the prominent neural response to energy onset at stimulus onset. It is also the case that the perception of pitch is not limited to sound onset; the perception lasts as long as the note producing it. This suggests that consonance studies should also consider the sustained activity that appears after the transient components die away. The current MEG study shows how energy-balanced sounds can focus the response waves on the consonance-dissonance distinction rather than energy changes and how source modeling techniques can be used to measure the sustained field associated with extended consonant and dissonant sounds. The study shows that musical dyads evoke distinct transient and sustained neuromagnetic responses in auditory cortex. The form of the response depends on both whether the dyads are consonant or dissonant and whether the listeners are musical or nonmusical. The results also show that auditory cortex requires more time for the early transient processing of dissonant dyads than it does for consonant dyads and that the continuous representation of temporal regularity in auditory cortex might be modulated by processes beyond auditory cortex.

NEW & NOTEWORTHY We report a magnetoencephalography (MEG) study on transient and sustained cortical consonance processing. Stimuli were long-duration, energy-balanced, musical dyads that were either consonant or dissonant. Spatiotemporal source analysis revealed specific transient and sustained neuromagnetic activity in response to the dyads; in particular, the morphology of the responses was shaped by the dyad’s consonance and the listener’s musicality. Our results also suggest that the sustained representation of stimulus regularity might be modulated by processes beyond auditory cortex.

The Dialog of Primary and Non-primary Auditory Cortex at the 'Cocktail Party'

In this issue of Neuron, O'Sullivan et al. (2019) measured electro-cortical responses to "cocktail party" speech mixtures in neurosurgical patients and demonstrated that the selective enhancement of attended speech is achieved through the adaptive weighting of primary auditory cortex output by non-primary auditory cortex.

Brain dynamics and connectivity networks under natural auditory stimulation

The analysis of functional magnetic resonance imaging (fMRI) data is challenging when subjects are under exposure to natural sensory stimulation. In this study, a two-stage approach was developed to enable the identification of connectivity networks involved in the processing of information in the brain under natural sensory stimulation. In the first stage, the degree of concordance between the results of inter-subject and intra-subject correlation analyses is assessed statistically. The microstructurally (i.e., cytoarchitectonically) defined brain areas are designated either as concordant in which the results of both correlation analyses are in agreement, or as discordant in which one analysis method shows a higher proportion of supra-threshold voxels than does the other. In the second stage, connectivity networks are identified using the time courses of supra-threshold voxels in brain areas contingent upon the classifications derived in the first stage. In an empirical study, fMRI data were collected from 40 young adults (19 males, average age 22.76 ± 3.25), who underwent auditory stimulation involving sound clips of human voices and animal vocalizations under two operational conditions (i.e., eyes-closed and eyes-open). The operational conditions were designed to assess confounding effects due to auditory instructions or visual perception. The proposed two-stage analysis demonstrated that stress modulation (affective) and language networks in the limbic and cortical structures were respectively engaged during sound stimulation, and presented considerable variability among subjects. The network involved in regulating visuomotor control was sensitive to the eyes-open instruction, and presented only small variations among subjects. A high degree of concordance was observed between the two analyses in the primary auditory cortex which was highly sensitive to the pitch of sound clips. Our results have indicated that brain areas can be identified as concordant or discordant based on the two correlation analyses. This may further facilitate the search for connectivity networks involved in the processing of information under natural sensory stimulation.

Modeling and MEG evidence of early consonance processing in auditory cortex

Pitch is a fundamental attribute of auditory perception. The interaction of concurrent pitches gives rise to a sensation that can be characterized by its degree of consonance or dissonance. In this work, we propose that human auditory cortex (AC) processes pitch and consonance through a common neural network mechanism operating at early cortical levels. First, we developed a new model of neural ensembles incorporating realistic neuronal and synaptic parameters to assess pitch processing mechanisms at early stages of AC. Next, we designed a magnetoencephalography (MEG) experiment to measure the neuromagnetic activity evoked by dyads with varying degrees of consonance or dissonance. MEG results show that dissonant dyads evoke a pitch onset response (POR) with a latency up to 36 ms longer than consonant dyads. Additionally, we used the model to predict the processing time of concurrent pitches; here, consonant pitch combinations were decoded faster than dissonant combinations, in line with the experimental observations. Specifically, we found a striking match between the predicted and the observed latency of the POR as elicited by the dyads. These novel results suggest that consonance processing starts early in human auditory cortex and may share the network mechanisms that are responsible for (single) pitch processing.

Spectro-temporal templates unify the pitch percepts of resolved and unresolved harmonics

Pitch is a fundamental attribute in auditory perception involved in source identification and segregation, music, and speech understanding. Pitch percepts are intimately related to harmonic resolvability of sound. When harmonics are well-resolved, the induced pitch is usually salient and precise, and several models relying on autocorrelations or harmonic spectral templates can account for these percepts. However, when harmonics are not completely resolved, the pitch percept becomes less salient, poorly discriminated, with upper range limited to a few hundred hertz, and spectral templates fail to convey percept since only temporal cues are available. Here, a biologically-motivated model is presented that combines spectral and temporal cues to account for both percepts. The model explains how temporal analysis to estimate the pitch of the unresolved harmonics is performed by bandpass filters implemented by resonances in dendritic trees of neurons in the early auditory pathway. It is demonstrated that organizing and exploiting such dendritic tuning can occur spontaneously in response to white noise. This paper then shows how temporal cues of unresolved harmonics may be integrated with spectrally resolved harmonics, creating spectro-temporal harmonic templates for all pitch percepts. Finally, the model extends its account of monaural pitch percepts to pitches evoked by dichotic binaural stimuli.

Hierarchy of speech-driven spectrotemporal receptive fields in human auditory cortex

Existing data indicate that cortical speech processing is hierarchically organized. Numerous studies have shown that early auditory areas encode fine acoustic details while later areas encode abstracted speech patterns. However, it remains unclear precisely what speech information is encoded across these hierarchical levels. Estimation of speech-driven spectrotemporal receptive fields (STRFs) provides a means to explore cortical speech processing in terms of acoustic or linguistic information associated with characteristic spectrotemporal patterns. Here, we estimate STRFs from cortical responses to continuous speech in fMRI. Using a novel approach based on filtering randomly-selected spectrotemporal modulations (STMs) from aurally-presented sentences, STRFs were estimated for a group of listeners and categorized using a data-driven clustering algorithm. 'Behavioral STRFs' highlighting STMs crucial for speech recognition were derived from intelligibility judgments. Clustering revealed that STRFs in the supratemporal plane represented a broad range of STMs, while STRFs in the lateral temporal lobe represented circumscribed STM patterns important to intelligibility. Detailed analysis recovered a bilateral organization with posterior-lateral regions preferentially processing STMs associated with phonological information and anterior-lateral regions preferentially processing STMs associated with word- and phrase-level information. Regions in lateral Heschl's gyrus preferentially processed STMs associated with vocalic information (pitch).

Distributed representation of vocalization pitch in marmoset primary auditory cortex

The pitch of vocalizations is a key communication feature aiding recognition of individuals and separating sound sources in complex acoustic environments. The neural representation of the pitch of periodic sounds is well defined. However, many natural sounds, like complex vocalizations, contain rich, aperiodic or not strictly periodic frequency content and/or include high-frequency components, but still evoke a strong sense of pitch. Indeed, such sounds are the rule, not the exception but the cortical mechanisms for encoding pitch of such sounds are unknown. We investigated how neurons in the high-frequency representation of primary auditory cortex (A1) of marmosets encoded changes in pitch of four natural vocalizations, two centred around a dominant frequency similar to the neuron's best sensitivity and two around a much lower dominant frequency. Pitch was varied over a fine range that can be used by marmosets to differentiate individuals. The responses of most high-frequency A1 neurons were sensitive to pitch changes in all four vocalizations, with a smaller proportion of the neurons showing pitch-insensitive responses. Classically defined excitatory drive, from the neuron's monaural frequency response area, predicted responses to changes in vocalization pitch in <30% of neurons suggesting most pitch tuning observed is not simple frequency-level response. Moreover, 39% of A1 neurons showed call-invariant tuning of pitch. These results suggest that distributed activity across A1 can represent the pitch of natural sounds over a fine, functionally relevant range, and exhibits pitch tuning for vocalizations within and outside the classical neural tuning area.

Cortical Coding of Auditory Features

How the cerebral cortex encodes auditory features of biologically important sounds, including speech and music, is one of the most important questions in auditory neuroscience. The pursuit to understand related neural coding mechanisms in the mammalian auditory cortex can be traced back several decades to the early exploration of the cerebral cortex. Significant progress in this field has been made in the past two decades with new technical and conceptual advances. This article reviews the progress and challenges in this area of research.

Functional Imaging of Audio-Visual Selective Attention in Monkeys and Humans: How do Lapses in Monkey Performance Affect Cross-Species Correspondences?

The cross-species correspondences and differences in how attention modulates brain responses in humans and animal models are poorly understood. We trained 2 monkeys to perform an audio–visual selective attention task during functional magnetic resonance imaging (fMRI), rewarding them to attend to stimuli in one modality while ignoring those in the other. Monkey fMRI identified regions strongly modulated by auditory or visual attention. Surprisingly, auditory attention-related modulations were much more restricted in monkeys than humans performing the same tasks during fMRI. Further analyses ruled out trivial explanations, suggesting that labile selective-attention performance was associated with inhomogeneous modulations in wide cortical regions in the monkeys. The findings provide initial insights into how audio–visual selective attention modulates the primate brain, identify sources for “lost” attention effects in monkeys, and carry implications for modeling the neurobiology of human cognition with nonhuman animals.

Cortical processing of pitch: Model-based encoding and decoding of auditory fMRI responses to real-life sounds

Pitch is a perceptual attribute related to the fundamental frequency (or periodicity) of a sound. So far, the cortical processing of pitch has been investigated mostly using synthetic sounds. However, the complex harmonic structure of natural sounds may require different mechanisms for the extraction and analysis of pitch. This study investigated the neural representation of pitch in human auditory cortex using model-based encoding and decoding analyses of high field (7 T) functional magnetic resonance imaging (fMRI) data collected while participants listened to a wide range of real-life sounds. Specifically, we modeled the fMRI responses as a function of the sounds' perceived pitch height and salience (related to the fundamental frequency and the harmonic structure respectively), which we estimated with a computational algorithm of pitch extraction (de Cheveigné and Kawahara, 2002). First, using single-voxel fMRI encoding, we identified a pitch-coding region in the antero-lateral Heschl's gyrus (HG) and adjacent superior temporal gyrus (STG). In these regions, the pitch representation model combining height and salience predicted the fMRI responses comparatively better than other models of acoustic processing and, in the right hemisphere, better than pitch representations based on height/salience alone. Second, we assessed with model-based decoding that multi-voxel response patterns of the identified regions are more informative of perceived pitch than the remainder of the auditory cortex. Further multivariate analyses showed that complementing a multi-resolution spectro-temporal sound representation with pitch produces a small but significant improvement to the decoding of complex sounds from fMRI response patterns. In sum, this work extends model-based fMRI encoding and decoding methods - previously employed to examine the representation and processing of acoustic sound features in the human auditory system - to the representation and processing of a relevant perceptual attribute such as pitch. Taken together, the results of our model-based encoding and decoding analyses indicated that the pitch of complex real life sounds is extracted and processed in lateral HG/STG regions, at locations consistent with those indicated in several previous fMRI studies using synthetic sounds. Within these regions, pitch-related sound representations reflect the modulatory combination of height and the salience of the pitch percept.

Gaze-Stabilizing Central Vestibular Neurons Project Asymmetrically to Extraocular Motoneuron Pools

Within reflex circuits, specific anatomical projections allow central neurons to relay sensations to effectors that generate movements. A major challenge is to relate anatomical features of central neural populations, such as asymmetric connectivity, to the computations the populations perform. To address this problem, we mapped the anatomy, modeled the function, and discovered a new behavioral role for a genetically defined population of central vestibular neurons in rhombomeres 5–7 of larval zebrafish. First, we found that neurons within this central population project preferentially to motoneurons that move the eyes downward. Concordantly, when the entire population of asymmetrically projecting neurons was stimulated collectively, only downward eye rotations were observed, demonstrating a functional correlate of the anatomical bias. When these neurons are ablated, fish failed to rotate their eyes following either nose-up or nose-down body tilts. This asymmetrically projecting central population thus participates in both upward and downward gaze stabilization. In addition to projecting to motoneurons, central vestibular neurons also receive direct sensory input from peripheral afferents. To infer whether asymmetric projections can facilitate sensory encoding or motor output, we modeled differentially projecting sets of central vestibular neurons. Whereas motor command strength was independent of projection allocation, asymmetric projections enabled more accurate representation of nose-up stimuli. The model shows how asymmetric connectivity could enhance the representation of imbalance during nose-up postures while preserving gaze stabilization performance. Finally, we found that central vestibular neurons were necessary for a vital behavior requiring maintenance of a nose-up posture: swim bladder inflation. These observations suggest that asymmetric connectivity in the vestibular system facilitates representation of ethologically relevant stimuli without compromising reflexive behavior.

SIGNIFICANCE STATEMENT Interneuron populations use specific anatomical projections to transform sensations into reflexive actions. Here we examined how the anatomical composition of a genetically defined population of balance interneurons in the larval zebrafish relates to the computations it performs. First, we found that the population of interneurons that stabilize gaze preferentially project to motoneurons that move the eyes downward. Next, we discovered through modeling that such projection patterns can enhance the encoding of nose-up sensations without compromising gaze stabilization. Finally, we found that loss of these interneurons impairs a vital behavior, swim bladder inflation, that relies on maintaining a nose-up posture. These observations suggest that anatomical specialization permits neural circuits to represent relevant features of the environment without compromising behavior.

Amplitude modulation rate dependent topographic organization of the auditory steady-state response in human auditory cortex

Periodic modulations of an acoustic feature, such as amplitude over a certain frequency range, leads to phase locking of neural responses to the envelope of the modulation. Using electrophysiological methods this neural activity pattern, also called the auditory steady-state response (aSSR), is visible following frequency transformation of the evoked response as a clear spectral peak at the modulation frequency. Despite several studies employing the aSSR that show, for example, strongest responses for ∼40 Hz and an overall right-hemispheric dominance, it has not been investigated so far to what extent within auditory cortex different modulation frequencies elicit aSSRs at a homogenous source or whether the localization of the aSSR is topographically organized in a systematic manner. The latter would be suggested by previous neuroimaging works in monkeys and humans showing a periodotopic organization within and across distinct auditory fields. However, the sluggishness of the signal from these neuroimaging works prohibit inferences with regards to the fine-temporal features of the neural response. In the present study, we employed amplitude-modulated (AM) sounds over a range between 4 and 85 Hz to elicit aSSRs while recording brain activity via magnetoencephalography (MEG). Using beamforming and a fine spatially resolved grid restricted to auditory cortical processing regions, our study revealed a topographic representation of the aSSR that depends on AM rate, in particular in the medial-lateral (bilateral) and posterior-anterior (right auditory cortex) direction. In summary, our findings confirm previous studies that showing different AM rates to elicit maximal response in distinct neural populations. They extend these findings however by also showing that these respective neural ensembles in auditory cortex actually phase lock their activity over a wide modulation frequency range.

Cortical Correlates of the Auditory Frequency-Following and Onset Responses: EEG and fMRI Evidence

The frequency-following response (FFR) is a measure of the brain's periodic sound encoding. It is of increasing importance for studying the human auditory nervous system due to numerous associations with auditory cognition and dysfunction. Although the FFR is widely interpreted as originating from brainstem nuclei, a recent study using MEG suggested that there is also a right-lateralized contribution from the auditory cortex at the fundamental frequency (Coffey et al., 2016b). Our objectives in the present work were to validate and better localize this result using a completely different neuroimaging modality and to document the relationships between the FFR, the onset response, and cortical activity. Using a combination of EEG, fMRI, and diffusion-weighted imaging, we show that activity in the right auditory cortex is related to individual differences in FFR-fundamental frequency (f₀) strength, a finding that was replicated with two independent stimulus sets, with and without acoustic energy at the fundamental frequency. We demonstrate a dissociation between this FFR-f₀-sensitive response in the right and an area in left auditory cortex that is sensitive to individual differences in the timing of initial response to sound onset. Relationships to timing and their lateralization are supported by parallels in the microstructure of the underlying white matter, implicating a mechanism involving neural conduction efficiency. These data confirm that the FFR has a cortical contribution and suggest ways in which auditory neuroscience may be advanced by connecting early sound representation to measures of higher-level sound processing and cognitive function.

SIGNIFICANCE STATEMENT

The frequency-following response (FFR) is an EEG signal that is used to explore how the auditory system encodes temporal regularities in sound and is related to differences in auditory function between individuals. It is known that brainstem nuclei contribute to the FFR, but recent findings of an additional cortical source are more controversial. Here, we use fMRI to validate and extend the prediction from MEG data of a right auditory cortex contribution to the FFR. We also demonstrate a dissociation between FFR-related cortical activity from that related to the latency of the response to sound onset, which is found in left auditory cortex. The findings provide a clearer picture of cortical processes for analysis of sound features.

Language related differences of the sustained response evoked by natural speech sounds

In tonal languages, such as Mandarin Chinese, the pitch contour of vowels discriminates lexical meaning, which is not the case in non-tonal languages such as German. Recent data provide evidence that pitch processing is influenced by language experience. However, there are still many open questions concerning the representation of such phonological and language-related differences at the level of the auditory cortex (AC). Using magnetoencephalography (MEG), we recorded transient and sustained auditory evoked fields (AEF) in native Chinese and German speakers to investigate language related phonological and semantic aspects in the processing of acoustic stimuli. AEF were elicited by spoken meaningful and meaningless syllables, by vowels, and by a French horn tone. Speech sounds were recorded from a native speaker and showed frequency-modulations according to the pitch-contours of Mandarin. The sustained field (SF) evoked by natural speech signals was significantly larger for Chinese than for German listeners. In contrast, the SF elicited by a horn tone was not significantly different between groups. Furthermore, the SF of Chinese subjects was larger when evoked by meaningful syllables compared to meaningless ones, but there was no significant difference regarding whether vowels were part of the Chinese phonological system or not. Moreover, the N100m gave subtle but clear evidence that for Chinese listeners other factors than purely physical properties play a role in processing meaningful signals. These findings show that the N100 and the SF generated in Heschl’s gyrus are influenced by language experience, which suggests that AC activity related to specific pitch contours of vowels is influenced in a top-down fashion by higher, language related areas. Such interactions are in line with anatomical findings and neuroimaging data, as well as with the dual-stream model of language of Hickok and Poeppel that highlights the close and reciprocal interaction between superior temporal gyrus and sulcus.

Correlation between the frequency difference limen and an index based on principal component analysis of the frequency-following response of normal hearing listeners

Subcortical phase locking tends to reflect performance differences in tasks related to pitch perception across different types of populations. Enhancement or attenuation in its strength may correspond to population excellence or deficiency in pitch perception. However, it is still unclear whether differences in perceptual capability among individuals with normal hearing can be predicted by subcortical phase locking. In this study, we examined the brain-behavior relationship between frequency-following responses (FFRs) evoked by pure/sweeping tones and frequency difference limens (FDLs). FFRs are considered to reflect subcortical phase locking, and FDLs are a psychophysical measure of behavioral performance in pitch discrimination. Traditional measures of FFR strength were found to be poorly correlated with FDL. Here, we introduced principal component analysis into FFR analysis and extracted an FFR component that was correlated with individual pitch discrimination. The absolute value of the score of this FFR principal component (but not the original score) was negatively correlated with FDL, regardless of stimulus type. The topographic distribution of this component was relatively constant across individuals and across stimulus types, and the inferior colliculus was identified as its origin. The findings suggest that subcortical phase locking at certain but not all FFR generators carries the neural information required for the prediction of individual pitch perception among humans with normal hearing.

Cortical auditory evoked potentials (CAEPs) represent neural cues relevant to pitch perception

Components of auditory event-related potentials (ERPs) may represent various aspects of the cortical processing of pitch. However, evidence hints an earlier representation of pitch perception in auditory ERPs of cortical origin. In this study, we examined whether earlier waves in cortical auditory evoked potentials (CAEPs) might reflect pitch-relevant features of both listeners and stimuli. CAEPs were elicited by pure tones and sweeping tones, and individual behavioral performance in pitch discrimination reflected by frequency difference limen (FDL) was also measured. Results show that CAEPs evoked by sweeping tones significantly correlate to FDL around ~50 ms, but CAEPs evoked by pure tones do not. Also, CAEPs are significantly affected by pitch-shift direction around ~130 ms. CAEPs evoked by ascending sweeping tones are larger in magnitude than those evoked by descending ones. Therefore, listeners' personal attributes relevant to pitch perception have already been reflected at a very early stage of cortical auditory processing, whilst certain pitch-related features of stimuli are recognized and represented at a later stage.

Functional Topography of Human Auditory Cortex

Functional and anatomical studies have clearly demonstrated that auditory cortex is populated by multiple subfields. However, functional characterization of those fields has been largely the domain of animal electrophysiology, limiting the extent to which human and animal research can inform each other. In this study, we used high-resolution functional magnetic resonance imaging to characterize human auditory cortical subfields using a variety of low-level acoustic features in the spectral and temporal domains. Specifically, we show that topographic gradients of frequency preference, or tonotopy, extend along two axes in human auditory cortex, thus reconciling historical accounts of a tonotopic axis oriented medial to lateral along Heschl's gyrus and more recent findings emphasizing tonotopic organization along the anterior-posterior axis. Contradictory findings regarding topographic organization according to temporal modulation rate in acoustic stimuli, or "periodotopy," are also addressed. Although isolated subregions show a preference for high rates of amplitude-modulated white noise (AMWN) in our data, large-scale "periodotopic" organization was not found. Organization by AM rate was correlated with dominant pitch percepts in AMWN in many regions. In short, our data expose early auditory cortex chiefly as a frequency analyzer, and spectral frequency, as imposed by the sensory receptor surface in the cochlea, seems to be the dominant feature governing large-scale topographic organization across human auditory cortex.

SIGNIFICANCE STATEMENT

In this study, we examine the nature of topographic organization in human auditory cortex with fMRI. Topographic organization by spectral frequency (tonotopy) extended in two directions: medial to lateral, consistent with early neuroimaging studies, and anterior to posterior, consistent with more recent reports. Large-scale organization by rates of temporal modulation (periodotopy) was correlated with confounding spectral content of amplitude-modulated white-noise stimuli. Together, our results suggest that the organization of human auditory cortex is driven primarily by its response to spectral acoustic features, and large-scale periodotopy spanning across multiple regions is not supported. This fundamental information regarding the functional organization of early auditory cortex will inform our growing understanding of speech perception and the processing of other complex sounds.

Effects of Electrical Stimulation in the Inferior Colliculus on Frequency Discrimination by Rhesus Monkeys and Implications for the Auditory Midbrain Implant

Understanding the relationship between the auditory selectivity of neurons and their contribution to perception is critical to the design of effective auditory brain prosthetics. These prosthetics seek to mimic natural activity patterns to achieve desired perceptual outcomes. We measured the contribution of inferior colliculus (IC) sites to perception using combined recording and electrical stimulation. Monkeys performed a frequency-based discrimination task, reporting whether a probe sound was higher or lower in frequency than a reference sound. Stimulation pulses were paired with the probe sound on 50% of trials (0.5-80 μA, 100-300 Hz, n = 172 IC locations in 3 rhesus monkeys). Electrical stimulation tended to bias the animals' judgments in a fashion that was coarsely but significantly correlated with the best frequency of the stimulation site compared with the reference frequency used in the task. Although there was considerable variability in the effects of stimulation (including impairments in performance and shifts in performance away from the direction predicted based on the site's response properties), the results indicate that stimulation of the IC can evoke percepts correlated with the frequency-tuning properties of the IC. Consistent with the implications of recent human studies, the main avenue for improvement for the auditory midbrain implant suggested by our findings is to increase the number and spatial extent of electrodes, to increase the size of the region that can be electrically activated, and to provide a greater range of evoked percepts.

SIGNIFICANCE STATEMENT

Patients with hearing loss stemming from causes that interrupt the auditory pathway after the cochlea need a brain prosthetic to restore hearing. Recently, prosthetic stimulation in the human inferior colliculus (IC) was evaluated in a clinical trial. Thus far, speech understanding was limited for the subjects and this limitation is thought to be partly due to challenges in harnessing the sound frequency representation in the IC. Here, we tested the effects of IC stimulation in monkeys trained to report the sound frequencies they heard. Our results indicate that the IC can be used to introduce a range of frequency percepts and suggest that placement of a greater number of electrode contacts may improve the effectiveness of such implants.

Insights on the Neuromagnetic Representation of Temporal Asymmetry in Human Auditory Cortex

Communication sounds are typically asymmetric in time and human listeners are highly sensitive to this short-term temporal asymmetry. Nevertheless, causal neurophysiological correlates of auditory perceptual asymmetry remain largely elusive to our current analyses and models. Auditory modelling and animal electrophysiological recordings suggest that perceptual asymmetry results from the presence of multiple time scales of temporal integration, central to the auditory periphery. To test this hypothesis we recorded auditory evoked fields (AEF) elicited by asymmetric sounds in humans. We found a strong correlation between perceived tonal salience of ramped and damped sinusoids and the AEFs, as quantified by the amplitude of the N100m dynamics. The N100m amplitude increased with stimulus half-life time, showing a maximum difference between the ramped and damped stimulus for a modulation half-life time of 4 ms which is greatly reduced at 0.5 ms and 32 ms. This behaviour of the N100m closely parallels psychophysical data in a manner that: i) longer half-life times are associated with a stronger tonal percept, and ii) perceptual differences between damped and ramped are maximal at 4 ms half-life time. Interestingly, differences in evoked fields were significantly stronger in the right hemisphere, indicating some degree of hemispheric specialisation. Furthermore, the N100m magnitude was successfully explained by a pitch perception model using multiple scales of temporal integration of auditory nerve activity patterns. This striking correlation between AEFs, perception, and model predictions suggests that the physiological mechanisms involved in the processing of pitch evoked by temporal asymmetric sounds are reflected in the N100m.

Learning Pitch with STDP: A Computational Model of Place and Temporal Pitch Perception Using Spiking Neural Networks

Pitch perception is important for understanding speech prosody, music perception, recognizing tones in tonal languages, and perceiving speech in noisy environments. The two principal pitch perception theories consider the place of maximum neural excitation along the auditory nerve and the temporal pattern of the auditory neurons’ action potentials (spikes) as pitch cues. This paper describes a biophysical mechanism by which fine-structure temporal information can be extracted from the spikes generated at the auditory periphery. Deriving meaningful pitch-related information from spike times requires neural structures specialized in capturing synchronous or correlated activity from amongst neural events. The emergence of such pitch-processing neural mechanisms is described through a computational model of auditory processing. Simulation results show that a correlation-based, unsupervised, spike-based form of Hebbian learning can explain the development of neural structures required for recognizing the pitch of simple and complex tones, with or without the fundamental frequency. The temporal code is robust to variations in the spectral shape of the signal and thus can explain the phenomenon of pitch constancy.

Pitch is the perceptual correlate of sound frequency. Our auditory system has a sophisticated mechanism to process and perceive the neural information corresponding to pitch. This mechanism employs both the place and the temporal pattern of pitch-evoked neural events. Based on the known functions of the auditory system, we develop a computational model of pitch perception using a network of neurons with modifiable connections. We demonstrate that a well-known neural learning rule that is based on the timing of the neural events can identify and strengthen the neuronal connections that are most effective for the extraction of pitch. By providing an insight into how the auditory system interprets pitch information, the results of our study can be used to develop improved sound processing strategies for cochlear implants. In cochlear implant hearing, auditory percept is generated by stimulating the auditory neurons with controlled electrical impulses, enhancing which with the help of the model would lead to a better representation of pitch and would subsequently improve music perception and speech understanding in noisy environments in cochlear implant users.

A biological rationale for musical consonance

The basis of musical consonance has been debated for centuries without resolution. Three interpretations have been considered: (i) that consonance derives from the mathematical simplicity of small integer ratios; (ii) that consonance derives from the physical absence of interference between harmonic spectra; and (iii) that consonance derives from the advantages of recognizing biological vocalization and human vocalization in particular. Whereas the mathematical and physical explanations are at odds with the evidence that has now accumulated, biology provides a plausible explanation for this central issue in music and audition.

The role of inhibition in a computational model of an auditory cortical neuron during the encoding of temporal information

In auditory cortex, temporal information within a sound is represented by two complementary neural codes: a temporal representation based on stimulus-locked firing and a rate representation, where discharge rate co-varies with the timing between acoustic events but lacks a stimulus-synchronized response. Using a computational neuronal model, we find that stimulus-locked responses are generated when sound-evoked excitation is combined with strong, delayed inhibition. In contrast to this, a non-synchronized rate representation is generated when the net excitation evoked by the sound is weak, which occurs when excitation is coincident and balanced with inhibition. Using single-unit recordings from awake marmosets (Callithrix jacchus), we validate several model predictions, including differences in the temporal fidelity, discharge rates and temporal dynamics of stimulus-evoked responses between neurons with rate and temporal representations. Together these data suggest that feedforward inhibition provides a parsimonious explanation of the neural coding dichotomy observed in auditory cortex.

From where to what: a neuroanatomically based evolutionary model of the emergence of speech in humans

In the brain of primates, the auditory cortex connects with the frontal lobe via the temporal pole (auditory ventral stream; AVS) and via the inferior parietal lobule (auditory dorsal stream; ADS). The AVS is responsible for sound recognition, and the ADS for sound-localization, voice detection and audio-visual integration. I propose that the primary role of the ADS in monkeys/apes is the perception and response to contact calls. These calls are exchanged between tribe members (e.g., mother-offspring) and are used for monitoring location. Perception of contact calls occurs by the ADS detecting a voice, localizing it, and verifying that the corresponding face is out of sight. The auditory cortex then projects to parieto-frontal visuospatial regions (visual dorsal stream) for searching the caller, and via a series of frontal lobe-brainstem connections, a contact call is produced in return. Because the human ADS processes also speech production and repetition, I further describe a course for the development of speech in humans. I propose that, due to duplication of a parietal region and its frontal projections, and strengthening of direct frontal-brainstem connections, the ADS converted auditory input directly to vocal regions in the frontal lobe, which endowed early Hominans with partial vocal control. This enabled offspring to modify their contact calls with intonations for signaling different distress levels to their mother. Vocal control could then enable question-answer conversations, by offspring emitting a low-level distress call for inquiring about the safety of objects, and mothers responding with high- or low-level distress calls. Gradually, the ADS and the direct frontal-brainstem connections became more robust and vocal control became more volitional. Eventually, individuals were capable of inventing new words and offspring were capable of inquiring about objects in their environment and learning their names via mimicry.

From where to what: a neuroanatomically based evolutionary model of the emergence of speech in humans

In the brain of primates, the auditory cortex connects with the frontal lobe via the temporal pole (auditory ventral stream; AVS) and via the inferior parietal lobe (auditory dorsal stream; ADS). The AVS is responsible for sound recognition, and the ADS for sound-localization, voice detection and integration of calls with faces. I propose that the primary role of the ADS in non-human primates is the detection and response to contact calls. These calls are exchanged between tribe members (e.g., mother-offspring) and are used for monitoring location. Detection of contact calls occurs by the ADS identifying a voice, localizing it, and verifying that the corresponding face is out of sight. Once a contact call is detected, the primate produces a contact call in return via descending connections from the frontal lobe to a network of limbic and brainstem regions. Because the ADS of present day humans also performs speech production, I further propose an evolutionary course for the transition from contact call exchange to an early form of speech. In accordance with this model, structural changes to the ADS endowed early members of the genus Homo with partial vocal control. This development was beneficial as it enabled offspring to modify their contact calls with intonations for signaling high or low levels of distress to their mother. Eventually, individuals were capable of participating in yes-no question-answer conversations. In these conversations the offspring emitted a low-level distress call for inquiring about the safety of objects (e.g., food), and his/her mother responded with a high- or low-level distress call to signal approval or disapproval of the interaction. Gradually, the ADS and its connections with brainstem motor regions became more robust and vocal control became more volitional. Speech emerged once vocal control was sufficient for inventing novel calls.

From where to what: a neuroanatomically based evolutionary model of the emergence of speech in humans

In the brain of primates, the auditory cortex connects with the frontal lobe via the temporal pole (auditory ventral stream; AVS) and via the inferior parietal lobe (auditory dorsal stream; ADS). The AVS is responsible for sound recognition, and the ADS for sound-localization, voice detection and integration of calls with faces. I propose that the primary role of the ADS in non-human primates is the detection and response to contact calls. These calls are exchanged between tribe members (e.g., mother-offspring) and are used for monitoring location. Detection of contact calls occurs by the ADS identifying a voice, localizing it, and verifying that the corresponding face is out of sight. Once a contact call is detected, the primate produces a contact call in return via descending connections from the frontal lobe to a network of limbic and brainstem regions. Because the ADS of present day humans also performs speech production, I further propose an evolutionary course for the transition from contact call exchange to an early form of speech. In accordance with this model, structural changes to the ADS endowed early members of the genus Homo with partial vocal control. This development was beneficial as it enabled offspring to modify their contact calls with intonations for signaling high or low levels of distress to their mother. Eventually, individuals were capable of participating in yes-no question-answer conversations. In these conversations the offspring emitted a low-level distress call for inquiring about the safety of objects (e.g., food), and his/her mother responded with a high- or low-level distress call to signal approval or disapproval of the interaction. Gradually, the ADS and its connections with brainstem motor regions became more robust and vocal control became more volitional. Speech emerged once vocal control was sufficient for inventing novel calls.

Auditory cortical processing in real-world listening: the auditory system going real

The auditory sense of humans transforms intrinsically senseless pressure waveforms into spectacularly rich perceptual phenomena: the music of Bach or the Beatles, the poetry of Li Bai or Omar Khayyam, or more prosaically the sense of the world filled with objects emitting sounds that is so important for those of us lucky enough to have hearing. Whereas the early representations of sounds in the auditory system are based on their physical structure, higher auditory centers are thought to represent sounds in terms of their perceptual attributes. In this symposium, we will illustrate the current research into this process, using four case studies. We will illustrate how the spectral and temporal properties of sounds are used to bind together, segregate, categorize, and interpret sound patterns on their way to acquire meaning, with important lessons to other sensory systems as well.

Neural representation of concurrent harmonic sounds in monkey primary auditory cortex: implications for models of auditory scene analysis

The ability to attend to a particular sound in a noisy environment is an essential aspect of hearing. To accomplish this feat, the auditory system must segregate sounds that overlap in frequency and time. Many natural sounds, such as human voices, consist of harmonics of a common fundamental frequency (F0). Such harmonic complex tones (HCTs) evoke a pitch corresponding to their F0. A difference in pitch between simultaneous HCTs provides a powerful cue for their segregation. The neural mechanisms underlying concurrent sound segregation based on pitch differences are poorly understood. Here, we examined neural responses in monkey primary auditory cortex (A1) to two concurrent HCTs that differed in F0 such that they are heard as two separate "auditory objects" with distinct pitches. We found that A1 can resolve, via a rate-place code, the lower harmonics of both HCTs, a prerequisite for deriving their pitches and for their perceptual segregation. Onset asynchrony between the HCTs enhanced the neural representation of their harmonics, paralleling their improved perceptual segregation in humans. Pitches of the concurrent HCTs could also be temporally represented by neuronal phase-locking at their respective F0s. Furthermore, a model of A1 responses using harmonic templates could qualitatively reproduce psychophysical data on concurrent sound segregation in humans. Finally, we identified a possible intracortical homolog of the "object-related negativity" recorded noninvasively in humans, which correlates with the perceptual segregation of concurrent sounds. Findings indicate that A1 contains sufficient spectral and temporal information for segregating concurrent sounds based on differences in pitch.

Cortical pitch regions in humans respond primarily to resolved harmonics and are located in specific tonotopic regions of anterior auditory cortex

Pitch is a defining perceptual property of many real-world sounds, including music and speech. Classically, theories of pitch perception have differentiated between temporal and spectral cues. These cues are rendered distinct by the frequency resolution of the ear, such that some frequencies produce "resolved" peaks of excitation in the cochlea, whereas others are "unresolved," providing a pitch cue only via their temporal fluctuations. Despite longstanding interest, the neural structures that process pitch, and their relationship to these cues, have remained controversial. Here, using fMRI in humans, we report the following: (1) consistent with previous reports, all subjects exhibited pitch-sensitive cortical regions that responded substantially more to harmonic tones than frequency-matched noise; (2) the response of these regions was mainly driven by spectrally resolved harmonics, although they also exhibited a weak but consistent response to unresolved harmonics relative to noise; (3) the response of pitch-sensitive regions to a parametric manipulation of resolvability tracked psychophysical discrimination thresholds for the same stimuli; and (4) pitch-sensitive regions were localized to specific tonotopic regions of anterior auditory cortex, extending from a low-frequency region of primary auditory cortex into a more anterior and less frequency-selective region of nonprimary auditory cortex. These results demonstrate that cortical pitch responses are located in a stereotyped region of anterior auditory cortex and are predominantly driven by resolved frequency components in a way that mirrors behavior.

Behavioral correlates of auditory streaming in rhesus macaques

Perceptual representations of auditory stimuli (i.e., sounds) are derived from the auditory system's ability to segregate and group the spectral, temporal, and spatial features of auditory stimuli-a process called "auditory scene analysis". Psychophysical studies have identified several of the principles and mechanisms that underlie a listener's ability to segregate and group acoustic stimuli. One important psychophysical task that has illuminated many of these principles and mechanisms is the "streaming" task. Despite the wide use of this task to study psychophysical mechanisms of human audition, no studies have explicitly tested the streaming abilities of non-human animals using the standard methodologies employed in human-audition studies. Here, we trained rhesus macaques to participate in the streaming task using methodologies and controls similar to those presented in previous human studies. Overall, we found that the monkeys' behavioral reports were qualitatively consistent with those of human listeners, thus suggesting that this task may be a valuable tool for future neurophysiological studies.

Hemispheric asymmetry of primary auditory cortex and Heschl's gyrus in schizophrenia and nonpsychiatric brains

Heschl's gyrus (HG) is reported to have a normal left>right hemispheric volume asymmetry, and reduced asymmetry in schizophrenia. Primary auditory cortex (A1) occupies the caudal-medial surface of HG, but it is unclear if A1 has normal asymmetry, or whether its asymmetry is altered in schizophrenia. To address these issues, we compared bilateral gray matter volumes of HG and A1, and neuron density and number in A1, in autopsy brains from male subjects with or without schizophrenia. Comparison of diagnostic groups did not reveal altered gray matter volumes, neuron density, neuron number or hemispheric asymmetries in schizophrenia. With respect to hemispheric differences, HG displayed a clear left>right asymmetry of gray matter volume. Area A1 occupied nearly half of HG, but had less consistent volume asymmetry, that was clearly present only in a subgroup of archival brains from elderly subjects. Neuron counts, in layers IIIb-c and V-VI, showed that the A1 volume asymmetry reflected differences in neuron number, and was not caused simply by changes in neuron density. Our findings confirm previous reports of striking hemispheric asymmetry of HG, and additionally show evidence that A1 has a corresponding asymmetry, although less consistent than that of HG.

The harmonic organization of auditory cortex

A fundamental structure of sounds encountered in the natural environment is the harmonicity. Harmonicity is an essential component of music found in all cultures. It is also a unique feature of vocal communication sounds such as human speech and animal vocalizations. Harmonics in sounds are produced by a variety of acoustic generators and reflectors in the natural environment, including vocal apparatuses of humans and animal species as well as music instruments of many types. We live in an acoustic world full of harmonicity. Given the widespread existence of the harmonicity in many aspects of the hearing environment, it is natural to expect that it be reflected in the evolution and development of the auditory systems of both humans and animals, in particular the auditory cortex. Recent neuroimaging and neurophysiology experiments have identified regions of non-primary auditory cortex in humans and non-human primates that have selective responses to harmonic pitches. Accumulating evidence has also shown that neurons in many regions of the auditory cortex exhibit characteristic responses to harmonically related frequencies beyond the range of pitch. Together, these findings suggest that a fundamental organizational principle of auditory cortex is based on the harmonicity. Such an organization likely plays an important role in music processing by the brain. It may also form the basis of the preference for particular classes of music and voice sounds.

The what, where and how of auditory-object perception

The fundamental perceptual unit in hearing is the 'auditory object'. Similar to visual objects, auditory objects are the computational result of the auditory system's capacity to detect, extract, segregate and group spectrotemporal regularities in the acoustic environment; the multitude of acoustic stimuli around us together form the auditory scene. However, unlike the visual scene, resolving the component objects within the auditory scene crucially depends on their temporal structure. Neural correlates of auditory objects are found throughout the auditory system. However, neural responses do not become correlated with a listener's perceptual reports until the level of the cortex. The roles of different neural structures and the contribution of different cognitive states to the perception of auditory objects are not yet fully understood.

Neural representation of harmonic complex tones in primary auditory cortex of the awake monkey

Many natural sounds are periodic and consist of frequencies (harmonics) that are integer multiples of a common fundamental frequency (F0). Such harmonic complex tones (HCTs) evoke a pitch corresponding to their F0, which plays a key role in the perception of speech and music. "Pitch-selective" neurons have been identified in non-primary auditory cortex of marmoset monkeys. Noninvasive studies point to a putative "pitch center" located in a homologous cortical region in humans. It remains unclear whether there is sufficient spectral and temporal information available at the level of primary auditory cortex (A1) to enable reliable pitch extraction in non-primary auditory cortex. Here we evaluated multiunit responses to HCTs in A1 of awake macaques using a stimulus design employed in auditory nerve studies of pitch encoding. The F0 of the HCTs was varied in small increments, such that harmonics of the HCTs fell either on the peak or on the sides of the neuronal pure tone tuning functions. Resultant response-amplitude-versus-harmonic-number functions ("rate-place profiles") displayed a periodic pattern reflecting the neuronal representation of individual HCT harmonics. Consistent with psychoacoustic findings in humans, lower harmonics were better resolved in rate-place profiles than higher harmonics. Lower F0s were also temporally represented by neuronal phase-locking to the periodic waveform of the HCTs. Findings indicate that population responses in A1 contain sufficient spectral and temporal information for extracting the pitch of HCTs by neurons in downstream cortical areas that receive their input from A1.