Cortical representations of pitch in monkeys and humans

Pitch coding and pitch processing in the human brain

Neuroimaging studies have provided important information regarding how and where pitch is coded and processed in the human brain. Recordings of the frequency-following response (FFR), an electrophysiological measure of neural temporal coding in the brainstem, have shown that the precision of temporal pitch information is dependent on linguistic and musical experience, and can even be modified by short-term training. However, the FFR does not seem to represent the output of a pitch extraction process, and this raises questions regarding how the peripheral neural signal is processed to produce a unified sensation. Since stimuli with a wide variety of spectral and binaural characteristics can produce the same pitch, it has been suggested that there is a place in the ascending auditory pathway at which the representations converge. There is evidence from many different human neuroimaging studies that certain areas of auditory cortex are specifically sensitive to pitch, although the location is still a matter of debate. Taken together, the results suggest that the initial temporal pitch code in the auditory periphery is converted to a code based on neural firing rate in the brainstem. In the upper brainstem or auditory cortex, the information from the individual harmonics of complex tones is combined to form a general representation of pitch. This article is part of a Special Issue entitled Human Auditory Neuroimaging.

The role of harmonic resolvability in pitch perception in a vocal nonhuman primate, the common marmoset (Callithrix jacchus)

Pitch is one of the most fundamental percepts in the auditory system and can be extracted using either spectral or temporal information in an acoustic signal. Although pitch perception has been extensively studied in human subjects, it is far less clear how nonhuman primates perceive pitch. We have addressed this question in a series of behavioral studies in which marmosets, a vocal nonhuman primate species, were trained to discriminate complex harmonic tones differing in either spectral (fundamental frequency [f0]) or temporal envelope (repetition rate) cues. We found that marmosets used temporal envelope information to discriminate pitch for acoustic stimuli with higher-order harmonics and lower f0 values and spectral information for acoustic stimuli with lower-order harmonics and higher f0 values. We further measured frequency resolution in marmosets using a psychophysical task in which pure tone thresholds were measured as a function of notched noise masker bandwidth. Results show that only the first four harmonics are resolved at low f0 values and up to 16 harmonics are resolved at higher f0 values. Resolvability in marmosets is different from that in humans, where the first five to nine harmonics are consistently resolved across most f0 values, and is likely the result of a smaller marmoset cochlea. In sum, these results show that marmosets use two mechanisms to extract pitch (harmonic templates [spectral] for resolved harmonics, and envelope extraction [temporal] for unresolved harmonics) and that species differences in stimulus resolvability need to be taken into account when investigating and comparing mechanisms of pitch perception across animals.

The neural control of singing

Singing provides a unique opportunity to examine music performance—the musical instrument is contained wholly within the body, thus eliminating the need for creating artificial instruments or tasks in neuroimaging experiments. Here, more than two decades of voice and singing research will be reviewed to give an overview of the sensory-motor control of the singing voice, starting from the vocal tract and leading up to the brain regions involved in singing. Additionally, to demonstrate how sensory feedback is integrated with vocal motor control, recent functional magnetic resonance imaging (fMRI) research on somatosensory and auditory feedback processing during singing will be presented. The relationship between the brain and singing behavior will be explored also by examining: (1) neuroplasticity as a function of various lengths and types of training, (2) vocal amusia due to a compromised singing network, and (3) singing performance in individuals with congenital amusia. Finally, the auditory-motor control network for singing will be considered alongside dual-stream models of auditory processing in music and speech to refine both these theoretical models and the singing network itself.

Auditory cortex represents both pitch judgments and the corresponding acoustic cues

The neural processing of sensory stimuli involves a transformation of physical stimulus parameters into perceptual features, and elucidating where and how this transformation occurs is one of the ultimate aims of sensory neurophysiology. Recent studies have shown that the firing of neurons in early sensory cortex can be modulated by multisensory interactions [1-5], motor behavior [1, 3, 6, 7], and reward feedback [1, 8, 9], but it remains unclear whether neural activity is more closely tied to perception, as indicated by behavioral choice, or to the physical properties of the stimulus. We investigated which of these properties are predominantly represented in auditory cortex by recording local field potentials (LFPs) and multiunit spiking activity in ferrets while they discriminated the pitch of artificial vowels. We found that auditory cortical activity is informative both about the fundamental frequency (F0) of a target sound and also about the pitch that the animals appear to perceive given their behavioral responses. Surprisingly, although the stimulus F0 was well represented at the onset of the target sound, neural activity throughout auditory cortex frequently predicted the reported pitch better than the target F0.

Integration of auditory and tactile inputs in musical meter perception

Musicians often say that they not only hear but also "feel" music. To explore the contribution of tactile information to "feeling" music, we investigated the degree that auditory and tactile inputs are integrated in humans performing a musical meter-recognition task. Subjects discriminated between two types of sequences, "duple" (march-like rhythms) and "triple" (waltz-like rhythms), presented in three conditions: (1) unimodal inputs (auditory or tactile alone); (2) various combinations of bimodal inputs, where sequences were distributed between the auditory and tactile channels such that a single channel did not produce coherent meter percepts; and (3) bimodal inputs where the two channels contained congruent or incongruent meter cues. We first show that meter is perceived similarly well (70-85 %) when tactile or auditory cues are presented alone. We next show in the bimodal experiments that auditory and tactile cues are integrated to produce coherent meter percepts. Performance is high (70-90 %) when all of the metrically important notes are assigned to one channel and is reduced to 60 % when half of these notes are assigned to one channel. When the important notes are presented simultaneously to both channels, congruent cues enhance meter recognition (90 %). Performance dropped dramatically when subjects were presented with incongruent auditory cues (10 %), as opposed to incongruent tactile cues (60 %), demonstrating that auditory input dominates meter perception. These observations support the notion that meter perception is a cross-modal percept with tactile inputs underlying the perception of "feeling" music.

Neural mechanisms for the abstraction and use of pitch information in auditory cortex

Experiments in animals have provided an important complement to human studies of pitch perception by revealing how the activity of individual neurons represents harmonic complex and periodic sounds. Such studies have shown that the acoustical parameters associated with pitch are represented by the spiking responses of neurons in A1 (primary auditory cortex) and various higher auditory cortical fields. The responses of these neurons are also modulated by the timbre of sounds. In marmosets, a distinct region on the low-frequency border of primary and non-primary auditory cortex may provide pitch tuning that generalizes across timbre classes.

Abstract encoding of auditory objects in cortical activity patterns

The human brain is thought to process auditory objects along a hierarchical temporal "what" stream that progressively abstracts object information from the low-level structure (e.g., loudness) as processing proceeds along the middle-to-anterior direction. Empirical demonstrations of abstract object encoding, independent of low-level structure, have relied on speech stimuli, and non-speech studies of object-category encoding (e.g., human vocalizations) often lack a systematic assessment of low-level information (e.g., vocalizations are highly harmonic). It is currently unknown whether abstract encoding constitutes a general functional principle that operates for auditory objects other than speech. We combined multivariate analyses of functional imaging data with an accurate analysis of the low-level acoustical information to examine the abstract encoding of non-speech categories. We observed abstract encoding of the living and human-action sound categories in the fine-grained spatial distribution of activity in the middle-to-posterior temporal cortex (e.g., planum temporale). Abstract encoding of auditory objects appears to extend to non-speech biological sounds and to operate in regions other than the anterior temporal lobe. Neural processes for the abstract encoding of auditory objects might have facilitated the emergence of speech categories in our ancestors.

Efficient coding and statistically optimal weighting of covariance among acoustic attributes in novel sounds

To the extent that sensorineural systems are efficient, redundancy should be extracted to optimize transmission of information, but perceptual evidence for this has been limited. Stilp and colleagues recently reported efficient coding of robust correlation (r = .97) among complex acoustic attributes (attack/decay, spectral shape) in novel sounds. Discrimination of sounds orthogonal to the correlation was initially inferior but later comparable to that of sounds obeying the correlation. These effects were attenuated for less-correlated stimuli (r = .54) for reasons that are unclear. Here, statistical properties of correlation among acoustic attributes essential for perceptual organization are investigated. Overall, simple strength of the principal correlation is inadequate to predict listener performance. Initial superiority of discrimination for statistically consistent sound pairs was relatively insensitive to decreased physical acoustic/psychoacoustic range of evidence supporting the correlation, and to more frequent presentations of the same orthogonal test pairs. However, increased range supporting an orthogonal dimension has substantial effects upon perceptual organization. Connectionist simulations and Eigenvalues from closed-form calculations of principal components analysis (PCA) reveal that perceptual organization is near-optimally weighted to shared versus unshared covariance in experienced sound distributions. Implications of reduced perceptual dimensionality for speech perception and plausible neural substrates are discussed.

Does a pitch center exist in auditory cortex?

Pitch perception is an important component of hearing, allowing us to appreciate melodies and harmonies as well as recognize prosodic cues in speech. Multiple studies over the last decade have suggested that pitch is represented by a pitch-processing center in auditory cortex. However, recent data (Barker D, Plack CJ, Hall DA. Cereb Cortex. In press; Hall DA, Plack CJ. Cereb Cortex 19: 576-585, 2009) now challenge these previous claims of a human "pitch center."

Non-isomorphism in efficient coding of complex sound properties

To the extent that sensorineural systems are efficient, stimulus redundancy should be captured in ways that optimize information transmission. Consistent with this principle, neural representations of sounds have been proposed to become "non-isomorphic," increasingly abstract and decreasingly resembling the original (redundant) input. Here, non-isomorphism is tested in perceptual learning using AXB discrimination of novel sounds with two highly correlated complex acoustic properties and a randomly varying third dimension. Discrimination of sounds obeying the correlation became superior to that of sounds violating it despite widely varying physical acoustic properties, suggesting non-isomorphic representation of stimulus redundancy.

Learning a novel phonological contrast depends on interactions between individual differences and training paradigm design

Studies evaluating phonological contrast learning typically investigate either the predictiveness of specific pretraining aptitude measures or the efficacy of different instructional paradigms. However, little research considers how these factors interact--whether different students learn better from different types of instruction--and what the psychological basis for any interaction might be. The present study demonstrates that successfully learning a foreign-language phonological contrast for pitch depends on an interaction between individual differences in perceptual abilities and the design of the training paradigm. Training from stimuli with high acoustic-phonetic variability is generally thought to improve learning; however, we found high-variability training enhanced learning only for individuals with strong perceptual abilities. Learners with weaker perceptual abilities were actually impaired by high-variability training relative to a low-variability condition. A second experiment assessing variations on the high-variability training design determined that the property of this learning environment most detrimental to perceptually weak learners is the amount of trial-by-trial variability. Learners' perceptual limitations can thus override the benefits of high-variability training where trial-by-trial variability in other irrelevant acoustic-phonetic features obfuscates access to the target feature. These results demonstrate the importance of considering individual differences in pretraining aptitudes when evaluating the efficacy of any speech training paradigm.

Measurement of absolute auditory thresholds in the common marmoset (Callithrix jacchus)

The common marmoset is a small, arboreal, New World primate that has emerged as a promising non-human model system in auditory neuroscience. A complete understanding of the neuroethology of auditory processing in marmosets will include behavioral work examining how sounds are perceived by these animals. However, there have been few studies of the marmoset's hearing and perceptual abilities and the audiogram of this species has not been measured using modern psychophysical methods. The present experiment pairs psychophysics with an operant conditioning technique to examine perception of pure tone stimuli by marmosets using an active behavioral paradigm. Subjects were trained to lick at a feeding tube when they detected a sound. Correct responses provided access to a food reward. Pure tones of varying intensities were presented to subjects using the method of constant stimuli. Behavioral thresholds were calculated for each animal based on hit rate--threshold was defined by the tone intensity that the animal correctly identified 50% of the time. Results show that marmoset hearing is comparable to that of other New World monkeys, with a hearing range extending from about 125 Hz up to 36 kHz and a sensitivity peak around 7 kHz.

Reexamining the evidence for a pitch-sensitive region: a human fMRI study using iterated ripple noise

Human neuroimaging studies have identified a region of auditory cortex, lateral Heschl's gyrus (HG), that shows a greater response to iterated ripple noise (IRN) than to a Gaussian noise control. Based in part on results using IRN as a pitch-evoking stimulus, it has been argued that lateral HG is a general "pitch center." However, IRN contains slowly varying spectrotemporal modulations, unrelated to pitch, that are not found in the control stimulus. Hence, it is possible that the cortical response to IRN is driven in part by these modulations. The current study reports the first attempt to control for these modulations. This was achieved using a novel type of stimulus that was generated by processing IRN to remove the fine temporal structure (and thus the pitch) but leave the slowly varying modulations. This "no-pitch IRN" stimulus is referred to as IRNo. Results showed a widespread response to the spectrotemporal modulations across auditory cortex. When IRN was contrasted with IRNo rather than with Gaussian noise, the apparent effect of pitch was no longer statistically significant. Our findings raise the possibility that a cortical response unrelated to pitch could previously have been errantly attributed to pitch coding.

Cortical dynamics of acoustic and phonological processing in speech perception

In speech perception, a functional hierarchy has been proposed by recent functional neuroimaging studies: Core auditory areas on the dorsal plane of superior temporal gyrus (STG) are sensitive to basic acoustic characteristics, whereas downstream regions, specifically the left superior temporal sulcus (STS) and middle temporal gyrus (MTG) ventral to Heschl's gyrus (HG) are responsive to abstract phonological features. What is unclear so far is the relationship between the dorsal and ventral processes, especially with regard to whether low-level acoustic processing is modulated by high-level phonological processing. To address the issue, we assessed sensitivity of core auditory and downstream regions to acoustic and phonological variations by using within- and across-category lexical tonal continua with equal physical intervals. We found that relative to within-category variation, across-category variation elicited stronger activation in the left middle MTG (mMTG), apparently reflecting the abstract phonological representations. At the same time, activation in the core auditory region decreased, resulting from the top-down influences of phonological processing. These results support a hierarchical organization of the ventral acoustic-phonological processing stream, which originates in the right HG/STG and projects to the left mMTG. Furthermore, our study provides direct evidence that low-level acoustic analysis is modulated by high-level phonological representations, revealing the cortical dynamics of acoustic and phonological processing in speech perception. Our findings confirm the existence of reciprocal progression projections in the auditory pathways and the roles of both feed-forward and feedback mechanisms in speech perception.

Tracking vocal pitch through noise: neural correlates in nonprimary auditory cortex

In natural environments, a sound can be heard as stable despite the presence of other occasionally louder sounds. For example, when a portion in a voice is replaced by masking noise, the interrupted voice may still appear illusorily continuous. Previous research found that continuity illusions of simple interrupted sounds, such as tones, are accompanied by weaker activity in the primary auditory cortex (PAC) during the interruption than veridical discontinuity percepts of these sounds. Here, we studied whether continuity illusions of more natural and more complex sounds also emerge from this mechanism. We used psychophysics and functional magnetic resonance imaging in humans to measure simultaneously continuity ratings and blood oxygenation level-dependent activity to vowels that were partially replaced by masking noise. Consistent with previous results on tone continuity illusions, we found listeners' reports of more salient vowel continuity illusions associated with weaker activity in auditory cortex (compared with reports of veridical discontinuity percepts of physically identical stimuli). In contrast to the reduced activity to tone continuity illusions in PAC, this reduction was localized in the right anterolateral Heschl's gyrus, a region that corresponds more to the non-PAC. Our findings suggest that the ability to hear differently complex sounds as stable during other louder sounds may be attributable to a common suppressive mechanism that operates at different levels of sound representation in auditory cortex.

Cross-modal interactions during perception of audiovisual speech and nonspeech signals: an fMRI study

During speech communication, visual information may interact with the auditory system at various processing stages. Most noteworthy, recent magnetoencephalography (MEG) data provided first evidence for early and preattentive phonetic/phonological encoding of the visual data stream--prior to its fusion with auditory phonological features [Hertrich, I., Mathiak, K., Lutzenberger, W., & Ackermann, H. Time course of early audiovisual interactions during speech and non-speech central-auditory processing: An MEG study. Journal of Cognitive Neuroscience, 21, 259-274, 2009]. Using functional magnetic resonance imaging, the present follow-up study aims to further elucidate the topographic distribution of visual-phonological operations and audiovisual (AV) interactions during speech perception. Ambiguous acoustic syllables--disambiguated to /pa/ or /ta/ by the visual channel (speaking face)--served as test materials, concomitant with various control conditions (nonspeech AV signals, visual-only and acoustic-only speech, and nonspeech stimuli). (i) Visual speech yielded an AV-subadditive activation of primary auditory cortex and the anterior superior temporal gyrus (STG), whereas the posterior STG responded both to speech and nonspeech motion. (ii) The inferior frontal and the fusiform gyrus of the right hemisphere showed a strong phonetic/phonological impact (differential effects of visual /pa/ vs. /ta/) upon hemodynamic activation during presentation of speaking faces. Taken together with the previous MEG data, these results point at a dual-pathway model of visual speech information processing: On the one hand, access to the auditory system via the anterior supratemporal “what" path may give rise to direct activation of "auditory objects." On the other hand, visual speech information seems to be represented in a right-hemisphere visual working memory, providing a potential basis for later interactions with auditory information such as the McGurk effect.

Functional properties of human auditory cortical fields

While auditory cortex in non-human primates has been subdivided into multiple functionally specialized auditory cortical fields (ACFs), the boundaries and functional specialization of human ACFs have not been defined. In the current study, we evaluated whether a widely accepted primate model of auditory cortex could explain regional tuning properties of fMRI activations on the cortical surface to attended and non-attended tones of different frequency, location, and intensity. The limits of auditory cortex were defined by voxels that showed significant activations to non-attended sounds. Three centrally located fields with mirror-symmetric tonotopic organization were identified and assigned to the three core fields of the primate model while surrounding activations were assigned to belt fields following procedures similar to those used in macaque fMRI studies. The functional properties of core, medial belt, and lateral belt field groups were then analyzed. Field groups were distinguished by tonotopic organization, frequency selectivity, intensity sensitivity, contralaterality, binaural enhancement, attentional modulation, and hemispheric asymmetry. In general, core fields showed greater sensitivity to sound properties than did belt fields, while belt fields showed greater attentional modulation than core fields. Significant distinctions in intensity sensitivity and contralaterality were seen between adjacent core fields A1 and R, while multiple differences in tuning properties were evident at boundaries between adjacent core and belt fields. The reliable differences in functional properties between fields and field groups suggest that the basic primate pattern of auditory cortex organization is preserved in humans. A comparison of the sizes of functionally defined ACFs in humans and macaques reveals a significant relative expansion in human lateral belt fields implicated in the processing of speech.

Cortical structure predicts success in performing musical transformation judgments

Recognizing melodies by their interval structure, or "relative pitch," is a fundamental aspect of musical perception. By using relative pitch, we are able to recognize tunes regardless of the key in which they are played. We sought to determine the cortical areas important for relative pitch processing using two morphometric techniques. Cortical differences have been reported in musicians within right auditory cortex (AC), a region considered important for pitch-based processing, and we have previously reported a functional correlation between relative pitch processing in the anterior intraparietal sulcus (IPS). We addressed the hypothesis that regional variation of cortical structure within AC and IPS is related to relative pitch ability using two anatomical techniques, cortical thickness (CT) analysis and voxel-based morphometry (VBM) of magnetic resonance imaging data. Persons with variable amounts of formal musical training were tested on a melody transposition task, as well as two musical control tasks and a speech control task. We found that gray matter concentration and cortical thickness in right Heschl's sulcus and bilateral IPS both predicted relative pitch task performance and correlated to a lesser extent with performance on the two musical control tasks. After factoring out variance explained by musical training, only relative pitch performance was predicted by cortical structure in these regions. These results directly demonstrate the functional relevance of previously reported anatomical differences in the auditory cortex of musicians. The findings in the IPS provide further support for the existence of a multimodal network for systematic transformation of stimulus information in this region.

Cortical representation of natural complex sounds: effects of acoustic features and auditory object category

How the brain processes complex sounds, like voices or musical instrument sounds, is currently not well understood. The features comprising the acoustic profiles of such sounds are thought to be represented by neurons responding to increasing degrees of complexity throughout auditory cortex, with complete auditory "objects" encoded by neurons (or small networks of neurons) in anterior superior temporal regions. Although specialized voice and speech-sound regions have been proposed, it is unclear how other types of complex natural sounds are processed within this object-processing pathway. Using functional magnetic resonance imaging, we sought to demonstrate spatially distinct patterns of category-selective activity in human auditory cortex, independent of semantic content and low-level acoustic features. Category-selective responses were identified in anterior superior temporal regions, consisting of clusters selective for musical instrument sounds and for human speech. An additional subregion was identified that was particularly selective for the acoustic-phonetic content of speech. In contrast, regions along the superior temporal plane closer to primary auditory cortex were not selective for stimulus category, responding instead to specific acoustic features embedded in natural sounds, such as spectral structure and temporal modulation. Our results support a hierarchical organization of the anteroventral auditory-processing stream, with the most anterior regions representing the complete acoustic signature of auditory objects.

Sensitivity and selectivity of neurons in auditory cortex to the pitch, timbre, and location of sounds

We are able to rapidly recognize and localize the many sounds in our environment. We can describe any of these sounds in terms of various independent "features" such as their loudness, pitch, or position in space. However, we still know surprisingly little about how neurons in the auditory brain, specifically the auditory cortex, might form representations of these perceptual characteristics from the information that the ear provides about sound acoustics. In this article, the authors examine evidence that the auditory cortex is necessary for processing the pitch, timbre, and location of sounds, and document how neurons across multiple auditory cortical fields might represent these as trains of action potentials. They conclude by asking whether neurons in different regions of the auditory cortex might not be simply sensitive to each of these three sound features but whether they might be selective for one of them. The few studies that have examined neural sensitivity to multiple sound attributes provide only limited support for neural selectivity within auditory cortex. Providing an explanation of the neural basis of feature invariance is thus one of the major challenges to sensory neuroscience obtaining the ultimate goal of understanding how neural firing patterns in the brain give rise to perception.

Neural coding of periodicity in marmoset auditory cortex

Pitch, our perception of how high or low a sound is on a musical scale, crucially depends on a sound's periodicity. If an acoustic signal is temporally jittered so that it becomes aperiodic, the pitch will no longer be perceivable even though other acoustical features that normally covary with pitch are unchanged. Previous electrophysiological studies investigating pitch have typically used only periodic acoustic stimuli, and as such these studies cannot distinguish between a neural representation of pitch and an acoustical feature that only correlates with pitch. In this report, we examine in the auditory cortex of awake marmoset monkeys (Callithrix jacchus) the neural coding of a periodicity's repetition rate, an acoustic feature that covaries with pitch. We first examine if individual neurons show similar repetition rate tuning for different periodic acoustic signals. We next measure how sensitive these neural representations are to the temporal regularity of the acoustic signal. We find that neurons throughout auditory cortex covary their firing rate with the repetition rate of an acoustic signal. However, similar repetition rate tuning across acoustic stimuli and sensitivity to temporal regularity were generally only observed in a small group of neurons found near the anterolateral border of primary auditory cortex, the location of a previously identified putative pitch processing center. These results suggest that although the encoding of repetition rate is a general component of auditory cortical processing, the neural correlate of periodicity is confined to a special class of pitch-selective neurons within the putative pitch processing center of auditory cortex.

Sound localization cues in the marmoset monkey

The most important acoustic cues available to the brain for sound localization are produced by the interaction of sound with the animal's head and external ears. As a first step in understanding the relation between these cues and their neural representation in a vocal new-world primate, we measured head-related transfer functions (HRTFs) across frequency for a wide range of sound locations in three anesthetized marmoset monkeys. The HRTF magnitude spectrum has a broad resonance peak at 6-12 kHz that coincides with the frequency range of the major call types of this species. A prominent first spectral notch (FN) in the HRTF magnitude above this resonance was observed at most source locations. The center frequency of the FN increased monotonically from approximately 12 to 26 kHz with increases in elevation in the lateral field. In the frontal field FN frequency changed in a less orderly fashion with source position. From the HRTFs we derived interaural time (ITDs) and level differences (ILDs). ITDs and ILDs (below 12 kHz) varied as a function of azimuth between +/-250 micros and +/-20dB, respectively. A reflexive orienting behavioral paradigm was used to confirm that marmosets can orient to sound sources.

Communication disorders in speakers of tone languages: etiological bases and clinical considerations

Lexical tones are a phonetic contrast necessary for conveying meaning in a majority of the world's languages. Various hearing, speech, and language disorders affect the ability to perceive or produce lexical tones, thereby seriously impairing individuals' communicative abilities. The number of tone language speakers is increasing, even in otherwise English-speaking nations, yet insufficient emphasis has been placed on clinical assessment and rehabilitation of lexical tone disorders. The similarities and dissimilarities between lexical tones and other speech sounds make a richer scientific understanding of their physiological bases paramount to more effective remediation of speech and language disorders in general. Here we discuss the cognitive and biological bases of lexical tones, emphasizing the neural structures and networks that support their acquisition, perception, and cognitive representation. We present emerging research on lexical tone learning in the context of the clinical disorders of hearing, speech, and language that this body of research will help to address.

Coding of repetitive transients by auditory cortex on Heschl's gyrus

The capacity of auditory cortex on Heschl's gyrus (HG) to encode repetitive transients was studied in human patients undergoing surgical evaluation for medically intractable epilepsy. Multicontact depth electrodes were chronically implanted in gray matter of HG. Bilaterally presented stimuli were click trains varying in rate from 4 to 200 Hz. Averaged evoked potentials (AEPs) and event-related band power (ERBP), computed from responses at each of 14 recording sites, identified two auditory fields. A core field, which occupies posteromedial HG, was characterized by a robust polyphasic AEP on which could be superimposed a frequency following response (FFR). The FFR was prominent at click rates below approximately 50 Hz, decreased rapidly as click rate was increased, but could reliably be detected at click rates as high as 200 Hz. These data are strikingly similar to those obtained by others in the monkey under essentially the same stimulus conditions, indicating that mechanisms underlying temporal processing in the auditory core may be highly conserved across primate species. ERBP, which reflects increases or decreases of both phase-locked and non-phase-locked power within given frequency bands, showed stimulus-related increases in gamma band frequencies as high as 250 Hz. The AEPs recorded in a belt field anterolateral to the core were typically of low amplitude, showing little or no evidence of short-latency waves or an FFR, even at the lowest click rates used. The non-phase-locked component of the response extracted from the ERBP showed a robust, long-latency response occurring here in response to the highest click rates in the series.

Temporal coherence in the perceptual organization and cortical representation of auditory scenes

Just as the visual system parses complex scenes into identifiable objects, the auditory system must organize sound elements scattered in frequency and time into coherent "streams." Current neurocomputational theories of auditory streaming rely on tonotopic organization of the auditory system to explain the observation that sequential spectrally distant sound elements tend to form separate perceptual streams. Here, we show that spectral components that are well separated in frequency are no longer heard as separate streams if presented synchronously rather than consecutively. In contrast, responses from neurons in primary auditory cortex of ferrets show that both synchronous and asynchronous tone sequences produce comparably segregated responses along the tonotopic axis. The results argue against tonotopic separation per se as a neural correlate of stream segregation. Instead we propose a computational model of stream segregation that can account for the data by using temporal coherence as the primary criterion for predicting stream formation.

Pitch perception and auditory stream segregation: implications for hearing loss and cochlear implants

Pitch is important for speech and music perception, and may also play a crucial role in our ability to segregate sounds that arrive from different sources. This article reviews some basic aspects of pitch coding in the normal auditory system and explores the implications for pitch perception in people with hearing impairments and cochlear implants. Data from normal-hearing listeners suggest that the low-frequency, low-numbered harmonics within complex tones are of prime importance in pitch perception and in the perceptual segregation of competing sounds. The poorer frequency selectivity experienced by many hearing-impaired listeners leads to less access to individual harmonics, and the coding schemes currently employed in cochlear implants provide little or no representation of individual harmonics. These deficits in the coding of harmonic sounds may underlie some of the difficulties experienced by people with hearing loss and cochlear implants, and may point to future areas where sound representation in auditory prostheses could be improved.

Speaker normalization using cortical strip maps: a neural model for steady-state vowel categorization

Auditory signals of speech are speaker dependent, but representations of language meaning are speaker independent. The transformation from speaker-dependent to speaker-independent language representations enables speech to be learned and understood from different speakers. A neural model is presented that performs speaker normalization to generate a pitch-independent representation of speech sounds, while also preserving information about speaker identity. This speaker-invariant representation is categorized into unitized speech items, which input to sequential working memories whose distributed patterns can be categorized, or chunked, into syllable and word representations. The proposed model fits into an emerging model of auditory streaming and speech categorization. The auditory streaming and speaker normalization parts of the model both use multiple strip representations and asymmetric competitive circuits, thereby suggesting that these two circuits arose from similar neural designs. The normalized speech items are rapidly categorized and stably remembered by adaptive resonance theory circuits. Simulations use synthesized steady-state vowels from the Peterson and Barney [Peterson, G. E., and Barney, H.L., J. Acoust. Soc. Am. 24, 175-184 (1952).] vowel database and achieve accuracy rates similar to those achieved by human listeners. These results are compared to behavioral data and other speaker normalization models.

Sensitive response to low-frequency cochlear distortion products in the auditory midbrain

During auditory stimulation with several frequency components, distortion products (DPs) are generated as byproduct of nonlinear cochlear amplification. After generated, DP energy is reemitted into the ear channel where it can be measured as DP otoacoustic emission (DPOAE), and it also induces an excitatory response at cochlear places related to the DP frequencies. We measured responses of 91 inferior colliculus (IC) neurons in the gerbil during two-tone stimulation with frequencies well above the unit's receptive field but adequate to generate a distinct distortion product (f2-f1 or 2f1-f2) at the unit's characteristic frequency (CF). Neuronal responses to DPs could be accounted for by the simultaneously measured DPOAEs for DP frequencies >1.3 kHz. For DP frequencies <1.3 kHz (n = 25), there was a discrepancy between intracochlear DP magnitude and DPOAE level, and most neurons responded as if the intracochlear DP level was significantly higher than the DPOAE level in the ear channel. In 12% of those low-frequency neurons, responses to the DPs could be elicited even if the stimulus tone levels were below the threshold level of the neuron at CF. High intracochlear f2-f1 and 2f1-f2 DP-levels were verified by cancellation of the neuronal DP response with a third phase-adjusted tone stimulus at the DP frequency. A frequency-specific reduction of middle ear gain at low frequencies is possibly involved in the reduction of DPOAE level. The results indicate that pitch-related properties of complex stimuli may be produced partially by high intracochlear f2-f1 distortion levels.

Music perception, pitch, and the auditory system

The perception of music depends on many culture-specific factors, but is also constrained by properties of the auditory system. This has been best characterized for those aspects of music that involve pitch. Pitch sequences are heard in terms of relative as well as absolute pitch. Pitch combinations give rise to emergent properties not present in the component notes. In this review we discuss the basic auditory mechanisms contributing to these and other perceptual effects in music.

Pitch processing sites in the human auditory brain

Lateral Heschl's gyrus (HG), a subdivision of the human auditory cortex, is commonly believed to represent a general “pitch center,” responding selectively to the pitch of sounds, irrespective of their spectral characteristics. However, most neuroimaging investigations have used only one specialized pitch-evoking stimulus: iterated-ripple noise (IRN). The present study used a novel experimental design in which a range of different pitch-evoking stimuli were presented to the same listeners. Pitch sites were identified by searching for voxels that responded well to the range of pitch-evoking stimuli. The first result suggested that parts of the planum temporale are more relevant for pitch processing than lateral HG. In some listeners, pitch responses occurred elsewhere, such as the temporo-parieto-occipital junction or prefrontal cortex. The second result demonstrated a different pattern of response to the IRN and raises the possibility that features of IRN unrelated to pitch might contribute to the earlier results. In conclusion, it seems premature to assign special status to lateral HG solely on the basis of neuroactivation patterns. Further work should consider the functional roles of these multiple pitch processing sites within the proposed network.

Neural representation of spectral and temporal features of song in the auditory forebrain of zebra finches as revealed by functional MRI

Song perception in songbirds, just as music and speech perception in humans, requires processing the spectral and temporal structure found in the succession of song-syllables. Using functional magnetic resonance imaging and synthetic songs that preserved exclusively either the temporal or the spectral structure of natural song, we investigated how vocalizations are processed in the avian forebrain. We found bilateral and equal activation of the primary auditory region, field L. The more ventral regions of field L showed depressed responses to the synthetic songs that lacked spectral structure. These ventral regions included subarea L3, medial-ventral subarea L and potentially the secondary auditory region caudal medial nidopallium. In addition, field L as a whole showed unexpected increased responses to the temporally filtered songs and this increase was the largest in the dorsal regions. These dorsal regions included L1 and the dorsal subareas L and L2b. Therefore, the ventral region of field L appears to be more sensitive to the preservation of both spectral and temporal information in the context of song processing. We did not find any differences in responses to playback of the bird's own song vs other familiar conspecific songs. We also investigated the effect of three commonly used anaesthetics on the blood oxygen level-dependent response: medetomidine, urethane and isoflurane. The extent of the area activated and the stimulus selectivity depended on the type of anaesthetic. We discuss these results in the context of what is known about the locus of action of the anaesthetics, and reports of neural activity measured in electrophysiological experiments.

Volume of left Heschl's Gyrus and linguistic pitch learning

Research on the contributions of the human nervous system to language processing and learning has generally been focused on the association regions of the brain without considering the possible contribution of primary and adjacent sensory areas. We report a study examining the relationship between the anatomy of Heschl's Gyrus (HG), which includes predominately primary auditory areas and is often found to be associated with nonlinguistic pitch processing and language learning. Unlike English, most languages of the world use pitch patterns to signal word meaning. In the present study, native English-speaking adult subjects learned to incorporate foreign pitch patterns in word identification. Subjects who were less successful in learning showed a smaller HG volume on the left (especially gray matter volume), but not on the right, relative to learners who were successful. These results suggest that HG, typically shown to be associated with the processing of acoustic cues in nonspeech processing, is also involved in speech learning. These results also suggest that primary auditory regions may be important for encoding basic acoustic cues during the course of spoken language learning.

Role of auditory cortex in sound localization in the midsagittal plane

Although the auditory cortex is known to be essential for normal sound localization in the horizontal plane, its contribution to vertical localization has not so far been examined. In this study, we measured the acuity with which ferrets could discriminate between two speakers in the midsagittal plane before and after silencing activity bilaterally in the primary auditory cortex (A1). This was achieved either by subdural placement of Elvax implants containing the GABA A receptor agonist muscimol or by making aspiration lesions after determining the approximate location of A1 electrophysiologically. Psychometric functions and minimum audible angles were measured in the upper hemifield for 500-, 200-, and 40-ms noise bursts. Muscimol-Elvax inactivation of A1 produced a small but significant deficit in the animals' ability to localize brief (40-ms) sounds, which was reversed after removal of the Elvax implants. A similar deficit in vertical localization was observed after bilateral aspiration lesions of A1, whereas performance at longer sound durations was unaffected. Another group of ferrets received larger lesions, encompassing both primary and nonprimary auditory cortical areas, and showed a greater deficit with performance being impaired for long- and short-duration (500- and 40-ms, respectively) stimuli. These data suggest that the integrity of the auditory cortex is required to successfully utilize spectral localization cues, which are thought to provide the basis for vertical localization, and that multiple cortical fields, including A1, contribute to this task.

Detection of sinusoidal amplitude modulated sounds: deficits after bilateral lesions of auditory cortex in the rat

The ability of rats to detect the presence of sinusoidal amplitude modulation (AM) of a broadband noise carrier was determined before and after bilateral ablation of auditory cortex. The rats were trained to withdraw from a drinking spout to avoid a shock when they detected a modulation of the sound. Sensitivity was evaluated by testing the rats at progressively smaller depths of modulation. Psychophysical curves were produced to describe the limits of detection at modulation rates of 10, 100 and 1000Hz. Performance scores were based on the probability of withdrawal from the spout during AM (warning periods) relative to withdrawal during the un-modulated noise (safe periods). A threshold was defined as the depth of modulation that produced a score halfway between perfect avoidance and no avoidance (performance score=0.5). Bilateral auditory cortical lesions resulted in significant elevations in threshold for detection of AM at rates of 100 and 1000Hz. No significant shift was found at a modulation rate of 10Hz. The magnitude of the deficit for AM rates of 100 and 1000Hz was positively correlated with the size of the cortical lesion. Substantial deficits were found only in animals with lesions that included secondary as well as primary auditory cortical areas. The results show that the rat's auditory cortex is important for processing sinusoidal AM and that its contribution is most apparent at high modulation rates. The data suggest that the auditory cortex is a crucial structure for maintaining normal sensitivity to temporal modulation of an auditory stimulus.

The human 'pitch center' responds differently to iterated noise and Huggins pitch

A magnetoencephalographic marker for pitch analysis (the pitch onset response) has been reported for different types of pitch-evoking stimuli, irrespective of whether the acoustic cues for pitch are monaurally or binaurally produced. It is claimed that the pitch onset response reflects a common cortical representation for pitch, putatively in lateral Heschl's gyrus. The result of this functional MRI study sheds doubt on this assertion. We report a direct comparison between iterated ripple noise and Huggins pitch in which we reveal a different pattern of auditory cortical activation associated with each pitch stimulus, even when individual variability in structure-function relations is accounted for. Our results suggest it may be premature to assume that lateral Heschl's gyrus is a universal pitch center.

Neural coding strategies in auditory cortex

In contrast to the visual system, the auditory system has longer subcortical pathways and more spiking synapses between the peripheral receptors and the cortex. This unique organization reflects the needs of the auditory system to extract behaviorally relevant information from a complex acoustic environment using strategies different from those used by other sensory systems. The neural representations of acoustic information in auditory cortex can be characterized by three types: (1) isomorphic (faithful) representations of acoustic structures; (2) non-isomorphic transformations of acoustic features and (3) transformations from acoustical to perceptual dimensions. The challenge facing auditory neurophysiologists is to understand the nature of the latter two transformations. In this article, I will review recent studies from our laboratory regarding temporal discharge patterns in auditory cortex of awake marmosets and cortical representations of time-varying signals. Findings from these studies show that (1) firing patterns of neurons in auditory cortex are dependent on stimulus optimality and context and (2) the auditory cortex forms internal representations of sounds that are no longer faithful replicas of their acoustic structures.