Binaural fusion and the representation of virtual pitch in the human auditory cortex

Similar abilities of musicians and non-musicians to segregate voices by fundamental frequency

Musicians can sometimes achieve better speech recognition in noisy backgrounds than non-musicians, a phenomenon referred to as the "musician advantage effect." In addition, musicians are known to possess a finer sense of pitch than non-musicians. The present study examined the hypothesis that the latter fact could explain the former. Four experiments measured speech reception threshold for a target voice against speech or non-speech maskers. Although differences in fundamental frequency (ΔF0s) were shown to be beneficial even when presented to opposite ears (experiment 1), the authors' attempt to maximize their use by directing the listener's attention to the target F0 led to unexpected impairments (experiment 2) and the authors' attempt to hinder their use by generating uncertainty about the competing F0s led to practically negligible effects (experiments 3 and 4). The benefits drawn from ΔF0s showed surprisingly little malleability for a cue that can be used in the complete absence of energetic masking. In half of the experiments, musicians obtained better thresholds than non-musicians, particularly in speech-on-speech conditions, but they did not reliably obtain larger ΔF0 benefits. Thus, the data do not support the hypothesis that the musician advantage effect is based on greater ability to exploit ΔF0s.

Maps of the Auditory Cortex

One of the fundamental properties of the mammalian brain is that sensory regions of cortex are formed of multiple, functionally specialized cortical field maps (CFMs). Each CFM comprises two orthogonal topographical representations, reflecting two essential aspects of sensory space. In auditory cortex, auditory field maps (AFMs) are defined by the combination of tonotopic gradients, representing the spectral aspects of sound (i.e., tones), with orthogonal periodotopic gradients, representing the temporal aspects of sound (i.e., period or temporal envelope). Converging evidence from cytoarchitectural and neuroimaging measurements underlies the definition of 11 AFMs across core and belt regions of human auditory cortex, with likely homology to those of macaque. On a macrostructural level, AFMs are grouped into cloverleaf clusters, an organizational structure also seen in visual cortex. Future research can now use these AFMs to investigate specific stages of auditory processing, key for understanding behaviors such as speech perception and multimodal sensory integration.

Neural responses towards a speaker's feeling of (un)knowing

During interpersonal communication, listeners must rapidly evaluate verbal and vocal cues to arrive at an integrated meaning about the utterance and about the speaker, including a representation of the speaker's 'feeling of knowing' (i.e., how confident they are in relation to the utterance). In this study, we investigated the time course and neural responses underlying a listener's ability to evaluate speaker confidence from combined verbal and vocal cues. We recorded real-time brain responses as listeners judged statements conveying three levels of confidence with the speaker's voice (confident, close-to-confident, unconfident), which were preceded by meaning-congruent lexical phrases (e.g. I am positive, Most likely, Perhaps). Event-related potentials to utterances with combined lexical and vocal cues about speaker confidence were compared to responses elicited by utterances without the verbal phrase in a previous study (Jiang and Pell, 2015). Utterances with combined cues about speaker confidence elicited reduced, N1, P2 and N400 responses when compared to corresponding utterances without the phrase. When compared to confident statements, close-to-confident and unconfident expressions elicited reduced N1 and P2 responses and a late positivity from 900 to 1250 ms; unconfident and close-to-confident expressions were differentiated later in the 1250-1600 ms time window. The effect of lexical phrases on confidence processing differed for male and female participants, with evidence that female listeners incorporated information from the verbal and vocal channels in a distinct manner. Individual differences in trait empathy and trait anxiety also moderated neural responses during confidence processing. Our findings showcase the cognitive processing mechanisms and individual factors governing how we infer a speaker's mental (knowledge) state from the speech signal.

Left-hemisphere activation is associated with enhanced vocal pitch error detection in musicians with absolute pitch

The ability to process auditory feedback for vocal pitch control is crucial during speaking and singing. Previous studies have suggested that musicians with absolute pitch (AP) develop specialized left-hemisphere mechanisms for pitch processing. The present study adopted an auditory feedback pitch perturbation paradigm combined with ERP recordings to test the hypothesis whether the neural mechanisms of the left-hemisphere enhance vocal pitch error detection and control in AP musicians compared with relative pitch (RP) musicians and non-musicians (NM). Results showed a stronger N1 response to pitch-shifted voice feedback in the right-hemisphere for both AP and RP musicians compared with the NM group. However, the left-hemisphere P2 component activation was greater in AP and RP musicians compared with NMs and also for the AP compared with RP musicians. The NM group was slower in generating compensatory vocal reactions to feedback pitch perturbation compared with musicians, and they failed to re-adjust their vocal pitch after the feedback perturbation was removed. These findings suggest that in the earlier stages of cortical neural processing, the right hemisphere is more active in musicians for detecting pitch changes in voice feedback. In the later stages, the left-hemisphere is more active during the processing of auditory feedback for vocal motor control and seems to involve specialized mechanisms that facilitate pitch processing in the AP compared with RP musicians. These findings indicate that the left hemisphere mechanisms of AP ability are associated with improved auditory feedback pitch processing during vocal pitch control in tasks such as speaking or singing.

Further evidence of auditory extinction in aphasia

PURPOSE

Preliminary research (Shisler, 2005) suggests that auditory extinction in individuals with aphasia (IWA) may be connected to binding and attention. In this study, the authors expanded on previous findings on auditory extinction to determine the source of extinction deficits in IWA.

METHOD

Seventeen IWA (M(age) = 53.19 years) and 17 neurologically intact controls (M(age) = 55.18 years) participated. Auditory stimuli were spoken letters presented in a free-field listening environment. Stimuli were presented in single-stimulus stimulation (SSS) or double-simultaneous stimulation (DSS) trials across 5 conditions designed to determine whether extinction is related to binding, inefficient attention resource allocation, or overall deficits in attention. All participants completed all experimental conditions.

RESULTS

Significant extinction was demonstrated only by IWA when sounds were different, providing further evidence of auditory extinction. However, binding requirements did not appear to influence the IWA's performance.

CONCLUSIONS

Results indicate that, for IWA, auditory extinction may not be attributed to a binding deficit or inefficient attention resource allocation because of equivalent performance across all 5 conditions. Rather, overall attentional resources may be influential. Future research in aphasia should explore the effect of the stimulus presentation in addition to the continued study of attention treatment.

Electrophysiological responses to emotional prosody perception in cochlear implant users

The present electroencephalographic (EEG) study investigated the ability of cochlear implant (CI) users to recognize emotional prosody. Two CI speech-processing strategies were compared: the ACE (Advance Combination Encoder) and the newly developed MP3000. Semantically neutral sentences spoken in three different emotional prosodies (neutral, angry, happy) were presented to 20 post-lingually deafened CI users and age-matched normal-hearing controls. Event related potentials (ERPs) were recorded to study the N100 and the P200 responses. In addition, event-related spectral power modulations were calculated to study the brain activity corresponding to the recognition of prosody in earlier (0–400) as well as later (600–1200) part of the stimuli where the prosodic features differed maximally. CI users with MP3000 strategy showed a higher proportion of correctly recognized prosodic information compared to the ACE strategy users. Our ERP results demonstrated that emotional prosody elicited significant N100 and P200 peaks. Furthermore, the P200 amplitude in response to happy prosodic information was significantly more positive for the MP3000 strategy compared to the ACE strategy. On spectral power analysis, two typical gamma activities were observed in the MP3000 users only: (1) an early gamma activity in the 100–250 ms time window reflecting bottom–up attention regulation; and (2) a late gamma activity between 900 and 1100 ms post-stimulus onset, probably reflecting top–down cognitive control. Our study suggests that the MP3000 strategy is better than ACE in regard to happy prosody perception. Furthermore, we show that EEG is a useful tool that, in combination with behavioral analysis, can reveal differences between two CI processing strategies for coding of prosody-specific features of language.

Sequencing the Cortical Processing of Pitch-Evoking Stimuli using EEG Analysis and Source Estimation

Cues to pitch include spectral cues that arise from tonotopic organization and temporal cues that arise from firing patterns of auditory neurons. fMRI studies suggest a common pitch center is located just beyond primary auditory cortex along the lateral aspect of Heschl’s gyrus, but little work has examined the stages of processing for the integration of pitch cues. Using electroencephalography, we recorded cortical responses to high-pass filtered iterated rippled noise (IRN) and high-pass filtered complex harmonic stimuli, which differ in temporal and spectral content. The two stimulus types were matched for pitch saliency, and a mismatch negativity (MMN) response was elicited by infrequent pitch changes. The P1 and N1 components of event-related potentials (ERPs) are thought to arise from primary and secondary auditory areas, respectively, and to result from simple feature extraction. MMN is generated in secondary auditory cortex and is thought to act on feature-integrated auditory objects. We found that peak latencies of both P1 and N1 occur later in response to IRN stimuli than to complex harmonic stimuli, but found no latency differences between stimulus types for MMN. The location of each ERP component was estimated based on iterative fitting of regional sources in the auditory cortices. The sources of both the P1 and N1 components elicited by IRN stimuli were located dorsal to those elicited by complex harmonic stimuli, whereas no differences were observed for MMN sources across stimuli. Furthermore, the MMN component was located between the P1 and N1 components, consistent with fMRI studies indicating a common pitch region in lateral Heschl’s gyrus. These results suggest that while the spectral and temporal processing of different pitch-evoking stimuli involves different cortical areas during early processing, by the time the object-related MMN response is formed, these cues have been integrated into a common representation of pitch.

ERP correlates of pitch error detection in complex tone and voice auditory feedback with missing fundamental

Previous studies have shown that the pitch of a sound is perceived in the absence of its fundamental frequency (F0), suggesting that a distinct mechanism may resolve pitch based on a pattern that exists between harmonic frequencies. The present study investigated whether such a mechanism is active during voice pitch control. ERPs were recorded in response to +200 cents pitch shifts in the auditory feedback of self-vocalizations and complex tones with and without the F0. The absence of the fundamental induced no difference in ERP latencies. However, a right-hemisphere difference was found in the N1 amplitudes with larger responses to complex tones that included the fundamental compared to when it was missing. The P1 and N1 latencies were shorter in the left hemisphere, and the N1 and P2 amplitudes were larger bilaterally for pitch shifts in voice and complex tones compared with pure tones. These findings suggest hemispheric differences in neural encoding of pitch in sounds with missing fundamental. Data from the present study suggest that the right cortical auditory areas, thought to be specialized for spectral processing, may utilize different mechanisms to resolve pitch in sounds with missing fundamental. The left hemisphere seems to perform faster processing to resolve pitch based on the rate of temporal variations in complex sounds compared with pure tones. These effects indicate that the differential neural processing of pitch in the left and right hemispheres may enable the audio-vocal system to detect temporal and spectral variations in the auditory feedback for vocal pitch control.

Detection of virtual pitch up to 5kHz by mice

Natural sounds consist of a component at the fundamental frequency (f0) and its overtones. Pitch is perceived at f0, even when spectral energy at f0 is missing. This missing f0, or 'virtual pitch', is thought to be detected in the auditory cortex and related cortical areas, but the precise neural mechanisms are unknown. One possibility is that virtual pitch can be retrieved from the periodicity of sound waveforms. However, this mechanism requires the temporal accuracy in periodicity detection, and so far the detection of virtual pitch has only been demonstrated at frequencies lower than 1kHz. We investigated the ability of mice to detect virtual pitch up to 5kHz using a two-step sound discrimination test. In the first step of this test, mice were trained to discriminate between tone bursts at 2.5 and 5kHz. In the second step, we tested the ability of mice to discriminate between virtual pitches at 2.5kHz and at 5kHz. It was demonstrated that the performance of mice to discriminate between virtual pitches at 2.5 and 5kHz was significantly affected by previous discrimination learning between tone bursts, indicating that mice can detect virtual pitch up to 5kHz.

Investigating the Multimodal Nature of Human Communication

The multimodal nature of human communication has been well established. Yet few empirical studies have systematically examined the widely held belief that this form of perception is facilitated in comparison to unimodal or bimodal perception. In the current experiment we first explored the processing of unimodally presented facial expressions. Furthermore, auditory (prosodic and/or lexical-semantic) information was presented together with the visual information to investigate the processing of bimodal (facial and prosodic cues) and multimodal (facial, lexic, and prosodic cues) human communication. Participants engaged in an identity identification task, while event-related potentials (ERPs) were being recorded to examine early processing mechanisms as reflected in the P200 and N300 component. While the former component has repeatedly been linked to physical property stimulus processing, the latter has been linked to more evaluative “meaning-related” processing. A direct relationship between P200 and N300 amplitude and the number of information channels present was found. The multimodal-channel condition elicited the smallest amplitude in the P200 and N300 components, followed by an increased amplitude in each component for the bimodal-channel condition. The largest amplitude was observed for the unimodal condition. These data suggest that multimodal information induces clear facilitation in comparison to unimodal or bimodal information. The advantage of multimodal perception as reflected in the P200 and N300 components may thus reflect one of the mechanisms allowing for fast and accurate information processing in human communication.

Responses of auditory cortex to complex stimuli: functional organization revealed using intrinsic optical signals

We used optical imaging of intrinsic signals to study the large-scale organization of ferret auditory cortex in response to complex sounds. Cortical responses were collected during continuous stimulation by sequences of sounds with varying frequency, period, or interaural level differences. We used a set of stimuli that differ in spectral structure, but have the same periodicity and therefore evoke the same pitch percept (click trains, sinusoidally amplitude modulated tones, and iterated ripple noise). These stimuli failed to reveal a consistent periodotopic map across the auditory fields imaged. Rather, gradients of period sensitivity differed for the different types of periodic stimuli. Binaural interactions were studied both with single contralateral, ipsilateral, and diotic broadband noise bursts and with sequences of broadband noise bursts with varying level presented contralaterally, ipsilaterally, or in opposite phase to both ears. Contralateral responses were generally largest and ipsilateral responses were smallest when using single noise bursts, but the extent of the activated area was large and comparable in all three aural configurations. Modulating the amplitude in counter phase to the two ears generally produced weaker modulation of the optical signals than the modulation produced by the monaural stimuli. These results suggest that binaural interactions seen in cortex are most likely predominantly due to subcortical processing. Thus our optical imaging data do not support the theory that the primary or nonprimary cortical fields imaged are topographically organized to form consistent maps of systematically varying sensitivity either to stimulus pitch or to simple binaural properties of the acoustic stimuli.

Ghost resonance in a pool of heterogeneous neurons

We numerically study the subharmonic response of a heterogeneous pool of neurons to a pair of independent inputs. The neurons are stimulated with periodic pulse trains of frequencies f(1)=2 Hz and f(2)=3 Hz, and with inharmonic pulses whose frequencies f(1) and f(2) are equally shifted an amount Delta f. When both inputs are subthreshold, we find that the neurons respond at a frequency equal to f(2)-f(1) in the harmonic situation (Delta f=0), that increases linearly with Delta f in the inharmonic case. Thus the neurons detect a frequency not present in the input; this effect is termed "ghost resonance". When one of the inputs is slightly suprathreshold the ghost resonance persists, but responses related with the frequency of the suprathreshold input also emerge. This behavior must be taken into account in experimental studies of signal integration and coincidence detection by neuronal pools.

Coincidence detection of inharmonic pulses in a nonlinear crystal

Two trains of light pulses at periods that are equally shifted from the harmonics of a missing fundamental are combined in a nonlinear crystal. As a result of a noncollinear phase-matched second-order nonlinear generation, a new train of pulses is obtained. When the temporal width of the input pulses is large, the frequency of the resulting pulse train follows the observations from classical experiments on the perception of virtual pitch by the brain. On the other hand, when the width of the input pulses is small, the generated pulse train exhibits much lower frequencies, analogous to those observed in the motor neural system. Our experimental setup allows us to explore, systematically and continuously, the transition between these two regimes, while at the same time to demonstrate that the functionalities that have been observed in the nervous system are similar to the ones we observe from coincidence detection in quadratic nonlinear systems.

The effects of stress and statistical cues on continuous speech segmentation: An event-related brain potential study

The study of the processes involved in speech segmentation has gained special relevance in recent years by trying to establish what type of information listeners use to segment the speech signal into words. An event-related brain potential experiment was conducted in order to understand how two of these cues (statistical and stress cues) interact. The experiment consisted of the presentation of artificial speech streams in which words were marked either by statistical cues alone, or by a combination of statistical and stress cues. As a baseline, comparison streams were also created with the same syllables but organized in random order. Results showed an N400 component that marks the on-line segmentation of speech into words, and an increased positivity (P2 component) for languages that include both types of cues. Possible implications of these results for the process of speech segmentation are discussed.

Cortical processing of quasi-periodic versus random noise sounds

The first objective was to confirm using auditory evoked potentials (AEPs) the findings of magnetoencephalographic studies, that quasi-periodic iterated rippled noise (IRN) elicits a population response in the human auditory cortex which is topographically distinct from that elicited by random noise with a similar overall frequency spectrum. AEPs were recorded at the onset of random noise from silence, at the transition from random noise to IRN with a period of 5 ms, and in the two complementary conditions, IRN onset from silence and the transition from IRN to random noise. An N1/P2 complex was recorded to all four stimuli, that to the transition to IRN being significantly the most anteriorly distributed on the scalp. The second objective was to determine whether the response to the transition to IRN was due to detection of its quasi-periodicity, rather than its spectral "ripples". Virtually no effect was found of applying a 2 kHz low- or high-pass filter, above which it is unlikely that the spectral ripples at intervals of 200 Hz would have been resolved on the cochlear partition. It is concluded that a substantial neuronal population in the auditory cortex is influenced by temporal regularity in sounds, and that this population is equally responsive to spectral frequencies below and above 2 kHz.

Neurophysiology and neuroanatomy of pitch perception: auditory cortex

We present original results and review literature from the past fifty years that address the role of primate auditory cortex in the following perceptual capacities: (1) the ability to perceive small differences between the pitches of two successive tones; (2) the ability to perceive the sign (i.e., direction) of the pitch difference [higher (+) vs. lower (-)]; and (3) the ability to abstract pitch constancy across changes in stimulus acoustics. Cortical mechanisms mediating pitch perception are discussed with respect to (1) gross and microanatomical distribution; and (2) candidate neural coding schemes. Observations by us and others suggest that (1) frequency-selective neurons in primary auditory cortex (A1) and surrounding fields play a critical role in fine-grained pitch discrimination at the perceptual level; (2) cortical mechanisms that detect pitch differences are neuroanatomically dissociable from those mediating pitch direction discrimination; (3) cortical mechanisms mediating perception of the "missing fundamental frequency (F0)" are neuroanatomically dissociable from those mediating pitch perception when F0 is present; (4) frequency-selective neurons in both right and left A1 contribute to pitch change detection and pitch direction discrimination; (5) frequency-selective neurons in right A1 are necessary for normal pitch direction discrimination; (6) simple codes for pitch that are based on single- and multiunit firing rates of frequency-selective neurons face both a "hyperacuity problem" and a "pitch constancy problem"-that is, frequency discrimination thresholds for pitch change direction and pitch direction discrimination are much smaller than neural tuning curves predict, and firing rate patterns change dramatically under conditions in which pitch percepts remain invariant; (7) cochleotopic organization of frequency-selective neurons bears little if any relevance to perceptual acuity and pitch constancy; and (8) simple temporal codes for pitch capable of accounting for pitches higher than a few hundred hertz have not been found in the auditory cortex. The cortical code for pitch is therefore not likely to be a function of simple rate profiles or synchronous temporal patterns. Studies motivated by interest in the neurophysiology and neuroanatomy of music perception have helped correct longstanding misconceptions about the functional role of auditory cortex in frequency discrimination and pitch perception. Advancing knowledge about the neural coding of pitch is of fundamental importance to the future design of neurobionic therapies for hearing loss.

Ghost stochastic resonance with distributed inputs in pulse-coupled electronic neurons

We study experimentally the phenomenon of ghost stochastic resonance in pulse-coupled excitable systems, for input signals distributed among different elements. Specifically, two excitable electronic circuits are driven by different sinusoidal signals that produce periodic spikes at distinct frequencies. Their outputs are sent to a third circuit that processes these spiking signals and is additionally perturbed by noise. When the input signals are harmonics of a certain fundamental (that is not present in the inputs) the processing circuit exhibits, for an optimal amount of noise, a resonant response at the frequency of the missing fundamental (ghost frequency). In contrast with the standard case in which the signals being directly integrated are sinusoidal, this behavior relies here on a coincidence-detection mechanism. When the input signals are homogeneously shifted in frequency, the processing circuit responds with pulse packages composed of spikes at a frequency that depends linearly on the frequency shift. Expressions for the dependence of the package period and duration on the frequency shift and spike width, respectively, are obtained. These results provide an experimental verification of a recently proposed mechanism of binaural pitch perception.

Neural response correlates of detection of monaurally and binaurally created pitches in humans

Recent magnetoencephalography (MEG) and functional magnetic resonance imaging studies of human auditory cortex are pointing to brain areas on lateral Heschl's gyrus as the 'pitch-processing center'. Here we describe results of a combined MEG-psychophysical study designed to investigate the timing of the formation of the percept of pitch and the generality of the hypothesized 'pitch-center'. We compared the cortical and behavioral responses to Huggins pitch (HP), a stimulus requiring binaural processing to elicit a pitch percept, with responses to tones embedded in noise (TN)-perceptually similar but physically very different signals. The stimuli were crafted to separate the electrophysiological responses to onset of the pitch percept from the onset of the initial stimulus. Our results demonstrate that responses to monaural pitch stimuli are affected by cross-correlational processes in the binaural pathway. Additionally, we show that MEG illuminates processes not simply observable in behavior. Crucially, the MEG data show that, although physically disparate, both HP and TN are mapped onto similar representations by 150 ms post-onset, and provide critical new evidence that the 'pitch onset response' reflects central pitch mechanisms, in agreement with models postulating a single, central pitch extractor.

Aphasia and auditory extinction: Preliminary evidence of binding

Background: McNeil, Odell, and Tseng (1991), and Murray and colleagues (Murray, 2000; Murray, Holland, & Beeson, 1997a, 1997b) have suggested that variability of performance in patients with aphasia may be due to nonlinguistic cognitive variables, such as attention (i.e., resources, capacity, effort), which affect language comprehension and production. Given the research that has supported the relationship between aphasia and attention deficits, it is important to determine what effect this breakdown in attention may have on cognitive processes for individuals with aphasia.Aims: This study aims to determine if auditory extinction is present in individuals with aphasia, and if so, if this is due to a breakdown in binding. If extinction is found for individuals with aphasia, it would further support the notion that auditory attention difficulties are present among individuals with aphasia, since visual and auditory research has attributed extinction to a breakdown in attention (Baylis, Driver, & Rafal, 1993; Deouell, Bentin, & Soroker, 2000; Deouell & Soroker, 2000). If binding is found to be deficient, the fact that individuals with both left and right hemisphere lesions demonstrate this phenomenon would lead to a number of implications regarding the relationship of attention and aphasia.Methods & Procedures: Auditory extinction, in which one stimulus is not perceived during double simultaneous stimulation (DSS) presentation, was examined in six individuals with aphasia (aged 42-74 years) and six age-matched healthy adults. Two different experiments were conducted in which the auditory stimuli, consisting of male and female voices speaking the letters "T" or "O", were systematically varied to investigate whether binding of identification to location contributes to extinction.Outcomes & Results: Participants with aphasia made more omission errors (extinction) than the control group, and extinction was significantly greater for binding versus nonbinding conditions, suggesting that binding may play a role in extinction for individuals with aphasia.Conclusions: These data provide preliminary results that auditory extinction exists in individuals with aphasia and may be due to deficits in binding together identification and localisation information. Research on this phenomenon and how it influences language would be a worthwhile endeavour for future studies. Moreover, little is known about assessment of auditory attention in patients with aphasia. Further research in this area can lead to advancements in theoretical and functional assessment for individuals with aphasia who have auditory attention and/or binding deficits and require speech-language pathology intervention.

Neural mechanism for binaural pitch perception via ghost stochastic resonance

We present a physiologically plausible binaural mechanism for the perception of the pitch of complex sounds via ghost stochastic resonance. In this scheme, two neurons are driven by noise and a different periodic signal each (with frequencies f(1)=kf(0) and f(2)=(k+1)f(0), where k>1), and their outputs (plus noise) are applied synaptically to a third neuron. Our numerical results, using the Morris-Lecar neuron model with chemical synapses explicitly considered, show that intermediate noise levels enhance the response of the third neuron at frequencies close to f(0), as in the cases previously described of ghost resonance. For the case of an inharmonic combination of inputs (f(1)=kf(0)+Deltaf and f(2)=(k+1)f(0)+Deltaf) noise is also seen to enhance the rates of most probable spiking for the third neuron at a frequency f(r)=f(0)+[Deltaf(k+12)]. In addition, we show that similar resonances can be observed as a function of the synaptic time constant. The suggested ghost-resonance-based stochastic mechanism can thus arise either at the peripheral level or at a higher level of neural processing in the perception of pitch.

MEG responses to rippled noise and Huggins pitch reveal similar cortical representations

The onset of pitch within an ongoing noise signal evokes a particular brain activity, the pitch onset response (POR). Using whole-head MEG, PORs to iterated rippled noise (IRN) and Huggins pitch (HP), representing prototypical pitch-in-noise signals, were measured in twenty subjects during a pitch identification task (333 Hz, 400 Hz, randomized). HP and IRN yielded similar responses, lateralized to the left hemisphere and peaking about 180 ms after pitch onset. The initial phase (140 ms) showed stronger activations to 400 than to 333 Hz whereas later stages (200-300 ms) showed target vs nontarget effects. These results suggest, first, that different pitches converge into a common cortical representation and, second, that the POR encompasses various successive processing stages.

Time course and hemispheric lateralization effects of complex pitch processing: evoked magnetic fields in response to rippled noise stimuli

To delineate the time course and processing stages of pitch encoding at the level of the supratemporal plane, the present study recorded evoked magnetic fields in response to rippled noise (RN) stimuli. RN largely masks simple tonotopic representations and addresses pitch processing within the temporal domain (periodicity encoding). Four dichotic stimulus types (111 or 133 Hz RN at one ear, white noise to the other one) were applied in randomized order during either visual distraction or selective auditory attention. Strictly periodic signals, noise-like events, and mixtures of both signals served as control conditions. (1) Attention-dependent ear x hemisphere interactions were observed within the time domain of the M50 field, indicating early streaming of auditory information. (2) M100 responses to strictly periodic stimuli were found lateralized to the right hemisphere. Furthermore, the higher-pitched stimuli yielded enhanced activation as compared to the lower-pitch signals (pitch scaling), conceivably reflecting sensory memory operations. (3) Besides right-hemisphere pitch scaling, the relatively late M100 component in association with the RN condition (latency = 136 ms) showed significantly stronger field strengths over the left hemisphere. Control experiments revealed this lateralization effect to be related to noise rather than pitch processing. Furthermore, subtle noise variations interacted with signal periodicity. Obviously, thus, complex task demands such as RN encoding give rise to functional segregation of auditory processing across the two hemispheres (left hemisphere: noise, right hemisphere: periodicity representation). The observed noise/periodicity interactions, furthermore, might reflect pitch-synchronous spectral evaluation at the level of the left supratemporal plane, triggered by right-hemisphere representation of signal periodicity.

Interaction of Excitatory and Inhibitory Frequency-receptive Fields in Determining Fundamental Frequency Sensitivity of Primary Auditory Cortex Neurons in Awake Cats

Harmonic complex tones produce pitch-height perception corresponding to the fundamental frequency (F0). This study investigates how the spectral cue of F0 is processed in neurons of the primary auditory cortex (A1) with sustained-response properties. We found F0-sensitive and -insensitive cells: the former discriminated between harmonics and noise, while the latter did not. F0-sensitive cells preferred F0s corresponding to the best frequency (BF) and 0.5 x BF. The F0-sensitivity to F0=0.5 x BF was preserved for missing F0, but abolished by eliminating both F0 and the second harmonic. The inhibitory subfield of the frequency-receptive field was restricted to the spectral region between the preferred harmonics in F0-sensitive cells, while it was frequency unspecific in F0-insensitive cells. We conclude that (i) A1 is well organized for discrimination between harmonics and noise; (ii) pitch-height is represented along with the tonotopic axis; (iii) all aspects of the sustained neural responses to harmonic and noise stimuli are consequences of spectral filtering; and (iv) although the observed cell behavior explains some psychophysical pitch perception behaviors, such as pitch-chroma (helical pitch perception with frequency elevation), pitch-level tolerance and adaptive behavior, F0-encoding in A1 remains at the incomplete perceptual level (dominance of the third to fifth harmonics for pitch strength is unexplainable by the cell behavior).

Evaluation of missing fundamental phenomenon in the human auditory cortex

OBJECTIVE

Harmonic complex tones consisting of four or more continuous harmonics of a certain stem frequency are perceived as the pitch of the fundamental frequency tone, it is referred to as the missing fundamental phenomenon (MFP). It is considered that the MFP is produced in the central auditory system, not in the periphery. However, it remains unclear where and how complex sounds is integrated. Using 306ch magnetoencephalography (MEG), we investigated when and where the MFP was integrated in the auditory cortex.

METHOD

We examined six subjects who were selected by MEG in 12 healthy right-handed adult volunteers with normal auditory sensation. Ears were randomly stimulated with five different complex tones consist of fundamental frequency tone and harmonic complex tones. The location and direction of equivalent current dipoles (ECD) were evaluated at P50 and N100 in the right temporal lobe by MEG. Dispersion of the source of ECD was respectively evaluated on their brain MRI.

RESULTS

Stimulation of ears with harmonic complex tones and the stem frequency tone revealed the localization of P50 and N100 ECD in the transverse temporal gyrus and their peripheral superior temporal gyrus. Although the sources of P50 ECD for harmonic complex tones and the fundamental tone were varied around the transverse temporal gyrus and superior temporal gyrus, the sources of N100 ECD were almost identical at the transverse temporal gyrus, demonstrating the MFP. This phenomenon were similarly observed, even when dichotic listening were stimulated.

CONCLUSION

These findings suggest that the MFP occurs in the transverse temporal gyrus and the superior temporal gyrus, which are the primary auditory cortex, between P50 and N100.

Auditory middle-latency components to fusion of speech elements forming an auditory object

OBJECTIVE

The purpose of this study was to define early brain activity associated with fusion of speech elements to form an auditory object in the middle-latency range preceding the F-Complex.

METHODS

Stimuli were binaural formant transition and base, that were presented separately or fused to form the vowel-consonant-vowel sequence /ada/. Eleven right-handed, adult, native Hebrew speakers listened to 2/s presentations, and the brain potentials from C(z) during the 250 msec following transition onset (in the responses to transition and to the fused word) or following the time it would have been presented (in the response to base alone) were recorded. The net-fusion response was extracted by subtracting the sum of potentials to the base and the formant transition from the potentials to the fused sound.

RESULTS

Auditory middle-latency components, comprising of 9 peaks and troughs were recorded in response to the base, to the formant transition and to the fused /ada/. In general, the responses to the fused object were significantly smaller in peak amplitude and in total activity (area under the curve) resulting in the difference waveform of the net-fusion response that also included 9 peaks, but with opposite polarities.

CONCLUSIONS

The early middle-latency components to fusion indicate that the fusion of speech elements to a word involves inhibition, occlusion or both. The results are in line with the uniqueness of speech perception and the early role of the auditory cortex in speech analysis.

Interpreting abnormality: an EEG and MEG study of P50 and the auditory paired-stimulus paradigm

Interpretation of neurophysiological differences between control and patient groups on the basis of scalp-recorded event-related brain potentials (ERPs), although common and promising, is often complicated in the absence of information on the distinct neural generators contributing to the ERP, particularly information regarding individual differences in the generators. For example, while sensory gating differences frequently observed in patients with schizophrenia in the P50 paired-click gating paradigm are typically interpreted as reflecting group differences in generator source strength, differences in the latency and/or orientation of P50 generators may also account for observed group differences. The present study examined how variability in source strength, amplitude, or orientation affects the P50 component of the scalp-recorded ERP. In Experiment 1, simulations examined the effect of changes in source strength, orientation, or latency in superior temporal gyrus (STG) dipoles on P50 recorded at Cz. In Experiment 2, within- and between-subject variability in left and right M50 STG dipole source strength, latency, and orientation was examined in 19 subjects. Given the frequently reported differences in left and right STG anatomy and function, substantial inter-subject and inter-hemispheric variability in these parameters were expected, with important consequences for how P50 at Cz reflects brain activity from relevant generators. In Experiment 1, simulated P50 responses were computed from hypothetical left- and right-hemisphere STG generators, with latency, amplitude, and orientation of the generators varied systematically. In Experiment 2, electroencephalographic (EEG) and magnetoencephalographic (MEG) data were collected from 19 subjects. Generators were modeled from the MEG data to assess and illustrate the generator variability evaluated parametrically in Experiment 1. In Experiment 1, realistic amounts of variability in generator latency, amplitude, and orientation produced ERPs in which P50 scoring was compromised and interpretation complicated. In Experiment 2, significant within and between subject variability was observed in the left and right hemisphere STG M50 sources. Given the variability in M50 source strength, orientation, and amplitude observed here in nonpatient subjects, future studies should examine whether group differences in P50 gating ratios typically observed for patient vs. control groups are specific to a particular hemisphere, as well as whether the group differences are due to differences in dipole source strength, latency, orientation, or a combination of these parameters. Present analyses focused on P50/M50 merely as an example of the broader need to evaluate scalp phenomena in light of underlying generators. The development and widespread use of EEG/MEG source localization methods will greatly enhance the interpretation and value of EEG/MEG data.

Tonotopic representation of missing fundamental complex sounds in the human auditory cortex

The N1m component of the auditory evoked magnetic field in response to tones and complex sounds was examined in order to clarify whether the tonotopic representation in the human secondary auditory cortex is based on perceived pitch or the physical frequency spectrum of the sound. The investigated stimulus parameters were the fundamental frequencies (F0 = 250, 500 and 1000 Hz), the spectral composition of the higher harmonics of the missing fundamental sounds (2nd to 5th, 6th to 9th and 10th to 13th harmonic) and the frequencies of pure tones corresponding to F0 and to the lowest component of each complex sound. Tonotopic gradients showed that high frequencies were more medially located than low frequencies for the pure tones and for the centre frequency of the complex tones. Furthermore, in the superior-inferior direction, the tonotopic gradients were different between pure tones and complex sounds. The results were interpreted as reflecting different processing in the auditory cortex for pure tones and complex sounds. This hypothesis was supported by the result of evoked responses to complex sounds having longer latencies. A more pronounced tonotopic representation in the right hemisphere gave evidence for right hemispheric dominance in spectral processing.

Studies of tonotopy based on wave N100 of the auditory evoked field are problematic

There is still dissension as to whether the auditory evoked field (AEF) reflects tonotopy in the auditory cortex. That notwithstanding, particularly the pronounced AEF wave occurring about 100 ms after stimulus onset (N100 m) is increasingly used for the investigation of issues such as cortical reorganization and representation of virtual pitch. Thus, it appears to be time for a critical revaluation of the supposed tonotopic organization of the N100 m generator. In the present magnetoencephalography study, the response to tonebursts of 500 ms duration, monaurally presented 60 dB above threshold, was recorded with a 37-channel axial gradiometer system over the hemisphere contralateral to the side of stimulation. The stimulus frequencies were 250, 500, 1000, and 2000 Hz. About 250 stimuli of each type were presented in random order in four independent sessions at intervals uniformly distributed between 2 and 2.8 s. An analysis of 19 hemispheres in 10 normal-hearing subjects showed a high intraindividual reproducibility, but also a substantial interindividual variability. In most cases, the dipole location either exhibited no significant frequency dependence at all, the dipoles for the four frequencies were not orderly aligned, or the data disagreed with the single-dipole model. In the few cases showing an arrangement of dipoles consistent with the assumption of an orderly tonotopic cortical map, the most relevant coordinate varied from subject to subject. Regarding theses results, it seems crucial to understand wave N100 m on the basis of individual subjects, whereas conclusions relying on mean dipole locations for groups of subjects are problematic.

The electrophysiological net response ('F-complex') to spatial fusion of speech elements forming an auditory object

OBJECTIVE

The purpose of this study was to define and analyze the brain activity associated with fusion of speech elements to form an auditory object and to study the effects of presenting the elements at different spatial locations (duplex stimulus).

METHODS

Stimuli were formant transitions (presented to the front, left or right of the subject) and base (presented to the front), that fused to result in V-C-V sequences /aga/ and /ada/. Ten right-handed, adult, native Hebrew speakers discriminated each fused stimulus, and the brain potentials associated with performance of the task were recorded from 21 electrodes. The net-fusion response, the 'F(fusion)-complex', was extracted by subtracting the sum of potentials to the base and formant transitions from the potentials to the fused sound. Low resolution electromagnetic tomography analysis (LORETA) was performed to assess the timing and brain location of the fusion process.

RESULTS

The 'F-complex', comprising of the difference N(1), P(2), N(2b) (FN(1), FP(2), FN(2b)) components could be identified for each of the stimuli and reflected a process indicating inhibition, occlusion or both, with right ear advantage in fusion. LORETA analyses indicate sequential processing of speech fusion in the temporal lobes, beginning with right prominence in FN(1) and FP(2) shifting to a more symmetrical pattern in FN(2).

CONCLUSIONS

The electrophysiological correlates of speech fusion highlight the uniqueness of speech perception and the brain areas involved in its analysis.

Functional cerebral asymmetries of pitch processing during dichotic stimulus application: a whole-head magnetoencephalography study

Dichotic listening (DL) studies indicate higher proficiency of the right cerebral hemisphere in processing the pitch of auditory events. Especially, acoustic stimuli of a rich harmonic structure such as square waves (complex tones) elicit a left ear advantage (LEA) under dichotic stimulus application. In order to investigate the timing of early sensory encoding at the level of the supratemporal plane, whole-head magnetoencephalography (MEG; 151 channels) recordings were performed in 20 right-handed subjects using an oddball paradigm based on dichotically applied complex tones. In contrast to electroencephalography (EEG) and event related potentials (ERP), this technique separately measures neuronal activity of left and right auditory cortex. Neuromagnetic responses were obtained both during preattentive stimulus processing, as well as during a pitch detection task. Rare stimuli presented to the left ear elicited a stronger magnetic analogue of mismatch negativity (MMNm) over both hemispheres and gave rise to shorter latencies of the contralateral mismatch fields than right ear deviants. In conclusion, the present data provide first evidence for functional laterality effects even at the level of preattentive pitch processing within the auditory cortex.

Human pitch perception is reflected in the timing of stimulus-related cortical activity

'Pitch' refers to a sound's subjective highness or lowness, as distinct from 'frequency,' which refers to a sound's physical structure. In speech, music and other natural contexts, complex tones are often perceived with a single pitch. Using whole-head magnetoencephalography (MEG) and stimuli that dissociate pitch from frequency, we studied cortical dynamics in normal individuals who extracted different pitches from the same tone complexes. Whereas all subjects showed similar spatial distributions in the magnitude of their brain responses to the stimuli, subjects who heard different pitches exhibited contrasting temporal patterns of brain activity in their right but not their left hemispheres. These data demonstrate a specific relationship between pitch perception and the timing (phase) of dynamic patterns of cortical activity.

Complex tone processing in primary auditory cortex of the awake monkey. II. Pitch versus critical band representation

Noninvasive neurophysiological studies in humans support the existence of an orthogonal spatial representation of pure tone frequency and complex tone pitch in auditory cortex [Langner et al., J. Comp. Physiol. A 181, 665-676 (1997)]. However, since this topographic organization is based on neuromagnetic responses evoked by wideband harmonic complexes (HCs) of variable fundamental frequency (f0), and thus interharmonic frequency separation (deltaF), critical band filtering effects due to differential resolvability of harmonics may have contributed to shaping these responses. To test this hypothesis, the present study examined responses evoked by three-component HCs of variable f0 in primary auditory cortex (A1) of the awake monkey. The center frequency of the HCs was fixed at the best frequency (BF) of the cortical site. Auditory evoked potential (AEP), multiunit activity, and current source density techniques were used to evaluate A1 responses as a function of f0 (=deltaF). Generally, amplitudes of nearly all response components increased with f0, such that maximal responses were evoked by HCs comprised of low-order resolved harmonics. Statistically significant increases in response amplitude typically occurred at deltaFs between 10% and 20% of center frequency, suggestive of critical bandlike behavior. Complex tone response amplitudes also reflected nonlinear summation in that they could not be predicted by the pure tone frequency sensitivity curves of the cortical sites. A mechanism accounting for the observed results is proposed which involves mutual lateral inhibitory interactions between responses evoked by stimulus components lying within the same critical band. As intracortical AEP components likely to be propagated to the scalp were also strongly modulated by deltaF, these findings indicate that noninvasive recordings of responses to complex sounds may require a consideration of critical band effects in their interpretation.

The perception of coherent and non-coherent auditory objects: a signature in gamma frequency band

The pertinence of gamma band activity in magnetoencephalographic and electroencephalographic recordings for the performance of a gestalt recognition process is a question at issue. We investigated the functional relevance of gamma band activity for the perception of auditory objects. An auditory experiment was performed as an analog to the Kanizsa experiment in the visual modality, comprising four different coherent and non-coherent stimuli. For the first time functional differences of evoked gamma band activity due to the perception of these stimuli were demonstrated by various methods (localization of sources, wavelet analysis and independent component analysis, ICA). Responses to coherent stimuli were found to have more features in common compared to non-coherent stimuli (e.g. closer located sources and smaller number of ICA components). The results point to the existence of a pitch processor in the auditory pathway.

Magnetoencephalographic studies of functional organization and plasticity of the human auditory cortex

Magnetoencephalography has proven to be a powerful noninvasive tool for investigating the functional organization of the human auditory cortex and its plastic changes. The first part of this review summarizes some recent experiments on the tonotopic organization, which can be observed not only in the slow auditory evoked fields, but also in the middle-latency and the steady-state fields. In the second part of this review, recent studies on plasticity of the auditory cortex are outlined. These studies showed that the cortical representation of tones may change within hours after a reversible "functional deafferentation" (short-term plasticity) and that early musical training leads to an expansion in the cortical representation of complex harmonic sounds (long-term plasticity).

Latency of the auditory evoked neuromagnetic field components: stimulus dependence and insights toward perception

This review will focus on investigations of the auditory evoked neuromagnetic field component, the M100, detectable in the magnetoencephalogram recorded during presentation of auditory stimuli, approximately 100 milliseconds after stimulus onset. In particular, the dependence of M100 latency on attributes of the stimulus, such as intensity, pitch and timbre will be discussed, along with evidence relating M100 latency observations to perceptual features of the stimuli. Comparison with investigation of the analogous electrical potential component, the N1, will be made. Parametric development of stimuli from pure tones through complex tones to speech elements will be made, allowing the influence of spectral pitch, virtual pitch and perceptual categorization to be delineated and suggesting implications for the role of such latency observations in the study of speech processing. The final section will deal with potential clinical applications offered by M100 latency measurements, as objective indices of normal and abnormal cortical processing.

Auditory processing in primate cerebral cortex

Auditory information is relayed from the ventral nucleus of the medial geniculate complex to a core of three primary or primary-like areas of auditory cortex that are cochleotopically organized and highly responsive to pure tones. Auditory information is then distributed from the core areas to a surrounding belt of about seven areas that are less precisely cochleotopic and generally more responsive to complex stimuli than tones. Recent studies indicate that the belt areas relay to the rostral and caudal divisions of a parabelt region at a third level of processing in the cortex lateral to the belt. The parabelt and belt regions have additional inputs from dorsal and magnocellular divisions of the medial geniculate complex and other parts of the thalamus. The belt and parabelt regions appear to be concerned with integrative and associative functions involved in pattern perception and object recognition. The parabelt fields connect with regions of temporal, parietal, and frontal cortex that mediate additional auditory functions, including space perception and auditory memory.

Click train encoding in primary auditory cortex of the awake monkey: evidence for two mechanisms subserving pitch perception

Multiunit activity (MUA) and current source density (CSD) patterns evoked by click trains are examined in primary auditory cortex (A1) of three awake monkeys. Temporal and spectral features of click trains are differentially encoded in A1. Encoding of temporal features occurs at rates of 100-200 Hz through phase-locked activity in the MUA and CSD, is independent of pulse polarity pattern, and occurs in high best frequency (BF) regions of A1. The upper limit of ensemble-wide phase-locking is about 400 Hz in the input to A1, as manifested in the cortical middle laminae CSD and MUA of thalamocortical fibers. In contrast, encoding of spectral features occurs in low BF regions, and resolves both the f0 and harmonics of the stimuli through local maxima of activity determined by the tonotopic organization of the recording sites. High-pass filtered click trains decrease spectral encoding in low BF regions without modifying phase-locked responses in high BF regions. These physiological responses parallel features of human pitch perception for click trains, and support the existence of two distinct physiological mechanisms involved in pitch perception: the first using resolved harmonic components and the second utilizing unresolved harmonics that is based on encoding stimulus waveform periodicity.

Dipole separability in a neuromagnetic source analysis

By studying the ability of a one-dipole model to explain the magnetic field actually resulting from two dipoles, minimum requirements for a successful separation of two dipoles were explored. Two dipoles in different depths generally require a much higher signal-to-noise ratio (SNR) than two dipoles in the same depth. For the latter condition, the dipole distance as well as the angles between the moments and the line connecting the dipole locations (connecting line) were systematically varied. A perpendicular orientation of the two dipoles turned out to be the most favorable condition: the minimum distance required for a separation of two dipoles was more than four times smaller than for a configuration with both moments oriented parallel to the connecting line. Separability of parallel dipoles was moderately enhanced if both moments assumed an orientation perpendicular to the connecting line. The separability of two antiparallel dipoles is not limited by concurrence with a one-dipole model, but by the low signal amplitudes resulting from a mutual cancellation of the fields arising from the two dipoles, and by concurrence with a quadrupole model. The results are presented so that quantitative conclusions about dipole separability can be derived for arbitrary SNR's. The study does not generally disprove the common believe that magnetoencephalography has a relatively poor spatial resolution, but it qualifies this view by suggesting that under favorable conditions two sources with a distance of only 1 cm may be resolvable.

Temporal dynamics of early perceptual processing

Recordings of electrical and magnetic brain responses to sensory stimulation provide high-resolution measures of the time course of early perceptual processing. Spatio-temporal analyses of brain activity patterns during the first 200 ms after stimulus presentation have characterized the timing of attentional selection processes and different stages of feature encoding and pattern analyses. Recent studies that incorporate blood flow neuroimaging techniques provide support for mechanisms of early selection of attended visual inputs in extrastriate cortical pathways. The spatial tuning properties of early auditory selection have also been delineated. Electrical and magnetic responses that index the encoding of higher-order pattern information have been identified in both visual and auditory modalities and localized to specific cortical areas.

Pitch vs. spectral encoding of harmonic complex tones in primary auditory cortex of the awake monkey

Neuromagnetic studies in humans and single-unit studies in monkeys have provided conflicting views regarding the role of primary auditory cortex (A1) in pitch encoding. While the former support a topographic organization based on the pitch of complex tones, single-unit studies support the classical tonotopic organization of A1 defined by the spectral composition of the stimulus. It is unclear whether the incongruity of these findings is due to limitations of noninvasive recordings or whether the discrepancy genuinely reflects pitch representation based on population encoding. To bridge these experimental approaches, we examined neuronal ensemble responses in A1 of the awake monkey using auditory evoked potential (AEP), multiple-unit activity (MUA) and current source density (CSD) techniques. Macaque monkeys can perceive the missing fundamental of harmonic complex tones and therefore serve as suitable animal models for studying neural encoding of pitch. Pure tones and harmonic complex tones missing the fundamental frequency (f0) were presented at 60 dB SPL to the ear contralateral to the hemisphere from which recordings were obtained. Laminar response profiles in A1 reflected the spectral content rather than the pitch (missing f0) of the compound stimuli. These findings are consistent with single-unit data and indicate that the cochleotopic organization is preserved at the level of A1. Thus, it appears that pitch encoding of multi-component sounds is more complex than suggested by noninvasive studies, which are based on the assumption of a single dipole generator within the superior temporal gyrus. These results support a pattern recognition mechanism of pitch encoding based on a topographic representation of stimulus spectral composition at the level of A1.

Two separate codes for missing-fundamental pitch in the human auditory cortex

Two auditory event-related potential components, the supratemporal N1 and the mismatch negativity (MMN), index traces encoding the missing-fundamental pitch. The present results suggest that these two codes derive from separate pitch extraction processes. Frequent 300-Hz and infrequent 600-Hz missing-fundamental tones were presented, in some stimulus blocks with short (150 ms), in others, with long (500 ms) stimulus durations. MMN, reflecting a preattentive change detection process, was elicited by infrequent missing-fundamental tones only in the long-duration condition. Correspondingly, subjects were able to detect these high-pitch missing-fundamental tones amongst similar low-pitch ones only when the stimulus duration was long. In addition, the MMN response peaked ca. 120 ms later for missing-fundamental tones than for pure tones of the fundamental frequency suggesting that missing-fundamental pitch resolving took substantially longer than extracting the spectral pitch. In contrast, a differential N1 response to the missing-fundamental pitch was found for both stimulus durations, with no substantial difference in peak latency between the pure and missing-fundamental tones. The contrasting features found for the two auditory cortical missing-fundamental pitch codes support the notion of two separate missing-fundamental pitch resolving mechanisms.