Human auditory cortical mechanisms of sound lateralization: II. Interaural time differences at sound onset

Original speech and its echo are segregated and separately processed in the human brain

Speech recognition crucially relies on slow temporal modulations (<16 Hz) in speech. Recent studies, however, have demonstrated that the long-delay echoes, which are common during online conferencing, can eliminate crucial temporal modulations in speech but do not affect speech intelligibility. Here, we investigated the underlying neural mechanisms. MEG experiments demonstrated that cortical activity can effectively track the temporal modulations eliminated by an echo, which cannot be fully explained by basic neural adaptation mechanisms. Furthermore, cortical responses to echoic speech can be better explained by a model that segregates speech from its echo than by a model that encodes echoic speech as a whole. The speech segregation effect was observed even when attention was diverted but would disappear when segregation cues, i.e., speech fine structure, were removed. These results strongly suggested that, through mechanisms such as stream segregation, the auditory system can build an echo-insensitive representation of speech envelope, which can support reliable speech recognition.

Neural Substrates and Models of Omission Responses and Predictive Processes

Predictive coding theories argue that deviance detection phenomena, such as mismatch responses and omission responses, are generated by predictive processes with possibly overlapping neural substrates. Molecular imaging and electrophysiology studies of mismatch responses and corollary discharge in the rodent model allowed the development of mechanistic and computational models of these phenomena. These models enable translation between human and non-human animal research and help to uncover fundamental features of change-processing microcircuitry in the neocortex. This microcircuitry is characterized by stimulus-specific adaptation and feedforward inhibition of stimulus-selective populations of pyramidal neurons and interneurons, with specific contributions from different interneuron types. The overlap of the substrates of different types of responses to deviant stimuli remains to be understood. Omission responses, which are observed both in corollary discharge and mismatch response protocols in humans, are underutilized in animal research and may be pivotal in uncovering the substrates of predictive processes. Omission studies comprise a range of methods centered on the withholding of an expected stimulus. This review aims to provide an overview of omission protocols and showcase their potential to integrate and complement the different models and procedures employed to study prediction and deviance detection.This approach may reveal the biological foundations of core concepts of predictive coding, and allow an empirical test of the framework's promise to unify theoretical models of attention and perception.

Properties of echoic memory revealed by auditory-evoked magnetic fields

We used auditory-evoked magnetic fields to investigate the properties of echoic memory. The sound stimulus was a repeated 1-ms click at 100 Hz for 500 ms, presented every 800 ms. The phase of the sound was shifted by inserting an interaural time delay of 0.49 ms to each side. Therefore, there were two sounds, lateralized to the left and right. According to the preceding sound, each sound was labeled as D (preceded by a different sound) or S (by the same sound). The D sounds were further grouped into 1D, 2D, and 3D, according to the number of preceding different sounds. The S sounds were similarly grouped to 1S and 2S. The results showed that the preceding event significantly affected the amplitude of the cortical response; although there was no difference between 1S and 2S, the amplitudes for D sounds were greater than those for S sounds. Most importantly, there was a significant amplitude difference between 1S and 1D. These results suggested that sensory memory was formed by a single sound, and was immediately replaced by new information. The constantly-updating nature of sensory memory is considered to enable it to act as a real-time monitor for new information.

The spatio-temporal dynamics of deviance and target detection in the passive and active auditory oddball paradigm: a sLORETA study

Background

Numerous studies have investigated the neural underpinnings of passive and active deviance and target detection in the well-known auditory oddball paradigm by means of event-related potentials (ERPs) or functional magnetic resonance imaging (fMRI). The present auditory oddball study investigates the spatio-temporal dynamics of passive versus active deviance and target detection by analyzing amplitude modulations of early and late ERPs while at the same time exploring the neural sources underling this modulation with standardized low-resolution brain electromagnetic tomography (sLORETA) .

Methods

A 64-channel EEG was recorded from twelve healthy right-handed participants while listening to ‘standards’ and ‘deviants’ (500 vs. 1000 Hz pure tones) during a passive (block 1) and an active (block 2) listening condition. During passive listening, participants had to simply listen to the tones. During active listening they had to attend and press a key in response to the deviant tones.

Results

Passive and active listening elicited an N1 component, a mismatch negativity (MMN) as difference potential (whose amplitudes were temporally overlapping with the N1) and a P3 component. N1/MMN and P3 amplitudes were significantly more pronounced for deviants as compared to standards during both listening conditions. Active listening augmented P3 modulation to deviants significantly compared to passive listening, whereas deviance detection as indexed by N1/MMN modulation was unaffected by the task. During passive listening, sLORETA contrasts (deviants > standards) revealed significant activations in the right superior temporal gyrus (STG) and the lingual gyri bilaterally (N1/MMN) as well as in the left and right insulae (P3). During active listening, significant activations were found for the N1/MMN in the right inferior parietal lobule (IPL) and for the P3 in multiple cortical regions (e.g., precuneus).

Discussion

The results provide evidence for the hypothesis that passive as well as active deviance and target detection elicit cortical activations in spatially distributed brain regions and neural networks including the ventral attention network (VAN), dorsal attention network (DAN) and salience network (SN). Based on the temporal activation of the neural sources underlying ERP modulations, a neurophysiological model of passive and active deviance and target detection is proposed which can be tested in future studies.

Sensorineural hearing loss degrades behavioral and physiological measures of human spatial selective auditory attention

Listeners with sensorineural hearing loss often have trouble understanding speech amid other voices. While poor spatial hearing is often implicated, direct evidence is weak; moreover, studies suggest that reduced audibility and degraded spectrotemporal coding may explain such problems. We hypothesized that poor spatial acuity leads to difficulty deploying selective attention, which normally filters out distracting sounds. In listeners with normal hearing, selective attention causes changes in the neural responses evoked by competing sounds, which can be used to quantify the effectiveness of attentional control. Here, we used behavior and electroencephalography to explore whether control of selective auditory attention is degraded in hearing-impaired (HI) listeners. Normal-hearing (NH) and HI listeners identified a simple melody presented simultaneously with two competing melodies, each simulated from different lateral angles. We quantified performance and attentional modulation of cortical responses evoked by these competing streams. Compared with NH listeners, HI listeners had poorer sensitivity to spatial cues, performed more poorly on the selective attention task, and showed less robust attentional modulation of cortical responses. Moreover, across NH and HI individuals, these measures were correlated. While both groups showed cortical suppression of distracting streams, this modulation was weaker in HI listeners, especially when attending to a target at midline, surrounded by competing streams. These findings suggest that hearing loss interferes with the ability to filter out sound sources based on location, contributing to communication difficulties in social situations. These findings also have implications for technologies aiming to use neural signals to guide hearing aid processing.

Cortical Representation of Interaural Time Difference Is Impaired by Deafness in Development: Evidence from Children with Early Long-term Access to Sound through Bilateral Cochlear Implants Provided Simultaneously

Accurate use of interaural time differences (ITDs) for spatial hearing may require access to bilateral auditory input during sensitive periods in human development. Providing bilateral cochlear implants (CIs) simultaneously promotes symmetrical development of bilateral auditory pathways but does not support normal ITD sensitivity. Thus, although binaural interactions are established by bilateral CIs in the auditory brainstem, potential deficits in cortical processing of ITDs remain. Cortical ITD processing in children with simultaneous bilateral CIs and normal hearing with similar time-in-sound was explored in the present study. Cortical activity evoked by bilateral stimuli with varying ITDs (0, ±0.4, ±1 ms) was recorded using multichannel electroencephalography. Source analyses indicated dominant activity in the right auditory cortex in both groups but limited ITD processing in children with bilateral CIs. In normal-hearing children, adult-like processing patterns were found underlying the immature P1 (∼100 ms) response peak with reduced activity in the auditory cortex ipsilateral to the leading ITD. Further, the left cortex showed a stronger preference than the right cortex for stimuli leading from the contralateral hemifield. By contrast, children with CIs demonstrated reduced ITD-related changes in both auditory cortices. Decreased parieto-occipital activity, possibly involved in spatial processing, was also revealed in children with CIs. Thus, simultaneous bilateral implantation in young children maintains right cortical dominance during binaural processing but does not fully overcome effects of deafness using present CI devices. Protection of bilateral pathways through simultaneous implantation might be capitalized for ITD processing with signal processing advances, which more consistently represent binaural timing cues.SIGNIFICANCE STATEMENT Multichannel electroencephalography demonstrated impairment of binaural processing in children who are deaf despite early access to bilateral auditory input by first finding that foundations for binaural hearing are normally established during early stages of cortical development. Although 4- to 7-year-old children with normal hearing had immature cortical responses, adult patterns in cortical coding of binaural timing cues were measured. Second, children receiving two cochlear implants in the same surgery maintained normal-like input from both ears, but this did not support significant effects of binaural timing cues in either auditory cortex. Deficits in parieto-occiptal areas further suggested impairment in spatial processing. Results indicate that cochlear implants working independently in each ear do not fully overcome deafness-related binaural processing deficits, even after long-term experience.

Lateralization and Binaural Interaction of Middle-Latency and Late-Brainstem Components of the Auditory Evoked Response

We used magnetoencephalography to examine lateralization and binaural interaction of the middle-latency and late-brainstem components of the auditory evoked response (the MLR and SN10, respectively). Click stimuli were presented either monaurally, or binaurally with left- or right-leading interaural time differences (ITDs). While early MLR components, including the N19 and P30, were larger for monaural stimuli presented contralaterally (by approximately 30 and 36 % in the left and right hemispheres, respectively), later components, including the N40 and P50, were larger ipsilaterally. In contrast, MLRs elicited by binaural clicks with left- or right-leading ITDs did not differ. Depending on filter settings, weak binaural interaction could be observed as early as the P13 but was clearly much larger for later components, beginning at the P30, indicating some degree of binaural linearity up to early stages of cortical processing. The SN10, an obscure late-brainstem component, was observed consistently in individuals and showed linear binaural additivity. The results indicate that while the MLR is lateralized in response to monaural stimuli-and not ITDs-this lateralization reverses from primarily contralateral to primarily ipsilateral as early as 40 ms post stimulus and is never as large as that seen with fMRI.

Neural realignment of spatially separated sound components

Natural auditory scenes often consist of several sound sources overlapping in time, but separated in space. Yet, location is not fully exploited in auditory grouping: spatially separated sounds can get perceptually fused into a single auditory object and this leads to difficulties in the identification and localization of concurrent sounds. Here, the brain mechanisms responsible for grouping across spatial locations were explored in magnetoencephalography (MEG) recordings. The results show that the cortical representation of a vowel spatially separated into two locations reflects the perceived location of the speech sound rather than the physical locations of the individual components. In other words, the auditory scene is neurally rearranged to bring components into spatial alignment when they were deemed to belong to the same object. This renders the original spatial information unavailable at the level of the auditory cortex and may contribute to difficulties in concurrent sound segregation.

Human cortical sensitivity to interaural time difference in high-frequency sounds

Human sound source localization relies on various acoustical cues one of the most important being the interaural time difference (ITD). ITD is best detected in the fine structure of low-frequency sounds but it may also contribute to spatial hearing at higher frequencies if extracted from the sound envelope. The human brain mechanisms related to this envelope ITD cue remain unexplored. Here, we tested the sensitivity of the human auditory cortex to envelope ITD in magnetoencephalography (MEG) recordings. We found two types of sensitivity to envelope ITD. First, the amplitude of the auditory cortical N1m response was smaller for zero envelope ITD than for long envelope ITDs corresponding to the sound being in opposite phase in the two ears. Second, the N1m response amplitude showed ITD-specific adaptation for both fine-structure and for envelope ITD. The auditory cortical sensitivity was weaker for envelope ITD in high-frequency sounds than for fine-structure ITD in low-frequency sounds but occurred within a range of ITDs that are encountered in natural conditions. Finally, the participants were briefly tested for their behavioral ability to detect envelope ITD. Interestingly, we found a correlation between the behavioral performance and the neural sensitivity to envelope ITD. In conclusion, our findings show that the human auditory cortex is sensitive to ITD in the envelope of high-frequency sounds and this sensitivity may have behavioral relevance.

Visual-induced expectations modulate auditory cortical responses

Active sensing has important consequences on multisensory processing (Schroeder et al., 2010). Here, we asked whether in the absence of saccades, the position of the eyes and the timing of transient color changes of visual stimuli could selectively affect the excitability of auditory cortex by predicting the “where” and the “when” of a sound, respectively. Human participants were recorded with magnetoencephalography (MEG) while maintaining the position of their eyes on the left, right, or center of the screen. Participants counted color changes of the fixation cross while neglecting sounds which could be presented to the left, right, or both ears. First, clear alpha power increases were observed in auditory cortices, consistent with participants' attention directed to visual inputs. Second, color changes elicited robust modulations of auditory cortex responses (“when” prediction) seen as ramping activity, early alpha phase-locked responses, and enhanced high-gamma band responses in the contralateral side of sound presentation. Third, no modulations of auditory evoked or oscillatory activity were found to be specific to eye position. Altogether, our results suggest that visual transience can automatically elicit a prediction of “when” a sound will occur by changing the excitability of auditory cortices irrespective of the attended modality, eye position or spatial congruency of auditory and visual events. To the contrary, auditory cortical responses were not significantly affected by eye position suggesting that “where” predictions may require active sensing or saccadic reset to modulate auditory cortex responses, notably in the absence of spatial orientation to sounds.

Change-related auditory P50: a MEG study

Changes in continuous sounds elicit a preattentive component that peaks at around 100ms (Change-N1m) on electroencephalograms or magnetoencephalograms (MEG). Change-N1m is thought to reflect brain activity relating to the automatic detection of changes, which facilitate processes for the execution of appropriate behavior in response to new environmental events. The aim of the present MEG study was to elucidate whether a component relating to auditory changes existed earlier than N1m. Change-related cortical responses were evoked by abrupt sound movement in a train of clicks at 100Hz. Sound movement was created by inserting an interaural time delay (ITD) of 0.15, 0.25, 0.35, and 0.45ms into the right ear. Ten out of 12 participants exhibited clear change-related cortical responses earlier than Change-N1m at around 60ms (Change-P50m). The results of source analysis showed that Change-P50m originated from the superior temporal gyrus of both hemispheres and that its location did not differ significantly from dipoles for the response to the sound onset. The magnitude of Change-P50m increased and the peak latency shortened with an increase in the ITD, similar to those of Change-N1m. These results suggest that change-related cortical activity is present as early as its onset latency at around 50ms.

Psychophysics and neuronal bases of sound localization in humans

Localization of sound sources is a considerable computational challenge for the human brain. Whereas the visual system can process basic spatial information in parallel, the auditory system lacks a straightforward correspondence between external spatial locations and sensory receptive fields. Consequently, the question how different acoustic features supporting spatial hearing are represented in the central nervous system is still open. Functional neuroimaging studies in humans have provided evidence for a posterior auditory "where" pathway that encompasses non-primary auditory cortex areas, including the planum temporale (PT) and posterior superior temporal gyrus (STG), which are strongly activated by horizontal sound direction changes, distance changes, and movement. However, these areas are also activated by a wide variety of other stimulus features, posing a challenge for the interpretation that the underlying areas are purely spatial. This review discusses behavioral and neuroimaging studies on sound localization, and some of the competing models of representation of auditory space in humans. This article is part of a Special Issue entitled Human Auditory Neuroimaging.

Differences in evoked potentials during the active processing of sound location and motion

Difference in the processing of motion and static sounds in the human cortex was studied by electroencephalography with subjects performing an active discrimination task. Sound bursts were presented in the acoustic free-field between 47° to the left and 47° to the right under three different stimulus conditions: (i) static, (ii) leftward motion, and (iii) rightward motion. In an active oddball design, subject was asked to detect target stimuli which were randomly embedded within a stream of frequently occurring non-target events (i.e. 'standards') and rare non-target stimuli (i.e. 'deviants'). The respective acoustic stimuli were presented in blocks with each stimulus type presented in either of three stimulus conditions: as target, as non-target, or as standard. The analysis focussed on the event related potentials evoked by the different stimulus types under the respective standard condition. Same as in previous studies, all three different acoustic stimuli elicited the obligatory P1/N1/P2 complex in the range of 50-200 ms. However, comparisons of ERPs elicited by static stimuli and both kinds of motion stimuli yielded differences as early as ~100 ms after stimulus-onset, i.e. at the level of the exogenous N1 and P2 components. Differences in signal amplitudes were also found in a time window 300-400 ms ('d300-400 ms' component in 'motion-minus-static' difference wave). For motion stimuli, the N1 amplitudes were larger over the hemisphere contralateral to the origin of motion, while for static stimuli N1 amplitudes over both hemispheres were in the same range. Contrary to the N1 component, the ERP in the 'd300-400 ms' time period showed stronger responses over the hemisphere contralateral to motion termination, with the static stimuli again yielding equal bilateral amplitudes. For the P2 component a motion-specific effect with larger signal amplitudes over the left hemisphere was found compared to static stimuli. The presently documented N1 components comply with the results of previous studies on auditory space processing and suggest a contralateral dominance during the process of cortical integration of spatial acoustic information. Additionally, the cortical activity in the 'd300-400 ms' time period indicates, that in addition to the motion origin (as reflected by the N1) also the direction of motion (leftward/ rightward motion) or rather motion termination is cortically encoded. These electrophysiological results are in accordance with the 'snap shot' hypothesis, assuming that auditory motion processing is not based on a genuine motion-sensitive system, but rather on a comparison process of spatial positions of motion origin (onset) and motion termination (offset). Still, specificities of the present P2 component provides evidence for additional motion-specific processes possibly associated with the evaluation of motion-specific attributes, i.e. motion direction and/or velocity which is preponderant in the left hemisphere.

Magnetoencephalography: From SQUIDs to neuroscience. Neuroimage 20th anniversary special edition

Magnetoencephalography (MEG), with its direct view to the cortex through the magnetically transparent skull, has developed from its conception in physics laboratories to a powerful tool of basic and clinical neuroscience. MEG provides millisecond time resolution and allows real-time tracking of brain activation sequences during sensory processing, motor planning and action, cognition, language perception and production, social interaction, and various brain disorders. Current-day neuromagnetometers house hundreds of SQUIDs, superconducting quantum interference devices, to pick up signals generated by concerted action of cortical neurons. Complementary MEG measures of neuronal involvement include evoked responses, modulation of cortical rhythms, properties of the on-going neural activity, and interareal connectivity. Future MEG breakthroughs in understanding brain dynamics are expected through advanced signal analysis and combined use of MEG with hemodynamic imaging (fMRI). Methodological development progresses most efficiently when linked with insightful neuroscientific questions.

MMN or no MMN: no magnitude of deviance effect on the MMN amplitude

Based on results showing that the "deviant-minus-standard" estimate of the mismatch negativity (MMN) amplitude increases with increasing amounts of deviance, it has been suggested that the MMN amplitude reflects the amount of difference between the neural representations of the standard and the deviant sound. However, the deviant-minus-standard waveform also includes an N1 difference. We tested the effects of the magnitude of deviance on MMN while minimizing this N1 confound. We found no significant magnitude of deviance effect on the genuine MMN amplitude. Thus we suggest that the average MMN amplitude does not reflect the difference between neural stimulus representations; rather it may index the percentage of detected deviants, each of which elicits an MMN response of uniform amplitude. These results are compatible with an explanation suggesting that MMN is involved in maintaining a neural representation of the auditory environment.

The influence of externalization and spatial cues on the generation of auditory brainstem responses and middle latency responses

The effect of externalization and spatial cues on the generation of auditory brainstem responses (ABRs) and middle latency responses (MLRs) was investigated in this study. Most previous evoked potential studies used click stimuli with variations of interaural time (ITDs) and interaural level differences (ILDs) which merely led to a lateralization of sound inside the subject's head. In contrast, in the present study potentials were elicited by a virtual acoustics stimulus paradigm with 'natural' spatial cues and compared to responses to a diotic, non-externalized reference stimulus. Spatial sound directions were situated on the horizontal plane (corresponding to variations in ITD, ILD, and spectral cues) or the midsagittal plane (variation of spectral cues only). An optimized chirp was used which had proven to be advantageous over the click since it compensates for basilar membrane dispersion. ABRs and MLRs were recorded from 32 scalp electrodes and both binaural potentials (B) and binaural difference potentials (BD, i.e., the difference between binaural and summed monaural responses) were investigated. The amplitudes of B and BD to spatial stimuli were not higher than those to the diotic reference. ABR amplitudes decreased and latencies increased with increasing laterality of the sound source. A rotating dipole source exhibited characteristic patterns in dependence on the stimulus laterality. For the MLR data, stimulus laterality was reflected in the latency of component N(a). In addition, dipole source analysis revealed a systematic magnitude increase for the dipole contralateral to the azimuthal position of the sound source. For the variation of elevation, the right dipole source showed a stronger activation for stimuli away from the horizontal plane. The results indicate that at the level of the brainstem and primary auditory cortex binaural interaction is mostly affected by interaural cues (ITD, ILD). Potentials evoked by stimuli with natural combinations of ITD, ILD, and spectral cues were not larger than those elicited by diotic chirps.

Cortical response to auditory motion suggests an asymmetry in the reliance on inter-hemispheric connections between the left and right auditory cortices

The aim of the current study was to measure the brain's response to auditory motion using electroencephalography (EEG) to gain insight into the mechanisms by which hemispheric lateralization for auditory spatial processing is established in the human brain. The onset of left- or rightward motion in an otherwise continuous sound was found to elicit a large response, which appeared to arise from higher-level nonprimary auditory areas. This motion onset response was strongly lateralized to the hemisphere contralateral to the direction of motion. The response latencies suggest that the ipsilateral response to the leftward motion was produced by indirect callosal projections from the opposite hemisphere, whereas the ipsilateral response to the rightward motion seemed to receive contributions from direct thalamocortical projections. These results suggest an asymmetry in the reliance on inter-hemispheric projections between the left and right auditory cortices for auditory spatial processing.

Auditory evoked magnetic fields in relation to interaural time delay and interaural correlation

The detection of interaural time differences (ITD) for sound localization depends on the similarity between the left and right ear signals, namely interaural correlation (IAC). Human localization performance deteriorates with decreasing IACs. In order to examine activity related to localization performance in the human cortex, auditory evoked magnetic fields to the ITD of bandpass noises with different IACs were analyzed. When the IAC was 0.95, the N1m amplitudes, i.e., the estimated equivalent current dipole moments, increased with increasing ITD. However the effect of ITD on the N1m amplitudes was not significant when the IAC was 0.5. When the ITD was 0.7 ms, the N1m amplitudes decreased with decreasing IACs. There were no systematic changes in the source location of N1m in the auditory cortex related to changes in ITD or IAC. The results suggest that localization performance is reflected in N1m amplitudes.

Interaural delay-dependent changes in the binaural difference potential of the human auditory brain stem response

Binaural difference potentials (BDs) are thought to be generated by neural units in the brain stem responding specifically to binaural stimulation. They are computed by subtracting the sum of monaural responses from the binaural response, BD = B - (L + R). BDs in dependency on the interaural time difference (ITD) have been measured and compared to the Jeffress model in a number of studies with conflicting results. The classical Jeffress model assuming binaural coincidence detector cells innervated by bilateral excitatory cells via two delay lines predicts a BD latency increase of ITD/2. A modification of the model using only a single delay line as found in birds yields a BD latency increase of ITD. The objective of this study is to measure BDs with a high signal-to-noise ratio for a large range of ITDs and to compare the data with the predictions of some models in the literature including that of Jeffress. Chirp evoked BDs were recorded for 17 ITDs in the range from 0 to 2 ms at a level of 40 dB nHL for four channels (A1, A2, PO9, PO10) from 11 normal hearing subjects. For each binaural condition 10,000 epochs were collected while 40,000 epochs were recorded for each of the two monaural conditions. Significant BD components are observed for ITDs up to 2 ms. The peak-to-peak amplitude of the first components of the BD, DP1-DN1, is monotonically decreasing with ITD. This is in contrast with click studies which reported a constant BD-amplitude for ITDs up to 1 ms. The latency of the BD-component DN1 is monotonically, but nonlinearly increasing with ITD. In the current study, DN1 latency is found to increase faster than ITD/2 but slower than ITD incompatible with either variant of the Jeffress model. To describe BD waveforms, the computational model proposed by Ungan et al. [Hearing Research 106, 66-82, 1997] using ipsilateral excitatory and contralateral inhibitory inputs to the binaural cells was implemented with only four parameters and successfully fitted to the BD data. Despite its simplicity the model predicts features which can be physiologically tested: the inhibitory input must arrive slightly before the excitatory input, and the duration of the inhibition must be considerably longer than the standard deviations of excitatory and inhibitory arrival times to the binaural cells. With these characteristics, the model can accurately describe BD amplitude and latency as a function of the ITD.

Effects of the frequency of interaural time difference in the human brain

The two cues to the horizontal location sound sources are interaural time differences and interaural level differences. For low-frequency tones, interaural time differences provide effective and unambiguous information. For higher frequency sounds, however, interaural time differences provide ambiguous cues. In order to evaluate the effect of frequency of interaural time differences in the human auditory cortex, the auditory evoked fields to different interaural time differences of pure tone were examined. The results showed that the N1m magnitudes varied with the interaural time differences when the frequency of the pure tone was 800 Hz. The N1m magnitudes, however, did not vary with the interaural time differences when the frequency of the pure tone was 1600 Hz. These results indicate that localization performance might be reflected in N1m magnitudes.

Neural response correlates of detection of monaurally and binaurally created pitches in humans

Recent magnetoencephalography (MEG) and functional magnetic resonance imaging studies of human auditory cortex are pointing to brain areas on lateral Heschl's gyrus as the 'pitch-processing center'. Here we describe results of a combined MEG-psychophysical study designed to investigate the timing of the formation of the percept of pitch and the generality of the hypothesized 'pitch-center'. We compared the cortical and behavioral responses to Huggins pitch (HP), a stimulus requiring binaural processing to elicit a pitch percept, with responses to tones embedded in noise (TN)-perceptually similar but physically very different signals. The stimuli were crafted to separate the electrophysiological responses to onset of the pitch percept from the onset of the initial stimulus. Our results demonstrate that responses to monaural pitch stimuli are affected by cross-correlational processes in the binaural pathway. Additionally, we show that MEG illuminates processes not simply observable in behavior. Crucially, the MEG data show that, although physically disparate, both HP and TN are mapped onto similar representations by 150 ms post-onset, and provide critical new evidence that the 'pitch onset response' reflects central pitch mechanisms, in agreement with models postulating a single, central pitch extractor.

Auditory evoked fields to variations of interaural time delay

Auditory motion can be simulated by presenting binaural sounds with time-varying interaural time delays. Human cortical responses to the rate of auditory motion were studied by recording auditory evoked magnetic fields with a 122-channel whole-head magnetometer. Auditory motion from central to right and then to central was produced by varying interaural time differences between ears. The results showed that the N1m latencies and amplitudes were not affected by the fluctuation of interaural time delay; however, the peak amplitude of P2m significantly increased as a function of fluctuation of the interaural time delay.

Spatial processing in human auditory cortex: the effects of 3D, ITD, and ILD stimulation techniques

Here, the perception of auditory spatial information as indexed by behavioral measures is linked to brain dynamics as reflected by the N1m response recorded with whole-head magnetoencephalography (MEG). Broadband noise stimuli with realistic spatial cues corresponding to eight direction angles in the horizontal plane were constructed via custom-made, individualized binaural recordings (BAR) and generic head-related transfer functions (HRTF). For comparison purposes, stimuli with impoverished acoustical cues were created via interaural time and level differences (ITDs and ILDs) and their combinations. MEG recordings in ten subjects revealed that the amplitude and the latency of the N1m exhibits directional tuning to sound location, with the amplitude of the right-hemispheric N1m being particularly sensitive to the amount of spatial cues in the stimuli. The BAR, HRTF, and combined ITD + ILD stimuli resulted both in a larger dynamic range and in a more systematic distribution of the N1m amplitude across stimulus angle than did the ITD or ILD stimuli alone. Further, the right-hemispheric source loci of the N1m responses for the BAR and HRTF stimuli were anterior to those for the ITD and ILD stimuli. In behavioral tests, we measured the ability of the subjects to localize BAR and HRTF stimuli in terms of azimuthal error and front-back confusions. We found that behavioral performance correlated positively with the amplitude of the N1m. Thus, the activity taking place already in the auditory cortex predicts behavioral sound detection of spatial stimuli, and the amount of spatial cues embedded in the signal are reflected in the activity of this brain area.

Human cortical representation of virtual auditory space: differences between sound azimuth and elevation

Sounds convolved with individual head-related transfer functions and presented through headphones can give very natural percepts of the three-dimensional auditory space. We recorded whole-scalp neuromagnetic responses to such stimuli to compare reactivity of the human auditory cortex to sound azimuth and elevation. The results suggest that the human auditory cortex analyses sound azimuth, based on both binaural and monaural localization cues, mainly in the hemisphere contralateral to the sound, whereas elevation in the anterior space and in the lateral auditory space in general, both strongly relying on monaural spectral cues, are analyzed in more detail in the right auditory cortex. The binaural interaural time and interaural intensity difference cues were processed in the auditory cortex around 100-150 ms and the monaural spectral cues later around 200-250 ms.

Cortical processing of speech sounds and their analogues in a spatial auditory environment

We used magnetoencephalographic (MEG) measurements to study how speech sounds presented in a realistic spatial sound environment are processed in human cortex. A spatial sound environment was created by utilizing head-related transfer functions (HRTFs), and using a vowel, a pseudo-vowel, and a wide-band noise burst as stimuli. The behaviour of the most prominent auditory response, the cortically generated N1m, was investigated above the left and right hemisphere. We found that the N1m responses elicited by the vowel and by the pseudo-vowel were much larger in amplitude than those evoked by the noise burst. Corroborating previous observations, we also found that cortical activity reflecting the processing of spatial sound was more pronounced in the right than in the left hemisphere for all of the stimulus types and that both hemispheres exhibited contralateral tuning to sound direction.

Right-hemisphere dominance for the processing of sound-source lateralization

Cortical processing of change in direction of a perceived sound source was investigated in 12 human subjects using whole-head magnetoencephalography. The German word "da" was presented either with or without 0.7 msec interaural time delays to create the impression of right- or left-lateralized or midline sources, respectively. Midline stimuli served as standards, and lateralized stimuli served as deviants in a mismatch paradigm. Two symmetrically linked dipoles fitted to the mismatch fields showed stronger moments in the hemisphere contralateral to the side of the deviant. The right dipole displayed equal latencies to both left and right deviants, whereas left dipole latencies were longer for ipsilateral than contralateral deviants. Frequency analysis between 20-70 Hz and statistical probability mapping revealed increased induced gamma-band activity at 53+/-2.5 Hz to both types of deviants. Right deviants elicited spectral amplitude enhancements in this frequency range, peaking at latencies of 160 and 240 msec. These effects were localized bilaterally over the angular gyri and posterior temporal regions. Coherence analysis suggested the existence of two separate interhemispheric networks. For left-lateralized deviants, both spectral amplitude enhancements at 110 and 220 msec and coherence increases were restricted to the right hemisphere. In conclusion, both mismatch dipole latencies at the supratemporal plane and gamma-band activity in posterior parietotemporal areas suggested a right hemisphere engagement in the processing of bidirectional sound-source shifts. In contrast, left-hemisphere regions responded predominantly to contralateral events. These findings may help to elucidate phenomena such as unilateral auditory neglect.

On the detection of auditory deviations: a pre-attentive activation model

The conscious perception of infrequent deviant sounds occurring in a series of frequent standard sounds may in part be based on the output of an obligatorily operating deviance detection system. This system encodes invariances inherent to the recent auditory stimulation into short-lived representations of auditory sensory memory and compares each actual input with these representations. The underlying processes may be regarded as preattentive in the sense that they do not rely on the explicit intention of a person to detect deviants and that they may be active even in the absence of attention (although they may be prone to attentional modulations). The output of this feature-specific preattentive deviance detection system fuses into an integrated mismatch signal that in turn may activate subsequent processes that result in the triggering of a motor response.

Temporal characteristics of auditory sensory memory: neuromagnetic evidence

We investigated the temporal dependencies of N100 m, the most prominent deflection of the auditory evoked response, using whole-head neuromagnetic recordings. Stimuli were presented singly or in pairs (tones in the pair were separated by 210 ms) at interstimulus intervals (ISIs) of 0.6-8.1 s. N100 m to single stimuli and to the first tone of the pair had similar temporal recovery functions, plateauing at ISIs of 6 s. N100 m to the second tone in the pair, which was smaller than that to the first except with short ISIs, plateaued with ISIs of about 4 s. Source analysis revealed that the N100 m could be decomposed into two sources separated by about 1 cm on the supratemporal plane. The recovery function of the posterior source was not affected by stimulus presentation, whereas that of the anterior source was. Activity in the anterior area appears to reflect the effects of temporal integration. We relate these results to auditory sensory memory.

Human auditory cortex is activated by omissions of auditory stimuli

Cortical signals associated with infrequent tone omissions were recorded from 9 healthy adults with a whole-head 122 channel neuromagnetometer. The stimulus sequence consisted of monaural (left or right) 50-ms 1-kHz tones repeated every 0.2 or 0.5 s, with 7% of the tones randomly omitted. Tones elicited typical responses in the supratemporal auditory cortices. Omissions evoked strong responses over temporal and frontal areas, independently of the side of stimulation, with peak amplitudes at 145-195 ms. Response amplitudes were 60% weaker when the subject was not attending to the stimuli. Omission responses originated in supratemporal auditory cortices bilaterally, indicating that auditory cortex plays an important role in the brain's modelling of temporal characteristics of the auditory environment. Additional activity was observed in the posterolateral frontal cortex and in the superior temporal sulcus, more often in the right than in the left hemisphere.

Information processing in the human brain: magnetoencephalographic approach

Rapid progress in effective methods to image brain functions has revolutionized neuroscience. It is now possible to study noninvasively in humans neural processes that were previously only accessible in experimental animals and in brain-injured patients. In this endeavor, positron emission tomography has been the leader, but the superconducting quantum interference device-based magnetoencephalography (MEG) is gaining a firm role, too. With the advent of instruments covering the whole scalp, MEG, typically with 5-mm spatial and 1-ms temporal resolution, allows neuroscientists to track cortical functions accurately in time and space. We present five representative examples of recent MEG studies in our laboratory that demonstrate the usefulness of whole-head magnetoencephalography in investigations of spatiotemporal dynamics of cortical signal processing.

Interaural time and level differences: integrated or separated processing?

The processing of internal differences in time (IDT) and sound pressure level (IDL) was studied by using the mismatch negativity auditory evoked potential (MMN), which is a probe of pre-attentive auditory sensory memory. In a passive oddball experiment, subjects were reading in a book while they were presented with a standard stimulus (P = 0.88) having no IDTs or IDLs and three different deviant stimuli revealing an IDT, IDL, or both IDT and IDL. The different deviants elicited MMNs of comparable latencies indicating that memory representations of the IDTs and IDLs have been established. The MMN amplitudes to the IDT-IDL deviant were larger than those to changes in either IDT or IDL only. Moreover, the time-courses, amplitudes, and topographies of the MMNs to the IDT-IDL deviants were very similar to the sum of the MMNs elicited by the IDT and IDL deviants. These findings suggest that the representations of the binaural location cues were (at least partly) processed in parallel. It is argued that separate azimuth representations exist for IDT and IDL at a cortical level.

Effects of lateralized cues on the processing of lateralized auditory stimuli

We examined the influence of auditory spatial cues on the processing of subsequently presented lateralized auditory target stimuli. In three experiments, effects of the position of the cue on event-related potentials (ERPs) and reaction times (RTs) to the target, which could occur at the same or at a different position to the cue, were examined. In Experiments I and II, stimulus-pairs were delivered via loudspeakers 13 degrees to the right or left of fixation. The cue was task-irrelevant in the sense that it never required a response, whereas the target was task-relevant. In Experiment I, the cue was not informative about the position of the succeeding target, that is, the target was presented at the same or at a different position to the cue with equal probability. In Experiment II, the cue predicted a target with high validity, occurring at the position opposite to the cue. In both experiments, the ERPs to targets presented at the same position as the preceding cue were negatively modulated compared with the ERPs to targets presented at a different position. This negative difference (Nd) between the ERPs to same and different position stimuli occurring between 200-300 ms relative to target onset was also obtained in situations where no overt behavior was required. Effects of cue position on RTs to the target were only observed in Experiment II, being shorter for validly predicted targets. These Nd effects either reflect refractoriness of location-specific exogeneous ERP components or involuntary attentional selection. In order to decide between these alternatives, a third experiment was performed, in which stimulus-pairs were presented via headphones to the left and right ear in attend and ignore condition. An Nd effect was obtained in the attend condition only. The absence of Nd effects in the ignore condition suggests that the Nd obtained in attend condition cannot completely be explained by refractoriness of exogeneous ERP components and thus reflects attentional processes.

Processing of auditory deviants with changes in one versus two stimulus dimensions

Auditory event-related potentials (ERPs) in response to 50-ms tones were recorded from the human scalp. A standard stimulus (p = .88) and three different deviants were randomly presented via earphones. There were two one-dimensional deviants (one frequency and one location deviant) and one two-dimensional deviant, with changes in both frequency and location. In one condition, subjects read a book and ignored the auditory stimuli, whereas in another condition they tried to discriminate deviants from standards. In the ignore condition, the two-dimensional deviant elicited an enhanced mismatch negativity (MMN) as compared with the MMNs elicited by the one-dimensional deviants. The temporal and the topographic distributions of the two-dimensional MMNs could be modeled by adding the one-dimensional MMNs. This activity of the MMNs probably results from the independent activity of separate neural populations generating the frequency and the location MMN. In the attend condition, the deviance-related ERP effects were not additive in the N2b and P3 range, implicating that the neural processes involved in the conscious detection of changes in location and frequency were not independent.

Human auditory cortical mechanisms of sound lateralisation: III. Monaural and binaural shift responses

Neuromagnetic responses were recorded over the whole head with a 122-channel gradiometer. A pair of 150-ms 1-kHz tones separated by an interval of 150 ms was presented to one ear every 2 s. The other ear received either no input, an identical pair simultaneous to the first, an identical pair alternating with the first or a continuous 600-ms tone. The 'monaural shift' condition in which stimuli alternated between ears produced a clear perception of changing lateralisation, but the evoked response could be explained as merely the sum of simple monaural onset and offset responses; thus we found no evidence for a separate response to interaural intensity difference in this condition. The 'binaural shift' condition, in which intensity changed in one ear while the other received a continuous tone, evoked a transient response (N130m) at a latency of about 130 ms. N130m was larger over the hemisphere contralateral to the direction of shift, and larger than the corresponding monaural response, whether to an onset or an offset. We concluded that N130m also was not a separate directional response, but was analogous to a simple monaural response, the prolonged latency being due to masking and the enhanced amplitude to facilitation by the sustained response to the continuous tone.

Effect of interaural time differences on middle-latency and late auditory evoked magnetic fields

To determine if interaural time differences (ITDs) in binaural stimuli affect the middle-latency auditory evoked fields (AEFs) in the same manner as they affect the N100m deflection, neuromagnetic responses were recorded over the whole head using a 122-channel SQUID magnetometer. Binaural stimuli were lateralized to three positions, left, midline, and right, on the basis of ITDs. The N100m was significantly larger to stimuli with contralaterally-leading ITDs than to stimuli with no, or with ipsilaterally-leading ITDs. Neither the P30m nor the P50m deflections of the middle-latency response were significantly affected by ITD, although the P30m showed a tendency, similar to but smaller than that of N100m, to be larger to stimuli with contralaterally-leading ITDs. In some subjects, the source location of the P50m was anterior and inferior to the sources of the P30m and N100m, which are generated in the superior surface of the temporal lobe. Sound-related muscular artifacts were seen in the posterior recording channels of one subject, and the contribution of this activity to the signals over the temporal area was determined.

Human auditory cortical mechanisms of sound lateralization: I. Interaural time differences within sound

Neuromagnetic responses to 600-ms binaural click trains, presented once every 1.1 s, were recorded with a 24-channel gradiometer from 6 healthy humans. During the first 300 ms, the left-ear stimulus led the right by 0.7 ms and the sound was lateralized to the left ear. At 300 ms, the interaural time difference (ITD) changed and the lateralization moved to one of 5 different locations between the ears. An N100m response peaked about 110 ms after the sound onset and an N130mc response (c to stress a response to the change) about 135 ms after the ITD change. The source locations of N100m and N130mc agreed with activity in the supratemporal auditory cortex; this was confirmed in one subject by superimposing MEG results on MR images. The sources of N100m and N130mc did not differ statistically significantly from each other, nor were there differences in N130mc sources to various lateralization changes. N130mc grew larger when the ITD change increased, in parallel with the increase in the change of the perceived location. We suggest that N130mc is analogous to N100m, but is delayed due to postmasking induced by the early part of the sound.