Functional imaging of auditory scene analysis

Our auditory system is constantly faced with the task of decomposing the complex mixture of sound arriving at the ears into perceptually independent streams constituting accurate representations of individual sound sources. This decomposition, termed auditory scene analysis, is critical for both survival and communication, and is thought to underlie both speech and music perception. The neural underpinnings of auditory scene analysis have been studied utilizing invasive experiments with animal models as well as non-invasive (MEG, EEG, and fMRI) and invasive (intracranial EEG) studies conducted with human listeners. The present article reviews human neurophysiological research investigating the neural basis of auditory scene analysis, with emphasis on two classical paradigms termed streaming and informational masking. Other paradigms - such as the continuity illusion, mistuned harmonics, and multi-speaker environments - are briefly addressed thereafter. We conclude by discussing the emerging evidence for the role of auditory cortex in remapping incoming acoustic signals into a perceptual representation of auditory streams, which are then available for selective attention and further conscious processing. This article is part of a Special Issue entitled Human Auditory Neuroimaging.

Time is of the essence for auditory scene analysis

Using computational models and stimuli that resemble natural acoustic signals, auditory scientists explore how we segregate competing streams of sound.

Role of pattern, regularity, and silent intervals in auditory stream segregation based on inter-aural time differences

Tone triplets separated by a pause (ABA_) are a popular tone-repetition pattern to study auditory stream segregation. Such triplets produce a galloping rhythm when integrated, but isochronous rhythms when segregated. Other patterns lacking a pause may produce less-prominent rhythmic differences but stronger streaming. Here, we evaluated whether this difference is readily explained by the presence of the pause and potentially associated with the reduction of adaptation, or whether there is contribution of tone pattern per se. Sequences with repetitive ABA_ and ABAA patterns were presented in magnetoencephalography. A and B tones were separated by differences in inter-aural time differences (ΔITD). Results showed that the stronger streaming of ABAA was associated with a more prominent release from the adaptation of the P(1)m in auditory cortex. We further compared behavioral streaming responses for patterns with and without pauses, and varied the position of the pause and pattern regularity. Results showed a major effect of the pauses' presence, but no prominent effects of tone pattern or pattern regularity. These results make a case for the existence of an early, primitive streaming mechanism that does not require an analysis of the tone pattern at later stages suggested by predictive-coding models of auditory streaming. The results are better explained by the simpler population-separation model and stress the previously observed role of neural adaptation for streaming perception.

Sustained BOLD and theta activity in auditory cortex are related to slow stimulus fluctuations rather than to pitch

Human functional MRI (fMRI) and magnetoencephalography (MEG) studies indicate a pitch-specific area in lateral Heschl's gyrus. Single-cell recordings in monkey suggest that sustained-firing, pitch-specific neurons are located lateral to primary auditory cortex. We reevaluated whether pitch strength contrasts reveal sustained pitch-specific responses in human auditory cortex. Sustained BOLD activity in auditory cortex was found for iterated rippled noise (vs. noise or silence) but not for regular click trains (vs. jittered click trains or silence). In contrast, iterated rippled noise and click trains produced similar pitch responses in MEG. Subsequently performed time-frequency analysis of the MEG data suggested that the dissociation of cortical BOLD activity between iterated rippled noise and click trains is related to theta band activity. It appears that both sustained BOLD and theta activity are associated with slow non-pitch-specific stimulus fluctuations. BOLD activity in the inferior colliculus was sustained for both stimulus types and varied neither with pitch strength nor with the presence of slow stimulus fluctuations. These results suggest that BOLD activity in auditory cortex is much more sensitive to slow stimulus fluctuations than to constant pitch, compromising the accessibility of the latter. In contrast, pitch-related activity in MEG can easily be separated from theta band activity related to slow stimulus fluctuations.

Transient bold activity locked to perceptual reversals of auditory streaming in human auditory cortex and inferior colliculus

Our auditory system separates and tracks temporally interleaved sound sources by organizing them into distinct auditory streams. This streaming phenomenon is partly determined by physical stimulus properties but additionally depends on the internal state of the listener. As a consequence, streaming perception is often bistable and reversals between one- and two-stream percepts may occur spontaneously or be induced by a change of the stimulus. Here, we used functional MRI to investigate perceptual reversals in streaming based on interaural time differences (ITD) that produce a lateralized stimulus perception. Listeners were continuously presented with two interleaved streams, which slowly moved apart and together again. This paradigm produced longer intervals between reversals than stationary bistable stimuli but preserved temporal independence between perceptual reversals and physical stimulus transitions. Results showed prominent transient activity synchronized with the perceptual reversals in and around the auditory cortex. Sustained activity in the auditory cortex was observed during intervals where the ΔITD could potentially produce streaming, similar to previous studies. A localizer-based analysis additionally revealed transient activity time locked to perceptual reversals in the inferior colliculus. These data suggest that neural activity associated with streaming reversals is not limited to the thalamo-cortical system but involves early binaural processing in the auditory midbrain, already.

Functional dissociation of transient and sustained fMRI BOLD components in human auditory cortex revealed with a streaming paradigm based on interaural time differences

A number of physiological studies suggest that feature-selective adaptation is relevant to the pre-processing for auditory streaming, the perceptual separation of overlapping sound sources. Most of these studies are focused on spectral differences between streams, which are considered most important for streaming. However, spatial cues also support streaming, alone or in combination with spectral cues, but physiological studies of spatial cues for streaming remain scarce. Here, we investigate whether the tuning of selective adaptation for interaural time differences (ITD) coincides with the range where streaming perception is observed. FMRI activation that has been shown to adapt depending on the repetition rate was studied with a streaming paradigm where two tones were differently lateralized by ITD. Listeners were presented with five different ΔITD conditions (62.5, 125, 187.5, 343.75, or 687.5 μs) out of an active baseline with no ΔITD during fMRI. The results showed reduced adaptation for conditions with ΔITD ≥ 125 μs, reflected by enhanced sustained BOLD activity. The percentage of streaming perception for these stimuli increased from approximately 20% for ΔITD = 62.5 μs to > 60% for ΔITD = 125 μs. No further sustained BOLD enhancement was observed when the ΔITD was increased beyond ΔITD = 125 μs, whereas the streaming probability continued to increase up to 90% for ΔITD = 687.5 μs. Conversely, the transient BOLD response, at the transition from baseline to ΔITD blocks, increased most prominently as ΔITD was increased from 187.5 to 343.75 μs. These results demonstrate a clear dissociation of transient and sustained components of the BOLD activity in auditory cortex.

Activity associated with stream segregation in human auditory cortex is similar for spatial and pitch cues

Streaming is a perceptual mechanism by which the brain segregates information from multiple sound sources in our environment and assigns them to distinct auditory streams. Examples for streaming cues are differences in frequency spectrum, pitch, or space, and potential neural correlates for streaming based on spectral and pitch cues have been identified in the auditory cortex. Here, magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) were used to evaluate if response enhancement in auditory cortex associated with streaming represents a general pattern that is independent of the stimulus cue. Interaural time differences (ITDs) were used as a spatial streaming cue and were compared with streaming based on fundamental frequency (f(0)) differences. The MEG results showed enhancement of the P(1)m after 60-90 ms that was similar during streaming based on ITD and pitch. Sustained fMRI activity was enhanced at identical sites in Heschl's gyrus and planum temporale for both cues; no topographical specificity for space or pitch was found for the streaming-associated enhancement. These results support the hypothesis of an early convergence of the neural representation for auditory streams that is independent of the acoustic cue that the streaming is based on.

BOLD responses in human auditory cortex are more closely related to transient MEG responses than to sustained ones

Blood oxygen level dependent-functional magnetic resonance imaging (BOLD-fMRI) and magnetoencephalographic (MEG) signals are both coupled to postsynaptic potentials, although their relationship is incompletely understood. Here, the wide range of BOLD-fMRI and MEG responses produced by auditory cortex was exploited to better understand the BOLD-fMRI/MEG relationship. Measurements of BOLD and MEG responses were made in the same subjects using the same stimuli for both modalities. The stimuli, 24-s sequences of click trains, had duty cycles of 2.5, 25, 72, and 100%. For the 2.5% sequence, the BOLD response was elevated throughout the sequence, whereas for 100%, it peaked after sequence onset and offset and showed a diminished elevation in between. On the finer timescale of MEG, responses at 2.5% consisted of a complex of transients, including N(1)m, to each click train of the sequence, whereas for 100% the only transients occurred at sequence onset and offset between which there was a sustained elevation in the MEG signal (a sustained field). A model that separately estimated the contributions of transient and sustained MEG signals to the BOLD response best fit BOLD measurements when the transient contribution was weighted 8- to 10-fold more than the sustained one. The findings suggest that BOLD responses in the auditory cortex are tightly coupled to the neural activity underlying transient, not sustained, MEG signals.

A sustained deviance response evoked by the auditory oddball paradigm

OBJECTIVE

Previous studies have suggested that the MMN(m) is related to selective adaptation of the N(1m). Since selective adaptation has also been reported for the sustained field, we hypothesized a second deviance response in addition to the MMN(m). The present study evaluated the existence of this wave.

METHODS

Magnetoencephalography was used to record deviance responses for pure tones of 1000 and 1050Hz. Tone duration was 50, 150, or 600ms in separate sets. Our hypothesis was that a sustained deviance response would increase with tone duration.

RESULTS

The data revealed a sustained deviance response with a similar source configuration as the main MMN(m), but a distinct time course. The sustained deviance response increased with the tone duration, but less than the standard sustained field. Moreover, the sustained deviance response was already present for short (50ms) tones.

CONCLUSIONS

The MMN(m) is followed by a sustained deviance response in the oddball paradigm. While some characteristics of the response coincide with the sustained field, its growth with tone duration differs. The response could possibly be related to automatic orienting of attention, but further studies are required to explore its functional role.

SIGNIFICANCE

The sustained deviance response is a separate component--distinct from the MMN(m) and P3--that needs to be considered in the evaluation of data obtained with the auditory oddball paradigm.

Rate perception and the auditory 40-Hz steady-state fields evoked by two-tone sequences

The rate perception of tone sequences reflects the physical repetition rate for identical sound elements. More complex sequences are perceived at the physical rate or at lower rates, depending on perceptual organization. Here, we used magnetoencephalography and psychophysical studies to evaluate the possible relationship between rate perception of such rapid, 40-Hz tone trains and the 40-Hz steady-state response (SSR) in human primary auditory cortex. In Experiment 1, the 40-Hz SSR evoked by monotone sequences of 1000 and 600 Hz were compared to the response evoked by alternating-tone sequences of the same frequencies. The results showed that the 40-Hz SSR for the alternating-tones was attenuated compared to the monotones. In Experiment 2, frequency differences across a range of 25-300 Hz were studied. Compared to a 1000-Hz monotone sequence, the 40-Hz SSR was reduced. Amplitude reduction was most prominent for frequency differences of 200 Hz and more, which were generally perceived with half-the-physical rate. We discuss possible physiological mechanisms of this finding and its relationship to perception.

Neural correlates of auditory perceptual awareness under informational masking

Our ability to detect target sounds in complex acoustic backgrounds is often limited not by the ear's resolution, but by the brain's information-processing capacity. The neural mechanisms and loci of this "informational masking" are unknown. We combined magnetoencephalography with simultaneous behavioral measures in humans to investigate neural correlates of informational masking and auditory perceptual awareness in the auditory cortex. Cortical responses were sorted according to whether or not target sounds were detected by the listener in a complex, randomly varying multi-tone background known to produce informational masking. Detected target sounds elicited a prominent, long-latency response (50-250 ms), whereas undetected targets did not. In contrast, both detected and undetected targets produced equally robust auditory middle-latency, steady-state responses, presumably from the primary auditory cortex. These findings indicate that neural correlates of auditory awareness in informational masking emerge between early and late stages of processing within the auditory cortex.

The pulse-train auditory aftereffect and the perception of rapid amplitude modulations

Prolonged listening to a pulse train with repetition rates around 100 Hz induces a striking aftereffect, whereby subsequently presented sounds are heard with an unusually "metallic" timbre [Rosenblith et al., Science 106, 333-335 (1947)]. The mechanisms responsible for this auditory aftereffect are currently unknown. Whether the aftereffect is related to an alteration of the perception of temporal envelope fluctuations was evaluated. Detection thresholds for sinusoidal amplitude modulation (AM) imposed onto noise-burst carriers were measured for different AM frequencies (50-500 Hz), following the continuous presentation of a periodic pulse train, a temporally jittered pulse train, or an unmodulated noise. AM detection thresholds for AM frequencies of 100 Hz and above were significantly elevated compared to thresholds in quiet, following the presentation of the pulse-train inducers, and both induced a subjective auditory aftereffect. Unmodulated noise, which produced no audible aftereffect, left AM detection thresholds unchanged. Additional experiments revealed that, like the Rosenblith et al. aftereffect, the effect on AM thresholds does not transfer across ears, is not eliminated by protracted training, and can last several tens of seconds. The results suggest that the Rosenblith et al. aftereffect is related to a temporary alteration in the perception of fast temporal envelope fluctuations in sounds.

Representation of auditory-filter phase characteristics in the cortex of human listeners

Harmonic tone complexes with component phases, adjusted using a variant of a method proposed by Schroeder, can produce pure-tone masked thresholds differing by >20 dB. This phenomenon has been qualitatively explained by the phase characteristics of the auditory filters on the basilar membrane, which differently affect the flat envelopes of the Schroeder-phase maskers. We examined the influence of auditory-filter phase characteristics on the neural representation in the auditory cortex by investigating cortical auditory evoked fields (AEFs). We found that the P1m component exhibited larger amplitudes when a long-duration tone was presented in a repeating linearly downward sweeping (Schroeder positive, or m(+)) masker than in a repeating linearly upward sweeping (Schroeder negative, or m(-)) masker. We also examined the neural representation of short-duration tone pulses presented at different temporal positions within a single period of three maskers differing in their component phases (m(+), m(-), and sine phase m(0)). The P1m amplitude varied with the position of the tone pulse in the masker and depended strongly on the masker waveform. The neuromagnetic results in all cases were consistent with the perceptual data obtained with the same stimuli and with results from simulations of neural activity at the output of cochlear preprocessing. These findings demonstrate that phase effects in peripheral auditory processing are accurately reflected up to the level of the auditory cortex.