Proprioceptive cues modulate further processing of spatially congruent auditory information. a high-density EEG study

Multisensory integration and interaction occur when bimodal stimuli are presented as either spatially congruent or incongruent, but temporally coincident. We investigated whether proprioceptive cues interact with auditory attention to one of two sound sources in free-field. The participant's task was to attend to either the left or right speaker and to respond to occasional increased-bandwidth targets via a footswitch. We recorded high-density EEG in three experimental conditions: the participants either held the speakers in their hands (Hold), reached out close to them (Reach), or had their hands in their lap (Lap). In the last two conditions, the auditory event-related potentials (ERPs) revealed a prominent negativity around 200 ms post-stimulus (N2 wave) over fronto-central areas, which is a reliable index of further processing of spatial stimulus features in free-field. The N2 wave was markedly attenuated in the 'Hold' condition, which suggests that proprioceptive cues apparently solidify spatial information computed by the auditory system, in so doing alleviating the need for further processing of spatial coordinates solely based on auditory information.

Effects of spatial congruity on audio-visual multimodal integration

Spatial constraints on multisensory integration of auditory (A) and visual (V) stimuli were investigated in humans using behavioral and electrophysiological measures. The aim was to find out whether cross-modal interactions between A and V stimuli depend on their spatial congruity, as has been found for multisensory neurons in animal studies (Stein & Meredith, 1993). Randomized sequences of unimodal (A or V) and simultaneous bimodal (AV) stimuli were presented to right- or left-field locations while subjects made speeded responses to infrequent targets of greater intensity that occurred in either or both modalities. Behavioral responses to the bimodal stimuli were faster and more accurate than to the unimodal stimuli for both same-location and different-location AV pairings. The neural basis of this cross-modal facilitation was studied by comparing event-related potentials (ERPs) to the bimodal AV stimuli with the summed ERPs to the unimodal A and V stimuli. These comparisons revealed neural interactions localized to the ventral occipito-temporal cortex (at 190 msec) and to the superior temporal cortical areas (at 260 msec) for both same- and different-location AV pairings. In contrast, ERP interactions that differed according to spatial congruity included a phase and amplitude modulation of visual-evoked activity localized to the ventral occipito-temporal cortex at 100-400 msec and an amplitude modulation of activity localized to the superior temporal region at 260-280 msec. These results demonstrate overlapping but distinctive patterns of multisensory integration for spatially congruent and incongruent AV stimuli.

Variability in AC amplifier distortions: estimation and correction

AC amplifiers can introduce significant distortions into the low frequency and DC components of recorded electrophysiological data such as event-related potentials (ERPs). Methods for correcting such distortions (i.e., estimating the waveform of the original data) after the data have been amplified and recorded rely on an accurate estimate of the amplifier's time constant (TC). We show that the filter characteristics of AC amplifiers in at least some commercially available ERP recording instruments may vary considerably across individual channels, even when each houses an identical AC amplifier circuit. Clearly, distortion correction methods must take this variability into account. We propose an empirical means of estimating the correct TC value. This approach yields more accurate correction than those based on TCs calculated analytically.

An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings

Crossmodal integration was studied in humans by presenting random sequences of auditory (brief noise bursts), visual (flashes), and audiovisual (simultaneous noise bursts and flashes) stimuli from a central location at irregular intervals between 600 and 800 ms. The subjects' task was to press a button to infrequent and unpredictable (P=0.15) target stimuli that could be either a more intense noise burst, a brighter flash, or a combination of the two. In accordance with previous studies, behavioral data showed that bimodal target stimuli were responded to much faster and were identified more accurately than the unimodal target stimuli. The neural basis of this crossmodal interaction was investigated by subtracting the ERPs to the auditory (A) and the visual (V) stimuli alone from the ERP to the combined audiovisual (AV) stimuli (i.e. interaction=AV-(A+V)). Using this approach, we replicated previous reports of both early (at around 40 ms) and late (after 100 ms) ERP interaction effects. However, it appears that the very early interaction effects can be largely accounted for by an anticipatory ERP that precedes both the unimodal and bimodal stimuli. In calculating the ERP interaction this slow shift is subtracted twice, resulting in an apparent shift of the opposite polarity that may be confounded with actual crossmodal interactions.

Electrophysiological evidence for the "missing link" in crossmodal attention

Orienting attention involuntarily to the location of a sensory event influences responses to subsequent stimuli that appear in different modalities with one possible exception: orienting attention involuntarily to a sudden light sometimes fails to affect responses to subsequent sounds (e.g., Spence & Driver, 1997). Here we investigated the effects of involuntary attention to a brief flash on the processing of subsequent sounds in a design that eliminates stimulus-response compatibility effects and criterion shifts as confounding factors. In addition, the neural processes mediating crossmodal attention were studied by recording event-related brain potentials. Our data show that orienting attention to the location of a spatially nonpredictive visual cue modulates behavioural and neural responses to subsequent auditory targets when the stimulus onset asynchrony is short (between 100 and 300 ms). These findings are consistent with the hypothesis that involuntary shifts of attention are controlled by supramodal brain mechanisms rather than by modality-specific ones.

Human event-related potentials and distraction during selective listening

Event-related brain potentials were recorded from healthy human subjects while they attended to one of two auditory stimulus channels (defined by location, left and right of a fixation point, and pitch) in order to detect rare target events. The distracting properties of periodic noise (vs. continuous noise, experiment 1) and backward speech (vs. forward speech, experiment 2) presented from a third speaker located behind the subjects were investigated. A typical attention effect with a larger negativity for attended tones was observed in both experiments. Backward speech led to a significantly reduced target detection rate for the first four stimuli after onset of the distractor accompanied by a reduced event-related brain potential (ERP)-attention effect and a reduced fronto-central N2b component for the target stimuli. This indicates that irrelevant information leads to an attention decrement of about 1 s duration.

Involuntary orienting to sound improves visual perception

To perceive real-world objects and events, we need to integrate several stimulus features belonging to different sensory modalities. Although the neural mechanisms and behavioural consequences of intersensory integration have been extensively studied, the processes that enable us to pay attention to multimodal objects are still poorly understood. An important question is whether a stimulus in one sensory modality automatically attracts attention to spatially coincident stimuli that appear subsequently in other modalities, thereby enhancing their perceptual salience. The occurrence of an irrelevant sound does facilitate motor responses to a subsequent light appearing nearby. However, because participants in previous studies made speeded responses rather than psychophysical judgements, it remains unclear whether involuntary auditory attention actually affects the perceptibility of visual stimuli as opposed to postperceptual decision and response processes. Here we provide psychophysical evidence that a sudden sound improves the detectability of a subsequent flash appearing at the same location. These data show that the involuntary orienting of attention to sound enhances early perceptual processing of visual stimuli.

Spatial attention to central and peripheral auditory stimuli as indexed by event-related potentials

Young adult subjects attended selectively to brief noise bursts delivered in free-field via central and peripheral arrays of four loudspeakers each that were arranged along a semi-circle extending from the midline to 90 degrees right of center. Frequent "standard" stimuli (90%) and infrequent "target/deviant" stimuli (10%) of increased bandwidth were delivered at a fast rate in random order and equiprobably from all eight speakers. In separate runs, the subject's task was to selectively attend to the center or rightmost speaker, and to press a button to the infrequent "target" stimuli occurring at the designated (spatial) location. Behavioral detection rates and concurrently recorded event-related potentials (ERPs) indicated that auditory attention was deployed as a finely tuned gradient around the attended source. The attentional gradients were steeper for the central than the peripheral array, indicating that attention can be more sharply focused upon sound sources directly in front of the listener. The ERP data suggested that selection for location is accomplished in two distinct stages, with an initial broadly tuned filtering, followed by a more narrowly focused selection of attended-location deviants.

Intra-modal and cross-modal spatial attention to auditory and visual stimuli. An event-related brain potential study

This study investigated cross-modal interactions in spatial attention by means of recording event-related brain potentials (ERPs). Noise bursts and light flashes were presented in random order to both left and right field locations separated by 60 degrees in free-field. One group of subjects was instructed to attend selectively to the noise bursts (attend-auditory group), and a second group attended only to the flashes (attend-visual group). On different runs attention was directed to either the right or left field stimuli of the designated modality. In the attend-auditory group, noise bursts at the attended location elicited a broad, biphasic negativity (Nd) beginning at 70 ms. The cross-modal spatial attention effect on the auditory ERPs in the attend-visual group was very similar in morphology, but the Nd was reduced in amplitude relative to the intra-modal effect. In the attend-visual group, flashes at the attended location elicited enhanced early (100-200 ms) and late (200-350 ms) ERP components relative to unattended-location flashes. The cross-modal effect in the attend-auditory group included small but significant enhancements of early components of the visual ERPs. It was concluded that spatial attention has a multi-modal organization such that the processing of stimuli at attended locations is facilitated at an early, sensory level, even for stimuli of an unattended modality.

The gradient of spatial auditory attention in free field: an event-related potential study

Young adult subjects attended selectively to brief noise bursts delivered in free field via a horizontal array of seven loudspeakers spaced apart by 9 degrees of angle. Frequent "standard" stimuli (90%) and infrequent "target/deviant" stimuli (10%) of increased bandwidth were delivered at a fast rate in a random sequence equiprobably from each speaker. In separate runs, the subjects' task was to selectively attend to the leftmost, center, or rightmost speaker and to press a button to the infrequent "target" stimuli occurring at the designated spatial location. Behavioral detection rates and concurrently recorded event-related potentials (ERPs) indicated that auditory attention was deployed as a finely tuned gradient around the attended sound source, thus providing support for gradient models of auditory spatial attention. Furthermore, the ERP data suggested that the spatial focusing of attention was achieved in two distinct stages, with an early more broadly tuned filtering of inputs occurring over the first 80-200 msec after stimulus onset, followed by a more narrowly focused selection of attended-location deviants that began at around 250 msec and closely resembled the behavioral gradient of target detections.

Effects of spatial selective attention on the steady-state visual evoked potential in the 20-28 Hz range

Steady-state visual evoked potentials (SSVEPs) were recorded from the scalp of subjects who attended to a flickering LED display in one visual field while ignoring a similar display (flickering at a different frequency) in the opposite visual field. The flicker frequencies were 20.8 Hz in the left-field display and 27.8 Hz in the right-field display. The SSVEP to the flicker in either field was enhanced in amplitude when attention was directed to its location. The scalp distribution of this SSVEP enhancement was narrowly focused over the posterior scalp contralateral to the visual field of stimulation. A source analysis using Variable Resolution Electromagnetic Tomography (VARETA) indicated that the source current densities for the SSVEP attention effect had a focal origin in the contralateral parieto-occipital cortex.

Temporal dynamics of early perceptual processing

Recordings of electrical and magnetic brain responses to sensory stimulation provide high-resolution measures of the time course of early perceptual processing. Spatio-temporal analyses of brain activity patterns during the first 200 ms after stimulus presentation have characterized the timing of attentional selection processes and different stages of feature encoding and pattern analyses. Recent studies that incorporate blood flow neuroimaging techniques provide support for mechanisms of early selection of attended visual inputs in extrastriate cortical pathways. The spatial tuning properties of early auditory selection have also been delineated. Electrical and magnetic responses that index the encoding of higher-order pattern information have been identified in both visual and auditory modalities and localized to specific cortical areas.