Elastic Attention: Enhanced, then Sharpened Response to Auditory Input as Attentional Load Increases

Effect of Auditory Distractors on Speech Recognition and Listening Effort

OBJECTIVES

Everyday listening environments are filled with competing noise and distractors. Although significant research has examined the effect of competing noise on speech recognition and listening effort, little is understood about the effect of distraction. The framework for understanding effortful listening recognizes the importance of attention-related processes in speech recognition and listening effort; however, it underspecifies the role that they play, particularly with respect to distraction. The load theory of attention predicts that resources will be automatically allocated to processing a distractor, but only if perceptual load in the listening task is low enough. If perceptual load is high (i.e., listening in noise), then resources that would otherwise be allocated to processing a distractor are used to overcome the increased perceptual load and are unavailable for distractor processing. Although there is ample evidence for this theory in the visual domain, there has been little research investigating how the load theory of attention may apply to speech processing. In this study, we sought to measure the effect of distractors on speech recognition and listening effort and to evaluate whether the load theory of attention can be used to understand a listener's resource allocation in the presence of distractors.

DESIGN

Fifteen adult listeners participated in a monosyllabic words repetition task. Test stimuli were presented in quiet or in competing speech (+5 dB signal-to-noise ratio) and in distractor or no distractor conditions. In conditions with distractors, auditory distractors were presented before the target words on 24% of the trials in quiet and in noise. Percent-correct was recorded as speech recognition, and verbal response time (VRT) was recorded as a measure of listening effort.

RESULTS

A significant interaction was present for speech recognition, showing reduced speech recognition when distractors were presented in the quiet condition but no effect of distractors when noise was present. VRTs were significantly longer when distractors were present, regardless of listening condition.

CONCLUSIONS

Consistent with the load theory of attention, distractors significantly reduced speech recognition in the low-perceptual load condition (i.e., listening in quiet) but did not impact speech recognition scores in conditions of high perceptual load (i.e., listening in noise). The increases in VRTs in the presence of distractors in both low- and high-perceptual load conditions (i.e., quiet and noise) suggest that the load theory of attention may not apply to listening effort. However, the large effect of distractors on VRT in both conditions is consistent with the previous work demonstrating that distraction-related shifts of attention can delay processing of the target task. These findings also fit within the framework for understanding effortful listening, which proposes that involuntary attentional shifts result in a depletion of cognitive resources, leaving less resources readily available to process the signal of interest; resulting in increased listening effort (i.e., elongated VRT).

Disruption in neural phase synchrony is related to identification of inattentional deafness in real-world setting

Individuals often have reduced ability to hear alarms in real world situations (e.g., anesthesia monitoring, flying airplanes) when attention is focused on another task, sometimes with devastating consequences. This phenomenon is called inattentional deafness and usually occurs under critical high workload conditions. It is difficult to simulate the critical nature of these tasks in the laboratory. In this study, dry electroencephalography is used to investigate inattentional deafness in real flight while piloting an airplane. The pilots participating in the experiment responded to audio alarms while experiencing critical high workload situations. It was found that missed relative to detected alarms were marked by reduced stimulus evoked phase synchrony in theta and alpha frequencies (6-14 Hz) from 120 to 230 ms poststimulus onset. Correlation of alarm detection performance with intertrial coherence measures of neural phase synchrony showed different frequency and time ranges for detected and missed alarms. These results are consistent with selective attentional processes actively disrupting oscillatory coherence in sensory networks not involved with the primary task (piloting in this case) under critical high load conditions. This hypothesis is corroborated by analyses of flight parameters showing greater maneuvering associated with difficult phases of flight occurring during missed alarms. Our results suggest modulation of neural oscillation is a general mechanism of attention utilizing enhancement of phase synchrony to sharpen alarm perception during successful divided attention, and disruption of phase synchrony in brain networks when attentional demands of the primary task are great, such as in the case of inattentional deafness.

Layer specific sharpening of frequency tuning by selective attention in primary auditory cortex

Recent electrophysiological and neuroimaging studies provide converging evidence that attending to sounds increases the response selectivity of neuronal ensembles even at the first cortical stage of auditory stimulus processing in primary auditory cortex (A1). This is achieved by enhancement of responses in the regions that process attended frequency content, and by suppression of responses in the surrounding regions. The goals of our study were to define the extent to which A1 neuronal ensembles are involved in this process, determine its effect on the frequency tuning of A1 neuronal ensembles, and examine the involvement of the different cortical layers. To accomplish these, we analyzed laminar profiles of synaptic activity and action potentials recorded in A1 of macaques performing a rhythmic intermodal selective attention task. We found that the frequency tuning of neuronal ensembles was sharpened due to both increased gain at the preferentially processed or best frequency and increased response suppression at all other frequencies when auditory stimuli were attended. Our results suggest that these effects are due to a frequency-specific counterphase entrainment of ongoing delta oscillations, which predictively orchestrates opposite sign excitability changes across all of A1. This results in a net suppressive effect due to the large proportion of neuronal ensembles that do not specifically process the attended frequency content. Furthermore, analysis of laminar activation profiles revealed that although attention-related suppressive effects predominate the responses of supragranular neuronal ensembles, response enhancement is dominant in the granular and infragranular layers, providing evidence for layer-specific cortical operations in attentive stimulus processing.

Attention modulates cortical processing of pitch feedback errors in voice control

Considerable evidence has shown that unexpected alterations in auditory feedback elicit fast compensatory adjustments in vocal production. Although generally thought to be involuntary in nature, whether these adjustments can be influenced by attention remains unknown. The present event-related potential (ERP) study aimed to examine whether neurobehavioral processing of auditory-vocal integration can be affected by attention. While sustaining a vowel phonation and hearing pitch-shifted feedback, participants were required to either ignore the pitch perturbations, or attend to them with low (counting the number of perturbations) or high attentional load (counting the type of perturbations). Behavioral results revealed no systematic change of vocal response to pitch perturbations irrespective of whether they were attended or not. At the level of cortex, there was an enhancement of P2 response to attended pitch perturbations in the low-load condition as compared to when they were ignored. In the high-load condition, however, P2 response did not differ from that in the ignored condition. These findings provide the first neurophysiological evidence that auditory-motor integration in voice control can be modulated as a function of attention at the level of cortex. Furthermore, this modulatory effect does not lead to a general enhancement but is subject to attentional load.

Neural effects of cognitive control load on auditory selective attention

Whether and how working memory disrupts or alters auditory selective attention is unclear. We compared simultaneous event-related potentials (ERP) and functional magnetic resonance imaging (fMRI) responses associated with task-irrelevant sounds across high and low working memory load in a dichotic-listening paradigm. Participants performed n-back tasks (1-back, 2-back) in one ear (Attend ear) while ignoring task-irrelevant speech sounds in the other ear (Ignore ear). The effects of working memory load on selective attention were observed at 130-210ms, with higher load resulting in greater irrelevant syllable-related activation in localizer-defined regions in auditory cortex. The interaction between memory load and presence of irrelevant information revealed stronger activations primarily in frontal and parietal areas due to presence of irrelevant information in the higher memory load. Joint independent component analysis of ERP and fMRI data revealed that the ERP component in the N1 time-range is associated with activity in superior temporal gyrus and medial prefrontal cortex. These results demonstrate a dynamic relationship between working memory load and auditory selective attention, in agreement with the load model of attention and the idea of common neural resources for memory and attention.

Modulation of response patterns in human auditory cortex during a target detection task: an intracranial electrophysiology study

Selective attention enhances cortical activity representing an attended sound stream in human posterolateral superior temporal gyrus (PLST). It is unclear, however, what mechanisms are associated with a target detection task that necessitates sustained attention (vigilance) to a sound stream. We compared responses elicited by target and non-target sounds, and to sounds presented in a passive-listening paradigm. Subjects were neurosurgical patients undergoing invasive monitoring for medically refractory epilepsy. Stimuli were complex tones, band-limited noise bursts and speech syllables. High gamma cortical activity (70-150 Hz) was examined in all subjects using subdural grid electrodes implanted over PLST. Additionally, responses were measured from depth electrodes implanted within Heschl's gyrus (HG) in one subject. Responses to target sounds recorded from PLST were increased when compared to responses elicited by the same sounds when they were not-targets, and when they were presented during passive listening. Increases in high gamma activity to target sounds occurred during later portions (after 250 ms) of the response. These increases were related to the task and not to detailed stimulus characteristics. In contrast, earlier activity that did not vary across conditions did represent stimulus acoustic characteristics. Effects observed on PLST were not noted in HG. No consistent effects were noted in the averaged evoked potentials in either cortical region. We conclude that task dependence modulates later activity in PLST during vigilance. Later activity may represent feedback from higher cortical areas. Study of concurrently recorded activity from frontoparietal areas is necessary to further clarify task-related modulation of activity on PLST.

Cortisol rapidly affects amplitudes of heartbeat-evoked brain potentials--implications for the contribution of stress to an altered perception of physical sensations?

Little is known about the impact of stress and stress hormones on the processing of visceral-afferent signals. Clinical data suggest that cortisol may lower the threshold for interoceptive stimuli, while a pharmacological administration of cortisol decreases the sensitivity for physical symptoms. To clarify the role of cortisol for the processing of interoceptive signals, we investigated 16 healthy men on two occasions, once during the infusion of 4 mg of cortisol and once during the infusion of a placebo substance. Heartbeat-evoked potentials (HEP; derived from resting EEG and ECG, during open and closed eyes), which are psychophysiological indicators for the cortical processing of cardioceptive signals, were measured over 6-min periods once before, and four times after the infusion (1-7, 11-17, 21-27 and 31-37 min). We found that HEP amplitudes were higher during open than during closed eyes between 1 and 17 min after cortisol infusion. There was no effect of cortisol on heart rate. We conclude that cortisol may rapidly modulate the cortical processing of cardioceptive neural signals. These results may have relevance for the effects of stress on the development and maintenance of psychosomatic symptoms.

Source analysis of event-related potentials during pitch discrimination and pitch memory tasks

Our previous studies using fMRI have demonstrated that activations in human auditory cortex (AC) are strongly dependent on the characteristics of the task. The present study tested whether source estimation of scalp-recorded event-related potentials (ERPs) can be used to investigate task-dependent AC activations. Subjects were presented with frequency-varying two-part tones during pitch discrimination, pitch n-back memory, and visual tasks identical to our previous fMRI study (Rinne et al., J Neurosci 29:13338-13343, 2009). ERPs and their minimum-norm source estimates in AC were strongly modulated by task at 200-700 ms from tone onset. As in the fMRI study, the pitch discrimination and pitch memory tasks were associated with distinct AC activation patterns. In the pitch discrimination task, increased activity in the anterior AC was detected relatively late at 300-700 ms from tone onset. Therefore, this activity was probably not associated with enhanced pitch processing but rather with the actual discrimination process (comparison between the two parts of tone). Increased activity in more posterior areas associated with the pitch memory task, in turn, occurred at 200-700 ms suggesting that this activity was related to operations on pitch categories after pitch analysis was completed. Finally, decreased activity associated with the pitch memory task occurred at 150-300 ms consistent with the notion that, in the demanding pitch memory task, spectrotemporal analysis is actively halted as soon as category information has been obtained. These results demonstrate that ERP source analysis can be used to complement fMRI to investigate task-dependent activations of human AC.

Attention to natural auditory signals

The challenge of understanding how the brain processes natural signals is compounded by the fact that such signals are often tied closely to specific natural behaviors and natural environments. This added complexity is especially true for auditory communication signals that can carry information at multiple hierarchical levels, and often occur in the context of other competing communication signals. Selective attention provides a mechanism to focus processing resources on specific components of auditory signals, and simultaneously suppress responses to unwanted signals or noise. Although selective auditory attention has been well-studied behaviorally, very little is known about how selective auditory attention shapes the processing on natural auditory signals, and how the mechanisms of auditory attention are implemented in single neurons or neural circuits. Here we review the role of selective attention in modulating auditory responses to complex natural stimuli in humans. We then suggest how the current understanding can be applied to the study of selective auditory attention in the context natural signal processing at the level of single neurons and populations in animal models amenable to invasive neuroscience techniques. This article is part of a Special Issue entitled "Communication Sounds and the Brain: New Directions and Perspectives".

The effect of imagination on stimulation: the functional specificity of efference copies in speech processing

The computational role of efference copies is widely appreciated in action and perception research, but their properties for speech processing remain murky. We tested the functional specificity of auditory efference copies using magnetoencephalography recordings in an unconventional pairing: We used a classical cognitive manipulation (mental imagery--to elicit internal simulation and estimation) with a well-established experimental paradigm (one shot repetition--to assess neuronal specificity). Participants performed tasks that differentially implicated internal prediction of sensory consequences (overt speaking, imagined speaking, and imagined hearing) and their modulatory effects on the perception of an auditory (syllable) probe were assessed. Remarkably, the neural responses to overt syllable probes vary systematically, both in terms of directionality (suppression, enhancement) and temporal dynamics (early, late), as a function of the preceding covert mental imagery adaptor. We show, in the context of a dual-pathway model, that internal simulation shapes perception in a context-dependent manner.

Two-stage processing of sounds explains behavioral performance variations due to changes in stimulus contrast and selective attention: an MEG study

Selectively attending to task-relevant sounds whilst ignoring background noise is one of the most amazing feats performed by the human brain. Here, we studied the underlying neural mechanisms by recording magnetoencephalographic (MEG) responses of 14 healthy human subjects while they performed a near-threshold auditory discrimination task vs. a visual control task of similar difficulty. The auditory stimuli consisted of notch-filtered continuous noise masker sounds, and of 1020-Hz target tones occasionally (p = 0.1) replacing 1000-Hz standard tones of 300-ms duration that were embedded at the center of the notches, the widths of which were parametrically varied. As a control for masker effects, tone-evoked responses were additionally recorded without masker sound. Selective attention to tones significantly increased the amplitude of the onset M100 response at ~100 ms to the standard tones during presence of the masker sounds especially with notches narrower than the critical band. Further, attention modulated sustained response most clearly at 300-400 ms time range from sound onset, with narrower notches than in case of the M100, thus selectively reducing the masker-induced suppression of the tone-evoked response. Our results show evidence of a multiple-stage filtering mechanism of sensory input in the human auditory cortex: 1) one at early (~100 ms) latencies bilaterally in posterior parts of the secondary auditory areas, and 2) adaptive filtering of attended sounds from task-irrelevant background masker at longer latency (~300 ms) in more medial auditory cortical regions, predominantly in the left hemisphere, enhancing processing of near-threshold sounds.

Is the effect of tinnitus on auditory steady-state response amplitude mediated by attention?

Objectives: Auditory steady-state response (ASSR) amplitude enhancement effects have been reported in tinnitus patients. As ASSR amplitude is also enhanced by attention, the effect of tinnitus on ASSR amplitude could be interpreted as an effect of attention mediated by tinnitus. As N1 attention effects are significantly larger than those on the ASSR, if the effect of tinnitus on ASSR amplitude were due to attention, there should be similar amplitude enhancement effects in tinnitus for the N1 component of the auditory-evoked response. Methods: MEG recordings which were previously examined for the ASSR (Diesch et al., 2010a) were analyzed with respect to the N1m component. Like the ASSR previously, the N1m was analyzed in the source domain (source space projection). Stimuli were amplitude-modulated (AM) tones with one of three carrier frequencies matching the tinnitus frequency or a surrogate frequency 1½ octave above the audiometric edge frequency in controls, the audiometric edge frequency, and a frequency below the audiometric edge. Single AM-tones were presented in a single condition and superpositions of three AM-tones differing in carrier and modulation frequency in a composite condition. Results: In the earlier ASSR study (Diesch et al., 2010a), the ASSR amplitude in tinnitus patients, but not in controls, was significantly larger in the (surrogate) tinnitus condition than in the edge condition. Patients showed less evidence than controls of reciprocal inhibition of component ASSR responses in the composite condition. In the present study, N1m amplitudes elicited by stimuli located at the audiometric edge and at the (surrogate) tinnitus frequency were smaller than N1m amplitudes elicited by sub-edge tones both in patients and controls. The relationship of the N1m response in the composite condition to the N1m response in the single condition indicated that reciprocal inhibition among component N1m responses was reduced in patients compared against controls. Conclusions: In the present study, no evidence was found for an N1-amplitude enhancement effect in tinnitus. Compared to controls, reciprocal inhibition is reduced in tinnitus patients. Thus, as there is no effect on N1m that could potentially be attributed to attention, it seems unlikely that the enhancement effect of tinnitus on ASSR amplitude could be accounted for in terms of attention induced by tinnitus.

Attention-related modulation of auditory-cortex responses to speech sounds during dichotic listening

Event-related magnetic fields (ERFs) were measured with magnetoencephalography (MEG) in fifteen healthy right-handed participants listening to sequences of consonant-vowel syllable pairs delivered dichotically (one syllable presented to the left ear and another syllable simultaneously to the right ear). The participants were instructed to press a response button to occurrences of a particular target syllable. In a condition with no other instruction (the non-forced condition, NF), they showed the well-known right-ear advantage (REA), that is, the participants responded more often to target syllables delivered to the right ear than to targets delivered to the left ear. The same was true in the forced-right (FR) condition, where the participants were instructed to attend selectively to the right-ear syllables and respond only to targets among them. In the forced-left (FL) condition, where they were instructed to respond only to left-ear targets, they responded more often to targets in this ear than to targets in the right ear. At 300-500 ms from syllable pair onset, a sustained field (SF) in ERFs to the syllable pairs was stronger in the left auditory cortex than in the right auditory cortex in the NF and FR conditions, while the opposite was true in the FL condition. Thus selective attention during dichotic listening leads to stronger processing of speech sounds in the auditory cortex contralateral to the attended direction. Our results also suggest that the REA observed for dichotic speech may involve a bias of attention to the right side even when there is no instruction to do so. This supports Kinsbourne's (1970) model of attention bias as a general principle of laterality.