Intentional attention switching in dichotic listening: Exploring the efficiency of nonspatial and spatial selection

Does preparation help to switch auditory attention between simultaneous voices: Effects of switch probability and prevalence of conflict

Switching auditory attention to one of two (or more) simultaneous voices incurs a substantial performance overhead. Whether/when this voice 'switch cost' reduces when the listener has opportunity to prepare in silence is not clear-the findings on the effect of preparation on the switch cost range from (near) null to substantial. We sought to determine which factors are crucial for encouraging preparation and detecting its effect on the switch cost in a paradigm where participants categorized the number spoken by one of two simultaneous voices; the target voice, which changed unpredictably, was specified by a visual cue depicting the target's gender. First, we manipulated the probability of a voice switch. When 25% of trials were switches, increasing the preparation interval (50/800/1,400 ms) resulted in substantial (~50%) reduction in switch cost. No reduction was observed when 75% of trials were switches. Second, we examined the relative prevalence of low-conflict, 'congruent' trials (where the numbers spoken by the two voices were mapped onto the same response) and high-conflict, 'incongruent' trials (where the voices afforded different responses). 'Conflict prevalence' had a strong effect on selectivity-the incongruent-congruent difference ('congruence effect') was reduced in the 66%-incongruent condition relative to the 66%-congruent condition-but conflict prevalence did not discernibly interact with preparation and its effect on the switch cost. Thus, conditions where switches of target voice are relatively rare are especially conducive to preparation, possibly because attention is committed more strongly to (and/or disengaged less rapidly from) the perceptual features of target voice.

Selective spatial attention in lateralized multi-talker speech perception: EEG correlates and the role of age

Speech comprehension under dynamic cocktail party conditions requires auditory search for relevant speech content and focusing spatial attention on the target talker. Here, we investigated the development of these cognitive processes in a population of 329 participants aged 20-70 years. We used a multi-talker speech detection and perception task in which pairs of words (each consisting of a cue and a target word) were simultaneously presented from lateralized positions. Participants attended to predefined cue words and responded to the corresponding target. Task difficulty was varied by presenting cue and target stimuli at different intensity levels. Decline in performance was observed only in the oldest group (age range 53-70 years) and only in the most difficult condition. The EEG analysis of neurocognitive correlates of lateralized auditory attention and stimulus evaluation (N2ac, LPCpc, alpha power lateralization) revealed age-associated changes in focussing on and processing of task-relevant information, while no such deficits were found on early auditory search and target segregation. Irrespective of age, more challenging listening conditions were associated with an increased allocation of attentional resources.

Effects of Spatial Speech Presentation on Listener Response Strategy for Talker-Identification

This study investigates effects of spatial auditory cues on human listeners' response strategy for identifying two alternately active talkers (“turn-taking” listening scenario). Previous research has demonstrated subjective benefits of audio spatialization with regard to speech intelligibility and talker-identification effort. So far, the deliberate activation of specific perceptual and cognitive processes by listeners to optimize their task performance remained largely unexamined. Spoken sentences selected as stimuli were either clean or degraded due to background noise or bandpass filtering. Stimuli were presented via three horizontally positioned loudspeakers: In a non-spatial mode, both talkers were presented through a central loudspeaker; in a spatial mode, each talker was presented through the central or a talker-specific lateral loudspeaker. Participants identified talkers via speeded keypresses and afterwards provided subjective ratings (speech quality, speech intelligibility, voice similarity, talker-identification effort). In the spatial mode, presentations at lateral loudspeaker locations entailed quicker behavioral responses, which were significantly slower in comparison to a talker-localization task. Under clean speech, response times globally increased in the spatial vs. non-spatial mode (across all locations); these “response time switch costs,” presumably being caused by repeated switching of spatial auditory attention between different locations, diminished under degraded speech. No significant effects of spatialization on subjective ratings were found. The results suggested that when listeners could utilize task-relevant auditory cues about talker location, they continued to rely on voice recognition instead of localization of talker sound sources as primary response strategy. Besides, the presence of speech degradations may have led to increased cognitive control, which in turn compensated for incurring response time switch costs.

How Task Set and Task Switching Modulate Perceptual Processes: Is Recognition of Facial Emotion an Exception?

In Part 1 we review task-switching and other studies showing that, even with time for preparation, participants' ability to shift attention to a relevant attribute or object before the stimulus onset is limited: there is a 'residual cost'. In particular, several brain potential markers of perceptual encoding are delayed on task-switch trials, compared to task-repeat trials that require attention to the same attribute as before. Such effects have been documented even for a process often considered 'automatic' - visual word recognition: ERP markers of word frequency and word/nonword status are (1) delayed when the word recognition task follows a judgement of a perceptual property compared to repeating the lexical task, and (2) strongly attenuated during the perceptual judgements. Thus, even lexical access seems influenced by the task/attentional set. In Part 2, we report in detail a demonstration of what seems to be a special case, where task-set and a task switch have no such effect on perceptual encoding. Participants saw an outline letter superimposed on a face expressing neutral or negative emotion, and were auditorily cued to categorise the letter as vowel/consonant, or the face as emotional/neutral. ERPs exhibited a robust emotional-neutral difference (Emotional Expression Effect) no smaller or later when switching to the face task than when repeating it; in the first half of its time-course it did not vary with the task at all. The initial encoding of the valence of a fixated facial emotional expression appears to be involuntary and invariant, whatever the endogenous task/attentional set.

Do congruent lip movements facilitate speech processing in a dynamic audiovisual multi-talker scenario? An ERP study with older and younger adults

In natural conversations, visible mouth and lip movements play an important role in speech comprehension. There is evidence that visual speech information improves speech comprehension, especially for older adults and under difficult listening conditions. However, the neurocognitive basis is still poorly understood. The present EEG experiment investigated the benefits of audiovisual speech in a dynamic cocktail-party scenario with 22 (aged 20-34 years) younger and 20 (aged 55-74 years) older participants. We presented three simultaneously talking faces with a varying amount of visual speech input (still faces, visually unspecific and audiovisually congruent). In a two-alternative forced-choice task, participants had to discriminate target words ("yes" or "no") among two distractors (one-digit number words). In half of the experimental blocks, the target was always presented from a central position, in the other half, occasional switches to a lateral position could occur. We investigated behavioral and electrophysiological modulations due to age, location switches and the content of visual information, analyzing response times and accuracy as well as the P1, N1, P2, N2 event-related potentials (ERPs) and the contingent negative variation (CNV) in the EEG. We found that audiovisually congruent speech information improved performance and modulated ERP amplitudes in both age groups, suggesting enhanced preparation and integration of the subsequent auditory input. In the older group, larger amplitude measures were found in early phases of processing (P1-N1). Here, amplitude measures were reduced in response to audiovisually congruent stimuli. In later processing phases (P2-N2) we found decreased amplitude measures in the older group, while an amplitude reduction for audiovisually congruent compared to visually unspecific stimuli was still observable. However, these benefits were only observed as long as no location switches occurred, leading to enhanced amplitude measures in later processing phases (P2-N2). To conclude, meaningful visual information in a multi-talker setting, when presented from the expected location, is shown to be beneficial for both younger and older adults.

EEG correlates of spatial shifts of attention in a dynamic multi-talker speech perception scenario in younger and older adults

Speech perception under "cocktail-party" conditions critically depends on the focusing of attention toward the talker of interest. In dynamic auditory scenes, changes in talker settings require rapid shifts of attention, which is especially relevant when the position of a target talker switches from one location to another. Here, we explored electrophysiological correlates of shifts in spatial auditory attention, using a free-field speech perception task, in which sequences of short words (a company name, followed by a numeric value, e.g., "Bosch-6") were presented in the participants' left and right horizontal plane. Younger and older participants responded to the value of a pre-defined target company, while ignoring three simultaneously presented pairs of concurrent company names and values from different locations. All four stimulus pairs were spoken by different talkers, alternating from trial-to-trial. The location of the target company was within either the left or right hemisphere for a variable number of consecutive trials (between 3 and 42 trials) and then changed, switching from the left to the right hemispace or vice versa. Thus, when a switch occurred, the participants had to search for the new position of the target company among the concurrent streams of auditory information and re-focus their attention on the relevant location. As correlates of lateralized spatial auditory attention, the anterior contralateral N2 subcomponent (N2ac) and the posterior alpha power lateralization were analyzed in trials immediately before and after switches of the target location. Both measures were increased after switches, while only the increase in N2ac was related to better speech perception performance (i.e., a reduced post-switch decline in accuracy). While both age groups showed a similar pattern of switch-related attentional modulations, N2ac and alpha lateralization to the task-relevant stimulus (the target company's value) was overall greater in the younger, than older, group. The results suggest that N2ac and alpha lateralization reflect different attentional processes in multi-talker speech perception, the first being primarily associated with auditory search and the focusing of attention, and the second with the in-depth attentional processing of task-relevant information. Especially the second process appears to be prone to age-related cognitive decline.

The Effects of Switching Non-Spatial Attention During Conversational Turn Taking

This study examined the effect of a change in target voice on word recall during a multi-talker conversation. Two experiments were conducted using matrix sentences to assess the cost of a single endogenous switch in non-spatial attention. Performance in a yes-no recognition task was significantly worse when a target voice changed compared to when it remained the same after a turn-taking gap. We observed a decrease in target hit rate and sensitivity, and an increase in masker confusion errors following a change in voice. These results highlight the cognitive demands of not only engaging attention on a new talker, but also of disengaging attention from a previous target voice. This shows that exposure to a voice can have a biasing effect on attention that persists well after a turn-taking gap. A second experiment showed that there was no change in switching performance using different talker combinations. This demonstrates that switching costs were consistent and did not depend on the degree of acoustic differences in target voice characteristics.

Cross-modal transfer after auditory task-switching training

Task-switching training was shown to improve performance not only for the trained tasks (i.e., reduced performance costs resulting from the task switches), but also for structurally similar (near transfer) or even dissimilar tasks (far transfer). However, it is still unclear whether the improvement is specific to the trained input modality or whether cognitive control occurs at an amodal processing level enabling transfer of set-shifting abilities to different input modalities. In this study, training and transfer was assessed for an auditory task-switching paradigm in which spoken words from different semantic categories were presented dichotically requiring participants to switch between two auditory categorization tasks. Cross-modal transfer of task-switching training was assessed in terms of the performance costs in a visual task-switching situation using tasks that were structurally similar to the trained tasks. The 4-day training significantly reduced the costs resulting from mixing the two auditory tasks, as compared to both an active (auditory single-task training) and a passive control group (no training). More importantly, the auditory task-switching training was also found to reduce the mixing costs for untrained visual tasks, indicating cross-modal transfer. This finding suggests that the improvement resulting from task-switching training is not specific to the trained stimulus modality, but it seems to be driven by a cognitive control mechanism operating at an amodal processing level. The training did not reveal any far-transfer effects to working memory, inhibition, or fluid intelligence, suggesting that the modality-independent enhancement of set-shifting does not generalize to other cognitive control functions.

The binding of an auditory target location to a judgement: A two-component switching approach

In a two-component switching paradigm, in which participants switched between two auditory attention selection criteria (attention component: left vs. right ear) and two judgements (judgement component: number vs. letter judgement), we found high judgement switch costs in attention criterion repetitions, but low costs in attention criterion switches. This finding showed an interaction of components. Previous two-component switching studies observed differently emphasised interaction patterns. In the present study, we explored whether the strength of the interaction pattern reflects the strength of the binding of target location and judgement. Specifically, we investigated whether exogenous target location cueing led to weaker binding than endogenous cueing, and whether preparation for ear selection could influence the binding. Attention switches with auditory exogenous target location cues did not affect the component interaction pattern, whereas a prolonged preparation interval led to a more emphasised pattern. Binding between target location and judgement may therefore be rather automatic and may not necessarily require concurrent component processing. Sufficient time for target location switches with long preparation time may activate the previous trial’s episode or facilitate switches of the subsequent judgement.

An Extended Binaural Real-Time Auralization System With an Interface to Research Hearing Aids for Experiments on Subjects With Hearing Loss

Theory and implementation of acoustic virtual reality have matured and become a powerful tool for the simulation of entirely controllable virtual acoustic environments. Such virtual acoustic environments are relevant for various types of auditory experiments on subjects with normal hearing, facilitating flexible virtual scene generation and manipulation. When it comes to expanding the investigation group to subjects with hearing loss, choosing a reproduction system which offers a proper integration of hearing aids into the virtual acoustic scene is crucial. Current loudspeaker-based spatial audio reproduction systems rely on different techniques to synthesize a surrounding sound field, providing various possibilities for adaptation and extension to allow applications in the field of hearing aid-related research. Representing one option, the concept and implementation of an extended binaural real-time auralization system is presented here. This system is capable of generating complex virtual acoustic environments, including room acoustic simulations, which are reproduced as combined via loudspeakers and research hearing aids. An objective evaluation covers the investigation of different system components, a simulation benchmark analysis for assessing the processing performance, and end-to-end latency measurements.

Neural Switch Asymmetry in Feature-Based Auditory Attention Tasks

Active listening involves dynamically switching attention between competing talkers and is essential to following conversations in everyday environments. Previous investigations in human listeners have examined the neural mechanisms that support switching auditory attention within the acoustic featural cues of pitch and auditory space. Here, we explored the cortical circuitry underlying endogenous switching of auditory attention between pitch and spatial cues necessary to discern target from masker words. Because these tasks are of unequal difficulty, we expected an asymmetry in behavioral switch costs for hard-to-easy versus easy-to-hard switches, mirroring prior evidence from vision-based cognitive task-switching paradigms. We investigated the neural correlates of this behavioral switch asymmetry and associated cognitive control operations in the present auditory paradigm. Behaviorally, we observed no switch-cost asymmetry, i.e., no performance difference for switching from the more difficult attend-pitch to the easier attend-space condition (P→S) versus switching from easy-to-hard (S→P). However, left lateral prefrontal cortex activity, correlated with improved performance, was observed during a silent gap period when listeners switched attention from P→S, relative to switching within pitch cues. No such differential activity was seen for the analogous easy-to-hard switch. We hypothesize that this neural switch asymmetry reflects proactive cognitive control mechanisms that successfully reconfigured neurally-specified task parameters and resolved competition from other such "task sets," thereby obviating the expected behavioral switch-cost asymmetry. The neural switch activity observed was generally consistent with that seen in cognitive paradigms, suggesting that established cognitive models of attention switching may be productively applied to better understand similar processes in audition.

Intentional preparation of auditory attention-switches: Explicit cueing and sequential switch-predictability

In an auditory attention-switching paradigm, participants heard two simultaneously spoken number-words, each presented to one ear, and decided whether the target number was smaller or larger than 5 by pressing a left or right key. An instructional cue in each trial indicated which feature had to be used to identify the target number (e.g., female voice). Auditory attention-switch costs were found when this feature changed compared to when it repeated in two consecutive trials. Earlier studies employing this paradigm showed mixed results when they examined whether such cued auditory attention-switches can be prepared actively during the cue–stimulus interval. This study systematically assessed which preconditions are necessary for the advance preparation of auditory attention-switches. Three experiments were conducted that controlled for cue-repetition benefits, modality switches between cue and stimuli, as well as for predictability of the switch-sequence. Only in the third experiment, in which predictability for an attention-switch was maximal due to a pre-instructed switch-sequence and predictable stimulus onsets, active switch-specific preparation was found. These results suggest that the cognitive system can prepare auditory attention-switches, and this preparation seems to be triggered primarily by the memorised switching-sequence and valid expectations about the time of target onset.

Intentional switching in auditory selective attention: Exploring attention shifts with different reverberation times

Using a well-established binaural-listening paradigm the ability to intentionally switch auditory selective attention was examined under anechoic, low reverberation (0.8 s) and high reverberation (1.75 s) conditions. Twenty-three young, normal-hearing subjects were tested in a within-subject design to analyze influences of the reverberation times. Spoken word pairs by two speakers were presented simultaneously to subjects from two of eight azimuth positions. The stimuli consisted of a single number word, (i.e., 1 to 9), followed by either the direction "UP" or "DOWN" in German. Guided by a visual cue prior to auditory stimulus onset indicating the position of the target speaker, subjects were asked to identify whether the target number was numerically smaller or greater than five and to categorize the direction of the second word. Switch costs, (i.e. reaction time differences between a position switch of the target relative to a position repetition), were larger under the high reverberation condition. Furthermore, the error rates were highly dependent on reverberant energy and reverberation interacted with the congruence effect, (i.e. stimuli spoken by target and distractor may evoke the same answer (congruent) or different answers (incongruent)), indicating larger congruence effects under higher reverberation times.

Intentional switching in auditory selective attention: Exploring age-related effects in a spatial setup requiring speech perception

Using a binaural-listening paradigm, age-related differences in the ability to intentionally switch auditory selective attention between two speakers, defined by their spatial location, were examined. Therefore 40 normal-hearing participants (20 young, Ø 24.8years; 20 older Ø 67.8years) were tested. The spatial reproduction of stimuli was provided by headphones using head-related-transfer-functions of an artificial head. Spoken number words of two speakers were presented simultaneously to participants from two out of eight locations on the horizontal plane. Guided by a visual cue indicating the spatial location of the target speaker, the participants were asked to categorize the target's number word into smaller vs. greater than five while ignoring the distractor's speech. Results showed significantly higher reaction times and error rates for older participants. The relative influence of the spatial switch of the target-speaker (switch or repetition of speaker's direction in space) was identical across age groups. Congruency effects (stimuli spoken by target and distractor may evoke the same answer or different answers) were increased for older participants and depend on the target's position. Results suggest that the ability to intentionally switch auditory attention to a new cued location was unimpaired whereas it was generally harder for older participants to suppress processing the distractor's speech.

The flip side of the auditory spatial selection benefit: larger attentional mixing costs for target selection by ear than by gender in auditory task switching

In cued auditory task switching, one of two dichotically presented number words, spoken by a female and a male, had to be judged according to its numerical magnitude. One experimental group selected targets by speaker gender and another group by ear of presentation. In mixed-task blocks, the target-defining feature (male/female vs. left/right) was cued prior to each trial, but in pure blocks it remained constant. Compared to selection by gender, selection by ear led to better performance in pure blocks than in mixed blocks, resulting in larger "global" mixing costs for ear-based selection. Selection by ear also led to larger "local" switch costs in mixed blocks, but this finding was partially mediated by differential cue-repetition benefits. Together, the data suggest that requirements of attention shifting diminish the auditory spatial selection benefit.