Spatial release of cognitive load measured in a dual-task paradigm in normal-hearing and hearing-impaired listeners

An Exploratory Study of Walking, Listening, and Remembering in Younger and Middle-Aged Adults

PURPOSE

The purpose of this study was to assess how needing to listen and remember information while walking affects speech perception, memory task performance, and gait in younger and middle-aged adults.

METHOD

Four gait parameters (stride duration, step variability, whole-body center of mass acceleration, and mediolateral head acceleration) were measured when younger and middle-aged participants stood or walked on a treadmill while they simultaneously completed a speech-on-speech perception task and a preload memory task, singly and in combination.

RESULTS

Speech perception was significantly poorer for middle-aged than for younger participants. Performance on the speech perception measure did not differ significantly between walking and standing for either group of participants, but the additional cognitive load of the memory task reduced performance on the speech perception task. Memory task performance was significantly poorer when combined with the speech perception task than when measured in isolation for both participant groups, but no further declines were noted when participants were also walking. Mediolateral head acceleration, which has been linked to loss of balance, was significantly greater during multitask trials, as compared to when participants were only walking without being required to listen or remember. Post hoc analysis showed that dual- and multitask influences on mediolateral head acceleration were more prominent for middle-aged than for younger participants. Stride duration was longer in the multitask condition than when participants were only walking.

CONCLUSIONS

Results of this exploratory study indicate that gait may be impacted when individuals (both younger and middle-aged) are listening and remembering while walking. Data also substantiate prior findings of early age-related declines in the perception of speech in the presence of understandable speech maskers.

An ecological investigation of the capacity to follow simultaneous speech and preferential detection of ones’ own name

Many situations require focusing attention on one speaker, while monitoring the environment for potentially important information. Some have proposed that dividing attention among 2 speakers involves behavioral trade-offs, due to limited cognitive resources. However the severity of these trade-offs, particularly under ecologically-valid circumstances, is not well understood. We investigated the capacity to process simultaneous speech using a dual-task paradigm simulating task-demands and stimuli encountered in real-life. Participants listened to conversational narratives (Narrative Stream) and monitored a stream of announcements (Barista Stream), to detect when their order was called. We measured participants’ performance, neural activity, and skin conductance as they engaged in this dual-task. Participants achieved extremely high dual-task accuracy, with no apparent behavioral trade-offs. Moreover, robust neural and physiological responses were observed for target-stimuli in the Barista Stream, alongside significant neural speech-tracking of the Narrative Stream. These results suggest that humans have substantial capacity to process simultaneous speech and do not suffer from insufficient processing resources, at least for this highly ecological task-combination and level of perceptual load. Results also confirmed the ecological validity of the advantage for detecting ones’ own name at the behavioral, neural, and physiological level, highlighting the contribution of personal relevance when processing simultaneous speech.

Assessing and Modeling Spatial Release From Listening Effort in Listeners With Normal Hearing: Reference Ranges and Effects of Noise Direction and Age

Listening to speech in noisy environments is challenging and effortful. Factors like the signal-to-noise ratio (SNR), the spatial separation between target speech and noise interferer(s), and possibly also the listener's age might influence perceived listening effort (LE). This study measured and modeled the effect of the spatial separation of target speech and interfering stationary speech-shaped noise on the perceived LE and its relation to the age of the listeners. Reference ranges for the relationship between subjectively perceived LE and SNR for different noise azimuths were established. For this purpose, 70 listeners with normal hearing and from three age groups rated the perceived LE using the Adaptive Categorical Listening Effort Scaling method (ACALES, Krueger et al., 2017a) with speech from the front and noise from 0°, 90°, 135°, or 180° azimuth. Based on these data, the spatial release from listening effort (SRLE) was calculated. The noise azimuth had a strong effect on SRLE, with the highest release for 135°. The binaural speech intelligibility model (BSIM2020, Hauth et al., 2020) predicted SRLE very well at negative SNRs, but overestimated for positive SNRs. No significant effect of age was found on the respective subjective ratings. Therefore, the reference ranges were determined independently of age. These reference ranges can be used for the classification of LE measurements. However, when the increase of the perceived LE with SNR was analyzed, a significant age difference was found between the listeners of the youngest and oldest group when considering the upper range of the LE function.

A Clinical Paradigm for Listening Effort Assessment in Middle-Aged Listeners

Listening effort (LE) has been known to characterize speech recognition in noise regardless of hearing sensitivity and age. Whereas the behavioral measure of dual-task paradigm effectively manifests the cognitive cost that listeners exert when processing speech in background noise, there is no consensus as to a clinical procedure that might best express LE. In order to assess the cognitive load underlying speech recognition in noise and promote counselling for coping strategies, a feasible clinical paradigm is warranted. The ecological validity of such a paradigm might best be demonstrated in middle-aged adults, exhibiting intact hearing sensitivity on one hand, however, experiencing difficulties in degraded listening conditions, unaware of the implicated cognitive cost of speech recognition in noise. To this end, we constructed a dual-task paradigm that consists of a primary task of sentences-in-noise recognition and a secondary task of simple visual colored-shape matching. Research objective was to develop a clinical paradigm for the assessment of LE in middle-aged adults. Participants were 17 middle-aged adults (mean age of 52.81 years) and 23 young adults (mean age of 24.90 years). All participants had normal hearing according to age. Speech stimuli consisted of the Hebrew Matrix sentences in noise test. SRTn was obtained for 80% correct identification. Visual stimuli were colored geometric shapes. Outcome measures were obtained initially for each task separately, to establish performance ability, and then obtained simultaneously. Reaction time and accuracy in the secondary task were the defined metrics for LE. Results: LE was indicated for both groups, however, was more pronounced in the middle-aged, manifested in the visual accuracy and reaction time metrics. Both groups maintained the 80% correct recognition-in-noise in the dual-task, however, the middle-aged group necessitated a better SNR of 1.4dB than the normal hearing group. Moreover, the middle-aged group was taxed in a greater prolongation of reaction time, in order to uphold the correct recognition. Conclusion: a dual-task paradigm consisting of sentences-in-noise primary task combined with a simple secondary task successfully showed different manifestations of LE in middle-aged adults compared to young adults, thus approximating the use of such a paradigm in a clinical setting.

Dichotic listening performance and effort as a function of spectral resolution and interaural symmetry

One potential benefit of bilateral cochlear implants is reduced listening effort in speech-on-speech masking situations. However, the symmetry of the input across ears, possibly related to spectral resolution, could impact binaural benefits. Fifteen young adults with normal hearing performed digit recall with target and interfering digits presented to separate ears and attention directed to the target ear. Recall accuracy and pupil size over time (used as an index of listening effort) were measured for unprocessed, 16-channel vocoded, and 4-channel vocoded digits. Recall accuracy was significantly lower for dichotic (with interfering digits) than for monotic listening. Dichotic recall accuracy was highest when the target was less degraded and the interferer was more degraded. With matched target and interferer spectral resolution, pupil dilation was lower with more degradation. Pupil dilation grew more shallowly over time when the interferer had more degradation. Overall, interferer spectral resolution more strongly affected listening effort than target spectral resolution. These results suggest that interfering speech both lowers performance and increases listening effort, and that the relative spectral resolution of target and interferer affect the listening experience. Ignoring a clearer interferer is more effortful.

The role of spatial separation of two talkers' auditory stimuli in the listener's memory of running speech: listening effort in a non-noisy conversational setting

OBJECTIVE

To investigate the role of the spatial position of conversing talkers, that is, spatially separated or co-located, in the listener's short-term memory of running speech and listening effort.

DESIGN

In two experiments (between-subject), participants underwent a dual-task paradigm, including a listening (primary) task wherein male and female talkers spoke coherent texts. Talkers were either spatially separated or co-located (within-subject). As a secondary task, visually presented tasks were used. Experiment I involved a number-judgement task, and Experiment II entailed switching between number and letter-judgement task.

STUDY SAMPLE

Twenty-four young adults who reported normal hearing and normal or corrected to normal vision participated in each experiment. They were all students from the RWTH Aachen University.

RESULTS

In both experiments, similar short-term memory performance of running speech was found independently of talkers being spatially separated or co-located. Performance in the secondary tasks, however, differed between these two talkers' auditory stimuli conditions, indicating that spatially separated talkers imposed reduced listening effort compared to their co-location.

CONCLUSION

The findings indicated that auditory-perceptive information, such as the spatial position of talkers, plays a role in higher-level auditory cognition, that is, short-term memory of running speech, even when listening in quiet.

Speech perception in noise: Masking and unmasking

Speech perception is essential for daily communication. Background noise or concurrent talkers, on the other hand, can make it challenging for listeners to track the target speech (i.e., cocktail party problem). The present study reviews and compares existing findings on speech perception and unmasking in cocktail party listening environments in English and Mandarin Chinese. The review starts with an introduction section followed by related concepts of auditory masking. The next two sections review factors that release speech perception from masking in English and Mandarin Chinese, respectively. The last section presents an overall summary of the findings with comparisons between the two languages. Future research directions with respect to the difference in literature on the reviewed topic between the two languages are also discussed.

Additivity of segregation cues in simulated cocktail-party listening

An approach is borrowed from Measurement Theory [Krantz et al. (1971). Foundations of Measurement (Academic, New York), Vol. 1] to evaluate the interaction of voice fundamental frequency and spatial cues in the segregation of talkers in simulated cocktail-party listening. The goal is to find a mathematical expression whereby the combined effect of cues can be simply related to their individual effects. On each trial, the listener judged whether an interleaved sequence of four vowel triplets (heard over headphones) were spoken by the same (MMM) or different (FMF) talkers. The talkers had nominally different fundamental frequencies and spoke from nominally different locations (simulated using head-related transfer functions). Natural variation in these cues was simulated by adding a small, random perturbation to the nominal values independently for each vowel on each trial. Psychometric functions (PFs) relating d' performance to the difference in nominal values were obtained for the cues presented individually and in combination. The results revealed a synergistic interaction of cues wherein the PFs for cues presented in combination exceeded the simple vector sum of the PFs for the cues presented individually. The results are discussed in terms of their implications for possible emergent properties of cues affecting performance in simulated cocktail-party listening.

Hearing and speech processing in midlife

Middle-aged adults often report a decline in their ability to understand speech in adverse listening situations. However, there has been relatively little research devoted to identifying how early aging affects speech processing, as the majority of investigations into senescent changes in speech understanding compare performance in groups of young and older adults. This paper provides an overview of research on hearing and speech perception in middle-aged adults. Topics covered include both objective and subjective (self-perceived) hearing and speech understanding, listening effort, and audiovisual speech perception. This review ends with justification for future research needed to define the nature, consequences, and remediation of hearing problems in middle-aged adults.

Prediction of individual speech recognition performance in complex listening conditions

This study examined how well individual speech recognition thresholds in complex listening scenarios could be predicted by a current binaural speech intelligibility model. Model predictions were compared with experimental data measured for seven normal-hearing and 23 hearing-impaired listeners who differed widely in their degree of hearing loss, age, as well as performance in clinical speech tests. The experimental conditions included two masker types (multi-talker or two-talker maskers), and two spatial conditions (maskers co-located with the frontal target or symmetrically separated from the target). The results showed that interindividual variability could not be well predicted by a model including only individual audiograms. Predictions improved when an additional individual "proficiency factor" was derived from one of the experimental conditions or a standard speech test. Overall, the current model can predict individual performance relatively well (except in conditions high in informational masking), but the inclusion of age-related factors may lead to even further improvements.

Energetic and Informational Components of Speech-on-Speech Masking in Binaural Speech Intelligibility and Perceived Listening Effort

Speech perception in complex sound fields can greatly benefit from different unmasking cues to segregate the target from interfering voices. This study investigated the role of three unmasking cues (spatial separation, gender differences, and masker time reversal) on speech intelligibility and perceived listening effort in normal-hearing listeners. Speech intelligibility and categorically scaled listening effort were measured for a female target talker masked by two competing talkers with no unmasking cues or one to three unmasking cues. In addition to natural stimuli, all measurements were also conducted with glimpsed speech-which was created by removing the time-frequency tiles of the speech mixture in which the maskers dominated the mixture-to estimate the relative amounts of informational and energetic masking as well as the effort associated with source segregation. The results showed that all unmasking cues as well as glimpsing improved intelligibility and reduced listening effort and that providing more than one cue was beneficial in overcoming informational masking. The reduction in listening effort due to glimpsing corresponded to increases in signal-to-noise ratio of 8 to 18 dB, indicating that a significant amount of listening effort was devoted to segregating the target from the maskers. Furthermore, the benefit in listening effort for all unmasking cues extended well into the range of positive signal-to-noise ratios at which speech intelligibility was at ceiling, suggesting that listening effort is a useful tool for evaluating speech-on-speech masking conditions at typical conversational levels.

Benefit of binaural listening as revealed by speech intelligibility and listening effort

In contrast to the well-known benefits for speech intelligibility, the advantage afforded by binaural stimulus presentation for reducing listening effort has not been thoroughly examined. This study investigated spatial release of listening effort and its relation to binaural speech intelligibility in listeners with normal hearing. Psychometric functions for speech intelligibility of a frontal target talker masked by a stationary speech-shaped noise were estimated for several different noise azimuths, different degrees of reverberation, and by maintaining only interaural level or time differences. For each of these conditions, listening effort was measured using a categorical scaling procedure. The results revealed that listening effort was significantly reduced when target and masker were spatially separated in anechoic conditions. This effect extended well into the range of signal-to-noise ratios (SNRs) in which speech intelligibility was at ceiling, and disappeared only at the highest SNRs. In reverberant conditions, spatial release from listening effort was observed for high, but not low, direct-to-reverberant ratios. The findings suggest that listening effort assessment can be a useful method for revealing the benefits of spatial separation of sources under realistic listening conditions comprising favorable SNRs and low reverberation, which typically are not apparent by other means.

Behavioral Assessment of Listening Effort Using a Dual-Task Paradigm

Published investigations (n = 29) in which a dual-task experimental paradigm was employed to measure listening effort during speech understanding in younger and older adults were reviewed. A summary of the main findings reported in the articles is provided with respect to the participants’ age-group and hearing status. Effects of different signal characteristics, such as the test modality, on dual-task outcomes are evaluated, and associations with cognitive abilities and self-report measures of listening effort are described. Then, several procedural issues associated with the use of dual-task experiment paradigms are discussed. Finally, some issues that warrant future research are addressed. The review revealed large variability in the dual-task experimental paradigms that have been used to measure the listening effort expended during speech understanding. The differences in experimental procedures used across studies make it difficult to draw firm conclusions concerning the optimal choice of dual-task paradigm or the sensitivity of specific paradigms to different types of experimental manipulations. In general, the analysis confirmed that dual-task paradigms have been used successfully to measure differences in effort under different experimental conditions, in both younger and older adults. Several research questions that warrant further investigation in order to better understand and characterize the intricacies of dual-task paradigms were identified.

Effects of Hearing Impairment and Hearing Aid Amplification on Listening Effort: A Systematic Review

OBJECTIVES

To undertake a systematic review of available evidence on the effect of hearing impairment and hearing aid amplification on listening effort. Two research questions were addressed: Q1) does hearing impairment affect listening effort? and Q2) can hearing aid amplification affect listening effort during speech comprehension?

DESIGN

English language articles were identified through systematic searches in PubMed, EMBASE, Cinahl, the Cochrane Library, and PsycINFO from inception to August 2014. References of eligible studies were checked. The Population, Intervention, Control, Outcomes, and Study design strategy was used to create inclusion criteria for relevance. It was not feasible to apply a meta-analysis of the results from comparable studies. For the articles identified as relevant, a quality rating, based on the 2011 Grading of Recommendations Assessment, Development, and Evaluation Working Group guidelines, was carried out to judge the reliability and confidence of the estimated effects.

RESULTS

The primary search produced 7017 unique hits using the keywords: hearing aids OR hearing impairment AND listening effort OR perceptual effort OR ease of listening. Of these, 41 articles fulfilled the Population, Intervention, Control, Outcomes, and Study design selection criteria of: experimental work on hearing impairment OR hearing aid technologies AND listening effort OR fatigue during speech perception. The methods applied in those articles were categorized into subjective, behavioral, and physiological assessment of listening effort. For each study, the statistical analysis addressing research question Q1 and/or Q2 was extracted. In seven articles more than one measure of listening effort was provided. Evidence relating to Q1 was provided by 21 articles that reported 41 relevant findings. Evidence relating to Q2 was provided by 27 articles that reported 56 relevant findings. The quality of evidence on both research questions (Q1 and Q2) was very low, according to the Grading of Recommendations Assessment, Development, and Evaluation Working Group guidelines. We tested the statistical evidence across studies with nonparametric tests. The testing revealed only one consistent effect across studies, namely that listening effort was higher for hearing-impaired listeners compared with normal-hearing listeners (Q1) as measured by electroencephalographic measures. For all other studies, the evidence across studies failed to reveal consistent effects on listening effort.

CONCLUSION

In summary, we could only identify scientific evidence from physiological measurement methods, suggesting that hearing impairment increases listening effort during speech perception (Q1). There was no scientific, finding across studies indicating that hearing aid amplification decreases listening effort (Q2). In general, there were large differences in the study population, the control groups and conditions, and the outcome measures applied between the studies included in this review. The results of this review indicate that published listening effort studies lack consistency, lack standardization across studies, and have insufficient statistical power. The findings underline the need for a common conceptual framework for listening effort to address the current shortcomings.

The Spatial Release of Cognitive Load in Cocktail Party Is Determined by the Relative Levels of the Talkers

In a multi-talker situation, spatial separation between talkers reduces cognitive processing load: this is the "spatial release of cognitive load". The present study investigated the role played by the relative levels of the talkers on this spatial release of cognitive load. During the experiment, participants had to report the speech emitted by a target talker in the presence of a concurrent masker talker. The spatial separation (0° and 120° angular distance in azimuth) and the relative levels of the talkers (adverse, intermediate, and favorable target-to-masker ratio) were manipulated. The cognitive load was assessed with a prefrontal functional near-infrared spectroscopy. Data from 14 young normal-hearing listeners revealed that the target-to-masker ratio had a direct impact on the spatial release of cognitive load. Spatial separation significantly reduced the prefrontal activity only for the intermediate target-to-masker ratio and had no effect on prefrontal activity for the favorable and the adverse target-to-masker ratios. Therefore, the relative levels of the talkers might be a key point to determine the spatial release of cognitive load and more specifically the prefrontal activity induced by spatial cues in multi-talker situations.

Aging and the effect of target-masker alignment

Similarity between target and competing speech messages plays a large role in how easy or difficult it is to understand messages of interest. Much research on informational masking has used highly aligned target and masking utterances that are very similar semantically and syntactically. However, listeners rarely encounter situations in real life where they must understand one sentence in the presence of another (or more than one) highly aligned, syntactically similar competing sentence(s). The purpose of the present study was to examine the effect of syntactic/semantic similarity of target and masking speech in different spatial conditions among younger, middle-aged, and older adults. The results of this experiment indicate that differences in speech recognition between older and younger participants were largest when the masker surrounded the target and was more similar to the target, especially at more adverse signal-to-noise ratios. Differences among listeners and the effect of similarity were much less robust, and all listeners were relatively resistant to masking, when maskers were located on one side of the target message. The present results suggest that previous studies using highly aligned stimuli may have overestimated age-related speech recognition problems.