Listener characteristics differentially affect self-reported and physiological measures of effort associated with two challenging listening conditions

A Multimodal Approach to Measuring Listening Effort: A Systematic Review on the Effects of Auditory Task Demand on Physiological Measures and Their Relationship

OBJECTIVES

Listening effort involves the mental effort required to perceive an auditory stimulus, for example in noisy environments. Prolonged increased listening effort, for example due to impaired hearing ability, may increase risk of health complications. It is therefore important to identify valid and sensitive measures of listening effort. Physiological measures have been shown to be sensitive to auditory task demand manipulations and are considered to reflect changes in listening effort. Such measures include pupil dilation, alpha power, skin conductance level, and heart rate variability. The aim of the current systematic review was to provide an overview of studies to listening effort that used multiple physiological measures. The two main questions were: (1) what is the effect of changes in auditory task demand on simultaneously acquired physiological measures from various modalities? and (2) what is the relationship between the responses in these physiological measures?

DESIGN

Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, relevant articles were sought in PubMed, PsycInfo, and Web of Science and by examining the references of included articles. Search iterations with different combinations of psychophysiological measures were performed in conjunction with listening effort-related search terms. Quality was assessed using the Appraisal Tool for Cross-Sectional Studies.

RESULTS

A total of 297 articles were identified from three databases, of which 27 were included. One additional article was identified from reference lists. Of the total 28 included articles, 16 included an analysis regarding the relationship between the physiological measures. The overall quality of the included studies was reasonable.

CONCLUSIONS

The included studies showed that most of the physiological measures either show no effect to auditory task demand manipulations or a consistent effect in the expected direction. For example, pupil dilation increased, pre-ejection period decreased, and skin conductance level increased with increasing auditory task demand. Most of the relationships between the responses of these physiological measures were nonsignificant or weak. The physiological measures varied in their sensitivity to auditory task demand manipulations. One of the identified knowledge gaps was that the included studies mostly used tasks with high-performance levels, resulting in an underrepresentation of the physiological changes at lower performance levels. This makes it difficult to capture how the physiological responses behave across the full psychometric curve. Our results support the Framework for Understanding Effortful Listening and the need for a multimodal approach to listening effort. We furthermore discuss focus points for future studies.

Effects of hearing acuity on psychophysiological responses to effortful speech perception

In recent studies, psychophysiological measures have been used as markers of listening effort, but there is limited research on the effect of hearing loss on such measures. The aim of the current study was to investigate the effect of hearing acuity on physiological responses and subjective measures acquired during different levels of listening demand, and to investigate the relationship between these measures. A total of 125 participants (37 males and 88 females, age range 37-72 years, pure-tone average hearing thresholds at the best ear between -5.0 to 68.8 dB HL and asymmetry between ears between 0.0 and 87.5 dB) completed a listening task. A speech reception threshold (SRT) test was used with target sentences spoken by a female voice masked by male speech. Listening demand was manipulated using three levels of intelligibility: 20 % correct speech recognition, 50 %, and 80 % (IL20 %/IL50 %/IL80 %, respectively). During the task, peak pupil dilation (PPD), heart rate (HR), pre-ejection period (PEP), respiratory sinus arrhythmia (RSA), and skin conductance level (SCL) were measured. For each condition, subjective ratings of effort, performance, difficulty, and tendency to give up were also collected. Linear mixed effects models tested the effect of intelligibility level, hearing acuity, hearing asymmetry, and tinnitus complaints on the physiological reactivity (compared to baseline) and subjective measures. PPD and PEP reactivity showed a non-monotonic relationship with intelligibility level, but no such effects were found for HR, RSA, or SCL reactivity. Participants with worse hearing acuity had lower PPD at all intelligibility levels and showed lower PEP baseline levels. Additionally, PPD and SCL reactivity were lower for participants who reported suffering from tinnitus complaints. For IL80 %, but not IL50 % or IL20 %, participants with worse hearing acuity rated their listening effort to be relatively high compared to participants with better hearing. The reactivity of the different physiological measures were not or only weakly correlated with each other. Together, the results suggest that hearing acuity may be associated with altered sympathetic nervous system (re)activity. Research using psychophysiological measures as markers of listening effort to study the effect of hearing acuity on such measures are best served by the use of the PPD and PEP.

Input-related demands: vocoded sentences evoke different pupillometrics and subjective listening effort than sentences in speech-shaped noise

OBJECTIVES

The Framework for Effortful Listening (FUEL) suggests five input-related demands can alter listening effort: source, transmission, listener, message and context factors. We hypothesised that vocoded sentences represented a source factor degradation and sentences in speech-shaped noise represented a transmission factor degradation. We used pupillometry and a subjective scale to examine our hypothesis.

DESIGN

Participants listened to vocoded sentences and sentences in speech-shaped noise at several difficulty levels designed to produce similar word recognition abilities; they also listened to unprocessed sentences. Within-participant pupillometrics and subjective listening effort were analysed. Post-hoc analyses were performed to examine if word recognition accuracy differentially influenced pupil responses.

STUDY SAMPLES

Twenty young adults with normal hearing.

RESULTS

Baseline pupil diameter was significantly smaller, peak pupil dilation was significantly larger, peak pupil dilation latency was significantly shorter, and subjective listening effort was significantly greater for the vocoded sentences than the sentences-in-noise. Word recognition ability also affected pupillometrics, but only for the vocoded sentences.

CONCLUSIONS

Our findings suggest that source factor degradations result in greater listening effort than transmission factor degradations. Future research should address how clinical interventions tailored towards different input-related demands may lead to reduced listening effort and improve patient outcomes.

The Influence of Sensorineural Hearing Loss on the Relationship Between the Perception of Speech in Noise and Dysarthric Speech

PURPOSE

The ability to understand speech under adverse listening conditions is highly variable across listeners. Despite this, studies have found that listeners with normal hearing display consistency in their ability to perceive speech across different types of degraded speech, suggesting that, for at least these listeners, global skills may be involved in navigating the ambiguity in speech signals. However, there are substantial differences in the perceptual challenges faced by listeners with normal and impaired hearing. This study examines whether listeners with sensorineural hearing loss demonstrate the same type of consistency as normal-hearing listeners when processing neurotypical (i.e., control) speech that has been degraded by external noise and speech that is neurologically degraded such as dysarthria.

METHOD

Listeners with normal hearing (n = 31) and listeners with sensorineural hearing loss (n = 36) completed an intelligibility task with neurotypical speech in noise and with dysarthric speech in quiet.

RESULTS

Findings were consistent with previous work demonstrating a relationship between the ability to perceive neurotypical speech in noise and dysarthric speech for listeners with normal hearing, albeit at a higher intelligibility level than previously observed. This relationship was also observed for listeners with hearing loss, although listeners with more severe hearing losses performed better with dysarthric speech than with neurotypical speech in noise.

CONCLUSIONS

This study demonstrated a high level of consistency in intelligibility performance for listeners across two different types of degraded speech, even when those listeners were further challenged by the presence of sensorineural hearing loss. Clinical implications for both listeners with hearing loss and their communication partners with dysarthria are discussed.

Individual characteristics moderate listening effort in noisy classrooms

Comprehending the teacher's message when other students are chatting is challenging. Even though the sound environment is the same for a whole class, differences in individual performance can be observed, which might depend on a variety of personal factors and their specific interaction with the listening condition. This study was designed to explore the role of individual characteristics (reading comprehension, inhibitory control, noise sensitivity) when primary school children perform a listening comprehension task in the presence of a two-talker masker. The results indicated that this type of noise impairs children's accuracy, effort, and motivation during the task. Its specific impact depended on the level and was modulated by the child's characteristics. In particular, reading comprehension was found to support task accuracy, whereas inhibitory control moderated the effect of listening condition on the two measures of listening effort included in the study (response time and self-ratings), even though with a different pattern of association. A moderation effect of noise sensitivity on perceived listening effort was also observed. Understanding the relationship between individual characteristics and classroom sound environment has practical implications for the acoustic design of spaces promoting students' well-being, and supporting their learning performance.

Combining Multiple Psychophysiological Measures of Listening Effort: Challenges and Recommendations

About one-third of all recently published studies on listening effort have used at least one physiological measure, providing evidence of the popularity of such measures in listening effort research. However, the specific measures employed, as well as the rationales used to justify their inclusion, vary greatly between studies, leading to a literature that is fragmented and difficult to integrate. A unified approach that assesses multiple psychophysiological measures justified by a single rationale would be preferable because it would advance our understanding of listening effort. However, such an approach comes with a number of challenges, including the need to develop a clear definition of listening effort that links to specific physiological measures, customized equipment that enables the simultaneous assessment of multiple measures, awareness of problems caused by the different timescales on which the measures operate, and statistical approaches that minimize the risk of type-I error inflation. This article discusses in detail the various obstacles for combining multiple physiological measures in listening effort research and provides recommendations on how to overcome them.

The effect of telepractice on vocal turn-taking between a provider, children with cochlear implants, and caregivers: A preliminary report

OBJECTIVES

To examine the effect of telepractice on vocal turn-taking between one clinical provider and children with cochlear implants and their caregivers during child-centered auditory rehabilitation intervention.

METHODS

Seven dyads of children with cochlear implants (mean age 4:11 years) and their hearing mothers and one speech-language pathologist participated together in a telepractice session and an in-person intervention session. Dependent variables were vocalization rate, turn taking rate, rate of speech overlap per second, and between-speaker pause duration.

RESULTS

The speech-language pathologist and children had lower rates of vocalization in the telepractice session than the in-person session. However, maternal vocalization rate was higher in the telepractice than in-person session. The rate of turn-taking between the provider and children was lower in telepractice than in-person sessions but the rate of turn taking between mothers and children was higher in telepractice than in-person sessions. Between-speaker pause duration between children and the provider and between mothers and children was longer in telepractice than in-person sessions. Rate of speech overlap did not vary significantly by session type.

DISCUSSION

The quantity and temporal characteristics of vocal turn-taking were impacted by remote communication during tele-intervention suggesting a potential increase in the cognitive effort required of participants.

Be Quiet! Effects of Competing Speakers and Individual Characteristics on Listening Comprehension for Primary School Students

Students learn in noisy classrooms, where the main sources of noise are their own voices. In this sound environment, students are not equally at risk from background noise interference during lessons, due to the moderation effect of the individual characteristics on the listening conditions. This study investigates the effect of the number of competing speakers on listening comprehension and whether this is modulated by selective attention skills, working memory, and noise sensitivity. Seventy-one primary school students aged 10 to 13 years completed a sentence comprehension task in three listening conditions: quiet, two competing speakers, and four competing speakers. Outcome measures were accuracy, listening effort (response times and self-reported), motivation, and confidence in completing the task. Individual characteristics were assessed in quiet. Results showed that the number of competing speakers has no direct effects on the task, whilst the individual characteristics were found to moderate the effect of the listening conditions. Selective attention moderated the effects on accuracy and response times, working memory on motivation, and noise sensitivity on both perceived effort and confidence. Students with low cognitive abilities and high noise sensitivity were found to be particularly at risk in the condition with two competing speakers.

Sensory-Processing Sensitivity Predicts Fatigue From Listening, But Not Perceived Effort, in Young and Older Adults

PURPOSE

Listening-related fatigue is a potential negative consequence of challenges experienced during everyday listening and may disproportionately affect older adults. Contrary to expectation, we recently found that increased reports of listening-related fatigue were associated with better performance on a dichotic listening task. However, this link was found only in individuals who reported heightened sensitivity to a variety of physical, social, and emotional stimuli (i.e., increased "sensory-processing sensitivity" [SPS]). This study examined whether perceived effort may underlie the link between performance and fatigue.

METHOD

Two hundred six young adults, aged 18-30 years (Experiment 1), and 122 older adults, aged 60-80 years (Experiment 2), performed a dichotic listening task and were administered a series of questionnaires including the NASA Task Load Index of perceived effort, the Vanderbilt Fatigue Scale (measuring daily life listening-related fatigue), and the Highly Sensitive Person Scale (measuring SPS). Both experiments were completed online.

RESULTS

SPS predicted listening-related fatigue, but perceived effort during the listening task was not associated with SPS or listening-related fatigue in either age group. We were also unable to replicate the interaction between dichotic listening performance and SPS in either group. Exploratory analyses revealed contrasting effects of age; older adults found the dichotic listening task more effortful but indicated lower overall fatigue.

CONCLUSIONS

These findings suggest that SPS is a better predictor of listening-related fatigue than performance or effort ratings on a dichotic listening task. SPS may be an important factor in determining an individual's likelihood of experiencing listening-related fatigue irrespective of hearing or cognitive ability.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21893013.

Exploring the Correlations Between Measures of Listening Effort in Adults and Children: A Systematic Review with Narrative Synthesis

Listening effort (LE) describes the cognitive resources needed to process an auditory message. Our understanding of this notion remains in its infancy, hindering our ability to appreciate how it impacts individuals with hearing impairment effectively. Despite the myriad of proposed measurement tools, a validated method remains elusive. This is complicated by the seeming lack of association between tools demonstrated via correlational analyses. This review aims to systematically review the literature relating to the correlational analyses between different measures of LE. Five databases were used- PubMed, Cochrane, EMBASE, PsychINFO, and CINAHL. The quality of the evidence was assessed using the GRADE criteria and risk of bias with ROBINS-I/GRADE tools. Each statistically significant analysis was classified using an approved system for medical correlations. The final analyses included 48 papers, equating to 274 correlational analyses, of which 99 reached statistical significance (36.1%). Within these results, the most prevalent classifications were poor or fair. Moreover, when moderate or very strong correlations were observed, they tended to be dependent on experimental conditions. The quality of evidence was graded as very low. These results show that measures of LE are poorly correlated and supports the multi-dimensional concept of LE. The lack of association may be explained by considering where each measure operates along the effort perception pathway. Moreover, the fragility of significant correlations to specific conditions further diminishes the hope of finding an all-encompassing tool. Therefore, it may be prudent to focus on capturing the consequences of LE rather than the notion itself.

Effect of telepractice on pediatric cochlear implant users and provider vowel space: A preliminary report

Clear speaking styles are goal-oriented modifications in which talkers adapt acoustic-phonetic characteristics of speech to compensate for communication challenges. Do children with hearing loss and a clinical provider modify speech characteristics during telepractice to adjust for remote communication? The study examined the effect of telepractice (tele-) on vowel production in seven (mean age 4:11 years, SD 1:2 years) children with cochlear implants (CIs) and a provider. The first (F1) and second (F2) formant frequencies of /i/, /ɑ/, and /u/ vowels were measured in child and provider speech during one in-person and one tele-speech-language intervention, order counterbalanced. Child and provider vowel space areas (VSA) were calculated. The results demonstrated an increase in F2 formant frequency for /i/ vowel in child and provider speech and an increase in F1 formant frequency for /ɑ/ vowel in the provider speech during tele- compared to in-person intervention. An expansion of VSA was found in child and provider speech in tele- compared to in-person intervention. In children, the earlier age of CI activation was associated with larger VSA in both tele- and in-person intervention. The results suggest that the children and the provider adjust vowel articulation in response to remote communication during telepractice.

Beyond Speech Intelligibility: Quantifying Behavioral and Perceived Listening Effort in Response to Dysarthric Speech

PURPOSE

This study investigated whether listener processing of dysarthric speech requires the recruitment of more cognitive resources (i.e., higher levels of listening effort) than neurotypical speech. We also explored relationships between behavioral listening effort, perceived listening effort, and objective measures of word transcription accuracy.

METHOD

A word recall paradigm was used to index behavioral listening effort. The primary task involved word transcription, whereas a memory task involved recalling words from previous sentences. Nineteen listeners completed the paradigm twice, once while transcribing dysarthric speech and once while transcribing neurotypical speech. Perceived listening effort was rated using a visual analog scale.

RESULTS

Results revealed significant effects of dysarthria on the likelihood of correct word recall, indicating that the transcription of dysarthric speech required higher levels of behavioral listening effort relative to neurotypical speech. There was also a significant relationship between transcription accuracy and measures of behavioral listening effort, such that listeners who were more accurate in understanding dysarthric speech exhibited smaller changes in word recall when listening to dysarthria. The subjective measure of perceived listening effort did not have a statistically significant correlation with measures of behavioral listening effort or transcription accuracy.

CONCLUSIONS

Results suggest that cognitive resources, particularly listeners' working memory capacity, are more taxed when deciphering dysarthric versus neurotypical speech. An increased demand on these resources may affect a listener's ability to remember aspects of their conversations with people with dysarthria, even when the speaker is fully intelligible.

Adding noise is a confounded nuisance

A wide variety of research and clinical assessments involve presenting speech stimuli in the presence of some kind of noise. Here, I selectively review two theoretical perspectives and discuss ways in which these perspectives may help researchers understand the consequences for listeners of adding noise to a speech signal. I argue that adding noise changes more about the listening task than merely making the signal more difficult to perceive. To fully understand the effects of an added noise on speech perception, we must consider not just how much the noise affects task difficulty, but also how it affects all of the systems involved in understanding speech: increasing message uncertainty, modifying attentional demand, altering affective response, and changing motivation to perform the task.

Quantifying the Effects of Motivation on Listening Effort: A Systematic Review and Meta-Analysis

Motivation influences the amount of listening effort (LE) exerted or experienced under challenging conditions, such as in high-noise environments. This systematic review and meta-analysis is the first to quantify the effects of motivation on LE. The review was pre-registered in PROSPERO, and performed in accordance with PRISMA guidelines. Eligible studies examined the influence of motivation or individual traits (related to motivation) on LE in adults. Motivational factors, coded as independent variables, included financial reward, evaluative threat, perceived competence, feedback, and individual traits. LE outcomes were categorized as subjective, behavioral, or physiological. The quality of evidence was assessed using an adaptation of the Cochrane Collaboration Risk of Bias Tool. Nested random-effects meta-analyses were performed to quantify and compare the influence of motivational factors across LE outcomes. After assessing 3,532 records, 48 studies met the inclusion criteria and 43 were included in the meta-analyses. Risk of bias was high, for example, many studies lacked sample size justification. Motivational factors had a small-to-medium effect (mean Cohen's d = 0.34, range: 0.11-0.72) on LE. When LE outcomes were considered collectively, an external manipulation of motivation (perceived competence) produced a larger mean effect size compared with individual traits. Some combinations of motivational factors and LE outcomes produced more robust effects than others, for example, evaluative threat and subjective LE outcomes. Although wide prediction intervals and high risk of bias mean that significant positive effects cannot be guaranteed, these findings provide useful guidance on the selection of motivational factors and LE outcomes for future research.

Personality captures dissociations of subjective versus objective hearing in noise

Acoustic noise is pervasive in human environments. Some individuals are more tolerant to noise than others. We demonstrate the explanatory potential of Big-5 personality traits neuroticism (being emotionally unstable) and extraversion (being enthusiastic, outgoing) on subjective self-report and objective psycho-acoustic metrics of hearing in noise in two samples (total N = 1103). Under statistical control for demographics and in agreement with pre-registered hypotheses, lower neuroticism and higher extraversion independently explained superior self-reported noise resistance, speech-hearing ability and acceptable background noise levels. Surprisingly, objective speech-in-noise recognition instead increased with higher levels of neuroticism. In turn, the bias in subjectively overrating one's own hearing in noise decreases with higher neuroticism but increases with higher extraversion. Of benefit to currently underspecified frameworks of hearing in noise and tailored audiological treatments, these results show that personality explains inter-individual differences in coping with acoustic noise, which is a ubiquitous source of distraction and a health hazard.

Psychophysiological responses to potentially annoying heating, ventilation, and air conditioning noise during mentally demanding work

Exposure to noise-or unwanted sound-is considered a major public health issue in the United States and internationally. Previous work has shown that even acute noise exposure can influence physiological response in humans and that individuals differ markedly in their susceptibility to noise. Recent research also suggests that specific acoustic properties of noise may have distinct effects on human physiological response. Much of the existing research on physiological response to noise consists of laboratory studies using very simple acoustic stimuli-like white noise or tone bursts-or field studies of longer-term workplace noise exposure that may neglect acoustic properties of the noise entirely. By using laboratory exposure to realistic heating, ventilation, and air conditioning (HVAC) noise, the current study explores the interaction between acoustic properties of annoying noise and individual response to working in occupational noise. This study assessed autonomic response to two acoustically distinct noises while participants performed cognitively demanding work. Results showed that the two HVAC noises affected physiological arousal in different ways. Individual differences in physiological response to noise as a function of noise sensitivity were also observed. Further research is necessary to link specific acoustic characteristics with differential physiological responses in humans.

Social observation increases the cardiovascular response of hearing-impaired listeners during a speech reception task

Certain cardiovascular measures allow for distinction between sympathetic and parasympathetic nervous system activity. Applied during listening, these measures may provide a novel and complementary insight into listening effort. To date, few studies have implemented cardiovascular measures of listening effort and seldom have these included hearing-impaired participants. These studies have generally measured changes in cardiovascular parameters while manipulating environmental factors, such as listening difficulty. Yet, listening effort is also known to be moderated by individual factors, including the importance of performing successfully. In this study, we aimed to manipulate success importance by adding observers to the traditional laboratory set-up. Twenty-nine hearing-impaired participants performed a speech reception task both alone and in the presence of two observers. Auditory stimuli consisted of Danish Hearing in Noise Test (HINT) sentences masked by four-talker babble. Sentences were delivered at two individually adapted signal-to-noise ratios, corresponding to 50 and 80% of sentences correct. We measured change scores, relative to baseline, of pre-ejection period, two indices of heart rate variability, heart rate and blood pressure (systolic, diastolic, and mean arterial pressure). After each condition, participants rated their effort investment, stress, tendency to give up and preference to change the situation to improve audibility. A multivariate analysis revealed that cardiovascular reactivity increased in the presence of the observers, compared to when the task was performed alone. More specifically, systolic, diastolic, and mean arterial blood pressure increased while observed. Interestingly, participants' subjective ratings were sensitive only to intelligibility level, not the observation state. This study was the first to report results from a range of different cardiovascular variables measured from hearing-impaired participants during a speech reception task. Due to the timing of the observers' presence, we were not able to conclusively attribute these physiological changes to being task related. Therefore, instead of representing listening effort, we suggest that the increased cardiovascular response detected during observation reveals increased physiological stress associated with potential evaluation.