The Moderating Effect of Success Importance on the Relationship Between Listening Demand and Listening Effort

A Multimodal Approach to Measuring Listening Effort: A Systematic Review on the Effects of Auditory Task Demand on Physiological Measures and Their Relationship

OBJECTIVES

Listening effort involves the mental effort required to perceive an auditory stimulus, for example in noisy environments. Prolonged increased listening effort, for example due to impaired hearing ability, may increase risk of health complications. It is therefore important to identify valid and sensitive measures of listening effort. Physiological measures have been shown to be sensitive to auditory task demand manipulations and are considered to reflect changes in listening effort. Such measures include pupil dilation, alpha power, skin conductance level, and heart rate variability. The aim of the current systematic review was to provide an overview of studies to listening effort that used multiple physiological measures. The two main questions were: (1) what is the effect of changes in auditory task demand on simultaneously acquired physiological measures from various modalities? and (2) what is the relationship between the responses in these physiological measures?

DESIGN

Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, relevant articles were sought in PubMed, PsycInfo, and Web of Science and by examining the references of included articles. Search iterations with different combinations of psychophysiological measures were performed in conjunction with listening effort-related search terms. Quality was assessed using the Appraisal Tool for Cross-Sectional Studies.

RESULTS

A total of 297 articles were identified from three databases, of which 27 were included. One additional article was identified from reference lists. Of the total 28 included articles, 16 included an analysis regarding the relationship between the physiological measures. The overall quality of the included studies was reasonable.

CONCLUSIONS

The included studies showed that most of the physiological measures either show no effect to auditory task demand manipulations or a consistent effect in the expected direction. For example, pupil dilation increased, pre-ejection period decreased, and skin conductance level increased with increasing auditory task demand. Most of the relationships between the responses of these physiological measures were nonsignificant or weak. The physiological measures varied in their sensitivity to auditory task demand manipulations. One of the identified knowledge gaps was that the included studies mostly used tasks with high-performance levels, resulting in an underrepresentation of the physiological changes at lower performance levels. This makes it difficult to capture how the physiological responses behave across the full psychometric curve. Our results support the Framework for Understanding Effortful Listening and the need for a multimodal approach to listening effort. We furthermore discuss focus points for future studies.

Effects of hearing acuity on psychophysiological responses to effortful speech perception

In recent studies, psychophysiological measures have been used as markers of listening effort, but there is limited research on the effect of hearing loss on such measures. The aim of the current study was to investigate the effect of hearing acuity on physiological responses and subjective measures acquired during different levels of listening demand, and to investigate the relationship between these measures. A total of 125 participants (37 males and 88 females, age range 37-72 years, pure-tone average hearing thresholds at the best ear between -5.0 to 68.8 dB HL and asymmetry between ears between 0.0 and 87.5 dB) completed a listening task. A speech reception threshold (SRT) test was used with target sentences spoken by a female voice masked by male speech. Listening demand was manipulated using three levels of intelligibility: 20 % correct speech recognition, 50 %, and 80 % (IL20 %/IL50 %/IL80 %, respectively). During the task, peak pupil dilation (PPD), heart rate (HR), pre-ejection period (PEP), respiratory sinus arrhythmia (RSA), and skin conductance level (SCL) were measured. For each condition, subjective ratings of effort, performance, difficulty, and tendency to give up were also collected. Linear mixed effects models tested the effect of intelligibility level, hearing acuity, hearing asymmetry, and tinnitus complaints on the physiological reactivity (compared to baseline) and subjective measures. PPD and PEP reactivity showed a non-monotonic relationship with intelligibility level, but no such effects were found for HR, RSA, or SCL reactivity. Participants with worse hearing acuity had lower PPD at all intelligibility levels and showed lower PEP baseline levels. Additionally, PPD and SCL reactivity were lower for participants who reported suffering from tinnitus complaints. For IL80 %, but not IL50 % or IL20 %, participants with worse hearing acuity rated their listening effort to be relatively high compared to participants with better hearing. The reactivity of the different physiological measures were not or only weakly correlated with each other. Together, the results suggest that hearing acuity may be associated with altered sympathetic nervous system (re)activity. Research using psychophysiological measures as markers of listening effort to study the effect of hearing acuity on such measures are best served by the use of the PPD and PEP.

Mood shapes the impact of reward on perceived fatigue from listening

Knowledge of the underlying mechanisms of effortful listening could help to reduce cases of social withdrawal and mitigate fatigue, especially in older adults. However, the relationship between transient effort and longer term fatigue is likely to be more complex than originally thought. Here, we manipulated the presence/absence of monetary reward to examine the role of motivation and mood state in governing changes in perceived effort and fatigue from listening. In an online study, 185 participants were randomly assigned to either a "reward" (n = 91) or "no-reward" (n = 94) group and completed a dichotic listening task along with a series of questionnaires assessing changes over time in perceived effort, mood, and fatigue. Effort ratings were higher overall in the reward group, yet fatigue ratings in that group showed a shallower linear increase over time. Mediation analysis revealed an indirect effect of reward on fatigue ratings via perceived mood state; reward induced a more positive mood state which was associated with reduced fatigue. These results suggest that: (1) listening conditions rated as more "effortful" may be less fatiguing if the effort is deemed worthwhile, and (2) alterations to one's mood state represent a potential mechanism by which fatigue may be elicited during unrewarding listening situations.

Combining Cardiovascular and Pupil Features Using k-Nearest Neighbor Classifiers to Assess Task Demand, Social Context, and Sentence Accuracy During Listening

In daily life, both acoustic factors and social context can affect listening effort investment. In laboratory settings, information about listening effort has been deduced from pupil and cardiovascular responses independently. The extent to which these measures can jointly predict listening-related factors is unknown. Here we combined pupil and cardiovascular features to predict acoustic and contextual aspects of speech perception. Data were collected from 29 adults (mean  =  64.6 years, SD  =  9.2) with hearing loss. Participants performed a speech perception task at two individualized signal-to-noise ratios (corresponding to 50% and 80% of sentences correct) and in two social contexts (the presence and absence of two observers). Seven features were extracted per trial: baseline pupil size, peak pupil dilation, mean pupil dilation, interbeat interval, blood volume pulse amplitude, pre-ejection period and pulse arrival time. These features were used to train k-nearest neighbor classifiers to predict task demand, social context and sentence accuracy. The k-fold cross validation on the group-level data revealed above-chance classification accuracies: task demand, 64.4%; social context, 78.3%; and sentence accuracy, 55.1%. However, classification accuracies diminished when the classifiers were trained and tested on data from different participants. Individually trained classifiers (one per participant) performed better than group-level classifiers: 71.7% (SD  =  10.2) for task demand, 88.0% (SD  =  7.5) for social context, and 60.0% (SD  =  13.1) for sentence accuracy. We demonstrated that classifiers trained on group-level physiological data to predict aspects of speech perception generalized poorly to novel participants. Individually calibrated classifiers hold more promise for future applications.

Impact of SNR, peripheral auditory sensitivity, and central cognitive profile on the psychometric relation between pupillary response and speech performance in CI users

Despite substantial technical advances and wider clinical use, cochlear implant (CI) users continue to report high and elevated listening effort especially under challenging noisy conditions. Among all the objective measures to quantify listening effort, pupillometry is one of the most widely used and robust physiological measures. Previous studies with normally hearing (NH) and hearing-impaired (HI) listeners have shown that the relation between speech performance in noise and listening effort (as measured by peak pupil dilation) is not linear and exhibits an inverted-U shape. However, it is unclear whether the same psychometric relation exists in CI users, and whether individual differences in auditory sensitivity and central cognitive capacity affect this relation. Therefore, we recruited 17 post-lingually deaf CI adults to perform speech-in-noise tasks from 0 to 20 dB SNR with a 4 dB step size. Simultaneously, their pupillary responses and self-reported subjective effort were recorded. To characterize top-down and bottom-up individual variabilities, a spectro-temporal modulation task and a set of cognitive abilities were measured. Clinical word recognition in quiet and Quality of Life (QoL) were also collected. Results showed that at a group level, an inverted-U shape psychometric curve between task difficulty (SNR) and peak pupil dilation (PPD) was not observed. Individual shape of the psychometric curve was significantly associated with some individual factors: CI users with higher clinical word and speech-in-noise recognition showed a quadratic decrease of PPD over increasing SNRs; CI users with better non-verbal intelligence and lower QoL showed smaller average PPD. To summarize, individual differences in CI users had a significant impact on the psychometric relation between pupillary response and task difficulty, hence affecting the interpretation of pupillary response as listening effort (or engagement) at different task difficulty levels. Future research and clinical applications should further characterize the possible effects of individual factors (such as motivation or engagement) in modulating CI users' occurrence of 'tipping point' on their psychometric functions, and develop an individualized method for reliably quantifying listening effort using pupillometry.

Pupil Size Sensitivity to Listening Demand Depends on Motivational State

Motivation plays a role when a listener needs to understand speech under acoustically demanding conditions. Previous work has demonstrated pupil-linked arousal being sensitive to both listening demands and motivational state during listening. It is less clear how motivational state affects the temporal evolution of the pupil size and its relation to subsequent behavior. We used an auditory gap detection task (N = 33) to study the joint impact of listening demand and motivational state on the pupil size response and examine its temporal evolution. Task difficulty and a listener's motivational state were orthogonally manipulated through changes in gap duration and monetary reward prospect. We show that participants' performance decreased with task difficulty, but that reward prospect enhanced performance under hard listening conditions. Pupil size increased with both increased task difficulty and higher reward prospect, and this reward prospect effect was largest under difficult listening conditions. Moreover, pupil size time courses differed between detected and missed gaps, suggesting that the pupil response indicates upcoming behavior. Larger pre-gap pupil size was further associated with faster response times on a trial-by-trial within-participant level. Our results reiterate the utility of pupil size as an objective and temporally sensitive measure in audiology. However, such assessments of cognitive resource recruitment need to consider the individual's motivational state.

Brightening the Study of Listening Effort with Functional Near-Infrared Spectroscopy: A Scoping Review

Listening effort is a long-standing area of interest in auditory cognitive neuroscience. Prior research has used multiple techniques to shed light on the neurophysiological mechanisms underlying listening during challenging conditions. Functional near-infrared spectroscopy (fNIRS) is growing in popularity as a tool for cognitive neuroscience research, and its recent advances offer many potential advantages over other neuroimaging modalities for research related to listening effort. This review introduces the basic science of fNIRS and its uses for auditory cognitive neuroscience. We also discuss its application in recently published studies on listening effort and consider future opportunities for studying effortful listening with fNIRS. After reading this article, the learner will know how fNIRS works and summarize its uses for listening effort research. The learner will also be able to apply this knowledge toward generation of future research in this area.

Combining Multiple Psychophysiological Measures of Listening Effort: Challenges and Recommendations

About one-third of all recently published studies on listening effort have used at least one physiological measure, providing evidence of the popularity of such measures in listening effort research. However, the specific measures employed, as well as the rationales used to justify their inclusion, vary greatly between studies, leading to a literature that is fragmented and difficult to integrate. A unified approach that assesses multiple psychophysiological measures justified by a single rationale would be preferable because it would advance our understanding of listening effort. However, such an approach comes with a number of challenges, including the need to develop a clear definition of listening effort that links to specific physiological measures, customized equipment that enables the simultaneous assessment of multiple measures, awareness of problems caused by the different timescales on which the measures operate, and statistical approaches that minimize the risk of type-I error inflation. This article discusses in detail the various obstacles for combining multiple physiological measures in listening effort research and provides recommendations on how to overcome them.

Previous Mental Load and Incentives Influence Anticipatory Arousal as Indexed by the Baseline Pupil Diameter in a Speech-in-Noise Task

Listening effort and fatigue are common experiences when conversing in noisy environments. Much research has investigated listening effort in relation to listening demand using the speech-in-noise paradigm. Recent conceptualizations of listening effort postulate that mental fatigue should result in decreased arousal and a reluctance to invest further effort, particularly when the effort is not worthwhile. The aim of the study was to investigate the influence of fatigue on listening effort, in interaction with listening demands and motivation. To induce fatigue 30 adults with normal hearing completed a 40-minute long speech-in-noise task ("load sequence"). Pre- and post-load sequence listening effort was probed in easy and hard listening demands (individually adjusted signal-to-noise ratios); with high and low motivation (manipulated with monetary incentives). Subjective effort, estimated performance, and tendency to quit listening were collected using rating scales. Baseline pupil diameter and mean pupil dilation were recorded as indices of anticipatory arousal and objective effort. Self-reported effort and mean pupil dilation were overall larger during hard SNR as compared to easy SNR. Baseline pupil diameter declined from pre- to post-load sequence, suggesting an overall decrease in arousal. Monetary incentives had no influence on the baseline pupil diameter for the easy SNR condition, but for the hard SNR condition larger incentives led to larger baseline pupil diameter. These results suggest that anticipatory arousal may be influenced by fatigue and motivation effects. Models of listening effort should account for the independent influence of motivation and previous load on anticipatory arousal and effort in distinct parameters.

Linking Affective and Hearing Sciences-Affective Audiology

A growing number of health-related sciences, including audiology, have increasingly recognized the importance of affective phenomena. However, in audiology, affective phenomena are mostly studied as a consequence of hearing status. This review first addresses anatomical and functional bidirectional connections between auditory and affective systems that support a reciprocal affect-hearing relationship. We then postulate, by focusing on four practical examples (hearing public campaigns, hearing intervention uptake, thorough hearing evaluation, and tinnitus), that some important challenges in audiology are likely affect-related and that potential solutions could be developed by inspiration from affective science advances. We continue by introducing useful resources from affective science that could help audiology professionals learn about the wide range of affective constructs and integrate them into hearing research and clinical practice in structured and applicable ways. Six important considerations for good quality affective audiology research are summarized. We conclude that it is worthwhile and feasible to explore the explanatory power of emotions, feelings, motivations, attitudes, moods, and other affective processes in depth when trying to understand and predict how people with hearing difficulties perceive, react, and adapt to their environment.

Adding noise is a confounded nuisance

A wide variety of research and clinical assessments involve presenting speech stimuli in the presence of some kind of noise. Here, I selectively review two theoretical perspectives and discuss ways in which these perspectives may help researchers understand the consequences for listeners of adding noise to a speech signal. I argue that adding noise changes more about the listening task than merely making the signal more difficult to perceive. To fully understand the effects of an added noise on speech perception, we must consider not just how much the noise affects task difficulty, but also how it affects all of the systems involved in understanding speech: increasing message uncertainty, modifying attentional demand, altering affective response, and changing motivation to perform the task.

Need for cognition is associated with the interaction of reward and task-load on effort: A verification and extension study

Here, we work to provide nuance around the assumption that people will work for rewards. We examine whether individuals' inherent tendency to mobilize cognitive effort (need for cognition, NFC) moderates this effect. We re-analyzed our existing data to verify an effect reported by Sandra and Otto (2018) regarding the association between NFC and reward-induced cognitive effort expenditure, using a more ecological cognitive task design and adding a psychophysiological measure of effort. Specifically, distinct from their short time course visual task-switching paradigm, we used a relatively long course auditory comprehension task paradigm. We found that, consistent with the original study, increased cognitive effort in response to incentive reward depends on individual differences in cognitive motivation (need for cognition). We also found that, to observe consistent phenomena, different indices of effort (behavioral and psychophysiological) need to be considered when evaluating the relationship between the effort expenditure and cognitive motivation. Pupil dilation showed an advantage over reaction time in revealing mental effort mobilized over a prolonged cognitive task. Our results suggest that assessing cognitive motivation when planning a behavior-change program involving reward feedback for positive performance could help to optimize individuals' effort investment.

Neural Activity during Story Listening Is Synchronized across Individuals Despite Acoustic Masking

Older people with hearing problems often experience difficulties understanding speech in the presence of background sound. As a result, they may disengage in social situations, which has been associated with negative psychosocial health outcomes. Measuring listening (dis)engagement during challenging listening situations has received little attention thus far. We recruit young, normal-hearing human adults (both sexes) and investigate how speech intelligibility and engagement during naturalistic story listening is affected by the level of acoustic masking (12-talker babble) at different signal-to-noise ratios (SNRs). In Experiment 1, we observed that word-report scores were above 80% for all but the lowest SNR (-3 dB SNR) we tested, at which performance dropped to 54%. In Experiment 2, we calculated intersubject correlation (ISC) using EEG data to identify dynamic spatial patterns of shared neural activity evoked by the stories. ISC has been used as a neural measure of participants' engagement with naturalistic materials. Our results show that ISC was stable across all but the lowest SNRs, despite reduced speech intelligibility. Comparing ISC and intelligibility demonstrated that word-report performance declined more strongly with decreasing SNR compared to ISC. Our measure of neural engagement suggests that individuals remain engaged in story listening despite missing words because of background noise. Our work provides a potentially fruitful approach to investigate listener engagement with naturalistic, spoken stories that may be used to investigate (dis)engagement in older adults with hearing impairment.

Quantifying the Effects of Motivation on Listening Effort: A Systematic Review and Meta-Analysis

Motivation influences the amount of listening effort (LE) exerted or experienced under challenging conditions, such as in high-noise environments. This systematic review and meta-analysis is the first to quantify the effects of motivation on LE. The review was pre-registered in PROSPERO, and performed in accordance with PRISMA guidelines. Eligible studies examined the influence of motivation or individual traits (related to motivation) on LE in adults. Motivational factors, coded as independent variables, included financial reward, evaluative threat, perceived competence, feedback, and individual traits. LE outcomes were categorized as subjective, behavioral, or physiological. The quality of evidence was assessed using an adaptation of the Cochrane Collaboration Risk of Bias Tool. Nested random-effects meta-analyses were performed to quantify and compare the influence of motivational factors across LE outcomes. After assessing 3,532 records, 48 studies met the inclusion criteria and 43 were included in the meta-analyses. Risk of bias was high, for example, many studies lacked sample size justification. Motivational factors had a small-to-medium effect (mean Cohen's d = 0.34, range: 0.11-0.72) on LE. When LE outcomes were considered collectively, an external manipulation of motivation (perceived competence) produced a larger mean effect size compared with individual traits. Some combinations of motivational factors and LE outcomes produced more robust effects than others, for example, evaluative threat and subjective LE outcomes. Although wide prediction intervals and high risk of bias mean that significant positive effects cannot be guaranteed, these findings provide useful guidance on the selection of motivational factors and LE outcomes for future research.

Adults with cochlear implants can use prosody to determine the clausal structure of spoken sentences

Speech prosody, including pitch contour, word stress, pauses, and vowel lengthening, can aid the detection of the clausal structure of a multi-clause sentence and this, in turn, can help listeners determine the meaning. However, for cochlear implant (CI) users, the reduced acoustic richness of the signal raises the question of whether CI users may have difficulty using sentence prosody to detect syntactic clause boundaries within sentences or whether this ability is rescued by the redundancy of the prosodic features that normally co-occur at clause boundaries. Twenty-two CI users, ranging in age from 19 to 77 years old, recalled three types of sentences: sentences in which the prosodic pattern was appropriate to the location of a clause boundary within the sentence (congruent prosody), sentences with reduced prosodic information, or sentences in which the location of the clause boundary and the prosodic marking of a clause boundary were placed in conflict. The results showed the presence of congruent prosody to be associated with superior sentence recall and a reduced processing effort as indexed by the pupil dilation. The individual differences in a standard test of word recognition (consonant-nucleus-consonant score) were related to the recall accuracy as well as the processing effort. The outcomes are discussed in terms of the redundancy of the prosodic features, which normally accompany a clause boundary and processing effort.

Effortful listening: Sympathetic activity varies as a function of listening demand but parasympathetic activity does not

Research on listening effort has used various physiological measures to examine the biological correlates of listening effort but a systematic examination of the impact of listening demand on cardiac autonomic nervous system activity is still lacking. The presented study aimed to close this gap by assessing cardiac sympathetic and parasympathetic responses to variations in listening demand. For this purpose, 45 participants performed four speech-in-noise tasks differing in listening demand-manipulated as signal-to-noise ratio varying between +23 dB and -16 dB-while their pre-ejection period and respiratory sinus arrythmia responses were assessed. Cardiac responses showed the expected effect of listening demand on sympathetic activity, but failed to provide evidence for the expected listening demand impact on parasympathetic activity: Pre-ejection period reactivity increased with increasing listening demand across the three possible listening conditions and was low in the very high (impossible) demand condition, whereas respiratory sinus arrythmia did not show this pattern. These findings have two main implications. First, cardiac sympathetic responses seem to be the more sensitive correlate of the impact of task demand on listening effort compared to cardiac parasympathetic responses. Second, very high listening demand may lead to disengagement and correspondingly low effort and reduced cardiac sympathetic response.

Social observation increases the cardiovascular response of hearing-impaired listeners during a speech reception task

Certain cardiovascular measures allow for distinction between sympathetic and parasympathetic nervous system activity. Applied during listening, these measures may provide a novel and complementary insight into listening effort. To date, few studies have implemented cardiovascular measures of listening effort and seldom have these included hearing-impaired participants. These studies have generally measured changes in cardiovascular parameters while manipulating environmental factors, such as listening difficulty. Yet, listening effort is also known to be moderated by individual factors, including the importance of performing successfully. In this study, we aimed to manipulate success importance by adding observers to the traditional laboratory set-up. Twenty-nine hearing-impaired participants performed a speech reception task both alone and in the presence of two observers. Auditory stimuli consisted of Danish Hearing in Noise Test (HINT) sentences masked by four-talker babble. Sentences were delivered at two individually adapted signal-to-noise ratios, corresponding to 50 and 80% of sentences correct. We measured change scores, relative to baseline, of pre-ejection period, two indices of heart rate variability, heart rate and blood pressure (systolic, diastolic, and mean arterial pressure). After each condition, participants rated their effort investment, stress, tendency to give up and preference to change the situation to improve audibility. A multivariate analysis revealed that cardiovascular reactivity increased in the presence of the observers, compared to when the task was performed alone. More specifically, systolic, diastolic, and mean arterial blood pressure increased while observed. Interestingly, participants' subjective ratings were sensitive only to intelligibility level, not the observation state. This study was the first to report results from a range of different cardiovascular variables measured from hearing-impaired participants during a speech reception task. Due to the timing of the observers' presence, we were not able to conclusively attribute these physiological changes to being task related. Therefore, instead of representing listening effort, we suggest that the increased cardiovascular response detected during observation reveals increased physiological stress associated with potential evaluation.

Cortical alpha oscillations in cochlear implant users reflect subjective listening effort during speech-in-noise perception

Listening to speech in noise is effortful for individuals with hearing loss, even if they have received a hearing prosthesis such as a hearing aid or cochlear implant (CI). At present, little is known about the neural functions that support listening effort. One form of neural activity that has been suggested to reflect listening effort is the power of 8–12 Hz (alpha) oscillations measured by electroencephalography (EEG). Alpha power in two cortical regions has been associated with effortful listening—left inferior frontal gyrus (IFG), and parietal cortex—but these relationships have not been examined in the same listeners. Further, there are few studies available investigating neural correlates of effort in the individuals with cochlear implants. Here we tested 16 CI users in a novel effort-focused speech-in-noise listening paradigm, and confirm a relationship between alpha power and self-reported effort ratings in parietal regions, but not left IFG. The parietal relationship was not linear but quadratic, with alpha power comparatively lower when effort ratings were at the top and bottom of the effort scale, and higher when effort ratings were in the middle of the scale. Results are discussed in terms of cognitive systems that are engaged in difficult listening situations, and the implication for clinical translation.

Listening effort and fatigue in native and non-native primary school children

Background noise makes listening effortful and may lead to fatigue. This may compromise classroom learning, especially for children with a non-native background. In the current study, we used pupillometry to investigate listening effort and fatigue during listening comprehension under typical (0 dB signal-to-noise ratio [SNR]) and favorable (+10 dB SNR) listening conditions in 63 Swedish primary school children (7-9 years of age) performing a narrative speech-picture verification task. Our sample comprised both native (n = 25) and non-native (n = 38) speakers of Swedish. Results revealed greater pupil dilation, indicating more listening effort, in the typical listening condition compared with the favorable listening condition, and it was primarily the non-native speakers who contributed to this effect (and who also had lower performance accuracy than the native speakers). Furthermore, the native speakers had greater pupil dilation during successful trials, whereas the non-native speakers showed greatest pupil dilation during unsuccessful trials, especially in the typical listening condition. This set of results indicates that whereas native speakers can apply listening effort to good effect, non-native speakers may have reached their effort ceiling, resulting in poorer listening comprehension. Finally, we found that baseline pupil size decreased over trials, which potentially indicates more listening-related fatigue, and this effect was greater in the typical listening condition compared with the favorable listening condition. Collectively, these results provide novel insight into the underlying dynamics of listening effort, fatigue, and listening comprehension in typical classroom conditions compared with favorable classroom conditions, and they demonstrate for the first time how sensitive this interplay is to language experience.

The effect of monetary reward on listening effort and sentence recognition

Recently we showed that higher reward results in increased pupil dilation during listening (listening effort). Remarkably, this effect was not accompanied with improved speech reception. Still, increased listening effort may reflect more in-depth processing, potentially resulting in a better memory representation of speech. Here, we investigated this hypothesis by also testing the effect of monetary reward on recognition memory performance. Twenty-four young adults performed speech reception threshold (SRT) tests, either hard or easy, in which they repeated sentences uttered by a female talker masked by a male talker. We recorded the pupil dilation response during listening. Participants could earn a high or low reward and the four conditions were presented in a blocked fashion. After each SRT block, participants performed a visual sentence recognition task. In this task, the sentences that were presented in the preceding SRT task were visually presented in random order and intermixed with unfamiliar sentences. Participants had to indicate whether they had previously heard the sentence or not. The SRT and sentence recognition were affected by task difficulty but not by reward. Contrary to our previous results, peak pupil dilation did not reflect effects of reward. However, post-hoc time course analysis (GAMMs) revealed that in the hard SRT task, the pupil response was larger for high than low reward. We did not observe an effect of reward on visual sentence recognition. Hence, the current results provide no conclusive evidence that the effect of monetary reward on the pupil response relates to the memory encoding of speech.

The Principle of Least Effort and Comprehension of Spoken Sentences by Younger and Older Adults

There is considerable evidence that listeners' understanding of a spoken sentence need not always follow from a full analysis of the words and syntax of the utterance. Rather, listeners may instead conduct a superficial analysis, sampling some words and using presumed plausibility to arrive at an understanding of the sentence meaning. Because this latter strategy occurs more often for sentences with complex syntax that place a heavier processing burden on the listener than sentences with simpler syntax, shallow processing may represent a resource conserving strategy reflected in reduced processing effort. This factor may be even more important for older adults who as a group are known to have more limited working memory resources. In the present experiment, 40 older adults (M age = 75.5 years) and 20 younger adults (M age = 20.7) were tested for comprehension of plausible and implausible sentences with a simpler subject-relative embedded clause structure or a more complex object-relative embedded clause structure. Dilation of the pupil of the eye was recorded as an index of processing effort. Results confirmed greater comprehension accuracy for plausible than implausible sentences, and for sentences with simpler than more complex syntax, with both effects amplified for the older adults. Analysis of peak pupil dilations for implausible sentences revealed a complex three-way interaction between age, syntactic complexity, and plausibility. Results are discussed in terms of models of sentence comprehension, and pupillometry as an index of intentional task engagement.

Disentangling listening effort and memory load beyond behavioural evidence: Pupillary response to listening effort during a concurrent memory task

Recent research has demonstrated that pupillometry is a robust measure for quantifying listening effort. However, pupillary responses in listening situations where multiple cognitive functions are engaged and sustained over a period of time remain hard to interpret. This limits our conceptualisation and understanding of listening effort in realistic situations, because rarely in everyday life are people challenged by one task at a time. Therefore, the purpose of this experiment was to reveal the dynamics of listening effort in a sustained listening condition using a word repeat and recall task. Words were presented in quiet and speech-shaped noise at different signal-to-noise ratios (SNR): 0dB, 7dB, 14dB and quiet. Participants were presented with lists of 10 words, and required to repeat each word after its presentation. At the end of the list, participants either recalled as many words as possible or moved on to the next list. Simultaneously, their pupil dilation was recorded throughout the whole experiment. When only word repeating was required, peak pupil dilation (PPD) was bigger in 0dB versus other conditions; whereas when recall was required, PPD showed no difference among SNR levels and PPD in 0dB was smaller than repeat-only condition. Baseline pupil diameter and PPD followed different variation patterns across the 10 serial positions within a block for conditions requiring recall: baseline pupil diameter built up progressively and plateaued in the later positions (but shot up when listeners were recalling the previously heard words from memory); PPD decreased at a pace quicker than in repeat-only condition. The current findings demonstrate that additional cognitive load during a speech intelligibility task could disturb the well-established relation between pupillary response and listening effort. Both the magnitude and temporal pattern of task-evoked pupillary response differ greatly in complex listening conditions, urging for more listening effort studies in complex and realistic listening situations.

Financial reward has differential effects on behavioural and self-report measures of listening effort

OBJECTIVES

To investigate the effects of listening demands and motivation on listening effort (LE) in a novel speech recognition task.

DESIGN

We manipulated listening demands and motivation using vocoded speech and financial reward, respectively, and measured task performance (correct response rate) and indices of LE (response times (RTs), subjective ratings of LE and likelihood of giving up). Effects of inter-individual differences in cognitive skills and personality on task performance and LE were also assessed within the context of the Cognitive Energetics Theory (CET).

STUDY SAMPLE

Twenty-four participants with normal-hearing (age range: 19 - 33 years, 6 male).

RESULTS

High listening demands decreased the correct response rate and increased RTs, self-rated LE and self-rated likelihood of giving up. High financial reward increased subjective LE ratings only. Mixed-effects modelling showed small fixed effects for competitiveness on LE measured using RTs. Small fixed effects were found for cognitive skills (lexical decision RTs and backwards digit span) on LE measured using RTs and correct response rate, respectively.

CONCLUSIONS

The effects of listening demands on LE in the speech recognition task aligned with CET, whereas predictions regarding the influence of motivation, cognitive skills and personality were only partially supported.

The everyday acoustic environment and its association with human heart rate: evidence from real-world data logging with hearing aids and wearables

We investigate the short-term association between multidimensional acoustic characteristics of everyday ambient sound and continuous mean heart rate. We used in-market data from hearing aid users who logged ambient acoustics via smartphone-connected hearing aids and continuous mean heart rate in 5 min intervals from their own wearables. We find that acoustic characteristics explain approximately 4% of the fluctuation in mean heart rate throughout the day. Specifically, increases in ambient sound pressure intensity are significantly related to increases in mean heart rate, corroborating prior laboratory and short-term real-world data. In addition, increases in ambient sound quality-that is, more favourable signal to noise ratios-are associated with decreases in mean heart rate. Our findings document a previously unrecognized mixed influence of everyday sounds on cardiovascular stress, and that the relationship is more complex than is seen from an examination of sound intensity alone. Thus, our findings highlight the relevance of ambient environmental sound in models of human ecophysiology.

Energy investment and motivation: The additive impact of task demand and reward value on exerted force in hand grip tasks

According to motivational intensity theory, individuals are motivated to conserve energy when pursuing goals. They should invest only the energy required for success and disengage if success is not important enough to justify the required energy. We tested this hypothesis in five experiments assessing exerted muscle force in isometric hand grip tasks as indicator of energy investment. Our results provided mixed evidence for motivational intensity theory. Corroborating its predictions, energy investment was a function of task demand. However, we did not find evidence for the predicted disengagement, and we observed that participants exerted in most conditions more force than required. Furthermore, the data could be better explained by a model that predicted an additive effect of task demand and success importance than by models drawing on motivational intensity theory’s predictions. These results illustrate the strong link between energy investment and task demand but challenge motivational intensity theory’s primacy of energy conservation.

Reward Enhances Online Participants' Engagement With a Demanding Auditory Task

Online recruitment platforms are increasingly used for experimental research. Crowdsourcing is associated with numerous benefits but also notable constraints, including lack of control over participants' environment and engagement. In the context of auditory experiments, these limitations may be particularly detrimental to threshold-based tasks that require effortful listening. Here, we ask whether incorporating a performance-based monetary bonus improves speech reception performance of online participants. In two experiments, participants performed an adaptive matrix-type speech-in-noise task (where listeners select two key words out of closed sets). In Experiment 1, our results revealed worse performance in online (N = 49) compared with in-lab (N = 81) groups. Specifically, relative to the in-lab cohort, significantly fewer participants in the online group achieved very low thresholds. In Experiment 2 (N = 200), we show that a monetary reward improved listeners' thresholds to levels similar to those observed in the lab setting. Overall, the results suggest that providing a small performance-based bonus increases participants' task engagement, facilitating a more accurate estimation of auditory ability under challenging listening conditions.

Investigating the Influences of Task Demand and Reward on Cardiac Pre-Ejection Period Reactivity During a Speech-in-Noise Task

OBJECTIVES

Effort investment during listening varies as a function of task demand and motivation. Several studies have manipulated both these factors to elicit and measure changes in effort associated with listening. The cardiac pre-ejection period (PEP) is a relatively novel measure in the field of cognitive hearing science. This measure, which reflects sympathetic nervous system activity on the heart, has previously been implemented during a tone discrimination task but not during a speech-in-noise task. Therefore, the primary goal of this study was to explore the influences of signal to noise ratio (SNR) and monetary reward level on PEP reactivity during a speech-in-noise task.

DESIGN

Thirty-two participants with normal hearing (mean age = 22.22 years, SD = 3.03) were recruited at VU University Medical Center. Participants completed a Dutch speech-in-noise test with a single-interfering-talker masking noise. Six fixed SNRs, selected to span the entire psychometric performance curve, were presented in a block-wise fashion. Participants could earn a low (€0.20) or high (€5.00) reward by obtaining a score of ≥70% of words correct in each block. The authors analyzed PEP reactivity: the change in PEP measured during the task, relative to the baseline during rest. Two separate methods of PEP analysis were used, one including data from the whole task block and the other including data obtained during presentation of the target sentences only. After each block, participants rated their effort investment, performance, tendency to give up, and the perceived difficulty of the task. They also completed the need for recovery questionnaire and the reading span test, which are indices of additional factors (fatigue and working memory capacity, respectively) that are known to influence listening effort.

RESULTS

Average sentence perception scores ranged from 2.73 to 91.62%, revealing a significant effect of SNR. In addition, an improvement in performance was elicited by the high, compared to the low reward level. A linear relationship between SNR and PEP reactivity was demonstrated: at the lower SNRs PEP reactivity was the most negative, indicating greater effort investment compared to the higher SNRs. The target stimuli method of PEP analysis was more sensitive to this effect than the block-wise method. Contrary to expectations, no significant impact of reward on PEP reactivity was found in the present dataset. Also, there was no physiological evidence that participants were disengaged, even when performance was poor. A significant correlation between need for recovery scores and average PEP reactivity was demonstrated, indicating that a lower need for recovery was associated with less effort investment.

CONCLUSIONS

This study successfully implemented the measurement of PEP during a standard speech-in-noise test and included two distinct methods of PEP analysis. The results revealed for the first time that PEP reactivity varies linearly with task demand during a speech-in-noise task, although the effect size was small. No effect of reward on PEP was demonstrated. Finally, participants with a higher need for recovery score invested more effort, as shown by average PEP reactivity, than those with a lower need for recovery score.

Hearing and speech processing in midlife

Middle-aged adults often report a decline in their ability to understand speech in adverse listening situations. However, there has been relatively little research devoted to identifying how early aging affects speech processing, as the majority of investigations into senescent changes in speech understanding compare performance in groups of young and older adults. This paper provides an overview of research on hearing and speech perception in middle-aged adults. Topics covered include both objective and subjective (self-perceived) hearing and speech understanding, listening effort, and audiovisual speech perception. This review ends with justification for future research needed to define the nature, consequences, and remediation of hearing problems in middle-aged adults.

Comparisons of the Sensitivity and Reliability of Multiple Measures of Listening Effort

OBJECTIVES

The objective of this study was to evaluate the sensitivity and reliability of one subjective (rating scale) and three objective (dual-task paradigm, pupillometry, and skin conductance response amplitude) measures of listening effort across multiple signal to noise ratios (SNRs).

DESIGN

Twenty adults with normal hearing attended two sessions and listened to sentences presented in quiet and in stationary noise at three different SNRs: 0, -3, and -5 dB. Listening effort was assessed by examining change in reaction time (dual-task paradigm), change in peak to peak pupil diameter (pupillometry), and change in mean skin conductance response amplitude; self-reported listening effort on a scale from 0 to 100 was also evaluated. Responses were averaged within each SNR and based on three word recognition ability categories (≤50%, 51% to 71%, and >71%) across all SNRs. Measures were considered reliable if there were no significant changes between sessions, and intraclass correlation coefficients were a minimum of 0.40. Effect sizes were calculated to compare the sensitivity of the measures.

RESULTS

Intraclass correlation coefficient values indicated fair-to-moderate reliability for all measures while individual measurement sensitivity was variable. Self-reports were sensitive to listening effort but were less reliable, given that subjective effort was greater during the dual task than either of the physiologic measures. The dual task was sensitive to a narrow range of word recognition abilities but was less reliable as it exhibited a global decrease in reaction time across sessions. Pupillometry was consistently sensitive and reliable to changes in listening effort. Skin conductance response amplitude was not sensitive or reliable while the participants listened to the sentences. Skin conductance response amplitude during the verbal response was sensitive to poor (≤50%) speech recognition abilities; however, it was less reliable as there was a significant change in amplitude across sessions.

CONCLUSIONS

In this study, pupillometry was the most sensitive and reliable objective measure of listening effort. Intersession variability significantly influenced the other objective measures of listening effort, which suggests challenges for cross-study comparability. Therefore, intraclass correlation coefficients combined with other statistical tests more fully describe the reliability of measures of listening effort across multiple difficulties. Minimizing intersession variability will increase measurement sensitivity. Further work toward standardized methods and analysis will strengthen our understanding of the reliability and sensitivity of measures of listening effort and better facilitate cross-modal and cross-study comparisons.

Listening effort: Are we measuring cognition or affect, or both?

Listening effort is increasingly recognized as a factor in communication, particularly for and with nonnative speakers, for the elderly, for individuals with hearing impairment and/or for those working in noise. However, as highlighted by McGarrigle et al., International Journal of Audiology, 2014, 53, 433-445, the term "listening effort" encompasses a wide variety of concepts, including the engagement and control of multiple possibly distinct neural systems for information processing, and the affective response to the expenditure of those resources in a given context. Thus, experimental or clinical methods intended to objectively quantify listening effort may ultimately reflect a complex interaction between the operations of one or more of those information processing systems, and/or the affective and motivational response to the demand on those systems. Here we examine theoretical, behavioral, and psychophysiological factors related to resolving the question of what we are measuring, and why, when we measure "listening effort." This article is categorized under: Linguistics > Language in Mind and Brain Psychology > Theory and Methods Psychology > Attention Psychology > Emotion and Motivation.

Listening Effort: How the Cognitive Consequences of Acoustic Challenge Are Reflected in Brain and Behavior

Everyday conversation frequently includes challenges to the clarity of the acoustic speech signal, including hearing impairment, background noise, and foreign accents. Although an obvious problem is the increased risk of making word identification errors, extracting meaning from a degraded acoustic signal is also cognitively demanding, which contributes to increased listening effort. The concepts of cognitive demand and listening effort are critical in understanding the challenges listeners face in comprehension, which are not fully predicted by audiometric measures. In this article, the authors review converging behavioral, pupillometric, and neuroimaging evidence that understanding acoustically degraded speech requires additional cognitive support and that this cognitive load can interfere with other operations such as language processing and memory for what has been heard. Behaviorally, acoustic challenge is associated with increased errors in speech understanding, poorer performance on concurrent secondary tasks, more difficulty processing linguistically complex sentences, and reduced memory for verbal material. Measures of pupil dilation support the challenge associated with processing a degraded acoustic signal, indirectly reflecting an increase in neural activity. Finally, functional brain imaging reveals that the neural resources required to understand degraded speech extend beyond traditional perisylvian language networks, most commonly including regions of prefrontal cortex, premotor cortex, and the cingulo-opercular network. Far from being exclusively an auditory problem, acoustic degradation presents listeners with a systems-level challenge that requires the allocation of executive cognitive resources. An important point is that a number of dissociable processes can be engaged to understand degraded speech, including verbal working memory and attention-based performance monitoring. The specific resources required likely differ as a function of the acoustic, linguistic, and cognitive demands of the task, as well as individual differences in listeners' abilities. A greater appreciation of cognitive contributions to processing degraded speech is critical in understanding individual differences in comprehension ability, variability in the efficacy of assistive devices, and guiding rehabilitation approaches to reducing listening effort and facilitating communication.

Effects of Task Difficulty on Neural Processes Underlying Semantics: An Event-Related Potentials Study

Purpose Improving the ability to listen efficiently in noisy environments is a critical goal for hearing rehabilitation. However, understanding of the impact of difficult listening conditions on language processing is limited. The current study evaluated the neural processes underlying semantics in challenging listening conditions. Method Thirty adults with normal hearing completed an auditory sentence processing task in 4-talker babble. Event-related brain potentials were elicited by the final word in high- or low-context sentences, where the final word was either highly expected or not expected, followed by a 4-alternative forced-choice response with either longer (1,000 ms), middle (700 ms), or shorter (400 ms) response time deadlines (RTDs). Results Behavioral accuracy was reduced, and reactions times were faster for shorter RTDs. N400 amplitudes, reflecting ease of lexical access, were larger when elicited by target words in low-context sentences followed by shorter compared with longer RTDs. Conclusions These results reveal that more neural resources are allocated for semantic processing/lexical access when listening difficulty increases. Differences between RTDs may reflect increased attentional allocation for shorter RTDs. These findings suggest that situational listening demands can impact the demands for cognitive resources engaged in language processing, which could significantly impact listener experiences across environments.

A Tipping Point in Listening Effort: Effects of Linguistic Complexity and Age-Related Hearing Loss on Sentence Comprehension

In recent years, there has been a growing interest in the relationship between effort and performance. Early formulations implied that, as the challenge of a task increases, individuals will exert more effort, with resultant maintenance of stable performance. We report an experiment in which normal-hearing young adults, normal-hearing older adults, and older adults with age-related mild-to-moderate hearing loss were tested for comprehension of recorded sentences that varied the comprehension challenge in two ways. First, sentences were constructed that expressed their meaning either with a simpler subject-relative syntactic structure or a more computationally demanding object-relative structure. Second, for each sentence type, an adjectival phrase was inserted that created either a short or long gap in the sentence between the agent performing an action and the action being performed. The measurement of pupil dilation as an index of processing effort showed effort to increase with task difficulty until a difficulty tipping point was reached. Beyond this point, the measurement of pupil size revealed a commitment of effort by the two groups of older adults who failed to keep pace with task demands as evidenced by reduced comprehension accuracy. We take these pupillometry data as revealing a complex relationship between task difficulty, effort, and performance that might not otherwise appear from task performance alone.

Psychophysiological measurement of affective responses during speech perception

When people make decisions about listening, such as whether to continue attending to a particular conversation or whether to wear their hearing aids to a particular restaurant, they do so on the basis of more than just their estimated performance. Recent research has highlighted the vital role of more subjective qualities such as effort, motivation, and fatigue. Here, we argue that the importance of these factors is largely mediated by a listener's emotional response to the listening challenge, and suggest that emotional responses to communication challenges may provide a crucial link between day-to-day communication stress and long-term health. We start by introducing some basic concepts from the study of emotion and affect. We then develop a conceptual framework to guide future research on this topic through examination of a variety of autonomic and peripheral physiological responses that have been employed to investigate both cognitive and affective phenomena related to challenging communication. We conclude by suggesting the need for further investigation of the links between communication difficulties, emotional response, and long-term health, and make some recommendations intended to guide future research on affective psychophysiology in speech communication.

Autonomic Nervous System Reactivity During Speech Repetition Tasks: Heart Rate Variability and Skin Conductance

Cognitive and emotional challenges may elicit a physiological stress response that can include arousal of the sympathetic nervous system (fight or flight response) and withdrawal of the parasympathetic nervous system (responsible for recovery and rest). This article reviews studies that have used measures of electrodermal activity (skin conductance) and heart rate variability (HRV) to index sympathetic and parasympathetic activity during auditory tasks. In addition, the authors present results from a new study with normal-hearing listeners examining the effects of speaking rate on changes in skin conductance and high-frequency HRV (HF-HRV). Sentence repetition accuracy for normal and fast speaking rates was measured in noise using signal to noise ratios that were adjusted to approximate 80% accuracy (+3 dB fast rate; 0 dB normal rate) while monitoring skin conductance and HF-HRV activity. A significant increase in skin conductance level (reflecting sympathetic nervous system arousal) and a decrease in HF-HRV (reflecting parasympathetic nervous system withdrawal) were observed with an increase in speaking rate indicating sensitivity of both measures to increased task demand. Changes in psychophysiological reactivity with increased auditory task demand may reflect differences in listening effort, but other person-related factors such as motivation and stress may also play a role. Further research is needed to understand how psychophysiological activity during listening tasks is influenced by the acoustic characteristics of stimuli, task demands, and by the characteristics and emotional responses of the individual.

Effort and Displeasure in People Who Are Hard of Hearing

Listening effort helps explain why people who are hard of hearing are prone to fatigue and social withdrawal. However, a one-factor model that cites only effort due to hardness of hearing is insufficient as there are many who lead happy lives despite their disability. This article explores other contributory factors, in particular motivational arousal and pleasure. The theory of rational motivational arousal predicts that some people forego listening comprehension because they believe it to be impossible and hence worth no effort at all. This is problematic. Why should the listening task be rated this way, given the availability of aids that reduce its difficulty? Two additional factors narrow the explanatory gap. First, we separate the listening task from the benefit derived as a consequence. The latter is temporally more distant, and is discounted as a result. The second factor is displeasure attributed to the listening task, which increases listening cost. Many who are hard of hearing enjoy social interaction. In such cases, the actual activity of listening is a benefit, not a cost. These people also reap the benefits of listening, but do not have to balance these against the displeasure of the task. It is suggested that if motivational harmony can be induced by training in somebody who is hard of hearing, then the obstacle to motivational arousal would be removed. This suggests a modified goal for health care professionals. Do not just teach those who are hard of hearing how to use hearing assistance devices. Teach them how to do so with pleasure and enjoyment.

Cortisol, Chromogranin A, and Pupillary Responses Evoked by Speech Recognition Tasks in Normally Hearing and Hard-of-Hearing Listeners: A Pilot Study

Pupillometry is one method that has been used to measure processing load expended during speech understanding. Notably, speech perception (in noise) tasks can evoke a pupil response. It is not known if there is concurrent activation of the sympathetic nervous system as indexed by salivary cortisol and chromogranin A (CgA) and whether such activation differs between normally hearing (NH) and hard-of-hearing (HH) adults. Ten NH and 10 adults with mild-to-moderate hearing loss (mean age 52 years) participated. Two speech perception tests were administered in random order: one in quiet targeting 100% correct performance and one in noise targeting 50% correct performance. Pupil responses and salivary samples for cortisol and CgA analyses were collected four times: before testing, after the two speech perception tests, and at the end of the session. Participants rated their perceived accuracy, effort, and motivation. Effects were examined using repeated-measures analyses of variance. Correlations between outcomes were calculated. HH listeners had smaller peak pupil dilations (PPDs) than NH listeners in the speech in noise condition only. No group or condition effects were observed for the cortisol data, but HH listeners tended to have higher cortisol levels across conditions. CgA levels were larger at the pretesting time than at the three other test times. Hearing impairment did not affect CgA. Self-rated motivation correlated most often with cortisol or PPD values. The three physiological indicators of cognitive load and stress (PPD, cortisol, and CgA) are not equally affected by speech testing or hearing impairment. Each of them seem to capture a different dimension of sympathetic nervous system activity.

A Taxonomy of Fatigue Concepts and Their Relation to Hearing Loss

Fatigue is common in individuals with a variety of chronic health conditions and can have significant negative effects on quality of life. Although limited in scope, recent work suggests persons with hearing loss may be at increased risk for fatigue, in part due to effortful listening that is exacerbated by their hearing impairment. However, the mechanisms responsible for hearing loss-related fatigue, and the efficacy of audiologic interventions for reducing fatigue, remain unclear. To improve our understanding of hearing loss-related fatigue, as a field it is important to develop a common conceptual understanding of this construct. In this article, the broader fatigue literature is reviewed to identify and describe core constructs, consequences, and methods for assessing fatigue and related constructs. Finally, the current knowledge linking hearing loss and fatigue is described and may be summarized as follows: Hearing impairment may increase the risk of subjective fatigue and vigor deficits; adults with hearing loss require more time to recover from fatigue after work and have more work absences; sustained, effortful, listening can be fatiguing; optimal methods for eliciting and measuring fatigue in persons with hearing loss remain unclear and may vary with listening condition; and amplification may minimize decrements in cognitive processing speed during sustained effortful listening. Future research is needed to develop reliable measurement methods to quantify hearing loss-related fatigue, explore factors responsible for modulating fatigue in people with hearing loss, and identify and evaluate potential interventions for reducing hearing loss-related fatigue.

Autonomic Nervous System Responses to Hearing-Related Demand and Evaluative Threat

PURPOSE

This paper consists of 2 parts. The purpose of Part 1 was to review the potential influence of internal (person-related) factors on listening effort. The purpose of Part 2 was to present, in support of Part 1, preliminary data illustrating the interactive effects of an external factor (task demand) and an internal factor (evaluative threat) on autonomic nervous system measures.

METHOD

For Part 1, we provided a brief narrative review of motivation and stress as modulators of listening effort. For Part 2, we described preliminary data from a study using a repeated-measures (2 × 2) design involving manipulations of task demand (high, low) and evaluative threat (high, low). The low-demand task consisted of repetition of sentences from a narrative. The high-demand task consisted of answering questions about the narrative, requiring both comprehension and recall. During the high evaluative threat condition, participants were filmed and told that their video recordings would be evaluated by a panel of experts. During the low evaluative threat condition, no filming occurred; participants were instructed to "do your best." Skin conductance (sympathetic nervous system activity) and heart rate variability (HRV, parasympathetic activity) were measured during the listening tasks. The HRV measure was the root mean square of successive differences of adjacent interbeat intervals. Twelve adults with hearing loss participated.

RESULTS

Skin conductance increased and HRV decreased relative to baseline (no task) for all listening conditions. Skin conductance increased significantly with an increase in evaluative threat, but only for the more demanding task. There was no significant change in HRV in response to increasing evaluative threat or task demand.

CONCLUSIONS

Listening effort may be influenced by factors other than task difficulty, as reviewed in Part 1. This idea is supported by the preliminary data indicating that the sympathetic nervous system response to task demand is modulated by social evaluative threat. More work is needed to determine the relative contributions of motivation and emotional stress on physiological responses during listening tasks.

Theta- and alpha-power enhancements in the electroencephalogram as an auditory delayed match-to-sample task becomes impossibly difficult

Recent studies have related enhancements of theta- (∼4-8 Hz) and alpha-power (∼8-13 Hz) to listening effort based on parallels between enhancement and task difficulty. In contrast, nonauditory works demonstrate that, although increases in difficulty are initially accompanied by increases in effort, effort decreases when a task becomes so difficult as to exceed one's ability. Given the latter, we examined whether theta- and alpha-power enhancements thought to reflect effortful listening show a quadratic trend across levels of listening difficulty from impossible to easy. Listeners (n = 14) performed an auditory delayed match-to-sample task with frequency-modulated tonal sweeps under impossible, difficult (at ∼70.7% correct threshold), and easy (well above threshold) conditions. Frontal midline theta-power and posterior alpha-power enhancements were observed during the retention interval, with greatest enhancement in the difficult condition. Independent component-based analyses of data suggest that theta-power enhancements stemmed from medial frontal sources at or near the anterior cingulate cortex, whereas alpha-power effects stemmed from occipital cortices. Results support the notion that theta- and alpha-power enhancements reflect effortful cognitive processes during listening, related to auditory working memory and the inhibition of task-irrelevant cortical processing regions, respectively. Theta- and alpha-power dynamics can be used to characterize the cognitive processes that make up effortful listening, including qualitatively different types of listening effort.

Toward a taxonomic model of attention in effortful listening

In recent years, there has been increasing interest in studying listening effort. Research on listening effort intersects with the development of active theories of speech perception and contributes to the broader endeavor of understanding speech perception within the context of neuroscientific theories of perception, attention, and effort. Due to the multidisciplinary nature of the problem, researchers vary widely in their precise conceptualization of the catch-all term listening effort. Very recent consensus work stresses the relationship between listening effort and the allocation of cognitive resources, providing a conceptual link to current cognitive neuropsychological theories associating effort with the allocation of selective attention. By linking listening effort to attentional effort, we enable the application of a taxonomy of external and internal attention to the characterization of effortful listening. More specifically, we use a vectorial model to decompose the demand causing listening effort into its mutually orthogonal external and internal components and map the relationship between demanded and exerted effort by means of a resource-limiting term that can represent the influence of motivation as well as vigilance and arousal. Due to its quantitative nature and easy graphical interpretation, this model can be applied to a broad range of problems dealing with listening effort. As such, we conclude that the model provides a good starting point for further research on effortful listening within a more differentiated neuropsychological framework.

Interpretation of physiological indicators of motivation: Caveats and recommendations

Motivation scientists employing physiological measures to gather information about motivation-related states are at risk of committing two fundamental errors: overstating the inferences that can be drawn from their physiological measures and circular reasoning. We critically discuss two complementary approaches, Cacioppo and colleagues' model of psychophysiological relations and construct validation theory, to highlight the conditions under which these errors are committed and provide guidance on how to avoid them. In particular, we demonstrate that the direct inference from changes in a physiological measure to changes in a motivation-related state requires the demonstration that the measure is not related to other relevant psychological states. We also point out that circular reasoning can be avoided by separating the definition of the motivation-related state from the hypotheses that are empirically tested.

Listening and Learning: Cognitive Contributions to the Rehabilitation of Older Adults With and Without Audiometrically Defined Hearing Loss

Here, we describe some of the ways in which aging negatively affects the way sensory input is transduced and processed within the aging brain and how cognitive work is involved when listening to a less-than-perfect signal. We also describe how audiologic rehabilitation, including hearing aid amplification and listening training, is used to reduce the amount of cognitive resources required for effective auditory communication and conclude with an example of how listening effort is being studied in research laboratories for the purpose(s) of informing clinical practice.