The Effects of Task Difficulty Predictability and Noise Reduction on Recall Performance and Pupil Dilation Responses

Attention Mobilization as a Modulator of Listening Effort: Evidence From Pupillometry

Listening to speech in noise can require substantial mental effort, even among younger normal-hearing adults. The task-evoked pupil response (TEPR) has been shown to track the increased effort exerted to recognize words or sentences in increasing noise. However, few studies have examined the trajectory of listening effort across longer, more natural, stretches of speech, or the extent to which expectations about upcoming listening difficulty modulate the TEPR. Seventeen younger normal-hearing adults listened to 60-s-long audiobook passages, repeated three times in a row, at two different signal-to-noise ratios (SNRs) while pupil size was recorded. There was a significant interaction between SNR, repetition, and baseline pupil size on sustained listening effort. At lower baseline pupil sizes, potentially reflecting lower attention mobilization, TEPRs were more sustained in the harder SNR condition, particularly when attention mobilization remained low by the third presentation. At intermediate baseline pupil sizes, differences between conditions were largely absent, suggesting these listeners had optimally mobilized their attention for both SNRs. Lastly, at higher baseline pupil sizes, potentially reflecting overmobilization of attention, the effect of SNR was initially reversed for the second and third presentations: participants initially appeared to disengage in the harder SNR condition, resulting in reduced TEPRs that recovered in the second half of the story. Together, these findings suggest that the unfolding of listening effort over time depends critically on the extent to which individuals have successfully mobilized their attention in anticipation of difficult listening conditions.

Hearing Aid Delay Effects on Neural Phase Locking

OBJECTIVES

This study was designed to examine the effects of hearing aid delay on the neural representation of the temporal envelope. It was hypothesized that the comb-filter effect would disrupt neural phase locking, and that shorter hearing aid delays would minimize this effect.

DESIGN

Twenty-one participants, ages 50 years and older, with bilateral mild-to-moderate sensorineural hearing loss were recruited through print advertisements in local senior newspapers. They were fitted with three different sets of hearing aids with average processing delays that ranged from 0.5 to 7 msec. Envelope-following responses (EFRs) were recorded to a 50-msec /da/ syllable presented through a speaker placed 1 meter in front of the participants while they wore the three sets of hearing aids with open tips. Phase-locking factor (PLF) and stimulus-to-response (STR) correlations were calculated from these recordings.

RESULTS

Recordings obtained while wearing hearing aids with a 0.5-msec processing delay showed higher PLF and STR correlations compared with those with either 5-msec or 7-msec delays. No differences were noted between recordings of hearing aids with 5-msec and 7-msec delays. The degree of difference between hearing aids was greater for individuals who had milder degrees of hearing loss.

CONCLUSIONS

Hearing aid processing delays disrupt phase locking due to mixing of processed and unprocessed sounds in the ear canal when using open domes. Given previous work showing that better phase locking correlates with better speech-in-noise performance, consideration should be given to reducing hearing aid processing delay in the design of hearing aid algorithms.

Impact of SNR, peripheral auditory sensitivity, and central cognitive profile on the psychometric relation between pupillary response and speech performance in CI users

Despite substantial technical advances and wider clinical use, cochlear implant (CI) users continue to report high and elevated listening effort especially under challenging noisy conditions. Among all the objective measures to quantify listening effort, pupillometry is one of the most widely used and robust physiological measures. Previous studies with normally hearing (NH) and hearing-impaired (HI) listeners have shown that the relation between speech performance in noise and listening effort (as measured by peak pupil dilation) is not linear and exhibits an inverted-U shape. However, it is unclear whether the same psychometric relation exists in CI users, and whether individual differences in auditory sensitivity and central cognitive capacity affect this relation. Therefore, we recruited 17 post-lingually deaf CI adults to perform speech-in-noise tasks from 0 to 20 dB SNR with a 4 dB step size. Simultaneously, their pupillary responses and self-reported subjective effort were recorded. To characterize top-down and bottom-up individual variabilities, a spectro-temporal modulation task and a set of cognitive abilities were measured. Clinical word recognition in quiet and Quality of Life (QoL) were also collected. Results showed that at a group level, an inverted-U shape psychometric curve between task difficulty (SNR) and peak pupil dilation (PPD) was not observed. Individual shape of the psychometric curve was significantly associated with some individual factors: CI users with higher clinical word and speech-in-noise recognition showed a quadratic decrease of PPD over increasing SNRs; CI users with better non-verbal intelligence and lower QoL showed smaller average PPD. To summarize, individual differences in CI users had a significant impact on the psychometric relation between pupillary response and task difficulty, hence affecting the interpretation of pupillary response as listening effort (or engagement) at different task difficulty levels. Future research and clinical applications should further characterize the possible effects of individual factors (such as motivation or engagement) in modulating CI users' occurrence of 'tipping point' on their psychometric functions, and develop an individualized method for reliably quantifying listening effort using pupillometry.

Time-specific Components of Pupil Responses Reveal Alternations in Effort Allocation Caused by Memory Task Demands During Speech Identification in Noise

Daily communication may be effortful due to poor acoustic quality. In addition, memory demands can induce effort, especially for long or complex sentences. In the current study, we tested the impact of memory task demands and speech-to-noise ratio on the time-specific components of effort allocation during speech identification in noise. Thirty normally hearing adults (15 females, mean age 42.2 years) participated. In an established auditory memory test, listeners had to listen to a list of seven sentences in noise, and repeat the sentence-final word after presentation, and, if instructed, recall the repeated words. We tested the effects of speech-to-noise ratio (SNR; -4 dB, +1 dB) and recall (Recall; Yes, No), on the time-specific components of pupil responses, trial baseline pupil size, and their dynamics (change) along the list. We found three components in the pupil responses (early, middle, and late). While the additional memory task (recall versus no recall) lowered all components' values, SNR (-4 dB versus +1 dB SNR) increased the middle and late component values. Increasing memory demands (Recall) progressively increased trial baseline and steepened decrease of the late component's values. Trial baseline increased most steeply in the condition of +1 dB SNR with recall. The findings suggest that adding a recall to the auditory task alters effort allocation for listening. Listeners are dynamically re-allocating effort from listening to memorizing under changing memory and acoustic demands. The pupil baseline and the time-specific components of pupil responses provide a comprehensive picture of the interplay of SNR and recall on effort.

Luminance effects on pupil dilation in speech-in-noise recognition

There is an increasing interest in the field of audiology and speech communication to measure the effort that it takes to listen in noisy environments, with obvious implications for populations suffering from hearing loss. Pupillometry offers one avenue to make progress in this enterprise but important methodological questions remain to be addressed before such tools can serve practical applications. Typically, cocktail-party situations may occur in less-than-ideal lighting conditions, e.g. a pub or a restaurant, and it is unclear how robust pupil dynamics are to luminance changes. In this study, we first used a well-known paradigm where sentences were presented at different signal-to-noise ratios (SNR), all conducive of good intelligibility. This enabled us to replicate findings, e.g. a larger and later peak pupil dilation (PPD) at adverse SNR, or when the sentences were misunderstood, and to investigate the dependency of the PPD on sentence duration. A second experiment reiterated two of the SNR levels, 0 and +14 dB, but measured at 0, 75, and 220 lux. The results showed that the impact of luminance on the SNR effect was non-monotonic (sub-optimal in darkness or in bright light), and as such, there is no trivial way to derive pupillary metrics that are robust to differences in background light, posing considerable constraints for applications of pupillometry in daily life. Our findings raise an under-examined but crucial issue when designing and understanding listening effort studies using pupillometry, and offer important insights to future clinical application of pupillometry across sites.

A Glimpse of Memory Through the Eyes: Pupillary Responses Measured During Encoding Reflect the Likelihood of Subsequent Memory Recall in an Auditory Free Recall Test

The aim of the current study was to investigate whether task-evoked pupillary responses measured during encoding, individual working memory capacity and noise reduction in hearing aids were associated with the likelihood of subsequently recalling an item in an auditory free recall test combined with pupillometry. Participants with mild to moderately severe symmetrical sensorineural hearing loss (n = 21) were included. The Sentence-final Word Identification and Recall (SWIR) test was administered in a background noise composed of sixteen talkers with noise reduction in hearing aids activated and deactivated. The task-evoked peak pupil dilation (PPD) was measured. The Reading Span (RS) test was used as a measure of individual working memory capacity. Larger PPD at a single trial level was significantly associated with higher likelihood of subsequently recalling a word, presumably reflecting the intensity of attention devoted during encoding. There was no clear evidence of a significant relationship between working memory capacity and subsequent memory recall, which may be attributed to the SWIR test and RS test being administered in different modalities, as well as differences in task characteristics. Noise reduction did not have a significant effect on subsequent memory recall. This may be due to the background noise not having a detrimental effect on attentional processing at the favorable signal-to-noise ratio levels at which the test was conducted.

A decrease in physiological arousal accompanied by stable behavioral performance reflects task habituation

Despite the evidence of a positive relationship between task demands and listening effort, the Framework for Understanding Effortful Listening (FUEL) highlights the important role of arousal on an individual’s choice to engage in challenging listening tasks. Previous studies have interpreted physiological responses in conjunction with behavioral responses as markers of task engagement. The aim of the current study was to investigate the effect of potential changes in physiological arousal, indexed by the pupil baseline, on task engagement over the course of an auditory recall test. Furthermore, the aim was to investigate whether working memory (WM) capacity and the signal-to-noise ratio (SNR) at which the test was conducted had an effect on changes in arousal. Twenty-one adult hearing aid users with mild to moderately severe symmetrical sensorineural hearing loss were included. The pupil baseline was measured during the Sentence-final Word Identification and Recall (SWIR) test, which was administered in a background noise composed of sixteen talkers. The Reading Span (RS) test was used as a measure of WM capacity. The findings showed that the pupil baseline decreased over the course of the SWIR test. However, recall performance remained stable, indicating that the participants maintained the necessary engagement level required to perform the task. These findings were interpreted as a decline in arousal as a result of task habituation. There was no effect of WM capacity or individual SNR level on the change in pupil baseline over time. A significant interaction was found between WM capacity and SNR level on the overall mean pupil baseline. Individuals with higher WM capacity exhibited an overall larger mean pupil baseline at low SNR levels compared to individuals with poorer WM capacity. This may be related to the ability of individuals with higher WM capacity to perform better than individual with poorer WM capacity in challenging listening conditions.

Relationship between Behavioral and Objective Measures of Sound Intensity in Normal-Hearing Listeners and Hearing-Aid Users: A Pilot Study

Background: For hearing-impaired individuals, hearing aids are clinically fit according to subjective measures of threshold and loudness. The goal of this study was to evaluate objective measures of loudness perception that might benefit hearing aid fitting. Method: Seventeen adult hearing aid users and 17 normal-hearing adults participated in the study. Outcome measures including categorical loudness scaling, cortical auditory evoked potentials (CAEPs), and pupillometry. Stimuli were 1-kHz tone bursts presented at 40, 60, and 80 dBA. Results: Categorical loudness scaling showed that loudness significantly increased with intensity for all participants (p < 0.05). For CAEPs, high intensity was associated with greater P1, N1, and P2 peak amplitude for all listeners (p < 0.05); a significant but small effect of hearing aid amplification was observed. For all participants, pupillometry showed significant effects of high intensity on pupil dilation (p < 0.05); there was no significant effect of hearing aid amplification. A Focused Principal Component analysis revealed significant correlations between subjective loudness and some of the objective measures. Conclusion: The present data suggest that intensity had a significant impact on loudness perception, CAEPs, and pupil response. The correlations suggest that pupillometry and/or CAEPs may be useful in determining comfortable amplification for hearing aids.

Listen Carefully protocol: an exploratory case-control study of the association between listening effort and cognitive function

INTRODUCTION

A growing body of evidence suggests that hearing loss is a significant and potentially modifiable risk factor for cognitive impairment. Although the mechanisms underlying the associations between cognitive decline and hearing loss are unclear, listening effort has been posited as one of the mechanisms involved with cognitive decline in older age. To date, there has been a lack of research investigating this association, particularly among adults with mild cognitive impairment (MCI).

METHODS AND ANALYSIS

15-25 cognitively healthy participants and 15-25 patients with MCI (age 40-85 years) will be recruited to participate in an exploratory study investigating the association between cognitive functioning and listening effort. Both behavioural and objective measures of listening effort will be investigated. The sentence-final word identification and recall (SWIR) test will be administered with single talker non-intelligible speech background noise while monitoring pupil dilation. Evaluation of cognitive function will be carried out in a clinical setting using a battery of neuropsychological tests. This study is considered exploratory and proof of concept, with information taken to help decide the validity of larger-scale trials.

ETHICS AND DISSEMINATION

Written approval exemption was obtained by the Scientific Ethics Committee in the central region of Denmark (De Videnskabsetiske Komiteer i Region Hovedstaden), reference 19042404, and the project is registered pre-results at clinicaltrials.gov, reference NCT04593290, Protocol ID 19042404. Study results will be disseminated in peer-reviewed journals and conferences.