Subjective and psychophysiological indexes of listening effort in a competing-talker task

A Multimodal Approach to Measuring Listening Effort: A Systematic Review on the Effects of Auditory Task Demand on Physiological Measures and Their Relationship

OBJECTIVES

Listening effort involves the mental effort required to perceive an auditory stimulus, for example in noisy environments. Prolonged increased listening effort, for example due to impaired hearing ability, may increase risk of health complications. It is therefore important to identify valid and sensitive measures of listening effort. Physiological measures have been shown to be sensitive to auditory task demand manipulations and are considered to reflect changes in listening effort. Such measures include pupil dilation, alpha power, skin conductance level, and heart rate variability. The aim of the current systematic review was to provide an overview of studies to listening effort that used multiple physiological measures. The two main questions were: (1) what is the effect of changes in auditory task demand on simultaneously acquired physiological measures from various modalities? and (2) what is the relationship between the responses in these physiological measures?

DESIGN

Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, relevant articles were sought in PubMed, PsycInfo, and Web of Science and by examining the references of included articles. Search iterations with different combinations of psychophysiological measures were performed in conjunction with listening effort-related search terms. Quality was assessed using the Appraisal Tool for Cross-Sectional Studies.

RESULTS

A total of 297 articles were identified from three databases, of which 27 were included. One additional article was identified from reference lists. Of the total 28 included articles, 16 included an analysis regarding the relationship between the physiological measures. The overall quality of the included studies was reasonable.

CONCLUSIONS

The included studies showed that most of the physiological measures either show no effect to auditory task demand manipulations or a consistent effect in the expected direction. For example, pupil dilation increased, pre-ejection period decreased, and skin conductance level increased with increasing auditory task demand. Most of the relationships between the responses of these physiological measures were nonsignificant or weak. The physiological measures varied in their sensitivity to auditory task demand manipulations. One of the identified knowledge gaps was that the included studies mostly used tasks with high-performance levels, resulting in an underrepresentation of the physiological changes at lower performance levels. This makes it difficult to capture how the physiological responses behave across the full psychometric curve. Our results support the Framework for Understanding Effortful Listening and the need for a multimodal approach to listening effort. We furthermore discuss focus points for future studies.

Effects of hearing acuity on psychophysiological responses to effortful speech perception

In recent studies, psychophysiological measures have been used as markers of listening effort, but there is limited research on the effect of hearing loss on such measures. The aim of the current study was to investigate the effect of hearing acuity on physiological responses and subjective measures acquired during different levels of listening demand, and to investigate the relationship between these measures. A total of 125 participants (37 males and 88 females, age range 37-72 years, pure-tone average hearing thresholds at the best ear between -5.0 to 68.8 dB HL and asymmetry between ears between 0.0 and 87.5 dB) completed a listening task. A speech reception threshold (SRT) test was used with target sentences spoken by a female voice masked by male speech. Listening demand was manipulated using three levels of intelligibility: 20 % correct speech recognition, 50 %, and 80 % (IL20 %/IL50 %/IL80 %, respectively). During the task, peak pupil dilation (PPD), heart rate (HR), pre-ejection period (PEP), respiratory sinus arrhythmia (RSA), and skin conductance level (SCL) were measured. For each condition, subjective ratings of effort, performance, difficulty, and tendency to give up were also collected. Linear mixed effects models tested the effect of intelligibility level, hearing acuity, hearing asymmetry, and tinnitus complaints on the physiological reactivity (compared to baseline) and subjective measures. PPD and PEP reactivity showed a non-monotonic relationship with intelligibility level, but no such effects were found for HR, RSA, or SCL reactivity. Participants with worse hearing acuity had lower PPD at all intelligibility levels and showed lower PEP baseline levels. Additionally, PPD and SCL reactivity were lower for participants who reported suffering from tinnitus complaints. For IL80 %, but not IL50 % or IL20 %, participants with worse hearing acuity rated their listening effort to be relatively high compared to participants with better hearing. The reactivity of the different physiological measures were not or only weakly correlated with each other. Together, the results suggest that hearing acuity may be associated with altered sympathetic nervous system (re)activity. Research using psychophysiological measures as markers of listening effort to study the effect of hearing acuity on such measures are best served by the use of the PPD and PEP.

Associations Between Auditory Working Memory, Self-Perceived Listening Effort, and Hearing Difficulty in Adults With Mild Traumatic Brain Injury

OBJECTIVES

Mild traumatic brain injury (TBI) can have persistent effects in the auditory domain (e.g., difficulty listening in noise), despite individuals having normal pure-tone auditory sensitivity. Individuals with a history of mild TBI often perceive hearing difficulty and greater listening effort in complex listening situations. The purpose of the present study was to examine self-perceived hearing difficulty, listening effort, and performance on an auditory processing test battery in adults with a history of mild TBI compared with a control group.

DESIGN

Twenty adults ages 20 to 53 years old participated divided into a mild TBI (n = 10) and control group (n = 10). Perceived hearing difficulties were measured using the Adult Auditory Processing Scale and the Hearing Handicap Inventory for Adults. Listening effort was measured using the National Aeronautics and Space Administration-Task Load Index. Listening effort ratings were obtained at baseline, after each auditory processing test, and at the completion of the test battery. The auditory processing test battery included (1) dichotic word recognition, (2) the 500-Hz masking level difference, (3) the Listening in Spatialized Noise-Sentences test, and (4) the Word Auditory Recognition and Recall Measure (WARRM).

RESULTS

Results indicated that individuals with a history of mild TBI perceived significantly greater degrees of hearing difficulty and listening effort than the control group. There were no significant group differences on two of the auditory processing tasks (dichotic word recognition or Listening in Spatialized Noise-Sentences). The mild TBI group exhibited significantly poorer performance on the 500-Hz MLD and the WARRM, a measure of auditory working memory, than the control group. Greater degrees of self-perceived hearing difficulty were significantly associated with greater listening effort and poorer auditory working memory. Greater listening effort was also significantly associated with poorer auditory working memory.

CONCLUSIONS

Results demonstrate that adults with a history of mild TBI may experience subjective hearing difficulty and listening effort when listening in challenging acoustic environments. Poorer auditory working memory on the WARRM task was observed for the adults with mild TBI and was associated with greater hearing difficulty and listening effort. Taken together, the present study suggests that conventional clinical audiometric battery alone may not provide enough information about auditory processing deficits in individuals with a history of mild TBI. The results support the use of a multifaceted battery of auditory processing tasks and subjective measures when evaluating individuals with a history of mild TBI.

Combining Cardiovascular and Pupil Features Using k-Nearest Neighbor Classifiers to Assess Task Demand, Social Context, and Sentence Accuracy During Listening

In daily life, both acoustic factors and social context can affect listening effort investment. In laboratory settings, information about listening effort has been deduced from pupil and cardiovascular responses independently. The extent to which these measures can jointly predict listening-related factors is unknown. Here we combined pupil and cardiovascular features to predict acoustic and contextual aspects of speech perception. Data were collected from 29 adults (mean  =  64.6 years, SD  =  9.2) with hearing loss. Participants performed a speech perception task at two individualized signal-to-noise ratios (corresponding to 50% and 80% of sentences correct) and in two social contexts (the presence and absence of two observers). Seven features were extracted per trial: baseline pupil size, peak pupil dilation, mean pupil dilation, interbeat interval, blood volume pulse amplitude, pre-ejection period and pulse arrival time. These features were used to train k-nearest neighbor classifiers to predict task demand, social context and sentence accuracy. The k-fold cross validation on the group-level data revealed above-chance classification accuracies: task demand, 64.4%; social context, 78.3%; and sentence accuracy, 55.1%. However, classification accuracies diminished when the classifiers were trained and tested on data from different participants. Individually trained classifiers (one per participant) performed better than group-level classifiers: 71.7% (SD  =  10.2) for task demand, 88.0% (SD  =  7.5) for social context, and 60.0% (SD  =  13.1) for sentence accuracy. We demonstrated that classifiers trained on group-level physiological data to predict aspects of speech perception generalized poorly to novel participants. Individually calibrated classifiers hold more promise for future applications.

Methods and evaluation of physiological measurements with acoustic stimuli-a systematic review

Objective. The detection of psychological loads, such as stress reactions, is receiving greater attention and social interest, as stress can have long-term effects on health O'Connor, Thayer and Vedhara (2021Ann. Rev. Psychol.72, 663-688). Acoustic stimuli, especially noise, are investigated as triggering factors. The application of physiological measurements in the detection of psychological loads enables the recording of a further quantitative dimension that goes beyond purely perceptive questionnaires. Thus, unconscious reactions to acoustic stimuli can also be captured. The numerous physiological signals and possible experimental designs with acoustic stimuli may quickly lead to a challenging implementation of the study and an increased difficulty in reproduction or comparison between studies. An unsuitable experimental design or processing of the physiological data may result in conclusions about psychological loads that are not valid anymore.Approach. The systematic review according to the preferred reporting items for systematic reviews and meta-analysis standard presented here is therefore intended to provide guidance and a basis for further studies in this field. For this purpose, studies were identified in which the participants' short-term physiological responses to acoustic stimuli were investigated in the context of a listening test in a laboratory study.Main Results. A total of 37 studies met these criteria and data items were analysed in terms of the experimental design (studied psychological load, independent variables/acoustic stimuli, participants, playback, scenario/context, duration of test phases, questionnaires for perceptual comparison) and the physiological signals (measures, calculated features, systems, data processing methods, data analysis methods, results). The overviews show that stress is the most studied psychological load in response to acoustic stimuli. An ECG/PPG system and the measurement of skin conductance were most frequently used for the detection of psychological loads. A critical aspect is the numerous different methods of experimental design, which prevent comparability of the results. In the future, more standardized methods are needed to achieve more valid analyses of the effects of acoustic stimuli.

The P300 Auditory Evoked Potential: A Physiological Measure of the Engagement of Cognitive Systems Contributing to Listening Effort?

OBJECTIVES

This study aimed to explore the potential of the P300 (P3b) as a physiological measure of the engagement of cognitive systems contributing to listening effort.

DESIGN

Nineteen right-handed young adults (mean age: 24.79 years) and 20 right-handed older adults (mean age: 58.90 years) with age-appropriate hearing were included. The P300 was recorded at Fz, Cz, and Pz using a two-stimulus oddball paradigm with the Flemish monosyllabic numbers "one" and "three" as standard and deviant stimuli, respectively. This oddball paradigm was conducted in three listening conditions, varying in listening demand: one quiet and two noisy listening conditions (+4 and -2 dB signal to noise ratio [SNR]). At each listening condition, physiological, behavioral, and subjective tests of listening effort were administered. P300 amplitude and latency served as a potential physiological measure of the engagement of cognitive systems contributing to listening effort. In addition, the mean reaction time to respond to the deviant stimuli was used as a behavioral listening effort measurement. Last, subjective listening effort was administered through a visual analog scale. To assess the effects of listening condition and age group on each of these measures, linear mixed models were conducted. Correlation coefficients were calculated to determine the relationship between the physiological, behavioral, and subjective measures.

RESULTS

P300 amplitude and latency, mean reaction time, and subjective scores significantly increased as the listening condition became more taxing. Moreover, a significant group effect was found for all physiological, behavioral, and subjective measures, favoring young adults. Last, no clear relationships between the physiological, behavioral, and subjective measures were found.

CONCLUSIONS

The P300 was considered a physiological measure of the engagement of cognitive systems contributing to listening effort. Because advancing age is associated with hearing loss and cognitive decline, more research is needed on the effects of all these variables on the P300 to further explore its usefulness as a listening effort measurement for research and clinical purposes.

The Effects of Noise and Simulated Conductive Hearing Loss on Physiological Response Measures During Interactive Conversations

PURPOSE

The purpose of this work was to study the effects of background noise and hearing attenuation associated with earplugs on three physiological measures, assumed to be markers of effort investment and arousal, during interactive communication.

METHOD

Twelve pairs of older people (average age of 63.2 years) with age-adjusted normal hearing took part in a face-to-face communication to solve a Diapix task. Communication was held in different levels of babble noise (0, 60, and 70 dBA) and with two levels of hearing attenuation (0 and 25 dB) in quiet. The physiological measures obtained included pupil size, heart rate variability, and skin conductance. In addition, subjective ratings of perceived communication success, frustration, and effort were obtained.

RESULTS

Ratings of perceived success, frustration, and effort confirmed that communication was more difficult in noise and with approximately 25-dB hearing attenuation and suggested that the implemented levels of noise and hearing attenuation resulted in comparable communication difficulties. Background noise at 70 dBA and hearing attenuation both led to an initial increase in pupil size (associated with effort), but only the effect of the background noise was sustained throughout the conversation. The 25-dB hearing attenuation led to a significant decrease of the high-frequency power of heart rate variability and a significant increase of skin conductance level, measured as the average z value of the electrodermal activity amplitude.

CONCLUSION

This study demonstrated that several physiological measures appear to be viable indicators of changing communication conditions, with pupillometry and cardiovascular as well as electrodermal measures potentially being markers of communication difficulty.

The impact of speech type on listening effort and intelligibility for native and non-native listeners

Listeners are routinely exposed to many different types of speech, including artificially-enhanced and synthetic speech, styles which deviate to a greater or lesser extent from naturally-spoken exemplars. While the impact of differing speech types on intelligibility is well-studied, it is less clear how such types affect cognitive processing demands, and in particular whether those speech forms with the greatest intelligibility in noise have a commensurately lower listening effort. The current study measured intelligibility, self-reported listening effort, and a pupillometry-based measure of cognitive load for four distinct types of speech: (i) plain i.e. natural unmodified speech; (ii) Lombard speech, a naturally-enhanced form which occurs when speaking in the presence of noise; (iii) artificially-enhanced speech which involves spectral shaping and dynamic range compression; and (iv) speech synthesized from text. In the first experiment a cohort of 26 native listeners responded to the four speech types in three levels of speech-shaped noise. In a second experiment, 31 non-native listeners underwent the same procedure at more favorable signal-to-noise ratios, chosen since second language listening in noise has a more detrimental effect on intelligibility than listening in a first language. For both native and non-native listeners, artificially-enhanced speech was the most intelligible and led to the lowest subjective effort ratings, while the reverse was true for synthetic speech. However, pupil data suggested that Lombard speech elicited the lowest processing demands overall. These outcomes indicate that the relationship between intelligibility and cognitive processing demands is not a simple inverse, but is mediated by speech type. The findings of the current study motivate the search for speech modification algorithms that are optimized for both intelligibility and listening effort.

Copresence Was Found to Be Related to Some Pupil Measures in Persons With Hearing Loss While They Performed a Speech-in-Noise Task

OBJECTIVES

To assess if a manipulation of copresence was related to speech-in-noise task performance, arousal, and effort of persons with hearing loss. Task-related arousal and effort were measured by means of pupillometry.

DESIGN

Twenty-nine participants (mean age: 64.6 years) with hearing loss (4-frequency pure-tone average [4F-PTA] of 50.2 dB HL [SD = 8.9 dB] in the right ear and 51.3 dB HL [SD = 8.7 dB] in the left ear; averaged across 0.5, 1, 2, and 4 kHz) listened to and repeated spoken Danish sentences that were masked by four streams of continuous speech. Participants were presented with blocks of 20 sentences, during which copresence was manipulated by having participants do the task either alone or accompanied by two observers who were recruited from a similar age group. The task was presented at two difficulty levels, which was accomplished by fixing the signal-to-noise ratio of the speech and masker to match the thresholds at which participants were estimated to correctly repeat 50% (difficult) or 80% (easy) of the sentences in a block. Performance was assessed based on whether or not sentences were repeated correctly. Measures of pupil size (baseline pupil size [BPS], peak pupil dilation [PPD], and mean pupil dilation [MPD]) were used to index arousal and effort. Participants also completed ratings of subjective effort and stress after each block of sentences and a self-efficacy for listening-questionnaire.

RESULTS

Task performance was not associated with copresence, but was found to be related to 4F-PTA. An increase in BPS was found for copresence conditions, compared to alone conditions. Furthermore, a post-hoc exploratory analysis revealed that the copresence conditions were associated with a significantly larger pupil size in the second half of the task-evoked pupil response (TEPR). No change in PPD or MPD did was detected between copresence and alone conditions. Self-efficacy, 4F-PTA, and age were not found to be related to the pupil data. Subjective ratings were sensitive to task difficulty but not copresence.

CONCLUSION

Copresence was not found to be related to speech-in-noise performance, PPD, or MPD in persons with HL but was associated with an increase in arousal (as indicated by a larger BPS). This could be related to premobilization of effort and/or discomfort in response to the observers' presence. Furthermore, an exploratory analysis of the pupil data showed that copresence was associated with greater pupil dilations in the second half of the TEPR. This may indicate that participants invested more effort during the speech-in-noise task while in the presence of the observers, but that this increase in effort may not necessarily have been related to listening itself. Instead, other speech-in-noise task-related processes, such as preparing to respond, could have been influenced by copresence.

Speech-in-noise perception in autistic adolescents with and without early language delay

Speech-in-noise perception seems aberrant in individuals with autism spectrum disorder (ASD). Potential aggravating factors are the level of linguistic skills and impairments in auditory temporal processing. Here, we investigated autistic adolescents with and without language delay as compared to non-autistic peers, and we assessed speech perception in steady-state noise, temporally modulated noise, and concurrent speech. We found that autistic adolescents with intact language capabilities and not those with language delay performed worse than NT peers on words-in-stationary-noise perception. For the perception of sentences in stationary noise, we did not observe significant group differences, although autistic adolescents with language delay tend to perform worse in comparison to their TD peers. We also found evidence for a robust deficit in speech-in-concurrent-speech processing in ASD independent of language ability, as well as an association between early language delay in ASD and inadequate temporal speech processing. We propose that reduced voice stream segregation and inadequate social attentional orienting in ASD result in disproportional informational masking of the speech signal. These findings indicate a speech-in-speech processing deficit in autistic adolescents with broad implications for the quality of social communication.

Music in noise recognition: An EEG study of listening effort in cochlear implant users and normal hearing controls

Despite the plethora of studies investigating listening effort and the amount of research concerning music perception by cochlear implant (CI) users, the investigation of the influence of background noise on music processing has never been performed. Given the typical speech in noise recognition task for the listening effort assessment, the aim of the present study was to investigate the listening effort during an emotional categorization task on musical pieces with different levels of background noise. The listening effort was investigated, in addition to participants' ratings and performances, using EEG features known to be involved in such phenomenon, that is alpha activity in parietal areas and in the left inferior frontal gyrus (IFG), that includes the Broca's area. Results showed that CI users performed worse than normal hearing (NH) controls in the recognition of the emotional content of the stimuli. Furthermore, when considering the alpha activity corresponding to the listening to signal to noise ratio (SNR) 5 and SNR10 conditions subtracted of the activity while listening to the Quiet condition-ideally removing the emotional content of the music and isolating the difficulty level due to the SNRs- CI users reported higher levels of activity in the parietal alpha and in the homologous of the left IFG in the right hemisphere (F8 EEG channel), in comparison to NH. Finally, a novel suggestion of a particular sensitivity of F8 for SNR-related listening effort in music was provided.

Listening Effort in Tinnitus: A Pilot Study Employing a Light EEG Headset and Skin Conductance Assessment during the Listening to a Continuous Speech Stimulus under Different SNR Conditions

Background noise elicits listening effort. What else is tinnitus if not an endogenous background noise? From such reasoning, we hypothesized the occurrence of increased listening effort in tinnitus patients during listening tasks. Such a hypothesis was tested by investigating some indices of listening effort through electroencephalographic and skin conductance, particularly parietal and frontal alpha and electrodermal activity (EDA). Furthermore, tinnitus distress questionnaires (THI and TQ12-I) were employed. Parietal alpha values were positively correlated to TQ12-I scores, and both were negatively correlated to EDA; Pre-stimulus frontal alpha correlated with the THI score in our pilot study; finally, results showed a general trend of increased frontal alpha activity in the tinnitus group in comparison to the control group. Parietal alpha during the listening to stimuli, positively correlated to the TQ12-I, appears to reflect a higher listening effort in tinnitus patients and the perception of tinnitus symptoms. The negative correlation between both listening effort (parietal alpha) and tinnitus symptoms perception (TQ12-I scores) with EDA levels could be explained by a less responsive sympathetic nervous system to prepare the body to expend increased energy during the "fight or flight" response, due to pauperization of energy from tinnitus perception.

Streamlining experiment design in cognitive hearing science using OpenSesame

Auditory science increasingly builds on concepts and testing paradigms originated in behavioral psychology and cognitive neuroscience - an evolution of which the resulting discipline is now known as cognitive hearing science. Experimental cognitive hearing science paradigms call for hybrid cognitive and psychobehavioral tests such as those relating the attentional system, working memory, and executive functioning to low-level auditory acuity or speech intelligibility. Building complex multi-stimuli experiments can rapidly become time-consuming and error-prone. Platform-based experiment design can help streamline the implementation of cognitive hearing science experimental paradigms, promote the standardization of experiment design practices, and ensure reliability and control. Here, we introduce a set of features for the open-source python-based OpenSesame platform that allows the rapid implementation of custom behavioral and cognitive hearing science tests, including complex multichannel audio stimuli while interfacing with various synchronous inputs/outputs. Our integration includes advanced audio playback capabilities with multiple loudspeakers, an adaptive procedure, compatibility with standard I/Os and their synchronization through implementation of the Lab Streaming Layer protocol. We exemplify the capabilities of this extended OpenSesame platform with an implementation of the three-alternative forced choice amplitude modulation detection test and discuss reliability and performance. The new features are available free of charge from GitHub: https://github.com/elus-om/BRM_OMEXP .

Facial Expressions as an Index of Listening Difficulty and Emotional Response

Knowledge about listening difficulty experienced during a task can be used to better understand speech perception processes, to guide amplification outcomes, and can be used by individuals to decide whether to participate in communication. Another factor affecting these decisions is individuals' emotional response which has not been measured objectively previously. In this study, we describe a novel method of measuring listening difficulty and affect of individuals in adverse listening situations using automatic facial expression algorithm. The purpose of our study was to determine if facial expressions of confusion and frustration are sensitive to changes in listening difficulty. We recorded speech recognition scores, facial expressions, subjective listening effort scores, and subjective emotional responses in 33 young participants with normal hearing. We used the signal-to-noise ratios of -1, +2, and +5 dB SNR and quiet conditions to vary the difficulty level. We found that facial expression of confusion and frustration increased with increase in difficulty level, but not with change in each level. We also found a relationship between facial expressions and both subjective emotion ratings and subjective listening effort. Emotional responses in the form of facial expressions show promise as a measure of affect and listening difficulty. Further research is needed to determine the specific contribution of affect to communication in challenging listening environments.

Brightening the Study of Listening Effort with Functional Near-Infrared Spectroscopy: A Scoping Review

Listening effort is a long-standing area of interest in auditory cognitive neuroscience. Prior research has used multiple techniques to shed light on the neurophysiological mechanisms underlying listening during challenging conditions. Functional near-infrared spectroscopy (fNIRS) is growing in popularity as a tool for cognitive neuroscience research, and its recent advances offer many potential advantages over other neuroimaging modalities for research related to listening effort. This review introduces the basic science of fNIRS and its uses for auditory cognitive neuroscience. We also discuss its application in recently published studies on listening effort and consider future opportunities for studying effortful listening with fNIRS. After reading this article, the learner will know how fNIRS works and summarize its uses for listening effort research. The learner will also be able to apply this knowledge toward generation of future research in this area.

Sensory-Processing Sensitivity Predicts Fatigue From Listening, But Not Perceived Effort, in Young and Older Adults

PURPOSE

Listening-related fatigue is a potential negative consequence of challenges experienced during everyday listening and may disproportionately affect older adults. Contrary to expectation, we recently found that increased reports of listening-related fatigue were associated with better performance on a dichotic listening task. However, this link was found only in individuals who reported heightened sensitivity to a variety of physical, social, and emotional stimuli (i.e., increased "sensory-processing sensitivity" [SPS]). This study examined whether perceived effort may underlie the link between performance and fatigue.

METHOD

Two hundred six young adults, aged 18-30 years (Experiment 1), and 122 older adults, aged 60-80 years (Experiment 2), performed a dichotic listening task and were administered a series of questionnaires including the NASA Task Load Index of perceived effort, the Vanderbilt Fatigue Scale (measuring daily life listening-related fatigue), and the Highly Sensitive Person Scale (measuring SPS). Both experiments were completed online.

RESULTS

SPS predicted listening-related fatigue, but perceived effort during the listening task was not associated with SPS or listening-related fatigue in either age group. We were also unable to replicate the interaction between dichotic listening performance and SPS in either group. Exploratory analyses revealed contrasting effects of age; older adults found the dichotic listening task more effortful but indicated lower overall fatigue.

CONCLUSIONS

These findings suggest that SPS is a better predictor of listening-related fatigue than performance or effort ratings on a dichotic listening task. SPS may be an important factor in determining an individual's likelihood of experiencing listening-related fatigue irrespective of hearing or cognitive ability.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21893013.

Responses in left inferior frontal gyrus are altered for speech-in-noise processing, but not for clear speech in autism

INTRODUCTION

Autistic individuals often have difficulties with recognizing what another person is saying in noisy conditions such as in a crowded classroom or a restaurant. The underlying neural mechanisms of this speech perception difficulty are unclear. In typically developed individuals, three cerebral cortex regions are particularly related to speech-in-noise perception: the left inferior frontal gyrus (IFG), the right insula, and the left inferior parietal lobule (IPL). Here, we tested whether responses in these cerebral cortex regions are altered in speech-in-noise perception in autism.

METHODS

Seventeen autistic adults and 17 typically developed controls (matched pairwise on age, sex, and IQ) performed an auditory-only speech recognition task during functional magnetic resonance imaging (fMRI). Speech was presented either with noise (noise condition) or without noise (no noise condition, i.e., clear speech).

RESULTS

In the left IFG, blood-oxygenation-level-dependent (BOLD) responses were higher in the control compared to the autism group for recognizing speech-in-noise compared to clear speech. For this contrast, both groups had similar response magnitudes in the right insula and left IPL. Additionally, we replicated previous findings that BOLD responses in speech-related and auditory brain regions (including bilateral superior temporal sulcus and Heschl's gyrus) for clear speech were similar in both groups and that voice identity recognition was impaired for clear and noisy speech in autism.

DISCUSSION

Our findings show that in autism, the processing of speech is particularly reduced under noisy conditions in the left IFG-a dysfunction that might be important in explaining restricted speech comprehension in noisy environments.

Exploring the Correlations Between Measures of Listening Effort in Adults and Children: A Systematic Review with Narrative Synthesis

Listening effort (LE) describes the cognitive resources needed to process an auditory message. Our understanding of this notion remains in its infancy, hindering our ability to appreciate how it impacts individuals with hearing impairment effectively. Despite the myriad of proposed measurement tools, a validated method remains elusive. This is complicated by the seeming lack of association between tools demonstrated via correlational analyses. This review aims to systematically review the literature relating to the correlational analyses between different measures of LE. Five databases were used- PubMed, Cochrane, EMBASE, PsychINFO, and CINAHL. The quality of the evidence was assessed using the GRADE criteria and risk of bias with ROBINS-I/GRADE tools. Each statistically significant analysis was classified using an approved system for medical correlations. The final analyses included 48 papers, equating to 274 correlational analyses, of which 99 reached statistical significance (36.1%). Within these results, the most prevalent classifications were poor or fair. Moreover, when moderate or very strong correlations were observed, they tended to be dependent on experimental conditions. The quality of evidence was graded as very low. These results show that measures of LE are poorly correlated and supports the multi-dimensional concept of LE. The lack of association may be explained by considering where each measure operates along the effort perception pathway. Moreover, the fragility of significant correlations to specific conditions further diminishes the hope of finding an all-encompassing tool. Therefore, it may be prudent to focus on capturing the consequences of LE rather than the notion itself.

The Involvement of Listening Effort in Explaining Bilingual Listening Under Adverse Listening Conditions

The current review examines listening effort to uncover how it is implicated in bilingual performance under adverse listening conditions. Various measures of listening effort, including physiological, behavioral, and subjective measures, have been employed to examine listening effort in bilingual children and adults. Adverse listening conditions, stemming from environmental factors, as well as factors related to the speaker or listener, have been examined. The existing literature, although relatively limited to date, points to increased listening effort among bilinguals in their nondominant second language (L2) compared to their dominant first language (L1) and relative to monolinguals. Interestingly, increased effort is often observed even when speech intelligibility remains unaffected. These findings emphasize the importance of considering listening effort alongside speech intelligibility. Building upon the insights gained from the current review, we propose that various factors may modulate the observed effects. These include the particular measure selected to examine listening effort, the characteristics of the adverse condition, as well as factors related to the particular linguistic background of the bilingual speaker. Critically, further research is needed to better understand the impact of these factors on listening effort. The review outlines avenues for future research that would promote a comprehensive understanding of listening effort in bilingual individuals.

BPACE: A Bayesian, Patient-Centered Procedure for Matrix Speech Tests in Noise

Matrix sentence tests in noise can be challenging to the listener and time-consuming. A trade-off should be found between testing time, listener's comfort and the precision of the results. Here, a novel test procedure based on an updated maximum likelihood method was developed and implemented in a German matrix sentence test. It determines the parameters of the psychometric function (threshold, slope, and lapse-rate) without constantly challenging the listener at the intelligibility threshold. A so-called "credible interval" was used as a mid-run estimate of reliability and can be used as a termination criterion for the test. The procedure was evaluated and compared to a STAIRCASE procedure in a study with 20 cochlear implant patients and 20 normal hearing participants. The proposed procedure offers comparable accuracy and reliability to the reference method, but with a lower listening effort, as rated by the listeners (- 1.8 points on a 10-point scale). Test duration can be reduced by 1.3 min on average when a credible interval of 2 dB is used as the termination criterion instead of testing 30 sentences. Particularly, normal hearing listeners and well performing, cochlear implant users can benefit from shorter test duration. Although the novel procedure was developed for a German test, it can easily be applied to tests in any other language.

The Effect of the Use of Hearing Aids in Elders: Perspectives

Older adults with hearing loss have difficulties during conversation with others because an elevated auditory threshold reduces speech intelligibility, especially in noisy environments. Listening and comprehension often become exhausting tasks for hearing-impaired elders, resulting in social isolation and depression. The aim of the present study was to investigate the advantages of hearing aid use in relation to relief from listening-related fatigue, which is still controversial. Participants included a sample of 49 hearing-impaired elders affected by presbycusis for whom hearing aids were prescribed. The Modified Fatigue Impact Scale was used to assess cognitive, physical and psychosocial fatigue. The vitality subscale of the Short Form Health Survey 36 and a single item of the multi-dimensional Speech, Spatial and Quality Hearing Scale (“Do you have to put a lot of effort to hear what is being said in conversation with others?”) were also used. The Cognitive Failures Questionnaire was used to investigate daily errors related to lack of memory and reduced mindedness. Hearing aids rehabilitation resulted in improved speech intelligibility in competing noise, and a significant reduction in cognitive and psychosocial fatigue and listening effort in conversation. Vitality was also improved and a significant reduction in the Cognitive Failures Questionnaire scores was observed. Findings from the study indicate that the use of hearing aids in older impaired-listeners provide them not only with an increased auditory function but also with a reduction in listening-related fatigue and mindedness.

Interdisciplinary Approaches to the Study of Listening Effort in Young Children with Cochlear Implants

Very early bilateral implantation is thought to significantly reduce the attentional effort required to acquire spoken language, and consequently offer a profound improvement in quality of life. Despite the early intervention, however, auditory and communicative outcomes in children with cochlear implants remain poorer than in hearing children. The distorted auditory input via the cochlear implants requires more auditory attention resulting in increased listening effort and fatigue. Listening effort and fatigue may critically affect attention to speech, and in turn language processing, which may help to explain the variation in language and communication abilities. However, measuring attention to speech and listening effort is demanding in infants and very young children. Three objective techniques for measuring listening effort are presented in this paper that may address the challenges of testing very young and/or uncooperative children with cochlear implants: pupillometry, electroencephalography, and functional near-infrared spectroscopy. We review the studies of listening effort that used these techniques in paediatric populations with hearing loss, and discuss potential benefits of the systematic evaluation of listening effort in these populations.

Effortful listening: Sympathetic activity varies as a function of listening demand but parasympathetic activity does not

Research on listening effort has used various physiological measures to examine the biological correlates of listening effort but a systematic examination of the impact of listening demand on cardiac autonomic nervous system activity is still lacking. The presented study aimed to close this gap by assessing cardiac sympathetic and parasympathetic responses to variations in listening demand. For this purpose, 45 participants performed four speech-in-noise tasks differing in listening demand-manipulated as signal-to-noise ratio varying between +23 dB and -16 dB-while their pre-ejection period and respiratory sinus arrythmia responses were assessed. Cardiac responses showed the expected effect of listening demand on sympathetic activity, but failed to provide evidence for the expected listening demand impact on parasympathetic activity: Pre-ejection period reactivity increased with increasing listening demand across the three possible listening conditions and was low in the very high (impossible) demand condition, whereas respiratory sinus arrythmia did not show this pattern. These findings have two main implications. First, cardiac sympathetic responses seem to be the more sensitive correlate of the impact of task demand on listening effort compared to cardiac parasympathetic responses. Second, very high listening demand may lead to disengagement and correspondingly low effort and reduced cardiac sympathetic response.

Cognitive and Physiological Measures of Listening Effort During Degraded Speech Perception: Relating Dual-Task and Pupillometry Paradigms

Purpose Listening effort is quickly becoming an important metric for assessing speech perception in less-than-ideal situations. However, the relationship between the construct of listening effort and the measures used to assess it remains unclear. We compared two measures of listening effort: a cognitive dual task and a physiological pupillometry task. We sought to investigate the relationship between these measures of effort and whether engaging effort impacts speech accuracy. Method In Experiment 1, 30 participants completed a dual task and a pupillometry task that were carefully matched in stimuli and design. The dual task consisted of a spoken word recognition task and a visual match-to-sample task. In the pupillometry task, pupil size was monitored while participants completed a spoken word recognition task. Both tasks presented words at three levels of listening difficulty (unmodified, eight-channel vocoding, and four-channel vocoding) and provided response feedback on every trial. We refined the pupillometry task in Experiment 2 (n = 31); crucially, participants no longer received response feedback. Finally, we ran a new group of subjects on both tasks in Experiment 3 (n = 30). Results In Experiment 1, accuracy in the visual task decreased with increased signal degradation in the dual task, but pupil size was sensitive to accuracy and not vocoding condition. After removing feedback in Experiment 2, changes in pupil size were predicted by listening condition, suggesting the task was now sensitive to engaged effort. Both tasks were sensitive to listening difficulty in Experiment 3, but there was no relationship between the tasks and neither task predicted speech accuracy. Conclusions Consistent with previous work, we found little evidence for a relationship between different measures of listening effort. We also found no evidence that effort predicts speech accuracy, suggesting that engaging more effort does not lead to improved speech recognition. Cognitive and physiological measures of listening effort are likely sensitive to different aspects of the construct of listening effort. Supplemental Material https://doi.org/10.23641/asha.16455900.

Social observation increases the cardiovascular response of hearing-impaired listeners during a speech reception task

Certain cardiovascular measures allow for distinction between sympathetic and parasympathetic nervous system activity. Applied during listening, these measures may provide a novel and complementary insight into listening effort. To date, few studies have implemented cardiovascular measures of listening effort and seldom have these included hearing-impaired participants. These studies have generally measured changes in cardiovascular parameters while manipulating environmental factors, such as listening difficulty. Yet, listening effort is also known to be moderated by individual factors, including the importance of performing successfully. In this study, we aimed to manipulate success importance by adding observers to the traditional laboratory set-up. Twenty-nine hearing-impaired participants performed a speech reception task both alone and in the presence of two observers. Auditory stimuli consisted of Danish Hearing in Noise Test (HINT) sentences masked by four-talker babble. Sentences were delivered at two individually adapted signal-to-noise ratios, corresponding to 50 and 80% of sentences correct. We measured change scores, relative to baseline, of pre-ejection period, two indices of heart rate variability, heart rate and blood pressure (systolic, diastolic, and mean arterial pressure). After each condition, participants rated their effort investment, stress, tendency to give up and preference to change the situation to improve audibility. A multivariate analysis revealed that cardiovascular reactivity increased in the presence of the observers, compared to when the task was performed alone. More specifically, systolic, diastolic, and mean arterial blood pressure increased while observed. Interestingly, participants' subjective ratings were sensitive only to intelligibility level, not the observation state. This study was the first to report results from a range of different cardiovascular variables measured from hearing-impaired participants during a speech reception task. Due to the timing of the observers' presence, we were not able to conclusively attribute these physiological changes to being task related. Therefore, instead of representing listening effort, we suggest that the increased cardiovascular response detected during observation reveals increased physiological stress associated with potential evaluation.

Listening Effort Is Not the Same as Speech Intelligibility Score

Listening effort is a valuable and important notion to measure because it is among the primary complaints of people with hearing loss. It is tempting and intuitive to accept speech intelligibility scores as a proxy for listening effort, but this link is likely oversimplified and lacks actionable explanatory power. This study was conducted to explain the mechanisms of listening effort that are not captured by intelligibility scores, using sentence-repetition tasks where specific kinds of mistakes were prospectively planned or analyzed retrospectively. Effort measured as changes in pupil size among 20 listeners with normal hearing and 19 listeners with cochlear implants. Experiment 1 demonstrates that mental correction of misperceived words increases effort even when responses are correct. Experiment 2 shows that for incorrect responses, listening effort is not a function of the proportion of words correct but is rather driven by the types of errors, position of errors within a sentence, and the need to resolve ambiguity, reflecting how easily the listener can make sense of a perception. A simple taxonomy of error types is provided that is both intuitive and consistent with data from these two experiments. The diversity of errors in these experiments implies that speech perception tasks can be designed prospectively to elicit the mistakes that are more closely linked with effort. Although mental corrective action and number of mistakes can scale together in many experiments, it is possible to dissociate them to advance toward a more explanatory (rather than correlational) account of listening effort.

EEG power spectral dynamics associated with listening in adverse conditions

Adverse listening conditions increase the demand on cognitive resources needed for speech comprehension. In an exploratory study, we aimed to identify independent power spectral features in the EEG useful for studying the cognitive processes involved in this effortful listening. Listeners performed the coordinate response measure task with a single-talker masker at a 0-dB signal-to-noise ratio. Sounds were left unfiltered or degraded with low-pass filtering. Independent component analysis (ICA) was used to identify independent components (ICs) in the EEG data, the power spectral dynamics of which were then analyzed. Frontal midline theta, left frontal, right frontal, left mu, right mu, left temporal, parietal, left occipital, central occipital, and right occipital clusters of ICs were identified. All IC clusters showed some significant listening-related changes in their power spectrum. This included sustained theta enhancements, gamma enhancements, alpha enhancements, alpha suppression, beta enhancements, and mu rhythm suppression. Several of these effects were absent or negligible using traditional channel analyses. Comparison of filtered to unfiltered speech revealed a stronger alpha suppression in the parietal and central occipital clusters of ICs for the filtered speech condition. This not only replicates recent findings showing greater alpha suppression as listening difficulty increases but also suggests that such alpha-band effects can stem from multiple cortical sources. We lay out the advantages of the ICA approach over the restrictive analyses that have been used as of late in the study of listening effort. We also make suggestions for moving into hypothesis-driven studies regarding the power spectral features that were revealed.

Listener characteristics differentially affect self-reported and physiological measures of effort associated with two challenging listening conditions

Listeners vary in their ability to understand speech in adverse conditions. Differences in both cognitive and linguistic capacities play a role, but increasing evidence suggests that such factors may contribute differentially depending on the listening challenge. Here, we used multilevel modeling to evaluate contributions of individual differences in age, hearing thresholds, vocabulary, selective attention, working memory capacity, personality traits, and noise sensitivity to variability in measures of comprehension and listening effort in two listening conditions. A total of 35 participants completed a battery of cognitive and linguistic tests as well as a spoken story comprehension task using (1) native-accented English speech masked by speech-shaped noise and (2) nonnative accented English speech without masking. Masker levels were adjusted individually to ensure each participant would show (close to) equivalent word recognition performance across the two conditions. Dependent measures included comprehension tests results, self-rated effort, and electrodermal, cardiovascular, and facial electromyographic measures associated with listening effort. Results showed varied patterns of responsivity across different dependent measures as well as across listening conditions. In particular, results suggested that working memory capacity may play a greater role in the comprehension of nonnative accented speech than noise-masked speech, while hearing acuity and personality may have a stronger influence on physiological responses affected by demands of understanding speech in noise. Furthermore, electrodermal measures may be more strongly affected by affective response to noise-related interference while cardiovascular responses may be more strongly affected by demands on working memory and lexical access.

Pupil Dilation Is Sensitive to Semantic Ambiguity and Acoustic Degradation

Speech comprehension is challenged by background noise, acoustic interference, and linguistic factors, such as the presence of words with more than one meaning (homonyms and homophones). Previous work suggests that homophony in spoken language increases cognitive demand. Here, we measured pupil dilation—a physiological index of cognitive demand—while listeners heard high-ambiguity sentences, containing words with more than one meaning, or well-matched low-ambiguity sentences without ambiguous words. This semantic-ambiguity manipulation was crossed with an acoustic manipulation in two experiments. In Experiment 1, sentences were masked with 30-talker babble at 0 and +6 dB signal-to-noise ratio (SNR), and in Experiment 2, sentences were heard with or without a pink noise masker at –2 dB SNR. Speech comprehension was measured by asking listeners to judge the semantic relatedness of a visual probe word to the previous sentence. In both experiments, comprehension was lower for high- than for low-ambiguity sentences when SNRs were low. Pupils dilated more when sentences included ambiguous words, even when no noise was added (Experiment 2). Pupil also dilated more when SNRs were low. The effect of masking was larger than the effect of ambiguity for performance and pupil responses. This work demonstrates that the presence of homophones, a condition that is ubiquitous in natural language, increases cognitive demand and reduces intelligibility of speech heard with a noisy background.

NeuroDante: Poetry Mentally Engages More Experts but Moves More Non-Experts, and for Both the Cerebral Approach Tendency Goes Hand in Hand with the Cerebral Effort

Neuroaesthetics, the science studying the biological underpinnings of aesthetic experience, recently extended its area of investigation to literary art; this was the humus where neurocognitive poetics blossomed. Divina Commedia represents one of the most important, famous and studied poems worldwide. Poetry stimuli are characterized by elements (meter and rhyme) promoting the processing fluency, a core aspect of neuroaesthetics theories. In addition, given the evidence of different neurophysiological reactions between experts and non-experts in response to artistic stimuli, the aim of the present study was to investigate, in poetry, a different neurophysiological cognitive and emotional reaction between Literature (L) and Non-Literature (NL) students. A further aim was to investigate whether neurophysiological underpinnings would support explanation of behavioral data. Investigation methods employed: self-report assessments (recognition, appreciation, content recall) and neurophysiological indexes (approach/withdrawal (AW), cerebral effort (CE) and galvanic skin response (GSR)). The main behavioral results, according to fluency theories in aesthetics, suggested in the NL but not in the L group that the appreciation/liking went hand by hand with the self-declared recognition and with the content recall. The main neurophysiological results were: (i) higher galvanic skin response in NL, whilst higher CE values in L; (ii) a positive correlation between AW and CE indexes in both groups. The present results extended previous evidence relative to figurative art also to auditory poetry stimuli, suggesting an emotional attenuation “expertise-specific” showed by experts, but increased cognitive processing in response to the stimuli.

Comparisons of the Sensitivity and Reliability of Multiple Measures of Listening Effort

OBJECTIVES

The objective of this study was to evaluate the sensitivity and reliability of one subjective (rating scale) and three objective (dual-task paradigm, pupillometry, and skin conductance response amplitude) measures of listening effort across multiple signal to noise ratios (SNRs).

DESIGN

Twenty adults with normal hearing attended two sessions and listened to sentences presented in quiet and in stationary noise at three different SNRs: 0, -3, and -5 dB. Listening effort was assessed by examining change in reaction time (dual-task paradigm), change in peak to peak pupil diameter (pupillometry), and change in mean skin conductance response amplitude; self-reported listening effort on a scale from 0 to 100 was also evaluated. Responses were averaged within each SNR and based on three word recognition ability categories (≤50%, 51% to 71%, and >71%) across all SNRs. Measures were considered reliable if there were no significant changes between sessions, and intraclass correlation coefficients were a minimum of 0.40. Effect sizes were calculated to compare the sensitivity of the measures.

RESULTS

Intraclass correlation coefficient values indicated fair-to-moderate reliability for all measures while individual measurement sensitivity was variable. Self-reports were sensitive to listening effort but were less reliable, given that subjective effort was greater during the dual task than either of the physiologic measures. The dual task was sensitive to a narrow range of word recognition abilities but was less reliable as it exhibited a global decrease in reaction time across sessions. Pupillometry was consistently sensitive and reliable to changes in listening effort. Skin conductance response amplitude was not sensitive or reliable while the participants listened to the sentences. Skin conductance response amplitude during the verbal response was sensitive to poor (≤50%) speech recognition abilities; however, it was less reliable as there was a significant change in amplitude across sessions.

CONCLUSIONS

In this study, pupillometry was the most sensitive and reliable objective measure of listening effort. Intersession variability significantly influenced the other objective measures of listening effort, which suggests challenges for cross-study comparability. Therefore, intraclass correlation coefficients combined with other statistical tests more fully describe the reliability of measures of listening effort across multiple difficulties. Minimizing intersession variability will increase measurement sensitivity. Further work toward standardized methods and analysis will strengthen our understanding of the reliability and sensitivity of measures of listening effort and better facilitate cross-modal and cross-study comparisons.

The Impact of Speech Recognition Testing on State Anxiety in Young, Middle-Age, and Older Adults

Purpose This study assessed state anxiety as a function of speech recognition testing using three clinical measures of speech in noise and one clinical measure of dichotic speech recognition. Method Thirty young adults, 30 middle-age adults, and 25 older adults participated. State anxiety was measured pre- and post-speech recognition testing using the State-Trait Anxiety Inventory. Speech recognition was measured with the Revised Speech Perception in Noise Test, the Quick Speech-in-Noise Test, the Words-in-Noise Test, and the Dichotic Digits Test (DDT). Results Speech recognition performance was as expected: Older adults performed significantly poorer on all measures as compared to the young adults and significantly poorer on the Revised Speech Perception in Noise Test, the Quick Speech-in-Noise Test, and the Words-in-Noise Test as compared to the middle-age adults. On average, State-Trait Anxiety Inventory scores increased posttesting, with the middle-age adults exhibiting significantly greater increases in state anxiety as compared to the young and older adults. Increases in state anxiety were significantly greater for the DDT relative to the speech-in-noise tests for the middle-age adults only. Poorer DDT recognition performance was associated with higher levels of state anxiety. Conclusions Increases in state anxiety were observed after speech-in-noise and dichotic listening testing for all groups, with significant increases seen for the young and middle-age adults. Although the exact mechanisms could not be determined, multiple factors likely influenced the observed increases in state anxiety, including task difficulty, individual proficiency, and age.

Perceived listening effort in children with hearing loss: listening to a dysphonic voice in quiet and in noise

AIM

The present study investigates the effect of signal degradation on perceived listening effort in children with hearing loss listening in a simulated class-room context. It also examines the associations between perceived listening effort, passage comprehension performance and executive functioning.

METHODS

Twenty-four children (aged 06:03-13:00 years) with hearing impairment using cochlear implant (CI) and/or hearing aids (HA) participated. The children made ratings of perceived listening effort after completing an auditory passage comprehension task. All children performed the task in four different listening conditions: listening to a typical (i.e. normal) voice in quiet, to a dysphonic voice in quiet, to a typical voice in background noise and to a dysphonic voice in background noise. In addition, the children completed a task assessing executive function.

RESULTS

Both voice quality and background noise increased perceived listening effort in children with CI/HA, but no interaction with executive function was seen.

CONCLUSION

Since increased listening effort seems to be a consequence of increased cognitive resource spending, it is likely that less resources will be available for these children not only to comprehend but also to learn in challenging listening environments such as classrooms.

Rapid adaptation to fully intelligible nonnative-accented speech reduces listening effort

In noisy settings or when listening to an unfamiliar talker or accent, it can be difficult to understand spoken language. This difficulty typically results in reductions in speech intelligibility, but may also increase the effort necessary to process the speech even when intelligibility is unaffected. In this study, we used a dual-task paradigm and pupillometry to assess the cognitive costs associated with processing fully intelligible accented speech, predicting that rapid perceptual adaptation to an accent would result in decreased listening effort over time. The behavioural and physiological paradigms provided converging evidence that listeners expend greater effort when processing nonnative- relative to native-accented speech, and both experiments also revealed an overall reduction in listening effort over the course of the experiment. Only the pupillometry experiment, however, revealed greater adaptation to nonnative- relative to native-accented speech. An exploratory analysis of the dual-task data that attempted to minimise practice effects revealed weak evidence for greater adaptation to the nonnative accent. These results suggest that even when speech is fully intelligible, resolving deviations between the acoustic input and stored lexical representations incurs a processing cost, and adaptation may attenuate this cost.

Brief Report: Speech-in-Noise Recognition and the Relation to Vocal Pitch Perception in Adults with Autism Spectrum Disorder and Typical Development

We tested the ability to recognise speech-in-noise and its relation to the ability to discriminate vocal pitch in adults with high-functioning autism spectrum disorder (ASD) and typically developed adults (matched pairwise on age, sex, and IQ). Typically developed individuals understood speech in higher noise levels as compared to the ASD group. Within the control group but not within the ASD group, better speech-in-noise recognition abilities were significantly correlated with better vocal pitch discrimination abilities. Our results show that speech-in-noise recognition is restricted in people with ASD. We speculate that perceptual impairments such as difficulties in vocal pitch perception might be relevant in explaining these difficulties in ASD.

Measures of Listening Effort Are Multidimensional

OBJECTIVES

Listening effort can be defined as the cognitive resources required to perform a listening task. The literature on listening effort is as confusing as it is voluminous: measures of listening effort rarely correlate with each other and sometimes result in contradictory findings. Here, we directly compared simultaneously recorded multimodal measures of listening effort. After establishing the reliability of the measures, we investigated validity by quantifying correlations between measures and then grouping-related measures through factor analysis.

DESIGN

One hundred and sixteen participants with audiometric thresholds ranging from normal to severe hearing loss took part in the study (age range: 55 to 85 years old, 50.3% male). We simultaneously measured pupil size, electroencephalographic alpha power, skin conductance, and self-report listening effort. One self-report measure of fatigue was also included. The signal to noise ratio (SNR) was adjusted at 71% criterion performance using sequences of 3 digits. The main listening task involved correct recall of a random digit from a sequence of six presented at a SNR where performance was around 82 to 93%. Test-retest reliability of the measures was established by retesting 30 participants 7 days after the initial session.

RESULTS

With the exception of skin conductance and the self-report measure of fatigue, interclass correlation coefficients (ICC) revealed good test-retest reliability (minimum ICC: 0.71). Weak or nonsignificant correlations were identified between measures. Factor analysis, using only the reliable measures, revealed four underlying dimensions: factor 1 included SNR, hearing level, baseline alpha power, and performance accuracy; factor 2 included pupillometry; factor 3 included alpha power (during speech presentation and during retention); factor 4 included self-reported listening effort and baseline alpha power.

CONCLUSIONS

The good ICC suggests that poor test reliability is not the reason for the lack of correlation between measures. We have demonstrated that measures traditionally used as indicators of listening effort tap into multiple underlying dimensions. We therefore propose that there is no "gold standard" measure of listening effort and that different measures of listening effort should not be used interchangeably. When choosing method(s) to measure listening effort, the nature of the task and aspects of increased listening demands that are of interest should be taken into account. The findings of this study provide a framework for understanding and interpreting listening effort measures.

Understanding Speech Amid the Jingle and Jangle: Recommendations for Improving Measurement Practices in Listening Effort Research

The latent constructs psychologists study are typically not directly accessible, so researchers must design measurement instruments that are intended to provide insights about those constructs. Construct validation-assessing whether instruments measure what they intend to-is therefore critical for ensuring that the conclusions we draw actually reflect the intended phenomena. Insufficient construct validation can lead to the jingle fallacy-falsely assuming two instruments measure the same construct because the instruments share a name (Thorndike, 1904)-and the jangle fallacy-falsely assuming two instruments measure different constructs because the instruments have different names (Kelley, 1927). In this paper, we examine construct validation practices in research on listening effort and identify patterns that strongly suggest the presence of jingle and jangle in the literature. We argue that the lack of construct validation for listening effort measures has led to inconsistent findings and hindered our understanding of the construct. We also provide specific recommendations for improving construct validation of listening effort instruments, drawing on the framework laid out in a recent paper on improving measurement practices (Flake & Fried, 2020). Although this paper addresses listening effort, the issues raised and recommendations presented are widely applicable to tasks used in research on auditory perception and cognitive psychology.

Validação de conteúdo de um instrumento para mensuração do esforço auditivo

RESUMO Objetivo: Validar o conteúdo de um instrumento para mensuração do esforço auditivo para indivíduos com perda auditiva. Método: Trata-se de um estudo de validação, desenvolvido em duas fases, sendo a fase 1 o planejamento e desenvolvimento da primeira versão do instrumento e a fase 2 a investigação das evidências de validade baseadas no conteúdo do instrumento e desenvolvimento da versão final para mensuração de esforço auditivo. Participaram dez profissionais com expertise na área audiológica, com mais de cinco anos de experiência. O instrumento a ser validado foi composto por três partes: I - “percepção de fala de logatomas e esforço auditivo”; II - “esforço auditivo e memória operacional”; e III - “percepção de sentenças sem sentido e memória operacional”, apresentadas de forma monoaural no silêncio e nas relações sinal-ruído +5dB, 0dB e -5dB. Foi realizada a análise descritiva das sugestões do comitê de fonoaudiólogos e do índice de validade de conteúdo individual e total. Resultados: Os resultados mostraram que as partes I e III do instrumento proposto atingiram o índice de validade de conteúdo total acima de 0,78, ou seja, os itens apresentados não necessitaram de modificações em seu constructo. Conclusão: As evidências de validade estudadas permitiram relevantes modificações e tornaram esse instrumento adequado ao seu constructo.

Listening effort: Are we measuring cognition or affect, or both?

Listening effort is increasingly recognized as a factor in communication, particularly for and with nonnative speakers, for the elderly, for individuals with hearing impairment and/or for those working in noise. However, as highlighted by McGarrigle et al., International Journal of Audiology, 2014, 53, 433-445, the term "listening effort" encompasses a wide variety of concepts, including the engagement and control of multiple possibly distinct neural systems for information processing, and the affective response to the expenditure of those resources in a given context. Thus, experimental or clinical methods intended to objectively quantify listening effort may ultimately reflect a complex interaction between the operations of one or more of those information processing systems, and/or the affective and motivational response to the demand on those systems. Here we examine theoretical, behavioral, and psychophysiological factors related to resolving the question of what we are measuring, and why, when we measure "listening effort." This article is categorized under: Linguistics > Language in Mind and Brain Psychology > Theory and Methods Psychology > Attention Psychology > Emotion and Motivation.

Moderate Reverberation Does Not Increase Subjective Fatigue, Subjective Listening Effort, or Behavioral Listening Effort in School-Aged Children

Background noise and reverberation levels in typical classrooms have negative effects on speech recognition, but their effects on listening effort and fatigue are less well understood. Based on the Framework for Understanding Effortful Listening, noise and reverberation would be expected to increase both listening effort and fatigue. However, previous investigations of the effects of reverberation for adults have resulted in mixed findings. Some discrepancies in the literature might be accounted for by methodological differences; behavioral and subjective indices of listening effort do not often align in adults. The effects of sustained listening on self-reported fatigue in school-aged children are also not well understood. The purposes of this project were to (1) evaluate the effects of noise and reverberation on listening effort in school-aged children using behavioral and subjective measures, (2) compare subjective and behavioral indices of listening effort, and (3) evaluate the effects of reverberation on self-reported fatigue. Twenty typically developing children (10-17 years old) participated. Participants completed dual-task testing in two rooms that varied in terms of reverberation, an audiometric sound booth and a moderately reverberant room. In each room, testing was completed in quiet and in two levels of background noise. Participants provided subjective ratings of listening effort after completing the dual-task in each listening condition. Subjective ratings of fatigue were completed before and after testing in each level of reverberation. Results revealed background noise, not reverberation, increased behavioral and subjective listening effort. Subjective ratings of perceived performance, ease of listening, and desire to control the listening situation revealed a similar pattern of results as word recognition performance, making them poor candidates for providing an indication of behavioral listening effort. However, ratings of time perception were moderately correlated with behavioral listening effort. Finally, sustained listening for approximately 25 min increased self-reported fatigue, although changes in fatigue were comparable in low and moderately reverberant environments. In total, these data offer no evidence that a moderate level of reverberation increases listening effort or fatigue, but the data do support the reduction of background noise in classrooms.

Effect of Auditory Task Type on Physiological and Subjective Measures of Listening Effort in Individuals With Normal Hearing

Purpose Listening effort has traditionally been measured using subjective rating scales and behavioral measures. Recent physiological measures of listening effort have utilized pupil dilation. Using a combination of physiological and subjective measures of listening effort, this study aimed to identify differences in listening effort during 2 auditory tasks: sentence recognition and word recognition. Method Pupil dilation and subjective ratings of listening effort were obtained for auditory tasks utilizing AzBio sentences recognition and Northwestern University Auditory Test No. 6 words recognition, across 3 listening situations: in quiet, at +6 dB signal-to-noise ratio, and at 0 dB signal-to-noise ratio. Task accuracy was recorded for each of the 6 conditions, as well as peak pupil dilation and a subjective rating of listening effort. Results A significant impact of listening situation (quiet vs. noise) and task type (sentence recognition vs. word recognition) on both physiological and subjective measures was found. There was a significant interaction between listening situation and task type, suggesting that contextual cues may only be beneficial when audibility is uncompromised. The current study found no correlation between the physiological and subjective measures, possibly suggesting that these measures analyze different aspects of cognitive effort in a listening task.

Listening Effort by Native and Nonnative Listeners Due to Noise, Reverberation, and Talker Foreign Accent During English Speech Perception

Purpose Understanding speech in complex realistic acoustic environments requires effort. In everyday listening situations, speech quality is often degraded due to adverse acoustics, such as excessive background noise level (BNL) and reverberation time (RT), or talker characteristics such as foreign accent ( Mattys, Davis, Bradlow, & Scott, 2012 ). In addition to factors affecting the quality of the input acoustic signals, listeners' individual characteristics such as language abilities can also make it more difficult and effortful to understand speech. Based on the Framework for Understanding Effortful Listening ( Pichora-Fuller et al., 2016 ), factors such as adverse acoustics, talker accent, and listener language abilities can all contribute to increasing listening effort. In this study, using both a dual-task paradigm and a self-report questionnaire, we seek to understand listening effort in a wide range of realistic classroom acoustic conditions as well as varying talker accent and listener English proficiency. Method One hundred fifteen native and nonnative adult listeners with normal hearing were tested in a dual task of speech comprehension and adaptive pursuit rotor (APR) under 15 acoustic conditions from combinations of BNLs and RTs. Listeners provided responses on the NASA Task Load Index (TLX) questionnaire immediately after completing the dual task under each acoustic condition. The NASA TLX surveyed 6 dimensions of perceived listening effort: mental demand, physical demand, temporal demand, effort, frustration, and perceived performance. Fifty-six listeners were tested with speech produced by native American English talkers; the other 59 listeners, with speech from native Mandarin Chinese talkers. Based on their 1st language learned during childhood, 3 groups of listeners were recruited: listeners who were native English speakers, native Mandarin Chinese speakers, and native speakers of other languages (e.g., Hindu, Korean, and Portuguese). Results Listening effort was measured objectively through the APR task performance and subjectively using the NASA TLX questionnaire. Performance on the APR task did not vary with changing acoustic conditions, but it did suggest increased listening effort for native listeners of other languages compared to the 2 other listener groups. From the NASA TLX, listeners reported feeling more frustrated and less successful in understanding Chinese-accented speech. Nonnative listeners reported more listening effort (i.e., physical demand, temporal demand, and effort) than native listeners in speech comprehension under adverse acoustics. When listeners' English proficiency was controlled, higher BNL was strongly related to a decrease in perceived performance, whereas such relationship with RT was much weaker. Nonnative listeners who shared the foreign talkers' accent reported no change in listening effort, whereas other listeners reported more difficulty in understanding the accented speech. Conclusions Adverse acoustics required more effortful listening as measured subjectively with a self-report NASA TLX. This subjective scale was more sensitive than a dual task that involved speech comprehension, which was beyond sentence recall. It was better at capturing the negative impacts on listening effort from acoustic factors (i.e., both BNL and RT), talker accent, and listener language abilities.

Emotionally conditioning the target-speech voice enhances recognition of the target speech under "cocktail-party" listening conditions

Under a noisy "cocktail-party" listening condition with multiple people talking, listeners can use various perceptual/cognitive unmasking cues to improve recognition of the target speech against informational speech-on-speech masking. One potential unmasking cue is the emotion expressed in a speech voice, by means of certain acoustical features. However, it was unclear whether emotionally conditioning a target-speech voice that has none of the typical acoustical features of emotions (i.e., an emotionally neutral voice) can be used by listeners for enhancing target-speech recognition under speech-on-speech masking conditions. In this study we examined the recognition of target speech against a two-talker speech masker both before and after the emotionally neutral target voice was paired with a loud female screaming sound that has a marked negative emotional valence. The results showed that recognition of the target speech (especially the first keyword in a target sentence) was significantly improved by emotionally conditioning the target speaker's voice. Moreover, the emotional unmasking effect was independent of the unmasking effect of the perceived spatial separation between the target speech and the masker. Also, (skin conductance) electrodermal responses became stronger after emotional learning when the target speech and masker were perceptually co-located, suggesting an increase of listening efforts when the target speech was informationally masked. These results indicate that emotionally conditioning the target speaker's voice does not change the acoustical parameters of the target-speech stimuli, but the emotionally conditioned vocal features can be used as cues for unmasking target speech.

Effects of Task Difficulty on Neural Processes Underlying Semantics: An Event-Related Potentials Study

Purpose Improving the ability to listen efficiently in noisy environments is a critical goal for hearing rehabilitation. However, understanding of the impact of difficult listening conditions on language processing is limited. The current study evaluated the neural processes underlying semantics in challenging listening conditions. Method Thirty adults with normal hearing completed an auditory sentence processing task in 4-talker babble. Event-related brain potentials were elicited by the final word in high- or low-context sentences, where the final word was either highly expected or not expected, followed by a 4-alternative forced-choice response with either longer (1,000 ms), middle (700 ms), or shorter (400 ms) response time deadlines (RTDs). Results Behavioral accuracy was reduced, and reactions times were faster for shorter RTDs. N400 amplitudes, reflecting ease of lexical access, were larger when elicited by target words in low-context sentences followed by shorter compared with longer RTDs. Conclusions These results reveal that more neural resources are allocated for semantic processing/lexical access when listening difficulty increases. Differences between RTDs may reflect increased attentional allocation for shorter RTDs. These findings suggest that situational listening demands can impact the demands for cognitive resources engaged in language processing, which could significantly impact listener experiences across environments.

Mensuração do esforço auditivo com o uso de um paradigma de tarefa dupla do Português Brasileiro: estudo-piloto

RESUMO Objetivo Mensurar o esforço auditivo com o uso de um paradigma de tarefa dupla de memória operacional e analisar a significância clínica do desempenho de indivíduos normo-ouvintes. Método Participaram 10 adultos jovens, entre 18 e 30 anos, de ambos os gêneros, normo-ouvintes classificados segundo a média quadritonal (500, 1000, 2000 e 4000Hz) e com nível sociocultural similar. Os participantes foram submetidos à anamnese audiológica, meatoscopia e audiometria tonal limiar. Para a mensuração do esforço auditivo, utilizou-se um paradigma de tarefa dupla, composto por tarefas de percepção de fala e memória operacional de logatomas, palavras reais e sentenças sem sentido. Anteriormente à mensuração, o paradigma de tarefa dupla foi realizado no silêncio com o intuito de treinar os participantes a desempenharem as tarefas adequadamente. Após a fase de treinamento, este paradigma foi realizado em duas situações de escuta distintas, nas relações sinal/ruído de +5 e -5dB, com o ruído do tipo White Noise. Resultados A comparação do desempenho por orelha, direita ou esquerda, nas duas relações sinal-ruído demonstrou efeito significante para as tarefas de percepção de fala de logatomas e sentenças sem sentido em ambas as orelhas, porém para a tarefa de esforço auditivo e memória operacional houve diferença significante apenas para a orelha direita. Conclusão Foi possível mensurar o esforço auditivo com o uso do paradigma proposto e este instrumento demonstrou ser sensível para a quantificação deste parâmetro auditivo.