1
|
Keur-Huizinga L, Kramer SE, de Geus EJC, Zekveld AA. A Multimodal Approach to Measuring Listening Effort: A Systematic Review on the Effects of Auditory Task Demand on Physiological Measures and Their Relationship. Ear Hear 2024; 45:1089-1106. [PMID: 38880960 PMCID: PMC11325958 DOI: 10.1097/aud.0000000000001508] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Accepted: 03/18/2024] [Indexed: 06/18/2024]
Abstract
OBJECTIVES Listening effort involves the mental effort required to perceive an auditory stimulus, for example in noisy environments. Prolonged increased listening effort, for example due to impaired hearing ability, may increase risk of health complications. It is therefore important to identify valid and sensitive measures of listening effort. Physiological measures have been shown to be sensitive to auditory task demand manipulations and are considered to reflect changes in listening effort. Such measures include pupil dilation, alpha power, skin conductance level, and heart rate variability. The aim of the current systematic review was to provide an overview of studies to listening effort that used multiple physiological measures. The two main questions were: (1) what is the effect of changes in auditory task demand on simultaneously acquired physiological measures from various modalities? and (2) what is the relationship between the responses in these physiological measures? DESIGN Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, relevant articles were sought in PubMed, PsycInfo, and Web of Science and by examining the references of included articles. Search iterations with different combinations of psychophysiological measures were performed in conjunction with listening effort-related search terms. Quality was assessed using the Appraisal Tool for Cross-Sectional Studies. RESULTS A total of 297 articles were identified from three databases, of which 27 were included. One additional article was identified from reference lists. Of the total 28 included articles, 16 included an analysis regarding the relationship between the physiological measures. The overall quality of the included studies was reasonable. CONCLUSIONS The included studies showed that most of the physiological measures either show no effect to auditory task demand manipulations or a consistent effect in the expected direction. For example, pupil dilation increased, pre-ejection period decreased, and skin conductance level increased with increasing auditory task demand. Most of the relationships between the responses of these physiological measures were nonsignificant or weak. The physiological measures varied in their sensitivity to auditory task demand manipulations. One of the identified knowledge gaps was that the included studies mostly used tasks with high-performance levels, resulting in an underrepresentation of the physiological changes at lower performance levels. This makes it difficult to capture how the physiological responses behave across the full psychometric curve. Our results support the Framework for Understanding Effortful Listening and the need for a multimodal approach to listening effort. We furthermore discuss focus points for future studies.
Collapse
Affiliation(s)
- Laura Keur-Huizinga
- Amsterdam UMC location Vrije Universiteit Amsterdam, Otolaryngology—Head and Neck Surgery, Ear & Hearing, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
- Department of Biological Psychology, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Sophia E. Kramer
- Amsterdam UMC location Vrije Universiteit Amsterdam, Otolaryngology—Head and Neck Surgery, Ear & Hearing, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
| | - Eco J. C. de Geus
- Department of Biological Psychology, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Adriana A. Zekveld
- Amsterdam UMC location Vrije Universiteit Amsterdam, Otolaryngology—Head and Neck Surgery, Ear & Hearing, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
| |
Collapse
|
2
|
Golob EJ, Olayo RC, Brown DMY, Mock JR. Relations Among Multiple Dimensions of Self-Reported Listening Effort in Response to an Auditory Psychomotor Vigilance Task. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2024:1-15. [PMID: 39116317 DOI: 10.1044/2024_jslhr-23-00465] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/10/2024]
Abstract
PURPOSE Listening effort is a broad construct, and there is no consensus on how to subdivide listening effort into dimensions. This project focuses on the subjective experience of effortful listening and tests if cognitive workload, mental fatigue, and mood are interrelated dimensions. METHOD Two online studies tested young adults (n = 74 and n = 195) and measured subjective workload, fatigue (subscales of fatigue and energy), and mood (subscales of positive and negative mood) before and after a challenging listening task. In the listening effort task, participants responded to intermittent 1-kHz target tones in continuous white noise for approximately 12 min. RESULTS Correlations and principal component analysis showed that fatigue and mood were distinct but interrelated constructs that weakly correlated with workload. Effortful listening provoked increased fatigue and decreased energy and positive mood yet did not influence negative mood or workload. CONCLUSIONS The findings suggest that self-reported listening effort has multiple dimensions that can have different responses to the same effortful listening episode. The results can help guide evidence-based development of clinical listening effort tests and may reveal mechanisms for how listening effort relates to quality of life in those with hearing impairment. SUPPLEMENTAL MATERIAL https://doi.org/10.23641/asha.26418976.
Collapse
Affiliation(s)
- Edward J Golob
- Department of Psychology, University of Texas, San Antonio
| | | | | | - Jeffrey R Mock
- Department of Psychology, University of Texas, San Antonio
| |
Collapse
|
3
|
Brown VA, Sewell K, Villanueva J, Strand JF. Noisy speech impairs retention of previously heard information only at short time scales. Mem Cognit 2024:10.3758/s13421-024-01583-y. [PMID: 38758512 DOI: 10.3758/s13421-024-01583-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/24/2024] [Indexed: 05/18/2024]
Abstract
When speech is presented in noise, listeners must recruit cognitive resources to resolve the mismatch between the noisy input and representations in memory. A consequence of this effortful listening is impaired memory for content presented earlier. In the first study on effortful listening, Rabbitt, The Quarterly Journal of Experimental Psychology, 20, 241-248 (1968; Experiment 2) found that recall for a list of digits was poorer when subsequent digits were presented with masking noise than without. Experiment 3 of that study extended this effect to more naturalistic, passage-length materials. Although the findings of Rabbitt's Experiment 2 have been replicated multiple times, no work has assessed the robustness of Experiment 3. We conducted a replication attempt of Rabbitt's Experiment 3 at three signal-to-noise ratios (SNRs). Results at one of the SNRs (Experiment 1a of the current study) were in the opposite direction from what Rabbitt, The Quarterly Journal of Experimental Psychology, 20, 241-248, (1968) reported - that is, speech was recalled more accurately when it was followed by speech presented in noise rather than in the clear - and results at the other two SNRs showed no effect of noise (Experiments 1b and 1c). In addition, reanalysis of a replication of Rabbitt's seminal finding in his second experiment showed that the effect of effortful listening on previously presented information is transient. Thus, effortful listening caused by noise appears to only impair memory for information presented immediately before the noise, which may account for our finding that noise in the second-half of a long passage did not impair recall of information presented in the first half of the passage.
Collapse
Affiliation(s)
- Violet A Brown
- Department of Psychology, Carleton College, Northfield, MN, USA.
| | - Katrina Sewell
- Department of Psychology, Carleton College, Northfield, MN, USA
| | - Jed Villanueva
- Department of Psychology, Carleton College, Northfield, MN, USA
| | - Julia F Strand
- Department of Psychology, Carleton College, Northfield, MN, USA
| |
Collapse
|
4
|
Keur-Huizinga L, Huizinga NA, Zekveld AA, Versfeld NJ, van de Ven SRB, van Dijk WAJ, de Geus EJC, Kramer SE. Effects of hearing acuity on psychophysiological responses to effortful speech perception. Hear Res 2024; 448:109031. [PMID: 38761554 DOI: 10.1016/j.heares.2024.109031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Revised: 05/02/2024] [Accepted: 05/13/2024] [Indexed: 05/20/2024]
Abstract
In recent studies, psychophysiological measures have been used as markers of listening effort, but there is limited research on the effect of hearing loss on such measures. The aim of the current study was to investigate the effect of hearing acuity on physiological responses and subjective measures acquired during different levels of listening demand, and to investigate the relationship between these measures. A total of 125 participants (37 males and 88 females, age range 37-72 years, pure-tone average hearing thresholds at the best ear between -5.0 to 68.8 dB HL and asymmetry between ears between 0.0 and 87.5 dB) completed a listening task. A speech reception threshold (SRT) test was used with target sentences spoken by a female voice masked by male speech. Listening demand was manipulated using three levels of intelligibility: 20 % correct speech recognition, 50 %, and 80 % (IL20 %/IL50 %/IL80 %, respectively). During the task, peak pupil dilation (PPD), heart rate (HR), pre-ejection period (PEP), respiratory sinus arrhythmia (RSA), and skin conductance level (SCL) were measured. For each condition, subjective ratings of effort, performance, difficulty, and tendency to give up were also collected. Linear mixed effects models tested the effect of intelligibility level, hearing acuity, hearing asymmetry, and tinnitus complaints on the physiological reactivity (compared to baseline) and subjective measures. PPD and PEP reactivity showed a non-monotonic relationship with intelligibility level, but no such effects were found for HR, RSA, or SCL reactivity. Participants with worse hearing acuity had lower PPD at all intelligibility levels and showed lower PEP baseline levels. Additionally, PPD and SCL reactivity were lower for participants who reported suffering from tinnitus complaints. For IL80 %, but not IL50 % or IL20 %, participants with worse hearing acuity rated their listening effort to be relatively high compared to participants with better hearing. The reactivity of the different physiological measures were not or only weakly correlated with each other. Together, the results suggest that hearing acuity may be associated with altered sympathetic nervous system (re)activity. Research using psychophysiological measures as markers of listening effort to study the effect of hearing acuity on such measures are best served by the use of the PPD and PEP.
Collapse
Affiliation(s)
- Laura Keur-Huizinga
- Amsterdam UMC location Vrije Universiteit Amsterdam, Otolaryngology - Head and Neck Surgery, Ear & Hearing, Amsterdam Public Health Research Institute, De Boelelaan 1117, Amsterdam, the Netherlands; Department of Biological Psychology, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands.
| | - Nicole A Huizinga
- Amsterdam UMC location Vrije Universiteit Amsterdam, Otolaryngology - Head and Neck Surgery, Ear & Hearing, Amsterdam Public Health Research Institute, De Boelelaan 1117, Amsterdam, the Netherlands; Department of Biological Psychology, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - Adriana A Zekveld
- Amsterdam UMC location Vrije Universiteit Amsterdam, Otolaryngology - Head and Neck Surgery, Ear & Hearing, Amsterdam Public Health Research Institute, De Boelelaan 1117, Amsterdam, the Netherlands
| | - Niek J Versfeld
- Amsterdam UMC location Vrije Universiteit Amsterdam, Otolaryngology - Head and Neck Surgery, Ear & Hearing, Amsterdam Public Health Research Institute, De Boelelaan 1117, Amsterdam, the Netherlands
| | - Sjors R B van de Ven
- Department of Biological Psychology, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - Wieke A J van Dijk
- Amsterdam UMC location Vrije Universiteit Amsterdam, Otolaryngology - Head and Neck Surgery, Ear & Hearing, Amsterdam Public Health Research Institute, De Boelelaan 1117, Amsterdam, the Netherlands
| | - Eco J C de Geus
- Department of Biological Psychology, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - Sophia E Kramer
- Amsterdam UMC location Vrije Universiteit Amsterdam, Otolaryngology - Head and Neck Surgery, Ear & Hearing, Amsterdam Public Health Research Institute, De Boelelaan 1117, Amsterdam, the Netherlands
| |
Collapse
|
5
|
Green GD, Jacewicz E, Santosa H, Arzbecker LJ, Fox RA. Evaluating Speaker-Listener Cognitive Effort in Speech Communication Through Brain-to-Brain Synchrony: A Pilot Functional Near-Infrared Spectroscopy Investigation. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2024; 67:1339-1359. [PMID: 38535722 DOI: 10.1044/2024_jslhr-23-00476] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/09/2024]
Abstract
PURPOSE We explore a new approach to the study of cognitive effort involved in listening to speech by measuring the brain activity in a listener in relation to the brain activity in a speaker. We hypothesize that the strength of this brain-to-brain synchrony (coupling) reflects the magnitude of cognitive effort involved in verbal communication and includes both listening effort and speaking effort. We investigate whether interbrain synchrony is greater in native-to-native versus native-to-nonnative communication using functional near-infrared spectroscopy (fNIRS). METHOD Two speakers participated, a native speaker of American English and a native speaker of Korean who spoke English as a second language. Each speaker was fitted with the fNIRS cap and told short stories. The native English speaker provided the English narratives, and the Korean speaker provided both the nonnative (accented) English and Korean narratives. In separate sessions, fNIRS data were obtained from seven English monolingual participants ages 20-24 years who listened to each speaker's stories. After listening to each story in native and nonnative English, they retold the content, and their transcripts and audio recordings were analyzed for comprehension and discourse fluency, measured in the number of hesitations and articulation rate. No story retellings were obtained for narratives in Korean (an incomprehensible language for English listeners). Utilizing fNIRS technique termed sequential scanning, we quantified the brain-to-brain synchronization in each speaker-listener dyad. RESULTS For native-to-native dyads, multiple brain regions associated with various linguistic and executive functions were activated. There was a weaker coupling for native-to-nonnative dyads, and only the brain regions associated with higher order cognitive processes and functions were synchronized. All listeners understood the content of all stories, but they hesitated significantly more when retelling stories told in accented English. The nonnative speaker hesitated significantly more often than the native speaker and had a significantly slower articulation rate. There was no brain-to-brain coupling during listening to Korean, indicating a break in communication when listeners failed to comprehend the speaker. CONCLUSIONS We found that effortful speech processing decreased interbrain synchrony and delayed comprehension processes. The obtained brain-based and behavioral patterns are consistent with our proposal that cognitive effort in verbal communication pertains to both the listener and the speaker and that brain-to-brain synchrony can be an indicator of differences in their cumulative communicative effort. SUPPLEMENTAL MATERIAL https://doi.org/10.23641/asha.25452142.
Collapse
Affiliation(s)
- Geoff D Green
- Department of Speech and Hearing Science, The Ohio State University, Columbus
| | - Ewa Jacewicz
- Department of Speech and Hearing Science, The Ohio State University, Columbus
| | | | - Lian J Arzbecker
- Department of Speech and Hearing Science, The Ohio State University, Columbus
| | - Robert A Fox
- Department of Speech and Hearing Science, The Ohio State University, Columbus
| |
Collapse
|
6
|
Brysbaert M. Designing and evaluating tasks to measure individual differences in experimental psychology: a tutorial. Cogn Res Princ Implic 2024; 9:11. [PMID: 38411837 PMCID: PMC10899130 DOI: 10.1186/s41235-024-00540-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2023] [Accepted: 02/18/2024] [Indexed: 02/28/2024] Open
Abstract
Experimental psychology is witnessing an increase in research on individual differences, which requires the development of new tasks that can reliably assess variations among participants. To do this, cognitive researchers need statistical methods that many researchers have not learned during their training. The lack of expertise can pose challenges not only in designing good, new tasks but also in evaluating tasks developed by others. To bridge the gap, this article provides an overview of test psychology applied to performance tasks, covering fundamental concepts such as standardization, reliability, norming and validity. It provides practical guidelines for developing and evaluating experimental tasks, as well as for combining tasks to better understand individual differences. To further address common misconceptions, the article lists 11 prevailing myths. The purpose of this guide is to provide experimental psychologists with the knowledge and tools needed to conduct rigorous and insightful studies of individual differences.
Collapse
Affiliation(s)
- Marc Brysbaert
- Department of Experimental Psychology, Ghent University, 9000, Ghent, Belgium.
| |
Collapse
|
7
|
Plain B, Pielage H, Kramer SE, Richter M, Saunders GH, Versfeld NJ, Zekveld AA, Bhuiyan TA. Combining Cardiovascular and Pupil Features Using k-Nearest Neighbor Classifiers to Assess Task Demand, Social Context, and Sentence Accuracy During Listening. Trends Hear 2024; 28:23312165241232551. [PMID: 38549351 PMCID: PMC10981225 DOI: 10.1177/23312165241232551] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2023] [Revised: 01/04/2024] [Accepted: 01/25/2024] [Indexed: 04/01/2024] Open
Abstract
In daily life, both acoustic factors and social context can affect listening effort investment. In laboratory settings, information about listening effort has been deduced from pupil and cardiovascular responses independently. The extent to which these measures can jointly predict listening-related factors is unknown. Here we combined pupil and cardiovascular features to predict acoustic and contextual aspects of speech perception. Data were collected from 29 adults (mean = 64.6 years, SD = 9.2) with hearing loss. Participants performed a speech perception task at two individualized signal-to-noise ratios (corresponding to 50% and 80% of sentences correct) and in two social contexts (the presence and absence of two observers). Seven features were extracted per trial: baseline pupil size, peak pupil dilation, mean pupil dilation, interbeat interval, blood volume pulse amplitude, pre-ejection period and pulse arrival time. These features were used to train k-nearest neighbor classifiers to predict task demand, social context and sentence accuracy. The k-fold cross validation on the group-level data revealed above-chance classification accuracies: task demand, 64.4%; social context, 78.3%; and sentence accuracy, 55.1%. However, classification accuracies diminished when the classifiers were trained and tested on data from different participants. Individually trained classifiers (one per participant) performed better than group-level classifiers: 71.7% (SD = 10.2) for task demand, 88.0% (SD = 7.5) for social context, and 60.0% (SD = 13.1) for sentence accuracy. We demonstrated that classifiers trained on group-level physiological data to predict aspects of speech perception generalized poorly to novel participants. Individually calibrated classifiers hold more promise for future applications.
Collapse
Affiliation(s)
- Bethany Plain
- Otolaryngology Head and Neck Surgery, Ear & Hearing, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam UMC, Amsterdam, the Netherlands
- Eriksholm Research Centre, Snekkersten, Denmark
| | - Hidde Pielage
- Otolaryngology Head and Neck Surgery, Ear & Hearing, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam UMC, Amsterdam, the Netherlands
- Eriksholm Research Centre, Snekkersten, Denmark
| | - Sophia E. Kramer
- Otolaryngology Head and Neck Surgery, Ear & Hearing, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam UMC, Amsterdam, the Netherlands
| | - Michael Richter
- School of Psychology, Liverpool John Moores University, Liverpool, UK
| | - Gabrielle H. Saunders
- Manchester Centre for Audiology and Deafness (ManCAD), University of Manchester, Manchester, UK
| | - Niek J. Versfeld
- Otolaryngology Head and Neck Surgery, Ear & Hearing, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam UMC, Amsterdam, the Netherlands
| | - Adriana A. Zekveld
- Otolaryngology Head and Neck Surgery, Ear & Hearing, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam UMC, Amsterdam, the Netherlands
| | | |
Collapse
|
8
|
Hunter CR, Abrahamyan H. Sensitivity, reliability and convergent validity of sequential dual-task measures of listening effort. Int J Audiol 2024; 63:30-39. [PMID: 36427054 DOI: 10.1080/14992027.2022.2145513] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2021] [Accepted: 11/04/2022] [Indexed: 11/27/2022]
Abstract
OBJECTIVE The aim of the current study was to assess the sensitivity, reliability and convergent validity of objective measures of listening effort collected in a sequential dual-task. DESIGN On each trial, participants viewed a set of digits and listened to a spoken sentence presented at one of a range of signal-to-noise ratios (SNR) and then typed the sentence-final word and recalled the digits. Listening effort measures included word response time, digit recall accuracy and digit response time. In Experiment 1, SNR on each trial was randomised. In Experiment 2, SNR varied in a blocked design, and in each block self-reported listening effort was also collected. STUDY SAMPLES Separate groups of 40 young adults participated in each experiment. RESULTS Effects of SNR were observed for all measures. Linear effects of SNR were generally observed even with word recognition accuracy factored out of the models. Among the objective measures, reliability was excellent, and repeated-measures correlations, though not between-subjects correlations, were nearly all significant. CONCLUSION The objective measures assessed appear to be sensitive and reliable indices of listening effort that are non-redundant with speech intelligibility and have strong within-participants convergent validity. Results support use of these measures in future studies of listening effort.
Collapse
Affiliation(s)
- Cynthia R Hunter
- Speech Perception, Cognition, and Hearing Laboratory, Department of Speech-Language-Hearing: Sciences and Disorders, University of Kansas, Lawrence, KS, USA
| | - Hayk Abrahamyan
- Language Perception Laboratory, Department of Psychology, State University of New York at Buffalo, Buffalo, NY, USA
| |
Collapse
|
9
|
Cui ME, Herrmann B. Eye Movements Decrease during Effortful Speech Listening. J Neurosci 2023; 43:5856-5869. [PMID: 37491313 PMCID: PMC10423048 DOI: 10.1523/jneurosci.0240-23.2023] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Revised: 06/09/2023] [Accepted: 07/18/2023] [Indexed: 07/27/2023] Open
Abstract
Hearing impairment affects many older adults but is often diagnosed decades after speech comprehension in noisy situations has become effortful. Accurate assessment of listening effort may thus help diagnose hearing impairment earlier. However, pupillometry-the most used approach to assess listening effort-has limitations that hinder its use in practice. The current study explores a novel way to assess listening effort through eye movements. Building on cognitive and neurophysiological work, we examine the hypothesis that eye movements decrease when speech listening becomes challenging. In three experiments with human participants from both sexes, we demonstrate, consistent with this hypothesis, that fixation duration increases and spatial gaze dispersion decreases with increasing speech masking. Eye movements decreased during effortful speech listening for different visual scenes (free viewing, object tracking) and speech materials (simple sentences, naturalistic stories). In contrast, pupillometry was less sensitive to speech masking during story listening, suggesting pupillometric measures may not be as effective for the assessments of listening effort in naturalistic speech-listening paradigms. Our results reveal a critical link between eye movements and cognitive load, suggesting that neural activity in the brain regions that support the regulation of eye movements, such as frontal eye field and superior colliculus, are modulated when listening is effortful.SIGNIFICANCE STATEMENT Assessment of listening effort is critical for early diagnosis of age-related hearing loss. Pupillometry is most used but has several disadvantages. The current study explores a novel way to assess listening effort through eye movements. We examine the hypothesis that eye movements decrease when speech listening becomes effortful. We demonstrate, consistent with this hypothesis, that fixation duration increases and gaze dispersion decreases with increasing speech masking. Eye movements decreased during effortful speech listening for different visual scenes (free viewing, object tracking) and speech materials (sentences, naturalistic stories). Our results reveal a critical link between eye movements and cognitive load, suggesting that neural activity in brain regions that support the regulation of eye movements are modulated when listening is effortful.
Collapse
Affiliation(s)
- M Eric Cui
- Rotman Research Institute, Baycrest Academy for Research and Education, North York, Ontario M6A 2E1, Canada
- Department of Psychology, University of Toronto, Toronto, Ontario M5S 1A1, Canada
| | - Björn Herrmann
- Rotman Research Institute, Baycrest Academy for Research and Education, North York, Ontario M6A 2E1, Canada
- Department of Psychology, University of Toronto, Toronto, Ontario M5S 1A1, Canada
| |
Collapse
|
10
|
Shatzer HE, Russo FA. Brightening the Study of Listening Effort with Functional Near-Infrared Spectroscopy: A Scoping Review. Semin Hear 2023; 44:188-210. [PMID: 37122884 PMCID: PMC10147513 DOI: 10.1055/s-0043-1766105] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/09/2023] Open
Abstract
Listening effort is a long-standing area of interest in auditory cognitive neuroscience. Prior research has used multiple techniques to shed light on the neurophysiological mechanisms underlying listening during challenging conditions. Functional near-infrared spectroscopy (fNIRS) is growing in popularity as a tool for cognitive neuroscience research, and its recent advances offer many potential advantages over other neuroimaging modalities for research related to listening effort. This review introduces the basic science of fNIRS and its uses for auditory cognitive neuroscience. We also discuss its application in recently published studies on listening effort and consider future opportunities for studying effortful listening with fNIRS. After reading this article, the learner will know how fNIRS works and summarize its uses for listening effort research. The learner will also be able to apply this knowledge toward generation of future research in this area.
Collapse
Affiliation(s)
- Hannah E. Shatzer
- Department of Psychology, Toronto Metropolitan University, Toronto, Canada
| | - Frank A. Russo
- Department of Psychology, Toronto Metropolitan University, Toronto, Canada
| |
Collapse
|
11
|
Richter M, Buhiyan T, Bramsløw L, Innes-Brown H, Fiedler L, Hadley LV, Naylor G, Saunders GH, Wendt D, Whitmer WM, Zekveld AA, Kramer SE. Combining Multiple Psychophysiological Measures of Listening Effort: Challenges and Recommendations. Semin Hear 2023; 44:95-105. [PMID: 37122882 PMCID: PMC10147512 DOI: 10.1055/s-0043-1767669] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/30/2023] Open
Abstract
About one-third of all recently published studies on listening effort have used at least one physiological measure, providing evidence of the popularity of such measures in listening effort research. However, the specific measures employed, as well as the rationales used to justify their inclusion, vary greatly between studies, leading to a literature that is fragmented and difficult to integrate. A unified approach that assesses multiple psychophysiological measures justified by a single rationale would be preferable because it would advance our understanding of listening effort. However, such an approach comes with a number of challenges, including the need to develop a clear definition of listening effort that links to specific physiological measures, customized equipment that enables the simultaneous assessment of multiple measures, awareness of problems caused by the different timescales on which the measures operate, and statistical approaches that minimize the risk of type-I error inflation. This article discusses in detail the various obstacles for combining multiple physiological measures in listening effort research and provides recommendations on how to overcome them.
Collapse
Affiliation(s)
- Michael Richter
- School of Psychology, Liverpool John Moores University, Liverpool, United Kingdom
| | | | - Lars Bramsløw
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
| | - Hamish Innes-Brown
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
- Hearing Systems Section, Department of Health Technology, Technical University of Denmark, Kongens Lyngby, Denmark
| | - Lorenz Fiedler
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
| | - Lauren V. Hadley
- Hearing Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom
| | - Graham Naylor
- Hearing Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom
| | - Gabrielle H. Saunders
- Manchester Centre for Audiology and Deafness, University of Manchester, Manchester, United Kingdom
| | - Dorothea Wendt
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
- Department of Health Technology, Technical University of Denmark, Lyngby, Denmark
| | - William M. Whitmer
- Hearing Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom
| | - Adriana A. Zekveld
- Section of Ear and Hearing, Department of Otolaryngology – Head and Neck Surgery, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
| | - Sophia E. Kramer
- Section of Ear and Hearing, Department of Otolaryngology – Head and Neck Surgery, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
| |
Collapse
|
12
|
Shepherd J. Conscious cognitive effort in cognitive control. WILEY INTERDISCIPLINARY REVIEWS. COGNITIVE SCIENCE 2023; 14:e1629. [PMID: 36263671 DOI: 10.1002/wcs.1629] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/27/2022] [Revised: 09/20/2022] [Accepted: 09/20/2022] [Indexed: 11/15/2022]
Abstract
Cognitive effort is thought to be familiar in everyday life, ubiquitous across multiple variations of task and circumstance, and integral to cost/benefit computations that are themselves central to the proper functioning of cognitive control. In particular, cognitive effort is thought to be closely related to the assessment of cognitive control's costs. I argue here that the construct of cognitive effort, as it is deployed in cognitive psychology and neuroscience, is problematically unclear. The result is that talk of cognitive effort may paper over significant disagreement regarding the nature of cognitive effort, and its key functions for cognitive control. I highlight key points of disagreement, and several open questions regarding what causes cognitive effort, what cognitive effort represents, cognitive effort's relationship to action, and cognitive effort's relationship to consciousness. I also suggest that pluralism about cognitive effort-that cognitive effort may manifest as a range of intentional or non-intentional actions the function of which is to promote greater success at paradigmatic cognitive control tasks-may be a fruitful and irenic way to conceive of cognitive effort. Finally, I suggest that recent trends in work on cognitive control suggests that we might fruitfully conceive of cognitive effort as one key node in a complex network of mental value, and that studying this complex network may illuminate the nature of cognitive control, and the role of consciousness in cognitive control's proper functioning. This article is categorized under: Philosophy > Consciousness Philosophy > Psychological Capacities Neuroscience > Cognition.
Collapse
Affiliation(s)
- Joshua Shepherd
- Carleton University, Ottawa, Ontario, Canada.,Facultat de Filosofia, Universität de Barcelona, Barcelona, Spain
| |
Collapse
|
13
|
Rovetti J, Sumantry D, Russo FA. Exposure to nonnative-accented speech reduces listening effort and improves social judgments of the speaker. Sci Rep 2023; 13:2808. [PMID: 36797318 PMCID: PMC9935874 DOI: 10.1038/s41598-023-29082-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Accepted: 01/30/2023] [Indexed: 02/18/2023] Open
Abstract
Prior research has revealed a native-accent advantage, whereby nonnative-accented speech is more difficult to process than native-accented speech. Nonnative-accented speakers also experience more negative social judgments. In the current study, we asked three questions. First, does exposure to nonnative-accented speech increase speech intelligibility or decrease listening effort, thereby narrowing the native-accent advantage? Second, does lower intelligibility or higher listening effort contribute to listeners' negative social judgments of speakers? Third and finally, does increased intelligibility or decreased listening effort with exposure to speech bring about more positive social judgments of speakers? To address these questions, normal-hearing adults listened to a block of English sentences with a native accent and a block with nonnative accent. We found that once participants were accustomed to the task, intelligibility was greater for nonnative-accented speech and increased similarly with exposure for both accents. However, listening effort decreased only for nonnative-accented speech, soon reaching the level of native-accented speech. In addition, lower intelligibility and higher listening effort was associated with lower ratings of speaker warmth, speaker competence, and willingness to interact with the speaker. Finally, competence ratings increased over time to a similar extent for both accents, with this relationship fully mediated by intelligibility and listening effort. These results offer insight into how listeners process and judge unfamiliar speakers.
Collapse
Affiliation(s)
- Joseph Rovetti
- grid.39381.300000 0004 1936 8884Department of Psychology, Western University, London, ON N6A 3K7 Canada ,Department of Psychology, Toronto Metropolitan University, Toronto, ON M5B 2K3 Canada
| | - David Sumantry
- Department of Psychology, Toronto Metropolitan University, Toronto, ON M5B 2K3 Canada
| | - Frank A. Russo
- Department of Psychology, Toronto Metropolitan University, Toronto, ON M5B 2K3 Canada
| |
Collapse
|
14
|
McGarrigle R, Mattys S. Sensory-Processing Sensitivity Predicts Fatigue From Listening, But Not Perceived Effort, in Young and Older Adults. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2023; 66:444-460. [PMID: 36657070 PMCID: PMC10023191 DOI: 10.1044/2022_jslhr-22-00374] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/24/2022] [Revised: 09/16/2022] [Accepted: 10/18/2022] [Indexed: 06/17/2023]
Abstract
PURPOSE Listening-related fatigue is a potential negative consequence of challenges experienced during everyday listening and may disproportionately affect older adults. Contrary to expectation, we recently found that increased reports of listening-related fatigue were associated with better performance on a dichotic listening task. However, this link was found only in individuals who reported heightened sensitivity to a variety of physical, social, and emotional stimuli (i.e., increased "sensory-processing sensitivity" [SPS]). This study examined whether perceived effort may underlie the link between performance and fatigue. METHOD Two hundred six young adults, aged 18-30 years (Experiment 1), and 122 older adults, aged 60-80 years (Experiment 2), performed a dichotic listening task and were administered a series of questionnaires including the NASA Task Load Index of perceived effort, the Vanderbilt Fatigue Scale (measuring daily life listening-related fatigue), and the Highly Sensitive Person Scale (measuring SPS). Both experiments were completed online. RESULTS SPS predicted listening-related fatigue, but perceived effort during the listening task was not associated with SPS or listening-related fatigue in either age group. We were also unable to replicate the interaction between dichotic listening performance and SPS in either group. Exploratory analyses revealed contrasting effects of age; older adults found the dichotic listening task more effortful but indicated lower overall fatigue. CONCLUSIONS These findings suggest that SPS is a better predictor of listening-related fatigue than performance or effort ratings on a dichotic listening task. SPS may be an important factor in determining an individual's likelihood of experiencing listening-related fatigue irrespective of hearing or cognitive ability. SUPPLEMENTAL MATERIAL https://doi.org/10.23641/asha.21893013.
Collapse
Affiliation(s)
- Ronan McGarrigle
- Department of Psychology, University of Bradford, United Kingdom
- Department of Psychology, University of York, United Kingdom
| | - Sven Mattys
- Department of Psychology, University of York, United Kingdom
| |
Collapse
|
15
|
Bsharat-Maalouf D, Degani T, Karawani H. The Involvement of Listening Effort in Explaining Bilingual Listening Under Adverse Listening Conditions. Trends Hear 2023; 27:23312165231205107. [PMID: 37941413 PMCID: PMC10637154 DOI: 10.1177/23312165231205107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2023] [Revised: 09/14/2023] [Accepted: 09/15/2023] [Indexed: 11/10/2023] Open
Abstract
The current review examines listening effort to uncover how it is implicated in bilingual performance under adverse listening conditions. Various measures of listening effort, including physiological, behavioral, and subjective measures, have been employed to examine listening effort in bilingual children and adults. Adverse listening conditions, stemming from environmental factors, as well as factors related to the speaker or listener, have been examined. The existing literature, although relatively limited to date, points to increased listening effort among bilinguals in their nondominant second language (L2) compared to their dominant first language (L1) and relative to monolinguals. Interestingly, increased effort is often observed even when speech intelligibility remains unaffected. These findings emphasize the importance of considering listening effort alongside speech intelligibility. Building upon the insights gained from the current review, we propose that various factors may modulate the observed effects. These include the particular measure selected to examine listening effort, the characteristics of the adverse condition, as well as factors related to the particular linguistic background of the bilingual speaker. Critically, further research is needed to better understand the impact of these factors on listening effort. The review outlines avenues for future research that would promote a comprehensive understanding of listening effort in bilingual individuals.
Collapse
Affiliation(s)
- Dana Bsharat-Maalouf
- Department of Communication Sciences and Disorders, University of Haifa, Haifa, Israel
| | - Tamar Degani
- Department of Communication Sciences and Disorders, University of Haifa, Haifa, Israel
| | - Hanin Karawani
- Department of Communication Sciences and Disorders, University of Haifa, Haifa, Israel
| |
Collapse
|
16
|
Huang H, Ricketts TA, Hornsby BWY, Picou EM. Effects of Critical Distance and Reverberation on Listening Effort in Adults. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2022; 65:4837-4851. [PMID: 36351258 DOI: 10.1044/2022_jslhr-22-00109] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
PURPOSE Mixed historical data on how listening effort is affected by reverberation and listener-to-speaker distance challenge existing models of listening effort. This study investigated the effects of reverberation and listener-to-speaker distance on behavioral and subjective measures of listening effort: (a) when listening at a fixed signal-to-noise ratio (SNR) and (b) at SNRs that were manipulated so that word recognition would be comparable across different reverberation times and listening distances. It was expected that increased reverberation would increase listening effort but only when listening outside critical distance. METHOD Nineteen adults (21-40 years) with no hearing loss completed a dual-task paradigm. The primary task was word recognition and the secondary task was timed word categorization; response times indexed behavioral listening effort. Additionally, participants provided subjective ratings in each condition. Testing was completed at two reverberation levels (moderate and high, RT30 = 469 and 1,223 ms, respectively) and at two listener-to-speaker distances (inside and outside critical distance for the test room, 1.25 and 4 m, respectively). RESULTS Increased reverberation and listening distances worsened word recognition performance and both behavioral and subjective listening effort. The effect of reverberation was exacerbated when listeners were outside critical distance. Subjective experience of listening effort persisted even when word recognition was comparable across conditions. CONCLUSIONS Longer reverberation times or listening outside the room's critical distance negatively affected behavioral and subjective listening effort. This study extends understanding of listening effort in reverberant rooms by highlighting the effect of listener's position relative to the room's critical distance.
Collapse
Affiliation(s)
- Haiping Huang
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN
| | - Todd A Ricketts
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN
| | - Benjamin W Y Hornsby
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN
| | - Erin M Picou
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN
| |
Collapse
|
17
|
Fletcher AR, Wisler AA, Gruver ER, Borrie SA. Beyond Speech Intelligibility: Quantifying Behavioral and Perceived Listening Effort in Response to Dysarthric Speech. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2022; 65:4060-4070. [PMID: 36198057 PMCID: PMC9940894 DOI: 10.1044/2022_jslhr-22-00136] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/03/2023]
Abstract
PURPOSE This study investigated whether listener processing of dysarthric speech requires the recruitment of more cognitive resources (i.e., higher levels of listening effort) than neurotypical speech. We also explored relationships between behavioral listening effort, perceived listening effort, and objective measures of word transcription accuracy. METHOD A word recall paradigm was used to index behavioral listening effort. The primary task involved word transcription, whereas a memory task involved recalling words from previous sentences. Nineteen listeners completed the paradigm twice, once while transcribing dysarthric speech and once while transcribing neurotypical speech. Perceived listening effort was rated using a visual analog scale. RESULTS Results revealed significant effects of dysarthria on the likelihood of correct word recall, indicating that the transcription of dysarthric speech required higher levels of behavioral listening effort relative to neurotypical speech. There was also a significant relationship between transcription accuracy and measures of behavioral listening effort, such that listeners who were more accurate in understanding dysarthric speech exhibited smaller changes in word recall when listening to dysarthria. The subjective measure of perceived listening effort did not have a statistically significant correlation with measures of behavioral listening effort or transcription accuracy. CONCLUSIONS Results suggest that cognitive resources, particularly listeners' working memory capacity, are more taxed when deciphering dysarthric versus neurotypical speech. An increased demand on these resources may affect a listener's ability to remember aspects of their conversations with people with dysarthria, even when the speaker is fully intelligible.
Collapse
Affiliation(s)
- Annalise R. Fletcher
- Department of Communicative Disorders and Deaf Education, Utah State University, Logan
| | - Alan A. Wisler
- Department of Mathematics and Statistics, Utah State University, Logan
| | - Emily R. Gruver
- Department of Audiology and Speech-Language Pathology, University of North Texas, Denton
| | - Stephanie A. Borrie
- Department of Communicative Disorders and Deaf Education, Utah State University, Logan
| |
Collapse
|
18
|
Winn MB, Wright RA. Reconsidering commonly used stimuli in speech perception experiments. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2022; 152:1394. [PMID: 36182291 DOI: 10.1121/10.0013415] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Accepted: 07/18/2022] [Indexed: 06/16/2023]
Abstract
This paper examines some commonly used stimuli in speech perception experiments and raises questions about their use, or about the interpretations of previous results. The takeaway messages are: 1) the Hillenbrand vowels represent a particular dialect rather than a gold standard, and English vowels contain spectral dynamics that have been largely underappreciated, 2) the /ɑ/ context is very common but not clearly superior as a context for testing consonant perception, 3) /ɑ/ is particularly problematic when testing voice-onset-time perception because it introduces strong confounds in the formant transitions, 4) /dɑ/ is grossly overrepresented in neurophysiological studies and yet is insufficient as a generalized proxy for "speech perception," and 5) digit tests and matrix sentences including the coordinate response measure are systematically insensitive to important patterns in speech perception. Each of these stimulus sets and concepts is described with careful attention to their unique value and also cases where they might be misunderstood or over-interpreted.
Collapse
Affiliation(s)
- Matthew B Winn
- Department of Speech-Language-Hearing Sciences, University of Minnesota, Minneapolis, Minnesota 55455, USA
| | - Richard A Wright
- Department of Linguistics, University of Washington, Seattle, Washington 98195, USA
| |
Collapse
|
19
|
Sherafati A, Dwyer N, Bajracharya A, Hassanpour MS, Eggebrecht AT, Firszt JB, Culver JP, Peelle JE. Prefrontal cortex supports speech perception in listeners with cochlear implants. eLife 2022; 11:e75323. [PMID: 35666138 PMCID: PMC9225001 DOI: 10.7554/elife.75323] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Accepted: 06/04/2022] [Indexed: 12/14/2022] Open
Abstract
Cochlear implants are neuroprosthetic devices that can restore hearing in people with severe to profound hearing loss by electrically stimulating the auditory nerve. Because of physical limitations on the precision of this stimulation, the acoustic information delivered by a cochlear implant does not convey the same level of acoustic detail as that conveyed by normal hearing. As a result, speech understanding in listeners with cochlear implants is typically poorer and more effortful than in listeners with normal hearing. The brain networks supporting speech understanding in listeners with cochlear implants are not well understood, partly due to difficulties obtaining functional neuroimaging data in this population. In the current study, we assessed the brain regions supporting spoken word understanding in adult listeners with right unilateral cochlear implants (n=20) and matched controls (n=18) using high-density diffuse optical tomography (HD-DOT), a quiet and non-invasive imaging modality with spatial resolution comparable to that of functional MRI. We found that while listening to spoken words in quiet, listeners with cochlear implants showed greater activity in the left prefrontal cortex than listeners with normal hearing, specifically in a region engaged in a separate spatial working memory task. These results suggest that listeners with cochlear implants require greater cognitive processing during speech understanding than listeners with normal hearing, supported by compensatory recruitment of the left prefrontal cortex.
Collapse
Affiliation(s)
- Arefeh Sherafati
- Department of Radiology, Washington University in St. LouisSt. LouisUnited States
| | - Noel Dwyer
- Department of Otolaryngology, Washington University in St. LouisSt. LouisUnited States
| | - Aahana Bajracharya
- Department of Otolaryngology, Washington University in St. LouisSt. LouisUnited States
| | | | - Adam T Eggebrecht
- Department of Radiology, Washington University in St. LouisSt. LouisUnited States
- Department of Electrical & Systems Engineering, Washington University in St. LouisSt. LouisUnited States
- Department of Biomedical Engineering, Washington University in St. LouisSt. LouisUnited States
- Division of Biology and Biomedical Sciences, Washington University in St. LouisSt. LouisUnited States
| | - Jill B Firszt
- Department of Otolaryngology, Washington University in St. LouisSt. LouisUnited States
| | - Joseph P Culver
- Department of Radiology, Washington University in St. LouisSt. LouisUnited States
- Department of Biomedical Engineering, Washington University in St. LouisSt. LouisUnited States
- Division of Biology and Biomedical Sciences, Washington University in St. LouisSt. LouisUnited States
- Department of Physics, Washington University in St. LouisSt. LouisUnited States
| | - Jonathan E Peelle
- Department of Otolaryngology, Washington University in St. LouisSt. LouisUnited States
| |
Collapse
|
20
|
Reduced Semantic Context and Signal-to-Noise Ratio Increase Listening Effort As Measured Using Functional Near-Infrared Spectroscopy. Ear Hear 2021; 43:836-848. [PMID: 34623112 DOI: 10.1097/aud.0000000000001137] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
OBJECTIVES Understanding speech-in-noise can be highly effortful. Decreasing the signal-to-noise ratio (SNR) of speech increases listening effort, but it is relatively unclear if decreasing the level of semantic context does as well. The current study used functional near-infrared spectroscopy to evaluate two primary hypotheses: (1) listening effort (operationalized as oxygenation of the left lateral PFC) increases as the SNR decreases and (2) listening effort increases as context decreases. DESIGN Twenty-eight younger adults with normal hearing completed the Revised Speech Perception in Noise Test, in which they listened to sentences and reported the final word. These sentences either had an easy SNR (+4 dB) or a hard SNR (-2 dB), and were either low in semantic context (e.g., "Tom could have thought about the sport") or high in context (e.g., "She had to vacuum the rug"). PFC oxygenation was measured throughout using functional near-infrared spectroscopy. RESULTS Accuracy on the Revised Speech Perception in Noise Test was worse when the SNR was hard than when it was easy, and worse for sentences low in semantic context than high in context. Similarly, oxygenation across the entire PFC (including the left lateral PFC) was greater when the SNR was hard, and left lateral PFC oxygenation was greater when context was low. CONCLUSIONS These results suggest that activation of the left lateral PFC (interpreted here as reflecting listening effort) increases to compensate for acoustic and linguistic challenges. This may reflect the increased engagement of domain-general and domain-specific processes subserved by the dorsolateral prefrontal cortex (e.g., cognitive control) and inferior frontal gyrus (e.g., predicting the sensory consequences of articulatory gestures), respectively.
Collapse
|
21
|
Colby S, McMurray B. Cognitive and Physiological Measures of Listening Effort During Degraded Speech Perception: Relating Dual-Task and Pupillometry Paradigms. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2021; 64:3627-3652. [PMID: 34491779 PMCID: PMC8642090 DOI: 10.1044/2021_jslhr-20-00583] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/01/2020] [Revised: 04/01/2021] [Accepted: 05/21/2021] [Indexed: 06/13/2023]
Abstract
Purpose Listening effort is quickly becoming an important metric for assessing speech perception in less-than-ideal situations. However, the relationship between the construct of listening effort and the measures used to assess it remains unclear. We compared two measures of listening effort: a cognitive dual task and a physiological pupillometry task. We sought to investigate the relationship between these measures of effort and whether engaging effort impacts speech accuracy. Method In Experiment 1, 30 participants completed a dual task and a pupillometry task that were carefully matched in stimuli and design. The dual task consisted of a spoken word recognition task and a visual match-to-sample task. In the pupillometry task, pupil size was monitored while participants completed a spoken word recognition task. Both tasks presented words at three levels of listening difficulty (unmodified, eight-channel vocoding, and four-channel vocoding) and provided response feedback on every trial. We refined the pupillometry task in Experiment 2 (n = 31); crucially, participants no longer received response feedback. Finally, we ran a new group of subjects on both tasks in Experiment 3 (n = 30). Results In Experiment 1, accuracy in the visual task decreased with increased signal degradation in the dual task, but pupil size was sensitive to accuracy and not vocoding condition. After removing feedback in Experiment 2, changes in pupil size were predicted by listening condition, suggesting the task was now sensitive to engaged effort. Both tasks were sensitive to listening difficulty in Experiment 3, but there was no relationship between the tasks and neither task predicted speech accuracy. Conclusions Consistent with previous work, we found little evidence for a relationship between different measures of listening effort. We also found no evidence that effort predicts speech accuracy, suggesting that engaging more effort does not lead to improved speech recognition. Cognitive and physiological measures of listening effort are likely sensitive to different aspects of the construct of listening effort. Supplemental Material https://doi.org/10.23641/asha.16455900.
Collapse
Affiliation(s)
- Sarah Colby
- Department of Psychological and Brain Sciences, The University of Iowa, Iowa City
| | - Bob McMurray
- Department of Psychological and Brain Sciences, The University of Iowa, Iowa City
| |
Collapse
|