Indices of Effortful Listening Can Be Mined from Existing Electroencephalographic Data

Validity and reliability of self-reported and neural measures of listening effort

Listening effort can be defined as a measure of cognitive resources used by listeners to perform a listening task. Various methods have been proposed to measure this effort, yet their reliability remains unestablished, a crucial step before their application in research or clinical settings. This study encompassed 32 participants undertaking speech-in-noise tasks across two sessions, approximately a week apart. They listened to sentences and word lists at varying signal-to-noise ratios (SNRs) (-9, -6, -3 and 0 dB), then retaining them for roughly 3 s. We evaluated the test-retest reliability of self-reported effort ratings, theta (4-7 Hz) and alpha (8-13 Hz) oscillatory power, suggested previously as neural markers of listening effort. Additionally, we examined the reliability of correct word percentages. Both relative and absolute reliability were assessed using intraclass correlation coefficients (ICC) and Bland-Altman analysis. We also computed the standard error of measurement (SEM) and smallest detectable change (SDC). Our findings indicated heightened frontal midline theta power for word lists compared to sentences during the retention phase under high SNRs (0 dB, -3 dB), likely indicating a greater memory load for word lists. We observed SNR's impact on alpha power in the right central region during the listening phase and frontal theta power during the retention phase in sentences. Overall, the reliability analysis demonstrated satisfactory between-session variability for correct words and effort ratings. However, neural measures (frontal midline theta power and right central alpha power) displayed substantial variability, even though group-level outcomes appeared consistent across sessions.

Effortful listening produces both enhancement and suppression of alpha in the EEG

Introduction

Adverse listening conditions can drive increased mental effort during listening. Neuromagnetic alpha oscillations (8-13 Hz) may index this listening effort, but inconsistencies regarding the direction of the relationship are abundant. We performed source analyses on high-density EEG data collected during a speech-on-speech listening task to address the possibility that opposing alpha power relationships among alpha producing brain sources drive this inconsistency.

Methods

Listeners (N=20) heard two simultaneously presented sentences of the form: Ready go to now. They either reported the color/number pair of a "Baron" call sign sentence (active: high effort), or ignored the stimuli (passive: low effort). Independent component analysis (ICA) was used to segregate temporally distinct sources in the EEG.

Results

Analysis of independent components (ICs) revealed simultaneous alpha enhancements (e.g., for somatomotor mu ICs) and suppressions (e.g., for left temporal ICs) for different brain sources. The active condition exhibited stronger enhancement for left somatomotor mu rhythm ICs, but stronger suppression for central occipital ICs.

Discussion

This study shows both alpha enhancement and suppression to be associated with increases in listening effort. Literature inconsistencies could partially relate to some source activities overwhelming others in scalp recordings.

Brightening the Study of Listening Effort with Functional Near-Infrared Spectroscopy: A Scoping Review

Listening effort is a long-standing area of interest in auditory cognitive neuroscience. Prior research has used multiple techniques to shed light on the neurophysiological mechanisms underlying listening during challenging conditions. Functional near-infrared spectroscopy (fNIRS) is growing in popularity as a tool for cognitive neuroscience research, and its recent advances offer many potential advantages over other neuroimaging modalities for research related to listening effort. This review introduces the basic science of fNIRS and its uses for auditory cognitive neuroscience. We also discuss its application in recently published studies on listening effort and consider future opportunities for studying effortful listening with fNIRS. After reading this article, the learner will know how fNIRS works and summarize its uses for listening effort research. The learner will also be able to apply this knowledge toward generation of future research in this area.

Finger tapping to different styles of music and changes in cortical oscillations

Music has been a therapeutic strategy proposed to improve impaired movement performance, but there remains a lack of understanding of how music impacts motor cortical activity. Thus, the purpose of this study is to use a time-frequency analysis (i.e., wavelet) of electroencephalographic (EEG) data to determine differences in motor and auditory cortical activity when moving to music at two different rates. Twenty healthy young adults tapped their index finger while electroencephalography was collected. There were three conditions (tapping in time with a tone and with two contrasting music styles), and each condition was repeated at two different rates (70 and 140 beats per minute). A time-frequency Morlet wavelet analysis was completed for electrodes of interest over the sensorimotor areas (FC3, FC4, FCz, C3, C4, Cz) and the primary auditory areas (T7, T8). Cluster-based permutation testing was applied to the electrodes of interest for all conditions. Results showed few differences between cortical oscillations when moving to music versus a tone. However, the two music conditions elicited a variety of distinct responses, particularly at the slower movement rate. These results suggest that music style and movement rate should be considered when designing therapeutic applications that include music to target motor performance.

Effects of auditory training on low-pass filtered speech perception and listening-related cognitive load

Studies supporting learning-induced reductions in listening-related cognitive load have lacked procedural learning controls, making it difficult to determine the extent to which effects arise from perceptual or procedural learning. Here, listeners were trained in the coordinate response measure (CRM) task under unfiltered (UT) or degraded low-pass filtered (FT) conditions. Improvements in low-pass filtered CRM performance were larger for FT. Both conditions showed training-related reductions in cognitive load as indexed by a secondary working memory task. However, only the FT condition showed a correlation between CRM improvement and secondary task performance, suggesting that effects can be driven by perceptual and procedural learning.

Listening effort: Are we measuring cognition or affect, or both?

Listening effort is increasingly recognized as a factor in communication, particularly for and with nonnative speakers, for the elderly, for individuals with hearing impairment and/or for those working in noise. However, as highlighted by McGarrigle et al., International Journal of Audiology, 2014, 53, 433-445, the term "listening effort" encompasses a wide variety of concepts, including the engagement and control of multiple possibly distinct neural systems for information processing, and the affective response to the expenditure of those resources in a given context. Thus, experimental or clinical methods intended to objectively quantify listening effort may ultimately reflect a complex interaction between the operations of one or more of those information processing systems, and/or the affective and motivational response to the demand on those systems. Here we examine theoretical, behavioral, and psychophysiological factors related to resolving the question of what we are measuring, and why, when we measure "listening effort." This article is categorized under: Linguistics > Language in Mind and Brain Psychology > Theory and Methods Psychology > Attention Psychology > Emotion and Motivation.

Antismoking Campaigns' Perception and Gender Differences: A Comparison among EEG Indices

Human factors' aim is to understand and evaluate the interactions between people and tasks, technologies, and environment. Among human factors, it is possible then to include the subjective reaction to external stimuli, due to individual's characteristics and states of mind. These processes are also involved in the perception of antismoking public service announcements (PSAs), the main tool for governments to contrast the first cause of preventable deaths in the world: tobacco addiction. In the light of that, in the present article, it has been investigated through the comparison of different electroencephalographic (EEG) indices a typical item known to be able of influencing PSA perception, that is gender. In order to investigate the neurophysiological underpinnings of such different perception, we tested two PSAs: one with a female character and one with a male character. Furthermore, the experimental sample was divided into men and women, as well as smokers and nonsmokers. The employed EEG indices were the mental engagement (ME: the ratio between beta activity and the sum of alpha and theta activity); the approach/withdrawal (AW: the frontal alpha asymmetry in the alpha band); and the frontal theta activity and the spectral asymmetry index (SASI: the ratio between beta minus theta and beta plus theta). Results suggested that the ME and the AW presented an opposite trend, with smokers showing higher ME and lower AW than nonsmokers. The ME and the frontal theta also evidenced a statistically significant interaction between the kind of the PSA and the gender of the observers; specifically, women showed higher ME and frontal theta activity for the male character PSA. This study then supports the usefulness of the ME and frontal theta for purposes of PSAs targeting on the basis of gender issues and of the ME and the AW and for purposes of PSAs targeting on the basis of smoking habits.

Confidence tracks sensory- and decision-related ERP dynamics during auditory detection

Recent research has focused on measuring neural correlates of metacognitive judgments in decision and post-decision processes during memory retrieval and categorization. However, many tasks (e.g., stimulus detection) may require monitoring of earlier sensory processing. Here, participants indicated which of two intervals contained an 80-ms pure tone embedded in white noise. One frequency (e.g., 1000 Hz) was presented on ∼80% of all trials (i.e., 'primary' trials). Another frequency (e.g., 2500 Hz) was presented on ∼20% of trials (i.e., 'probe' trials). The event-related potential (ERP) was used to investigate the processing stages related to confidence. Tone-locked N1, P2, and P3 amplitudes were larger for trials rated with high than low confidence. Interestingly, a P3-like late positivity for the tone-absent interval showed high amplitude for low confidence. No 'primary' vs. 'probe' differences were found. However, confidence rating differences between primary and probe trials were correlated with N1 and tone-present P3 amplitude differences. We suggest that metacognitive judgments can track both sensory- and decision-related processes (indexed by the N1 and P3, respectively). The particular processes on which confidence judgments are based likely depend upon the task an individual is faced with and the information at hand (e.g., presence or absence of a signal).

Sustained frontal midline theta enhancements during effortful listening track working memory demands

Recent studies demonstrate that frontal midline theta power (4-8 Hz) enhancements in the electroencephalogram (EEG) relate to effortful listening. It has been proposed that these enhancements reflect working memory demands. Here, the need to retain auditory information in working memory was manipulated in a 2-interval 2-alternative forced-choice delayed pitch discrimination task ("Which interval contained the higher pitch?"). On each trial, two square wave stimuli differing in pitch at an individual's ∼70.7% correct threshold were separated by a 3-second ISI. In a 'Roving' condition, the lowest pitch stimulus was randomly selected on each trial (uniform distribution from 840 to 1160 Hz). In a 'Fixed' condition, the lowest pitch was always 979 Hz. Critically, the 'Fixed' condition allowed one to know the correct response immediately following the first stimulus (e.g., if the first stimulus is 979 Hz, the second must be higher). In contrast, the 'Roving' condition required retention of the first tone for comparison to the second. Frontal midline theta enhancements during the ISI were only observed for the 'Roving' condition. Alpha (8-13 Hz) enhancements were apparent during the ISI, but did not differ significantly between conditions. Since conditions were matched for accuracy at threshold, results suggest that frontal midline theta enhancements will not always accompany difficult listening. Mixed results in the literature regarding frontal midline theta enhancements may be related to differences between tasks in regards to working memory demands. Alpha enhancements may reflect task general effortful listening processes.

Theta- and alpha-power enhancements in the electroencephalogram as an auditory delayed match-to-sample task becomes impossibly difficult

Recent studies have related enhancements of theta- (∼4-8 Hz) and alpha-power (∼8-13 Hz) to listening effort based on parallels between enhancement and task difficulty. In contrast, nonauditory works demonstrate that, although increases in difficulty are initially accompanied by increases in effort, effort decreases when a task becomes so difficult as to exceed one's ability. Given the latter, we examined whether theta- and alpha-power enhancements thought to reflect effortful listening show a quadratic trend across levels of listening difficulty from impossible to easy. Listeners (n = 14) performed an auditory delayed match-to-sample task with frequency-modulated tonal sweeps under impossible, difficult (at ∼70.7% correct threshold), and easy (well above threshold) conditions. Frontal midline theta-power and posterior alpha-power enhancements were observed during the retention interval, with greatest enhancement in the difficult condition. Independent component-based analyses of data suggest that theta-power enhancements stemmed from medial frontal sources at or near the anterior cingulate cortex, whereas alpha-power effects stemmed from occipital cortices. Results support the notion that theta- and alpha-power enhancements reflect effortful cognitive processes during listening, related to auditory working memory and the inhibition of task-irrelevant cortical processing regions, respectively. Theta- and alpha-power dynamics can be used to characterize the cognitive processes that make up effortful listening, including qualitatively different types of listening effort.

EEG activity as an objective measure of cognitive load during effortful listening: A study on pediatric subjects with bilateral, asymmetric sensorineural hearing loss

OBJECTIVES

Deaf subjects with hearing aids or cochlear implants generally find it challenging to understand speech in noisy environments where a great deal of listening effort and cognitive load are invested. In prelingually deaf children, such difficulties may have detrimental consequences on the learning process and, later in life, on academic performance. Despite the importance of such a topic, currently, there is no validated test for the assessment of cognitive load during audiological tasks. Recently, alpha and theta EEG rhythm variations in the parietal and frontal areas, respectively, have been used as indicators of cognitive load in adult subjects. The aim of the present study was to investigate, by means of EEG, the cognitive load of pediatric subjects affected by asymmetric sensorineural hearing loss as they were engaged in a speech-in-noise identification task.

METHODS

Seven children (4F and 3M, age range = 8-16 years) affected by asymmetric sensorineural hearing loss (i.e. profound degree on one side, mild-to-severe degree on the other side) and using a hearing aid only in their better ear, were included in the study. All of them underwent EEG recording during a speech-in-noise identification task: the experimental conditions were quiet, binaural noise, noise to the better hearing ear and noise to the poorer hearing ear. The subjects' Speech Recognition Thresholds (SRT) were also measured in each test condition. The primary outcome measures were: frontal EEG Power Spectral Density (PSD) in the theta band and parietal EEG PSD in the alpha band, as assessed before stimulus (word) onset.

RESULTS

No statistically significant differences were noted among frontal theta power levels in the four test conditions. However, parietal alpha power levels were significantly higher in the "binaural noise" and in the "noise to worse hearing ear" conditions than in the "quiet" and "noise to better hearing ear" conditions (p < 0.001). SRT scores were consistent with task difficulty, but did not correlate with alpha and theta power level variations.

CONCLUSION

This is the first time that EEG has been applied to children with sensorineural hearing loss with the purpose of studying the cognitive load during effortful listening. Significantly higher parietal alpha power levels in two of three noisy conditions, compared to the quiet condition, are consistent with increased cognitive load. Specifically, considering the time window of the analysis (pre-stimulus), parietal alpha power levels may be a measure of cognitive functions such as sustained attention and selective inhibition. In this respect, the significantly lower parietal alpha power levels in the most challenging listening condition (i.e. noise to the better ear) may be attributed to loss of attention and to the subsequent fatigue and "withdrawal" from the task at hand.