Best Practices and Advice for Using Pupillometry to Measure Listening Effort: An Introduction for Those Who Want to Get Started

Pupillary Responses for Cognitive Load Measurement to Classify Difficulty Levels in an Educational Video Game: Empirical Study

Background

A learning task recurrently perceived as easy (or hard) may cause poor learning results. Gamer data such as errors, attempts, or time to finish a challenge are widely used to estimate the perceived difficulty level. In other contexts, pupillometry is widely used to measure cognitive load (mental effort); hence, this may describe the perceived task difficulty.

Objective

This study aims to assess the use of task-evoked pupillary responses to measure the cognitive load measure for describing the difficulty levels in a video game. In addition, it proposes an image filter to better estimate baseline pupil size and to reduce the screen luminescence effect.

Methods

We conducted an experiment that compares the baseline estimated from our filter against that estimated from common approaches. Then, a classifier with different pupil features was used to classify the difficulty of a data set containing information from students playing a video game for practicing math fractions.

Results

We observed that the proposed filter better estimates a baseline. Mauchly’s test of sphericity indicated that the assumption of sphericity had been violated (χ²₁₄=0.05; P=.001); therefore, a Greenhouse-Geisser correction was used (ε=0.47). There was a significant difference in mean pupil diameter change (MPDC) estimated from different baseline images with the scramble filter (F_5,78=30.965; P<.001). Moreover, according to the Wilcoxon signed rank test, pupillary response features that better describe the difficulty level were MPDC (z=−2.15; P=.03) and peak dilation (z=−3.58; P<.001). A random forest classifier for easy and hard levels of difficulty showed an accuracy of 75% when the gamer data were used, but the accuracy increased to 87.5% when pupillary measurements were included.

Conclusions

The screen luminescence effect on pupil size is reduced with a scrambled filter on the background video game image. Finally, pupillary response data can improve classifier accuracy for the perceived difficulty of levels in educational video games.

GazeR: A Package for Processing Gaze Position and Pupil Size Data

Eye-tracking is widely used throughout the scientific community, from vision science and psycholinguistics to marketing and human-computer interaction. Surprisingly, there is little consistency and transparency in preprocessing steps, making replicability and reproducibility difficult. To increase replicability, reproducibility, and transparency, a package in R (a free and widely used statistical programming environment) called gazeR was created to read and preprocess two types of data: gaze position and pupil size. For gaze position data, gazeR has functions for reading in raw eye-tracking data, formatting it for analysis, converting from gaze coordinates to areas of interest, and binning and aggregating data. For data from pupillometry studies, the gazeR package has functions for reading in and merging multiple raw pupil data files, removing observations with too much missing data, eliminating artifacts, blink identification and interpolation, subtractive baseline correction, and binning and aggregating data. The package is open-source and freely available for download and installation: https://github.com/dmirman/gazer . We provide step-by-step analyses of data from two tasks exemplifying the package's capabilities.

Effect of Speech-to-Noise Ratio and Luminance on a Range of Current and Potential Pupil Response Measures to Assess Listening Effort

In hearing research, pupillometry is an established method of studying listening effort. The focus of this study was to evaluate several pupil measures extracted from the Task-Evoked Pupil Responses (TEPRs) in speech-in-noise test. A range of analysis approaches was applied to extract these pupil measures, namely (a) pupil peak dilation (PPD); (b) mean pupil dilation (MPD); (c) index of pupillary activity; (d) growth curve analysis (GCA); and (e) principal component analysis (PCA). The effect of signal-to-noise ratio (SNR; Data Set A: -20 dB, -10 dB, +5 dB SNR) and luminance (Data Set B: 0.1 cd/m2, 360 cd/m2) on the TEPRs were investigated. Data Sets A and B were recorded during a speech-in-noise test and included TEPRs from 33 and 27 normal-hearing native Dutch speakers, respectively. The main results were as follows: (a) A significant effect of SNR was revealed for all pupil measures extracted in the time domain (PPD, MPD, GCA, PCA); (b) Two time series analysis approaches (GCA, PCA) provided modeled temporal profiles of TEPRs (GCA); and time windows spanning subtasks performed in a speech-in-noise test (PCA); and (c) All pupil measures revealed a significant effect of luminance. In conclusion, multiple pupil measures showed similar effects of SNR, suggesting that effort may be reflected in multiple aspects of TEPR. Moreover, a direct analysis of the pupil time course seems to provide a more holistic view of TEPRs, yet further research is needed to understand and interpret its measures. Further research is also required to find pupil measures less sensitive to changes in luminance.

Pupillometry Assessment of Speech Recognition and Listening Experience in Adult Cochlear Implant Patients

Objective

The aim of the present study was to investigate the pupillary response to word identification in cochlear implant (CI) patients. Authors hypothesized that when task difficulty (i.e., addition of background noise) increased, pupil dilation markers such as the peak dilation or the latency of the peak dilation would increase in CI users, as already observed in normal-hearing and hearing-impaired subjects.

Methods

Pupillometric measures in 10 CI patients were combined to standard speech recognition scores used to evaluate CI outcomes, namely, speech audiometry in quiet and in noise at +10 dB signal-to-noise ratio (SNR). The main outcome measures of pupillometry were mean pupil dilation, maximal pupil dilation, dilation latency, and mean dilation during return to baseline or retention interval. Subjective hearing quality was evaluated by means of one self-reported fatigue questionnaire, and the Speech, Spatial, and Qualities (SSQ) of Hearing scale.

Results

All pupil dilation data were transformed to percent change in event-related pupil dilation (ERPD, %). Analyses show that the peak amplitudes for both mean pupil dilation and maximal pupil dilation were higher during the speech-in-noise test. Mean peak dilation was measured at 3.47 ± 2.29% noise vs. 2.19 ± 2.46 in quiet and maximal peak value was detected at 9.17 ± 3.25% in noise vs. 8.72 ± 2.93% in quiet. Concerning the questionnaires, the mean pupil dilation during the retention interval was significantly correlated with the spatial subscale score of the SSQ Hearing scale [r(8) = −0.84, p = 0.0023], and with the global score [r(8) = −0.78, p = 0.0018].

Conclusion

The analysis of pupillometric traces, obtained during speech audiometry in quiet and in noise in CI users, provided interesting information about the different processes engaged in this task. Pupillometric measures could be indicative of listening difficulty, phoneme intelligibility, and were correlated with general hearing experience as evaluated by the SSQ of Hearing scale. These preliminary results show that pupillometry constitutes a promising tool to improve objective quantification of CI performance in clinical settings.

Effects of blocked vs. interleaved administration mode on saccade preparatory set revealed using pupillometry

Eye movements have been used extensively to assess information processing and cognitive function. However, significant variability in saccade performance has been observed, which could arise from methodological variations across different studies. For example, prosaccades and antisaccades have been studied using either a blocked or interleaved design, which has a significant influence on error rates and latency. This is problematic as it makes it difficult to compare saccade performance across studies and may limit the ability to use saccades as a behavioural assay to assess neurocognitive function. Thus, the current study examined how administration mode influences saccade related preparatory activity by employing pupil size as a non-invasive proxy for neural activity related to saccade planning and execution. Saccade performance and pupil dynamics were examined in eleven participants as they completed pro- and antisaccades in blocked and interleaved paradigms. Results showed that administration mode significantly modulated saccade performance and preparatory activity. Reaction times were longer for both pro- and antisaccades in the interleaved condition, compared to the blocked condition (p < 0.05). Prosaccade pupil dilations were larger in the interleaved condition (p < 0.05), while antisaccade pupil dilations did not significantly differ between administration modes. Additionally, ROC analysis provided preliminary evidence that pupil size can effectively predict saccade directional errors prior to saccade onset. We propose that task-evoked pupil dilations reflect an increase in preparatory activity for prosaccades and the corresponding cognitive demands associated with interleaved administration mode. Overall, the results highlight the importance that administration mode plays in the design of neurocognitive tasks.

Pupillometry as an Objective Measure of Sustained Attention in Young and Older Listeners

The ability to sustain attention on a task-relevant sound source while avoiding distraction from concurrent sounds is fundamental to listening in crowded environments. We aimed to (a) devise an experimental paradigm with which this aspect of listening can be isolated and (b) evaluate the applicability of pupillometry as an objective measure of sustained attention in young and older populations. We designed a paradigm that continuously measured behavioral responses and pupillometry during 25-s trials. Stimuli contained a number of concurrent, spectrally distinct tone streams. On each trial, participants detected gaps in one of the streams while resisting distraction from the others. Behavior demonstrated increasing difficulty with time-on-task and with number/proximity of distractor streams. In young listeners (N = 20; aged 18 to 35 years), pupil diameter (on the group and individual level) was dynamically modulated by instantaneous task difficulty: Periods where behavioral performance revealed a strain on sustained attention were accompanied by increased pupil diameter. Only trials on which participants performed successfully were included in the pupillometry analysis so that the observed effects reflect task demands as opposed to failure to attend. In line with existing reports, we observed global changes to pupil dynamics in the older group (N = 19; aged 63 to 79 years) including decreased pupil diameter, limited dilation range, and reduced temporal variability. However, despite these changes, older listeners showed similar effects of attentive tracking to those observed in the young listeners. Overall, our results demonstrate that pupillometry can be a reliable and time-sensitive measure of attentive tracking over long durations in both young and (with caveats) older listeners.

Effortful listening under the microscope: Examining relations between pupillometric and subjective markers of effort and tiredness from listening

Effort during listening is commonly measured using the task-evoked pupil response (TEPR); a pupillometric marker of physiological arousal. However, studies to date report no association between TEPR and perceived effort. One possible reason for this is the way in which self-report effort measures are typically administered, namely as a single data point collected at the end of a testing session. Another possible reason is that TEPR might relate more closely to the experience of tiredness from listening than to effort per se. To examine these possibilities, we conducted two preregistered experiments that recorded subjective ratings of effort and tiredness from listening at multiple time points and examined their covariance with TEPR over the course of listening tasks varying in levels of acoustic and attentional demand. In both experiments, we showed a within-subject association between TEPR and tiredness from listening, but no association between TEPR and effort. The data also suggest that the effect of task difficulty on the experience of tiredness from listening may go undetected using the traditional approach of collecting a single data point at the end of a listening block. Finally, this study demonstrates the utility of a novel correlation analysis technique ("rmcorr"), which can be used to overcome statistical power constraints commonly found in the literature. Teasing apart the subjective and physiological mechanisms that underpin effortful listening is a crucial step toward addressing these difficulties in older and/or hearing-impaired individuals.

An exploratory Study of EEG Alpha Oscillation and Pupil Dilation in Hearing-Aid Users During Effortful listening to Continuous Speech

Individuals with hearing loss allocate cognitive resources to comprehend noisy speech in everyday life scenarios. Such a scenario could be when they are exposed to ongoing speech and need to sustain their attention for a rather long period of time, which requires listening effort. Two well-established physiological methods that have been found to be sensitive to identify changes in listening effort are pupillometry and electroencephalography (EEG). However, these measurements have been used mainly for momentary, evoked or episodic effort. The aim of this study was to investigate how sustained effort manifests in pupillometry and EEG, using continuous speech with varying signal-to-noise ratio (SNR). Eight hearing-aid users participated in this exploratory study and performed a continuous speech-in-noise task. The speech material consisted of 30-second continuous streams that were presented from loudspeakers to the right and left side of the listener (±30° azimuth) in the presence of 4-talker background noise (+180° azimuth). The participants were instructed to attend either to the right or left speaker and ignore the other in a randomized order with two different SNR conditions: 0 dB and -5 dB (the difference between the target and the competing talker). The effects of SNR on listening effort were explored objectively using pupillometry and EEG. The results showed larger mean pupil dilation and decreased EEG alpha power in the parietal lobe during the more effortful condition. This study demonstrates that both measures are sensitive to changes in SNR during continuous speech.

Executive Dysfunction after a Sport-Related Concussion Is Independent of Task-Based Symptom Burden

A sport-related concussion (SRC) results in short- and long-term deficits in oculomotor control; however, it is unclear whether this change reflects executive dysfunction and/or a performance decrement caused by an increase in task-based symptom burden. Here, individuals with a SRC - and age- and sex-matched controls - completed an antisaccade task (i.e., saccade mirror-symmetrical to a target) during the early (initial assessment ≤12 days) and later (follow-up assessment <30 days) stages of recovery. Antisaccades were used because they require top-down executive control and exhibit performance decrements following an SRC. Reaction time (RT) and directional errors were included with pupillometry, because pupil size in the antisaccade task has been shown to provide a neural proxy for executive control. In addition, the Sport-Concussion Assessment Tool (SCAT-5) symptom checklist was completed prior to and after each oculomotor assessment to identify a possible task-based increase in symptomology. The SRC group yielded longer initial assessment RTs, more directional errors, and larger task-evoked pupil dilations (TEPD) than the control group. At the follow-up assessment, RTs for the SRC and control group did not reliably differ; however, the former demonstrated more directional errors and larger TEPDs. SCAT-5 symptom severity scores did not vary from the pre- to post-oculomotor evaluation for either initial or follow-up assessments. Accordingly, an SRC imparts a persistent executive dysfunction to oculomotor planning independent of a task-based increase in symptom burden. These findings evince that antisaccades serve as an effective tool to identify subtle executive deficits during the early and later stages of SRC recovery.

Effects of acoustic and semantic cues on listening effort during native and non-native speech perception

Relative to native listeners, non-native listeners who are immersed in a second language environment experience increased listening effort and a reduced ability to successfully perform an additional task while listening. Previous research demonstrated that listeners can exploit a variety of intelligibility-enhancing cues to cope with adverse listening conditions. However, little is known about the implications of those speech perception strategies for listening effort. The current research aims to investigate by means of pupillometry how listening effort is modulated in native and non-native listeners by the availability of semantic context and acoustic enhancements during the comprehension of spoken sentences. For this purpose, semantic plausibility and speaking style were manipulated both separately and in combination during a speech perception task in noise. The signal to noise ratio was individually adjusted for each participant in order to target 50% intelligibility level. Behavioural results indicated that native and non-native listeners were equally able to fruitfully exploit both semantic and acoustic cues to aid their comprehension. Pupil data indicated that listening effort was reduced for both groups of listeners when acoustic enhancements were available, while the presence of a plausible semantic context did not lead to a reduction in listening effort.

Top-down modulation of neural envelope tracking: The interplay with behavioral, self-report and neural measures of listening effort

When listening to natural speech, our brain activity tracks the slow amplitude modulations of speech, also called the speech envelope. Moreover, recent research has demonstrated that this neural envelope tracking can be affected by top-down processes. The present study was designed to examine if neural envelope tracking is modulated by the effort that a person expends during listening. Five measures were included to quantify listening effort: two behavioral measures based on a novel dual-task paradigm, a self-report effort measure and two neural measures related to phase synchronization and alpha power. Electroencephalography responses to sentences, presented at a wide range of subject-specific signal-to-noise ratios, were recorded in thirteen young, normal-hearing adults. A comparison of the five measures revealed different effects of listening effort as a function of speech understanding. Reaction times on the primary task and self-reported effort decreased with increasing speech understanding. In contrast, reaction times on the secondary task and alpha power showed a peak-shaped behavior with highest effort at intermediate speech understanding levels. With regard to neural envelope tracking, we found that the reaction times on the secondary task and self-reported effort explained a small part of the variability in theta-band envelope tracking. Speech understanding was found to strongly modulate neural envelope tracking. More specifically, our results demonstrated a robust increase in envelope tracking with increasing speech understanding. The present study provides new insights in the relations among different effort measures and highlights the potential of neural envelope tracking to objectively measure speech understanding in young, normal-hearing adults.

Rapid adaptation to fully intelligible nonnative-accented speech reduces listening effort

In noisy settings or when listening to an unfamiliar talker or accent, it can be difficult to understand spoken language. This difficulty typically results in reductions in speech intelligibility, but may also increase the effort necessary to process the speech even when intelligibility is unaffected. In this study, we used a dual-task paradigm and pupillometry to assess the cognitive costs associated with processing fully intelligible accented speech, predicting that rapid perceptual adaptation to an accent would result in decreased listening effort over time. The behavioural and physiological paradigms provided converging evidence that listeners expend greater effort when processing nonnative- relative to native-accented speech, and both experiments also revealed an overall reduction in listening effort over the course of the experiment. Only the pupillometry experiment, however, revealed greater adaptation to nonnative- relative to native-accented speech. An exploratory analysis of the dual-task data that attempted to minimise practice effects revealed weak evidence for greater adaptation to the nonnative accent. These results suggest that even when speech is fully intelligible, resolving deviations between the acoustic input and stored lexical representations incurs a processing cost, and adaptation may attenuate this cost.

The relationship between talker acoustics, intelligibility, and effort in degraded listening conditions

Listening to degraded speech is associated with decreased intelligibility and increased effort. However, listeners are generally able to adapt to certain types of degradations. While intelligibility of degraded speech is modulated by talker acoustics, it is unclear whether talker acoustics also affect effort and adaptation. Moreover, it has been demonstrated that talker differences are preserved across spectral degradations, but it is not known whether this effect extends to temporal degradations and which acoustic-phonetic characteristics are responsible. In a listening experiment combined with pupillometry, participants were presented with speech in quiet as well as in masking noise, time-compressed, and noise-vocoded speech by 16 Southern British English speakers. Results showed that intelligibility, but not adaptation, was modulated by talker acoustics. Talkers who were more intelligible under noise-vocoding were also more intelligible under masking and time-compression. This effect was linked to acoustic-phonetic profiles with greater vowel space dispersion (VSD) and energy in mid-range frequencies, as well as slower speaking rate. While pupil dilation indicated increasing effort with decreasing intelligibility, this study also linked reduced effort in quiet to talkers with greater VSD. The results emphasize the relevance of talker acoustics for intelligibility and effort in degraded listening conditions.

Task-evoked pupil response for accurately recognized accented speech

Unfamiliar second-language (L2) accents present a common challenge to speech understanding. However, the extent to which accurately recognized unfamiliar L2-accented speech imposes a greater cognitive load than native speech remains unclear. The current study used pupillometry to assess cognitive load for native English listeners during the perception of intelligible Mandarin Chinese-accented English and American-accented English. Results showed greater pupil response (indicating greater cognitive load) for the unfamiliar L2-accented speech. These findings indicate that the mismatches between unfamiliar L2-accented speech and native listeners' linguistic representations impose greater cognitive load even when recognition accuracy is at ceiling.

Physiological regulation and social-emotional processing in female carriers of the FMR1 premutation

The FMR1 gene is associated with a wide range of clinical and cognitive phenotypes, ranging from intellectual disability and autism symptoms in fragile X syndrome (caused by the FMR1 full mutation), to a more varied, and still poorly understood range of clinical and cognitive phenotypes among carriers of the gene in its premutation state. Because the FMR1 premutation is relatively common among women (as high as 1 in 150), investigations of its phenotypic impact could have broad implications for understanding gene-behavior relationships underlying complex human traits, with potential clinical implications. This study investigated physiological regulation measured by pupillary responses, along with fixation patterns while viewing facial expressions among women who carry the FMR1 premutation (PM group; n = 47), to examine whether the FMR1 gene may relate to physiological regulation, social-emotional functioning, and social language skills (where subclinical differences have been previously reported among PM carriers that resemble those documented in autism-related conditions). Relative to controls (n = 25), the PM group demonstrated atypical pupillary responses and fixation patterns, controlling for IQ. In the PM group, pupillary response and fixation patterns were related to social cognition, social language abilities, and FMR1-related variation. Results indicate a pattern of atypical attention allocation among women who carry the FMR1 PM that could reflect different emotion-processing strategies mediated by autonomic dysregulation and the FMR1 gene. These findings lend insight into the FMR1 gene's potential contributions to complex human traits such as social emotional processing and social language.

Anticipatory Baseline Pupil Diameter Is Sensitive to Differences in Hearing Thresholds

Task-evoked changes in pupil dilation have long been used as a physiological index of cognitive effort. Unlike this response, that is measured during or after an experimental trial, the baseline pupil dilation (BPD) is a measure taken prior to an experimental trial. As such, it is considered to reflect an individual’s arousal level in anticipation of an experimental trial. We report data for 68 participants, ages 18 to 89, whose hearing acuity ranged from normal hearing to a moderate hearing loss, tested over a series 160 trials on an auditory sentence comprehension task. Results showed that BPDs progressively declined over the course of the experimental trials, with participants with poorer pure tone detection thresholds showing a steeper rate of decline than those with better thresholds. Data showed this slope difference to be due to participants with poorer hearing having larger BPDs than those with better hearing at the start of the experiment, but with their BPDs approaching that of the better hearing participants by the end of the 160 trials. A finding of increasing response accuracy over trials was seen as inconsistent with a fatigue or reduced task engagement account of the diminishing BPDs. Rather, the present results imply BPD as reflecting a heightened arousal level in poorer-hearing participants in anticipation of a task that demands accurate speech perception, a concern that dissipates over trials with task success. These data taken with others suggest that the baseline pupillary response may not reflect a single construct.

Autonomic Nervous System Correlates of Speech Categorization Revealed Through Pupillometry

Human perception requires the many-to-one mapping between continuous sensory elements and discrete categorical representations. This grouping operation underlies the phenomenon of categorical perception (CP)-the experience of perceiving discrete categories rather than gradual variations in signal input. Speech perception requires CP because acoustic cues do not share constant relations with perceptual-phonetic representations. Beyond facilitating perception of unmasked speech, we reasoned CP might also aid the extraction of target speech percepts from interfering sound sources (i.e., noise) by generating additional perceptual constancy and reducing listening effort. Specifically, we investigated how noise interference impacts cognitive load and perceptual identification of unambiguous (i.e., categorical) vs. ambiguous stimuli. Listeners classified a speech vowel continuum (/u/-/a/) at various signal-to-noise ratios (SNRs [unmasked, 0 and -5 dB]). Continuous recordings of pupil dilation measured processing effort, with larger, later dilations reflecting increased listening demand. Critical comparisons were between time-locked changes in eye data in response to unambiguous (i.e., continuum endpoints) tokens vs. ambiguous tokens (i.e., continuum midpoint). Unmasked speech elicited faster responses and sharper psychometric functions, which steadily declined in noise. Noise increased pupil dilation across stimulus conditions, but not straightforwardly. Noise-masked speech modulated peak pupil size (i.e., [0 and -5 dB] > unmasked). In contrast, peak dilation latency varied with both token and SNR. Interestingly, categorical tokens elicited earlier pupil dilation relative to ambiguous tokens. Our pupillary data suggest CP reconstructs auditory percepts under challenging listening conditions through interactions between stimulus salience and listeners' internalized effort and/or arousal.

Illuminating the Dark Ages: Pupil Dilation as a Measure of Expectancy Violation Across the Life Span

Mainly for methodological reasons, little is known about the course of development of early cognitive competencies diagnosed with the violation of expectation (VoE) method in infants. The goal of this research was to evaluate the use of pupillometry as a unified approach to assess expectancy violations during and beyond the "dark ages" between 1 and 3 years. We tested children aged 1-6 years and adults (N = 279) with pictures of animals combined with matching or mismatching animal sounds. All age groups exhibited significantly greater pupil dilation in mismatched than matched trials. We conclude that pupillometry is a viable alternative to the VoE method that, by contrast to the latter, can be used throughout the life span.

Effects of Baby Schema and Mere Exposure on Explicit and Implicit Face Processing

In an increasingly multicultural society, the way people perceive individuals from the same vs different ethnic groups greatly affects their own and societal well-being. Two psychological effects that influence these perceptions are the Mere-Exposure Effect (MRE), wherein familiarity with certain objects or persons suffices for people to develop a preference for them, and the Baby Schema (BS), a set of specific facial features that evokes caregiving behaviors and an affective orientation in adults. In the present study, we aimed to investigate whether these two effects play a role in implicit physiological responses to babies vs. adults faces belonging to participants in-group vs. out-group. In study 1, the pupillary diameter of 62 Caucasian participants (M = 31; F = 31) who observed adult and infant faces of different ethnic groups (Caucasian, Chinese) was measured. In study 2, brain waves of 38 Caucasian participants (M = 19; F = 19), who observed the same set of faces, were recorded using EEG. In both studies, adults explicit preferences (i.e., attitudes) toward faces were assessed using questionnaires. In Study 1, females showed greater attention to infant than adult faces (BS effect) in both pupils, regardless of the ethnic group of the face. By contrast, males attended to infant more than adult faces for out-group faces only (BS effect). In Study 2, greater left posterior-parietal alpha activation toward out-group compared to in-group adult faces was found in males (MRE). Participants with a low BS effect toward in-group baby faces exhibited greater left posterior alpha activation to out-group than in-group baby faces (MRE). These findings reveal how different levels of sensitivity to in-group infants may moderate perceptions of both in-group and out-group baby faces. Questionnaire measures on attitudes showed that males and females preferred in-group to out-group adult faces (MRE). Participants in Study 2 also reported a greater preference for infants than adults faces (BS effect). These findings explicate the roles of gender and the Baby Schema effect in moderating implicit processing of in-group and out-group faces, despite their lack in moderating explicit reports. Contradictory findings at the implicit (physiological) and explicit (self-report) levels suggest that differential processing of faces may occur at a non-conscious level.

Pupillometry Reveals That Context Benefit in Speech Perception Can Be Disrupted by Later-Occurring Sounds, Especially in Listeners With Cochlear Implants

Contextual cues can be used to improve speech recognition, especially for people with hearing impairment. However, previous work has suggested that when the auditory signal is degraded, context might be used more slowly than when the signal is clear. This potentially puts the hearing-impaired listener in a dilemma of continuing to process the last sentence when the next sentence has already begun. This study measured the time course of the benefit of context using pupillary responses to high- and low-context sentences that were followed by silence or various auditory distractors (babble noise, ignored digits, or attended digits). Participants were listeners with cochlear implants or normal hearing using a 12-channel noise vocoder. Context-related differences in pupil dilation were greater for normal hearing than for cochlear implant listeners, even when scaled for differences in pupil reactivity. The benefit of context was systematically reduced for both groups by the presence of the later-occurring sounds, including virtually complete negation when sentences were followed by another attended utterance. These results challenge how we interpret the benefit of context in experiments that present just one utterance at a time. If a listener uses context to “repair” part of a sentence, and later-occurring auditory stimuli interfere with that repair process, the benefit of context might not survive outside the idealized laboratory or clinical environment. Elevated listening effort in hearing-impaired listeners might therefore result not just from poor auditory encoding but also inefficient use of context and prolonged processing of misperceived utterances competing with perception of incoming speech.

Auditory attention switching with listening difficulty: Behavioral and pupillometric measures

Pupillometry has emerged as a useful tool for studying listening effort. Past work involving listeners with normal audiological thresholds has shown that switching attention between competing talker streams evokes pupil dilation indicative of listening effort [McCloy, Lau, Larson, Pratt, and Lee (2017). J. Acoust. Soc. Am. 141(4), 2440-2451]. The current experiment examines behavioral and pupillometric data from a two-stream target detection task requiring attention-switching between auditory streams, in two participant groups: audiometrically normal listeners who self-report difficulty localizing sound sources and/or understanding speech in reverberant or acoustically crowded environments, and their age-matched controls who do not report such problems. Three experimental conditions varied the number and type of stream segregation cues available. Participants who reported listening difficulty showed both behavioral and pupillometric signs of increased effort compared to controls, especially in trials where listeners had to switch attention between streams, or trials where only a single stream segregation cue was available.