The Pupil Dilation Response to Auditory Stimuli: Current State of Knowledge

Auditory Challenges and Listening Effort in School-Age Children With Autism: Insights From Pupillary Dynamics During Speech-in-Noise Perception

PURPOSE

This study aimed to investigate challenges in speech-in-noise (SiN) processing faced by school-age children with autism spectrum conditions (ASCs) and their impact on listening effort.

METHOD

Participants, including 23 Mandarin-speaking children with ASCs and 19 age-matched neurotypical (NT) peers, underwent sentence recognition tests in both quiet and noisy conditions, with a speech-shaped steady-state noise masker presented at 0-dB signal-to-noise ratio in the noisy condition. Recognition accuracy rates and task-evoked pupil responses were compared to assess behavioral performance and listening effort during auditory tasks.

RESULTS

No main effect of group was found on accuracy rates. Instead, significant effects emerged for autistic trait scores, listening conditions, and their interaction, indicating that higher trait scores were associated with poorer performance in noise. Pupillometric data revealed significantly larger and earlier peak dilations, along with more varied pupillary dynamics in the ASC group relative to the NT group, especially under noisy conditions. Importantly, the ASC group's peak dilation in quiet mirrored that of the NT group in noise. However, the ASC group consistently exhibited reduced mean dilations than the NT group.

CONCLUSIONS

Pupillary responses suggest a different resource allocation pattern in ASCs: An initial sharper and larger dilation may signal an intense, narrowed resource allocation, likely linked to heightened arousal, engagement, and cognitive load, whereas a subsequent faster tail-off may indicate a greater decrease in resource availability and engagement, or a quicker release of arousal and cognitive load. The presence of noise further accentuates this pattern. This highlights the unique SiN processing challenges children with ASCs may face, underscoring the importance of a nuanced, individual-centric approach for interventions and support.

Learning effects in speech-in-noise tasks: Effect of masker modulation and masking release

Previous research has shown that learning effects are present for speech intelligibility in temporally modulated (TM) noise, but not in stationary noise. The present study aimed to gain more insight into the factors that might affect the time course (the number of trials required to reach stable performance) and size [the improvement in the speech reception threshold (SRT)] of the learning effect. Two hypotheses were addressed: (1) learning effects are present in both TM and spectrally modulated (SM) noise and (2) the time course and size of the learning effect depend on the amount of masking release caused by either TM or SM noise. Eighteen normal-hearing adults (23-62 years) participated in SRT measurements, in which they listened to sentences in six masker conditions, including stationary, TM, and SM noise conditions. The results showed learning effects in all TM and SM noise conditions, but not for the stationary noise condition. The learning effect was related to the size of masking release: a larger masking release was accompanied by an increased time course of the learning effect and a larger learning effect. The results also indicate that speech is processed differently in SM noise than in TM noise.

Anisocoria After Direct Light Stimulus is Associated with Poor Outcomes Following Acute Brain Injury

BACKGROUND

Assessing pupil size and reactivity is the standard of care in neurocritically ill patients. Anisocoria observed in critically ill patients often prompts further investigation and treatment. This study explores anisocoria at rest and after light stimulus determined using quantitative pupillometry as a predictor of discharge modified Rankin Scale (mRS) scores.

METHODS

This analysis includes data from an international registry and includes patients with paired (left and right eye) quantitative pupillometry readings linked to discharge mRS scores. Anisocoria was defined as the absolute difference in pupil size using three common cut points (> 0.5 mm, > 1 mm, and > 2 mm). Nonparametric models were constructed to explore patient outcome using three predictors: the presence of anisocoria at rest (in ambient light); the presence of anisocoria after light stimulus; and persistent anisocoria (present both at rest and after light). The primary outcome was discharge mRS score associated with the presence of anisocoria at rest versus after light stimulus using the three commonly defined cut points.

RESULTS

This analysis included 152,905 paired observations from 6,654 patients with a mean age of 57.0 (standard deviation 17.9) years, and a median hospital stay of 5 (interquartile range 3-12) days. The mean admission Glasgow Coma Scale score was 12.7 (standard deviation 3.5), and the median discharge mRS score was 2 (interquartile range 0-4). The ranges for absolute differences in pupil diameters were 0-5.76 mm at rest and 0-6.84 mm after light. Using an anisocoria cut point of > 0.5 mm, patients with anisocoria after light had worse median mRS scores (2 [interquartile range 0-4]) than patients with anisocoria at rest (1 [interquartile range 0-3]; P < .0001). Patients with persistent anisocoria had worse median mRS scores (3 [interquartile range 1-4]) than those without persistent anisocoria (1 [interquartile range 0-3]; P < .0001). Similar findings were observed using a cut point for anisocoria of > 1 mm and > 2 mm.

CONCLUSIONS

Anisocoria after light is a new biomarker that portends worse outcome than anisocoria at rest. After further validation, anisocoria after light should be considered for inclusion as a reported and trended assessment value.

The Effects of Light Level and Signal-to-Noise Ratio on the Task-Evoked Pupil Response in a Speech-in-Noise Task

PURPOSE

There is increasing interest in the measurement of cognitive effort during listening tasks, for both research and clinical purposes. Quantification of task-evoked pupil responses (TEPRs) is a psychophysiological method that can be used to study cognitive effort. However, light level during cognitively demanding listening tasks may affect TEPRs, complicating interpretation of listening-related changes. The objective of this study was to examine the effects of light level on TEPRs during effortful listening across a range of signal-to-noise ratios (SNRs).

METHOD

Thirty-six adults without hearing loss were asked to repeat target sentences presented in background babble noise while their pupil diameter was recorded. Light level and SNRs were manipulated in a 4 × 4 repeated-measures design. Repeated-measures analyses of variance were used to measure the effects.

RESULTS

Peak and mean dilation were typically larger in more adverse SNR conditions (except for SNR -6 dB) and smaller in higher light levels. Differences in mean and peak dilation between SNR conditions were larger in dim light than in brighter light.

CONCLUSIONS

Brighter light conditions make TEPRs less sensitive to variations in listening effort across levels of SNR. Therefore, light level must be considered and reported in detail to ensure sensitivity of TEPRs and for comparisons of findings across different studies. It is recommended that TEPR testing be conducted in relatively low light conditions, considering both background illumination and screen luminance.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.25676538.

The Costs (and Benefits?) of Effortful Listening for Older Adults: Insights from Simultaneous Electrophysiology, Pupillometry, and Memory

Although the impact of acoustic challenge on speech processing and memory increases as a person ages, older adults may engage in strategies that help them compensate for these demands. In the current preregistered study, older adults (n = 48) listened to sentences-presented in quiet or in noise-that were high constraint with either expected or unexpected endings or were low constraint with unexpected endings. Pupillometry and EEG were simultaneously recorded, and subsequent sentence recognition and word recall were measured. Like young adults in prior work, we found that noise led to increases in pupil size, delayed and reduced ERP responses, and decreased recall for unexpected words. However, in contrast to prior work in young adults where a larger pupillary response predicted a recovery of the N400 at the cost of poorer memory performance in noise, older adults did not show an associated recovery of the N400 despite decreased memory performance. Instead, we found that in quiet, increases in pupil size were associated with delays in N400 onset latencies and increased recognition memory performance. In conclusion, we found that transient variation in pupil-linked arousal predicted trade-offs between real-time lexical processing and memory that emerged at lower levels of task demand in aging. Moreover, with increased acoustic challenge, older adults still exhibited costs associated with transient increases in arousal without the corresponding benefits.

Pupil Size and Eye Movements Differently Index Effort in Both Younger and Older Adults

The assessment of mental effort is increasingly relevant in neurocognitive and life span domains. Pupillometry, the measure of the pupil size, is often used to assess effort but has disadvantages. Analysis of eye movements may provide an alternative, but research has been limited to easy and difficult task demands in younger adults. An effort measure must be sensitive to the whole effort profile, including "giving up" effort investment, and capture effort in different age groups. The current study comprised three experiments in which younger (n = 66) and older (n = 44) adults listened to speech masked by background babble at different signal-to-noise ratios associated with easy, difficult, and impossible speech comprehension. We expected individuals to invest little effort for easy and impossible speech (giving up) but to exert effort for difficult speech. Indeed, pupil size was largest for difficult but lower for easy and impossible speech. In contrast, gaze dispersion decreased with increasing speech masking in both age groups. Critically, gaze dispersion during difficult speech returned to levels similar to easy speech after sentence offset, when acoustic stimulation was similar across conditions, whereas gaze dispersion during impossible speech continued to be reduced. These findings show that a reduction in eye movements is not a byproduct of acoustic factors, but instead suggest that neurocognitive processes, different from arousal-related systems regulating the pupil size, drive reduced eye movements during high task demands. The current data thus show that effort in one sensory domain (audition) differentially impacts distinct functional properties in another sensory domain (vision).

Arousal, interindividual differences and temporal binding a psychophysiological study

The sense of agency varies as a function of arousal in negative emotional contexts. As yet, it is unknown whether the same is true for positive affect, and how inter-individual characteristics might predict these effects. Temporal binding, an implicit measure of the sense of agency, was measured in 59 participants before and after watching either an emotionally neutral film clip or a positive film clip with high or low arousal. Analyses included participants' individual differences in subjective affective ratings, physiological arousal (pupillometry, skin conductance, heart rate), striatal dopamine levels via eye blink rates, and psychopathy. Linear mixed models showed that sexual arousal decreased temporal binding whereas calm pleasure had no facilitation effect on binding. Striatal dopamine levels were positively linked whereas subjective and physiological arousal may be negatively associated with binding towards actions. Psychopathic traits reduced the effect of high arousal on binding towards actions. These results provide evidence that individual differences influence the extent to which the temporal binding is affected by high arousing states with positive valence.

The pupil dilation response as an indicator of visual cue uncertainty and auditory outcome surprise

In everyday perception, we combine incoming sensory information with prior expectations. Expectations can be induced by cues that indicate the probability of following sensory events. The information provided by cues may differ and hence lead to different levels of uncertainty about which event will follow. In this experiment, we employed pupillometry to investigate whether the pupil dilation response to visual cues varies depending on the level of cue-associated uncertainty about a following auditory outcome. Also, we tested whether the pupil dilation response reflects the amount of surprise about the subsequently presented auditory stimulus. In each trial, participants were presented with a visual cue (face image) which was followed by an auditory outcome (spoken vowel). After the face cue, participants had to indicate by keypress which of three auditory vowels they expected to hear next. We manipulated the cue-associated uncertainty by varying the probabilistic cue-outcome contingencies: One face was most likely followed by one specific vowel (low cue uncertainty), another face was equally likely followed by either of two vowels (intermediate cue uncertainty) and the third face was followed by all three vowels (high cue uncertainty). Our results suggest that pupil dilation in response to task-relevant cues depends on the associated uncertainty, but only for large differences in the cue-associated uncertainty. Additionally, in response to the auditory outcomes, the pupil dilation scaled negatively with the cue-dependent probabilities, likely signalling the amount of surprise.

Pupil dilation as a function of pitch discrimination difficulty: A replication of Kahneman and Beatty, 1967

In the present paper, we carry out a replication of a seminal paper by Kahneman, D. & Beatty, J. (1967). Perception & Psychophysics, 2(3),101-105 for using pupillometry as an implicit measure of auditory processing load, specifically, non-verbal auditory processing. While numerous papers since have supported the notion that pupillometry is a fairly reliable index of processing load in general  (Zekveld, A. A., Koelewijn, T., and Kramer, S. E. (2018). Trends in Hearing, 22,1-25; Winn, M. B., Wendt, D., Koelewijn, T., and Kuchinsky, S. E. (2018). Trends in Hearing, 22,1-32), they typically have relied on memory recall, and/or more sophisticated cognitive tasks such as language comprehension or split attention. Kahneman and Beatty's paper, despite that it was published more than 50 years ago, continues to be the primary citation to support the claim that pupillometry is a reliable index of task difficulty for a simple non-verbal pitch discrimination task therefore giving us an implicit measure for listening effort (e.g.,Kramer, S. E., Lorens, A., Coninx, F., Zekveld, A. A., Piotrowska, A., & Skarzynski, H. (2013). Language and Cognitive Processes, 28(4),426-442; Schlemmer, K. B., Kulke, F., Kuchinke, L., & Van Der Meer, E. (2005). Psychophysiology, 42(4),465-472; Lisi, M., Bonato, M., and Zorzi, M. (2015). Biological Psychology, 112,39-45). This type of task takes very little explicit memory, is non-verbal, and relies heavily on more low-level, automatic perceptual processing. Using two different replication studies, one exact, and one modified, we only replicated the main result in the modified replication. The true replication failed to replicate on all nine statistical tests. Overall, our findings suggest that pupil dilation can be used as an implicit measure of task difficulty for a simple, non-semantic, auditory task, however, the robustness of the effect appears relatively weak in comparison with the original study, and the amount of variation across participants much greater.

Pupil diameter as an indicator of sound pair familiarity after statistically structured auditory sequence

Inspired by recent findings in the visual domain, we investigated whether the stimulus-evoked pupil dilation reflects temporal statistical regularities in sequences of auditory stimuli. We conducted two preregistered pupillometry experiments (experiment 1, n = 30, 21 females; experiment 2, n = 31, 22 females). In both experiments, human participants listened to sequences of spoken vowels in two conditions. In the first condition, the stimuli were presented in a random order and, in the second condition, the same stimuli were presented in a sequence structured in pairs. The second experiment replicated the first experiment with a modified timing and number of stimuli presented and without participants being informed about any sequence structure. The sound-evoked pupil dilation during a subsequent familiarity task indicated that participants learned the auditory vowel pairs of the structured condition. However, pupil diameter during the structured sequence did not differ according to the statistical regularity of the pair structure. This contrasts with similar visual studies, emphasizing the susceptibility of pupil effects during statistically structured sequences to experimental design settings in the auditory domain. In sum, our findings suggest that pupil diameter may serve as an indicator of sound pair familiarity but does not invariably respond to task-irrelevant transition probabilities of auditory sequences.

From pre-processing to advanced dynamic modeling of pupil data

The pupil of the eye provides a rich source of information for cognitive scientists, as it can index a variety of bodily states (e.g., arousal, fatigue) and cognitive processes (e.g., attention, decision-making). As pupillometry becomes a more accessible and popular methodology, researchers have proposed a variety of techniques for analyzing pupil data. Here, we focus on time series-based, signal-to-signal approaches that enable one to relate dynamic changes in pupil size over time with dynamic changes in a stimulus time series, continuous behavioral outcome measures, or other participants' pupil traces. We first introduce pupillometry, its neural underpinnings, and the relation between pupil measurements and other oculomotor behaviors (e.g., blinks, saccades), to stress the importance of understanding what is being measured and what can be inferred from changes in pupillary activity. Next, we discuss possible pre-processing steps, and the contexts in which they may be necessary. Finally, we turn to signal-to-signal analytic techniques, including regression-based approaches, dynamic time-warping, phase clustering, detrended fluctuation analysis, and recurrence quantification analysis. Assumptions of these techniques, and examples of the scientific questions each can address, are outlined, with references to key papers and software packages. Additionally, we provide a detailed code tutorial that steps through the key examples and figures in this paper. Ultimately, we contend that the insights gained from pupillometry are constrained by the analysis techniques used, and that signal-to-signal approaches offer a means to generate novel scientific insights by taking into account understudied spectro-temporal relationships between the pupil signal and other signals of interest.

Contributions of listening effort and intelligibility to cortical tracking of speech in adverse listening conditions

Cortical tracking of speech is vital for speech segmentation and is linked to speech intelligibility. However, there is no clear consensus as to whether reduced intelligibility leads to a decrease or an increase in cortical speech tracking, warranting further investigation of the factors influencing this relationship. One such factor is listening effort, defined as the cognitive resources necessary for speech comprehension, and reported to have a strong negative correlation with speech intelligibility. Yet, no studies have examined the relationship between speech intelligibility, listening effort, and cortical tracking of speech. The aim of the present study was thus to examine these factors in quiet and distinct adverse listening conditions. Forty-nine normal hearing adults listened to sentences produced casually, presented in quiet and two adverse listening conditions: cafeteria noise and reverberant speech. Electrophysiological responses were registered with electroencephalogram, and listening effort was estimated subjectively using self-reported scores and objectively using pupillometry. Results indicated varying impacts of adverse conditions on intelligibility, listening effort, and cortical tracking of speech, depending on the preservation of the speech temporal envelope. The more distorted envelope in the reverberant condition led to higher listening effort, as reflected in higher subjective scores, increased pupil diameter, and stronger cortical tracking of speech in the delta band. These findings suggest that using measures of listening effort in addition to those of intelligibility is useful for interpreting cortical tracking of speech results. Moreover, reading and phonological skills of participants were positively correlated with listening effort in the cafeteria condition, suggesting a special role of expert language skills in processing speech in this noisy condition. Implications for future research and theories linking atypical cortical tracking of speech and reading disorders are further discussed.

Input-related demands: vocoded sentences evoke different pupillometrics and subjective listening effort than sentences in speech-shaped noise

OBJECTIVES

The Framework for Effortful Listening (FUEL) suggests five input-related demands can alter listening effort: source, transmission, listener, message and context factors. We hypothesised that vocoded sentences represented a source factor degradation and sentences in speech-shaped noise represented a transmission factor degradation. We used pupillometry and a subjective scale to examine our hypothesis.

DESIGN

Participants listened to vocoded sentences and sentences in speech-shaped noise at several difficulty levels designed to produce similar word recognition abilities; they also listened to unprocessed sentences. Within-participant pupillometrics and subjective listening effort were analysed. Post-hoc analyses were performed to examine if word recognition accuracy differentially influenced pupil responses.

STUDY SAMPLES

Twenty young adults with normal hearing.

RESULTS

Baseline pupil diameter was significantly smaller, peak pupil dilation was significantly larger, peak pupil dilation latency was significantly shorter, and subjective listening effort was significantly greater for the vocoded sentences than the sentences-in-noise. Word recognition ability also affected pupillometrics, but only for the vocoded sentences.

CONCLUSIONS

Our findings suggest that source factor degradations result in greater listening effort than transmission factor degradations. Future research should address how clinical interventions tailored towards different input-related demands may lead to reduced listening effort and improve patient outcomes.

Effects of linguistic context and noise type on speech comprehension

Introduction

Understanding speech in background noise is an effortful endeavor. When acoustic challenges arise, linguistic context may help us fill in perceptual gaps. However, more knowledge is needed regarding how different types of background noise affect our ability to construct meaning from perceptually complex speech input. Additionally, there is limited evidence regarding whether perceptual complexity (e.g., informational masking) and linguistic complexity (e.g., occurrence of contextually incongruous words) interact during processing of speech material that is longer and more complex than a single sentence. Our first research objective was to determine whether comprehension of spoken sentence pairs is impacted by the informational masking from a speech masker. Our second objective was to identify whether there is an interaction between perceptual and linguistic complexity during speech processing.

Methods

We used multiple measures including comprehension accuracy, reaction time, and processing effort (as indicated by task-evoked pupil response), making comparisons across three different levels of linguistic complexity in two different noise conditions. Context conditions varied by final word, with each sentence pair ending with an expected exemplar (EE), within-category violation (WV), or between-category violation (BV). Forty young adults with typical hearing performed a speech comprehension in noise task over three visits. Each participant heard sentence pairs presented in either multi-talker babble or spectrally shaped steady-state noise (SSN), with the same noise condition across all three visits.

Results

We observed an effect of context but not noise on accuracy. Further, we observed an interaction of noise and context in peak pupil dilation data. Specifically, the context effect was modulated by noise type: context facilitated processing only in the more perceptually complex babble noise condition.

Discussion

These findings suggest that when perceptual complexity arises, listeners make use of the linguistic context to facilitate comprehension of speech obscured by background noise. Our results extend existing accounts of speech processing in noise by demonstrating how perceptual and linguistic complexity affect our ability to engage in higher-level processes, such as construction of meaning from speech segments that are longer than a single sentence.

Attention Mobilization as a Modulator of Listening Effort: Evidence From Pupillometry

Listening to speech in noise can require substantial mental effort, even among younger normal-hearing adults. The task-evoked pupil response (TEPR) has been shown to track the increased effort exerted to recognize words or sentences in increasing noise. However, few studies have examined the trajectory of listening effort across longer, more natural, stretches of speech, or the extent to which expectations about upcoming listening difficulty modulate the TEPR. Seventeen younger normal-hearing adults listened to 60-s-long audiobook passages, repeated three times in a row, at two different signal-to-noise ratios (SNRs) while pupil size was recorded. There was a significant interaction between SNR, repetition, and baseline pupil size on sustained listening effort. At lower baseline pupil sizes, potentially reflecting lower attention mobilization, TEPRs were more sustained in the harder SNR condition, particularly when attention mobilization remained low by the third presentation. At intermediate baseline pupil sizes, differences between conditions were largely absent, suggesting these listeners had optimally mobilized their attention for both SNRs. Lastly, at higher baseline pupil sizes, potentially reflecting overmobilization of attention, the effect of SNR was initially reversed for the second and third presentations: participants initially appeared to disengage in the harder SNR condition, resulting in reduced TEPRs that recovered in the second half of the story. Together, these findings suggest that the unfolding of listening effort over time depends critically on the extent to which individuals have successfully mobilized their attention in anticipation of difficult listening conditions.

Combining Cardiovascular and Pupil Features Using k-Nearest Neighbor Classifiers to Assess Task Demand, Social Context, and Sentence Accuracy During Listening

In daily life, both acoustic factors and social context can affect listening effort investment. In laboratory settings, information about listening effort has been deduced from pupil and cardiovascular responses independently. The extent to which these measures can jointly predict listening-related factors is unknown. Here we combined pupil and cardiovascular features to predict acoustic and contextual aspects of speech perception. Data were collected from 29 adults (mean  =  64.6 years, SD  =  9.2) with hearing loss. Participants performed a speech perception task at two individualized signal-to-noise ratios (corresponding to 50% and 80% of sentences correct) and in two social contexts (the presence and absence of two observers). Seven features were extracted per trial: baseline pupil size, peak pupil dilation, mean pupil dilation, interbeat interval, blood volume pulse amplitude, pre-ejection period and pulse arrival time. These features were used to train k-nearest neighbor classifiers to predict task demand, social context and sentence accuracy. The k-fold cross validation on the group-level data revealed above-chance classification accuracies: task demand, 64.4%; social context, 78.3%; and sentence accuracy, 55.1%. However, classification accuracies diminished when the classifiers were trained and tested on data from different participants. Individually trained classifiers (one per participant) performed better than group-level classifiers: 71.7% (SD  =  10.2) for task demand, 88.0% (SD  =  7.5) for social context, and 60.0% (SD  =  13.1) for sentence accuracy. We demonstrated that classifiers trained on group-level physiological data to predict aspects of speech perception generalized poorly to novel participants. Individually calibrated classifiers hold more promise for future applications.

Impact of SNR, peripheral auditory sensitivity, and central cognitive profile on the psychometric relation between pupillary response and speech performance in CI users

Despite substantial technical advances and wider clinical use, cochlear implant (CI) users continue to report high and elevated listening effort especially under challenging noisy conditions. Among all the objective measures to quantify listening effort, pupillometry is one of the most widely used and robust physiological measures. Previous studies with normally hearing (NH) and hearing-impaired (HI) listeners have shown that the relation between speech performance in noise and listening effort (as measured by peak pupil dilation) is not linear and exhibits an inverted-U shape. However, it is unclear whether the same psychometric relation exists in CI users, and whether individual differences in auditory sensitivity and central cognitive capacity affect this relation. Therefore, we recruited 17 post-lingually deaf CI adults to perform speech-in-noise tasks from 0 to 20 dB SNR with a 4 dB step size. Simultaneously, their pupillary responses and self-reported subjective effort were recorded. To characterize top-down and bottom-up individual variabilities, a spectro-temporal modulation task and a set of cognitive abilities were measured. Clinical word recognition in quiet and Quality of Life (QoL) were also collected. Results showed that at a group level, an inverted-U shape psychometric curve between task difficulty (SNR) and peak pupil dilation (PPD) was not observed. Individual shape of the psychometric curve was significantly associated with some individual factors: CI users with higher clinical word and speech-in-noise recognition showed a quadratic decrease of PPD over increasing SNRs; CI users with better non-verbal intelligence and lower QoL showed smaller average PPD. To summarize, individual differences in CI users had a significant impact on the psychometric relation between pupillary response and task difficulty, hence affecting the interpretation of pupillary response as listening effort (or engagement) at different task difficulty levels. Future research and clinical applications should further characterize the possible effects of individual factors (such as motivation or engagement) in modulating CI users' occurrence of 'tipping point' on their psychometric functions, and develop an individualized method for reliably quantifying listening effort using pupillometry.

Manipulation of phasic arousal by auditory cues is associated with subsequent changes in visual orienting to faces in infancy

This eye-tracking study investigated the effect of sound-induced arousal on social orienting under different auditory cue conditions in 5-month-old (n = 25; n = 13 males) and 10-month-old infants (n = 21; n = 14 males) participating in a spontaneous visual search task. Results showed: (1) larger pupil dilation discriminating between high and low volume (b = 0.02, p = 0.007), but not between social and non-social sounds (b = 0.004, p = 0.64); (2) faster visual orienting (b =  - 0.09, p < 0.001) and better social orienting at older age (b = 0.94, p < 0.001); (3) a fast habituation effect on social orienting after high-volume sounds (χ2(2) = 7.39, p = 0.025); (4) a quadratic association between baseline pupil size and target selection (b =  - 1.0, SE = 0.5, χ2(1) = 4.04, p = 0.045); (5) a positive linear association between pupil dilation and social orienting (b = 0.09, p = 0.039). Findings support adaptive gain theories of arousal, extending the link between phasic pupil dilation and task performance to spontaneous social orienting in infancy.

Pupil Size Sensitivity to Listening Demand Depends on Motivational State

Motivation plays a role when a listener needs to understand speech under acoustically demanding conditions. Previous work has demonstrated pupil-linked arousal being sensitive to both listening demands and motivational state during listening. It is less clear how motivational state affects the temporal evolution of the pupil size and its relation to subsequent behavior. We used an auditory gap detection task (N = 33) to study the joint impact of listening demand and motivational state on the pupil size response and examine its temporal evolution. Task difficulty and a listener's motivational state were orthogonally manipulated through changes in gap duration and monetary reward prospect. We show that participants' performance decreased with task difficulty, but that reward prospect enhanced performance under hard listening conditions. Pupil size increased with both increased task difficulty and higher reward prospect, and this reward prospect effect was largest under difficult listening conditions. Moreover, pupil size time courses differed between detected and missed gaps, suggesting that the pupil response indicates upcoming behavior. Larger pre-gap pupil size was further associated with faster response times on a trial-by-trial within-participant level. Our results reiterate the utility of pupil size as an objective and temporally sensitive measure in audiology. However, such assessments of cognitive resource recruitment need to consider the individual's motivational state.

Pupillometry reveals differences in cognitive demands of listening to face mask-attenuated speech

Face masks offer essential protection but also interfere with speech communication. Here, audio-only sentences spoken through four types of masks were presented in noise to young adult listeners. Pupil dilation (an index of cognitive demand), intelligibility, and subjective effort and performance ratings were collected. Dilation increased in response to each mask relative to the no-mask condition and differed significantly where acoustic attenuation was most prominent. These results suggest that the acoustic impact of the mask drives not only the intelligibility of speech, but also the cognitive demands of listening. Subjective effort ratings reflected the same trends as the pupil data.

Gender congruence and emotion effects in cross-modal associative learning: Insights from ERPs and pupillary responses

Social and emotional cues from faces and voices are highly relevant and have been reliably demonstrated to attract attention involuntarily. However, there are mixed findings as to which degree associating emotional valence to faces occurs automatically. In the present study, we tested whether inherently neutral faces gain additional relevance by being conditioned with either positive, negative, or neutral vocal affect bursts. During learning, participants performed a gender-matching task on face-voice pairs without explicit emotion judgments of the voices. In the test session on a subsequent day, only the previously associated faces were presented and had to be categorized regarding gender. We analyzed event-related potentials (ERPs), pupil diameter, and response times (RTs) of N = 32 subjects. Emotion effects were found in auditory ERPs and RTs during the learning session, suggesting that task-irrelevant emotion was automatically processed. However, ERPs time-locked to the conditioned faces were mainly modulated by the task-relevant information, that is, the gender congruence of the face and voice, but not by emotion. Importantly, these ERP and RT effects of learned congruence were not limited to learning but extended to the test session, that is, after removing the auditory stimuli. These findings indicate successful associative learning in our paradigm, but it did not extend to the task-irrelevant dimension of emotional relevance. Therefore, cross-modal associations of emotional relevance may not be completely automatic, even though the emotion was processed in the voice.

Phonological discrimination and contrast detection in pupillometry

Introduction

The perception of phonemes is guided by both low-level acoustic cues and high-level linguistic context. However, differentiating between these two types of processing can be challenging. In this study, we explore the utility of pupillometry as a tool to investigate both low- and high-level processing of phonological stimuli, with a particular focus on its ability to capture novelty detection and cognitive processing during speech perception.

Methods

Pupillometric traces were recorded from a sample of 22 Danish-speaking adults, with self-reported normal hearing, while performing two phonological-contrast perception tasks: a nonword discrimination task, which included minimal-pair combinations specific to the Danish language, and a nonword detection task involving the detection of phonologically modified words within sentences. The study explored the perception of contrasts in both unprocessed speech and degraded speech input, processed with a vocoder.

Results

No difference in peak pupil dilation was observed when the contrast occurred between two isolated nonwords in the nonword discrimination task. For unprocessed speech, higher peak pupil dilations were measured when phonologically modified words were detected within a sentence compared to sentences without the nonwords. For vocoded speech, higher peak pupil dilation was observed for sentence stimuli, but not for the isolated nonwords, although performance decreased similarly for both tasks.

Conclusion

Our findings demonstrate the complexity of pupil dynamics in the presence of acoustic and phonological manipulation. Pupil responses seemed to reflect higher-level cognitive and lexical processing related to phonological perception rather than low-level perception of acoustic cues. However, the incorporation of multiple talkers in the stimuli, coupled with the relatively low task complexity, may have affected the pupil dilation.

Neural correlation of speech envelope tracking for background noise in normal hearing

Everyday speech communication often occurs in environments with background noise, and the impact of noise on speech recognition can vary depending on factors such as noise type, noise intensity, and the listener's hearing ability. However, the extent to which neural mechanisms in speech understanding are influenced by different types and levels of noise remains unknown. This study aims to investigate whether individuals exhibit distinct neural responses and attention strategies depending on noise conditions. We recorded electroencephalography (EEG) data from 20 participants with normal hearing (13 males) and evaluated both neural tracking of speech envelopes and behavioral performance in speech understanding in the presence of varying types of background noise. Participants engaged in an EEG experiment consisting of two separate sessions. The first session involved listening to a 12-min story presented binaurally without any background noise. In the second session, speech understanding scores were measured using matrix sentences presented under speech-shaped noise (SSN) and Story noise background noise conditions at noise levels corresponding to sentence recognitions score (SRS). We observed differences in neural envelope correlation depending on noise type but not on its level. Interestingly, the impact of noise type on the variation in envelope tracking was more significant among participants with higher speech perception scores, while those with lower scores exhibited similarities in envelope correlation regardless of the noise condition. The findings suggest that even individuals with normal hearing could adopt different strategies to understand speech in challenging listening environments, depending on the type of noise.

The Effects of Noise and Simulated Conductive Hearing Loss on Physiological Response Measures During Interactive Conversations

PURPOSE

The purpose of this work was to study the effects of background noise and hearing attenuation associated with earplugs on three physiological measures, assumed to be markers of effort investment and arousal, during interactive communication.

METHOD

Twelve pairs of older people (average age of 63.2 years) with age-adjusted normal hearing took part in a face-to-face communication to solve a Diapix task. Communication was held in different levels of babble noise (0, 60, and 70 dBA) and with two levels of hearing attenuation (0 and 25 dB) in quiet. The physiological measures obtained included pupil size, heart rate variability, and skin conductance. In addition, subjective ratings of perceived communication success, frustration, and effort were obtained.

RESULTS

Ratings of perceived success, frustration, and effort confirmed that communication was more difficult in noise and with approximately 25-dB hearing attenuation and suggested that the implemented levels of noise and hearing attenuation resulted in comparable communication difficulties. Background noise at 70 dBA and hearing attenuation both led to an initial increase in pupil size (associated with effort), but only the effect of the background noise was sustained throughout the conversation. The 25-dB hearing attenuation led to a significant decrease of the high-frequency power of heart rate variability and a significant increase of skin conductance level, measured as the average z value of the electrodermal activity amplitude.

CONCLUSION

This study demonstrated that several physiological measures appear to be viable indicators of changing communication conditions, with pupillometry and cardiovascular as well as electrodermal measures potentially being markers of communication difficulty.

Auditory, Lexical, and Multitasking Demands Interactively Impact Listening Effort

PURPOSE

This study examined the extent to which acoustic, linguistic, and cognitive task demands interactively impact listening effort.

METHOD

Using a dual-task paradigm, on each trial, participants were instructed to perform either a single task or two tasks. In the primary word recognition task, participants repeated Northwestern University Auditory Test No. 6 words presented in speech-shaped noise at either an easier or a harder signal-to-noise ratio (SNR). The words varied in how commonly they occur in the English language (lexical frequency). In the secondary visual task, participants were instructed to press a specific key as soon as a number appeared on screen (simpler task) or one of two keys to indicate whether the visualized number was even or odd (more complex task).

RESULTS

Manipulation checks revealed that key assumptions of the dual-task design were met. A significant three-way interaction was observed, such that the expected effect of SNR on effort was only observable for words with lower lexical frequency and only when multitasking demands were relatively simpler.

CONCLUSIONS

This work reveals that variability across speech stimuli can influence the sensitivity of the dual-task paradigm for detecting changes in listening effort. In line with previous work, the results of this study also suggest that higher cognitive demands may limit the ability to detect expected effects of SNR on measures of effort. With implications for real-world listening, these findings highlight that even relatively minor changes in lexical and multitasking demands can alter the effort devoted to listening in noise.

The Influence of Hearing Loss on the Pupil Response to Degraded Speech

PURPOSE

Current evidence regarding the influence of hearing loss on the pupil response elicited by speech perception is inconsistent. This might be partially due to confounding effects of age. This study aimed to compare pupil responses in age-matched groups of normal-hearing (NH) and hard of hearing (HH) listeners during listening to speech.

METHOD

We tested the baseline pupil size and mean and peak pupil dilation response of 17 NH participants (Mage = 46 years; age range: 20-62 years) and 17 HH participants (Mage = 45 years; age range: 20-63 years) who were pairwise matched on age and educational level. Participants performed three speech perception tasks at a 50% intelligibility level: noise-vocoded speech and speech masked with either stationary noise or interfering speech. They also listened to speech presented in quiet.

RESULTS

Hearing loss was associated with poorer speech perception, except for noise-vocoded speech. In contrast to NH participants, performance of HH participants did not improve across trials for the interfering speech condition, and it decreased for speech in stationary noise. HH participants had a smaller mean pupil dilation in degraded speech conditions compared to NH participants, but not for speech in quiet. They also had a steeper decline in the baseline pupil size across trials. The baseline pupil size was smaller for noise-vocoded speech as compared to the other conditions. The normalized data showed an additional group effect on the baseline pupil response.

CONCLUSIONS

Hearing loss is associated with a smaller pupil response and steeper decline in baseline pupil size during the perception of degraded speech. This suggests difficulties of the HH participants to sustain their effort investment and performance across the test session.

Altered pupil responses to social and non-social stimuli in Shank3 mutant dogs

Pupillary response, an important process in visual perception and social and emotional cognition, has been widely studied for understanding the neural mechanisms of neuropsychiatric disorders. However, there have been few studies on pupil response to social and non-social stimuli in animal models of neurodevelopmental disorders including autism spectrum disorder (ASD) and attention deficit hyperactivity disorder. Here, we developed a pupilometer using a robust eye feature-detection algorithm for real-time pupillometry in dogs. In a pilot study, we found that a brief light flash induced a less-pronounced and slower pupil dilation response in gene-edited dogs carrying mutations in Shank3; mutations of its ortholog in humans were repeatedly identified in ASD patients. We further found that obnoxious, loud firecracker sound of 120 dB induced a stronger and longer pupil dilation response in Shank3 mutant dogs, whereas a high reward food induced a weaker pupillary response in Shank3 mutants than in wild-type control dogs. In addition, we found that Shank3 mutants showed compromised pupillary synchrony during dog-human interaction. These findings of altered pupil response in Shank3 mutant dogs recapitulate the altered sensory responses in ASD patients. Thus, this study demonstrates the validity and value of the pupilometer for dogs, and provides an effective paradigm for studying the underlying neural mechanisms of ASD and potentially other psychiatric disorders.

Copresence Was Found to Be Related to Some Pupil Measures in Persons With Hearing Loss While They Performed a Speech-in-Noise Task

OBJECTIVES

To assess if a manipulation of copresence was related to speech-in-noise task performance, arousal, and effort of persons with hearing loss. Task-related arousal and effort were measured by means of pupillometry.

DESIGN

Twenty-nine participants (mean age: 64.6 years) with hearing loss (4-frequency pure-tone average [4F-PTA] of 50.2 dB HL [SD = 8.9 dB] in the right ear and 51.3 dB HL [SD = 8.7 dB] in the left ear; averaged across 0.5, 1, 2, and 4 kHz) listened to and repeated spoken Danish sentences that were masked by four streams of continuous speech. Participants were presented with blocks of 20 sentences, during which copresence was manipulated by having participants do the task either alone or accompanied by two observers who were recruited from a similar age group. The task was presented at two difficulty levels, which was accomplished by fixing the signal-to-noise ratio of the speech and masker to match the thresholds at which participants were estimated to correctly repeat 50% (difficult) or 80% (easy) of the sentences in a block. Performance was assessed based on whether or not sentences were repeated correctly. Measures of pupil size (baseline pupil size [BPS], peak pupil dilation [PPD], and mean pupil dilation [MPD]) were used to index arousal and effort. Participants also completed ratings of subjective effort and stress after each block of sentences and a self-efficacy for listening-questionnaire.

RESULTS

Task performance was not associated with copresence, but was found to be related to 4F-PTA. An increase in BPS was found for copresence conditions, compared to alone conditions. Furthermore, a post-hoc exploratory analysis revealed that the copresence conditions were associated with a significantly larger pupil size in the second half of the task-evoked pupil response (TEPR). No change in PPD or MPD did was detected between copresence and alone conditions. Self-efficacy, 4F-PTA, and age were not found to be related to the pupil data. Subjective ratings were sensitive to task difficulty but not copresence.

CONCLUSION

Copresence was not found to be related to speech-in-noise performance, PPD, or MPD in persons with HL but was associated with an increase in arousal (as indicated by a larger BPS). This could be related to premobilization of effort and/or discomfort in response to the observers' presence. Furthermore, an exploratory analysis of the pupil data showed that copresence was associated with greater pupil dilations in the second half of the TEPR. This may indicate that participants invested more effort during the speech-in-noise task while in the presence of the observers, but that this increase in effort may not necessarily have been related to listening itself. Instead, other speech-in-noise task-related processes, such as preparing to respond, could have been influenced by copresence.

Eye Movements Decrease during Effortful Speech Listening

Hearing impairment affects many older adults but is often diagnosed decades after speech comprehension in noisy situations has become effortful. Accurate assessment of listening effort may thus help diagnose hearing impairment earlier. However, pupillometry-the most used approach to assess listening effort-has limitations that hinder its use in practice. The current study explores a novel way to assess listening effort through eye movements. Building on cognitive and neurophysiological work, we examine the hypothesis that eye movements decrease when speech listening becomes challenging. In three experiments with human participants from both sexes, we demonstrate, consistent with this hypothesis, that fixation duration increases and spatial gaze dispersion decreases with increasing speech masking. Eye movements decreased during effortful speech listening for different visual scenes (free viewing, object tracking) and speech materials (simple sentences, naturalistic stories). In contrast, pupillometry was less sensitive to speech masking during story listening, suggesting pupillometric measures may not be as effective for the assessments of listening effort in naturalistic speech-listening paradigms. Our results reveal a critical link between eye movements and cognitive load, suggesting that neural activity in the brain regions that support the regulation of eye movements, such as frontal eye field and superior colliculus, are modulated when listening is effortful.SIGNIFICANCE STATEMENT Assessment of listening effort is critical for early diagnosis of age-related hearing loss. Pupillometry is most used but has several disadvantages. The current study explores a novel way to assess listening effort through eye movements. We examine the hypothesis that eye movements decrease when speech listening becomes effortful. We demonstrate, consistent with this hypothesis, that fixation duration increases and gaze dispersion decreases with increasing speech masking. Eye movements decreased during effortful speech listening for different visual scenes (free viewing, object tracking) and speech materials (sentences, naturalistic stories). Our results reveal a critical link between eye movements and cognitive load, suggesting that neural activity in brain regions that support the regulation of eye movements are modulated when listening is effortful.

Energetic and informational masking place dissociable demands on listening effort: Evidence from simultaneous electroencephalography and pupillometrya)

The task of processing speech masked by concurrent speech/noise can pose a substantial challenge to listeners. However, performance on such tasks may not directly reflect the amount of listening effort they elicit. Changes in pupil size and neural oscillatory power in the alpha range (8-12 Hz) are prominent neurophysiological signals known to reflect listening effort; however, measurements obtained through these two approaches are rarely correlated, suggesting that they may respond differently depending on the specific cognitive demands (and, by extension, the specific type of effort) elicited by specific tasks. This study aimed to compare changes in pupil size and alpha power elicited by different types of auditory maskers (highly confusable intelligible speech maskers, speech-envelope-modulated speech-shaped noise, and unmodulated speech-shaped noise maskers) in young, normal-hearing listeners. Within each condition, the target-to-masker ratio was set at the participant's individually estimated 75% correct point on the psychometric function. The speech masking condition elicited a significantly greater increase in pupil size than either of the noise masking conditions, whereas the unmodulated noise masking condition elicited a significantly greater increase in alpha oscillatory power than the speech masking condition, suggesting that the effort needed to solve these respective tasks may have different neural origins.

Use of the first-acquired language modulates pupil size in the processing of island constraint violations

Introduction

Traditional studies of the population called "heritage speakers" (HS) have treated this group as distinct from other bilingual populations, e.g., simultaneous or late bilinguals (LB), focusing on group differences in the competencies of the first-acquired language or "heritage language". While several explanations have been proposed for such differences (e.g., incomplete acquisition, attrition, differential processing mechanisms), few have taken into consideration the individual variation that must occur, due to the fluctuation of factors such as exposure and use that characterize all bilinguals. In addition, few studies have used implicit measures, e.g., psychophysiological methods (ERPs; Eye-tracking), that can circumvent confounding variables such as resorting to conscious metalinguistic knowledge.

Methodology

This study uses pupillometry, a method that has only recently been used in psycholinguistic studies of bilingualism, to investigate pupillary responses to three syntactic island constructions in two groups of Spanish/English bilinguals: heritage speakers and late bilinguals. Data were analyzed using generalized additive mixed effects models (GAMMs) and two models were created and compared to one another: one with group (LB/HS) and the other with groups collapsed and current and historical use of Spanish as continuous variables.

Results

Results show that group-based models generally yield conflicting results while models collapsing groups and having usage as a predictor yield consistent ones. In particular, current use predicts sensitivity to L1 ungrammaticality across both HS and LB populations. We conclude that individual variation, as measured by use, is a critical factor tha must be taken into account in the description of the language competencies and processing of heritage and late bilinguals alike.

Towards personalized and optimized fitting of cochlear implants

A cochlear implant (CI) is a neurotechnological device that restores total sensorineural hearing loss. It contains a sophisticated speech processor that analyzes and transforms the acoustic input. It distributes its time-enveloped spectral content to the auditory nerve as electrical pulsed stimulation trains of selected frequency channels on a multi-contact electrode that is surgically inserted in the cochlear duct. This remarkable brain interface enables the deaf to regain hearing and understand speech. However, tuning of the large (>50) number of parameters of the speech processor, so-called "device fitting," is a tedious and complex process, which is mainly carried out in the clinic through 'one-size-fits-all' procedures. Current fitting typically relies on limited and often subjective data that must be collected in limited time. Despite the success of the CI as a hearing-restoration device, variability in speech-recognition scores among users is still very large, and mostly unexplained. The major factors that underly this variability incorporate three levels: (i) variability in auditory-system malfunction of CI-users, (ii) variability in the selectivity of electrode-to-auditory nerve (EL-AN) activation, and (iii) lack of objective perceptual measures to optimize the fitting. We argue that variability in speech recognition can only be alleviated by using objective patient-specific data for an individualized fitting procedure, which incorporates knowledge from all three levels. In this paper, we propose a series of experiments, aimed at collecting a large amount of objective (i.e., quantitative, reproducible, and reliable) data that characterize the three processing levels of the user's auditory system. Machine-learning algorithms that process these data will eventually enable the clinician to derive reliable and personalized characteristics of the user's auditory system, the quality of EL-AN signal transfer, and predictions of the perceptual effects of changes in the current fitting.

Speech recognition in noise task among children and young-adults: a pupillometry study

Introduction

Children experience unique challenges when listening to speech in noisy environments. The present study used pupillometry, an established method for quantifying listening and cognitive effort, to detect temporal changes in pupil dilation during a speech-recognition-in-noise task among school-aged children and young adults.

Methods

Thirty school-aged children and 31 young adults listened to sentences amidst four-talker babble noise in two signal-to-noise ratios (SNR) conditions: high accuracy condition (+10 dB and  + 6 dB, for children and adults, respectively) and low accuracy condition (+5 dB and + 2 dB, for children and adults, respectively). They were asked to repeat the sentences while pupil size was measured continuously during the task.

Results

During the auditory processing phase, both groups displayed pupil dilation; however, adults exhibited greater dilation than children, particularly in the low accuracy condition. In the second phase (retention), only children demonstrated increased pupil dilation, whereas adults consistently exhibited a decrease in pupil size. Additionally, the children's group showed increased pupil dilation during the response phase.

Discussion

Although adults and school-aged children produce similar behavioural scores, group differences in dilation patterns point that their underlying auditory processing differs. A second peak of pupil dilation among the children suggests that their cognitive effort during speech recognition in noise lasts longer than in adults, continuing past the first auditory processing peak dilation. These findings support effortful listening among children and highlight the need to identify and alleviate listening difficulties in school-aged children, to provide proper intervention strategies.

Streamlining experiment design in cognitive hearing science using OpenSesame

Auditory science increasingly builds on concepts and testing paradigms originated in behavioral psychology and cognitive neuroscience - an evolution of which the resulting discipline is now known as cognitive hearing science. Experimental cognitive hearing science paradigms call for hybrid cognitive and psychobehavioral tests such as those relating the attentional system, working memory, and executive functioning to low-level auditory acuity or speech intelligibility. Building complex multi-stimuli experiments can rapidly become time-consuming and error-prone. Platform-based experiment design can help streamline the implementation of cognitive hearing science experimental paradigms, promote the standardization of experiment design practices, and ensure reliability and control. Here, we introduce a set of features for the open-source python-based OpenSesame platform that allows the rapid implementation of custom behavioral and cognitive hearing science tests, including complex multichannel audio stimuli while interfacing with various synchronous inputs/outputs. Our integration includes advanced audio playback capabilities with multiple loudspeakers, an adaptive procedure, compatibility with standard I/Os and their synchronization through implementation of the Lab Streaming Layer protocol. We exemplify the capabilities of this extended OpenSesame platform with an implementation of the three-alternative forced choice amplitude modulation detection test and discuss reliability and performance. The new features are available free of charge from GitHub: https://github.com/elus-om/BRM_OMEXP .

On the relationship of arousal and attentional distraction by emotional novel sounds

Unexpected and task-irrelevant sounds can impair performance in a task. It has been shown that highly arousing emotional distractor sounds impaired performance less compared to moderately arousing neutral distractor sounds. The present study tests whether these differential emotion-related distraction effects are directly related to an enhancement of arousal evoked by processing of emotional distractor sounds. We disentangled costs of orienting of attention and benefits of increased arousal levels during the presentation of highly arousing emotional and moderately arousing neutral novel sounds that were embedded in a sequence of repeated standard sounds. We used sound-related pupil dilation responses as a marker of arousal and RTs as a marker of distraction in a visual categorization task in 57 healthy young adults. Multilevel analyses revealed increased RT and increased pupil dilation in response to novel vs. standard sounds. Emotional novel sounds reduced distraction effects on the behavioral level and increased pupil dilation responses compared to neutral novel sounds. Bayes Factors revealed strong evidence against an inverse proportional relationship between behavioral distraction effects and sound-related pupil dilation responses for emotional sounds. Given that the activity of the locus coeruleus has been linked to both changes in pupil diameter and arousal, it may embody an indirect relationship as a common antecedent by the release of norepinephrine into brain networks involved in attention control and control of the pupil. The present study provides new insights into the relation of changes in arousal and attentional distraction during the processing of emotional task-irrelevant novel sounds.

Time Scales and Moments of Listening Effort Revealed in Pupillometry

This article offers a collection of observations that highlight the value of time course data in pupillometry and points out ways in which these observations create deeper understanding of listening effort. The main message is that listening effort should be considered on a moment-to-moment basis rather than as a singular amount. A review of various studies and the reanalysis of data reveal distinct signatures of effort before a stimulus, during a stimulus, in the moments after a stimulus, and changes over whole experimental testing sessions. Collectively these observations motivate questions that extend beyond the "amount" of effort, toward understanding how long the effort lasts, and how precisely someone can allocate effort at specific points in time or reduce effort at other times. Apparent disagreements between studies are reconsidered as informative lessons about stimulus selection and the nature of pupil dilation as a reflection of decision making rather than the difficulty of sensory encoding.

A virtual reality study investigating the train illusion

The feeling of self-movement that occurs in the absence of physical motion is often referred to as vection, which is commonly exemplified using the train illusion analogy (TIA). Limited research exists on whether the TIA accurately exemplifies the experience of vection in virtual environments (VEs). Few studies complemented their vection research with participants' qualitative feedback or by recording physiological responses, and most studies used stimuli that contextually differed from the TIA. We investigated whether vection is experienced differently in a VE replicating the TIA compared to a VE depicting optic flow by recording subjective and physiological responses. Additionally, we explored participants' experience through an open question survey. We expected the TIA environment to induce enhanced vection compared to the optic flow environment. Twenty-nine participants were visually and audibly immersed in VEs that either depicted optic flow or replicated the TIA. Results showed optic flow elicited more compelling vection than the TIA environment and no consistent physiological correlates to vection were identified. The post-experiment survey revealed discrepancies between participants' quantitative and qualitative feedback. Although the dynamic content may outweigh the ecological relevance of the stimuli, it was concluded that more qualitative research is needed to understand participants' vection experience in VEs.

Language athletes: Dual-language code-switchers exhibit inhibitory control advantages

Recent studies have begun to examine bilingual cognition from more nuanced, experienced-based perspectives. The present study adds to this body of work by investigating the potential impact of code-switching on bilinguals’ inhibitory control abilities. Crucially, our bilingual participants originated from a predominantly dual-language environment, the interactional context which is believed to require (and therefore, potentially train) cognitive control processes related to goal-monitoring and inhibition. As such, 266 French Canadian bilinguals completed an online experiment wherein they were asked to complete a domain-general (Flanker) and a language-specific (bilingual Stroop) inhibitory control task, as well as extensive demographic and language background questionnaires. Stepwise multiple regressions (including various potential demographic and linguistic predictors) were conducted on the participants’ Flanker and Stroop effects. The results indicated that the bilinguals’ propensity to code-switch consistently yielded significant positive (but unidirectional) inhibitory control effects: dual-language bilinguals who reported more habitual French-to-English switching exhibited better goal-monitoring and inhibition abilities. For the language-specific task, the analysis also revealed that frequent unintentional code-switching may mitigate these inhibition skills. As such, the findings demonstrate that dual-language code-switchers may experience inhibitory control benefits, but only when their switching is self-reportedly deliberate. We conclude that the bilinguals’ interactional context is thus of primary importance, as the dual-language context is more conducive to intentional code-switching. Overall, the current study highlights the importance of considering individualistic language experience when it comes to examining potential bilingual executive functioning advantages.

Investigating the Reliability of Pupillometry as a Measure of Individualized Listening Effort

Recordings of the pupillary response have been used in numerous studies to assess listening effort during a speech-in-noise task. Most studies focused on averaged responses across listeners, whereas less is known about pupil dilation as an indicator of the individuals’ listening effort. The present study investigated the reliability of several pupil features as potential indicators of individual listening effort and the impact of different normalization procedures on the reliability. The pupil diameters of 31 normal-hearing listeners were recorded during multiple visits while performing a speech-in-noise task. The signal-to-noise ratios (SNRs) of the stimuli ranged from −12 dB to +4 dB. All listeners were measured twice at separate visits, and 11 were re-tested at a third visit. To examine the reliability of the pupil responses across visits, the intraclass correlation coefficient was applied to the peak and mean pupil dilation and to the temporal features of the pupil response, extracted using growth curve analysis. The reliability of the pupillary response was assessed in relation to SNR and different normalization procedures over multiple visits. The most reliable pupil features were the traditional mean and peak pupil dilation. The highest reliability results were obtained when the data were baseline-corrected and normalized to the individual pupil response range across all visits. Moreover, the present study results showed only a minor impact of the SNR and the number of visits on the reliability of the pupil response. Overall, the results may provide an important basis for developing a standardized test for pupillometry in the clinic.

Differentiating between Bayesian parameter learning and structure learning based on behavioural and pupil measures

Within predictive processing two kinds of learning can be distinguished: parameter learning and structure learning. In Bayesian parameter learning, parameters under a specific generative model are continuously being updated in light of new evidence. However, this learning mechanism cannot explain how new parameters are added to a model. Structure learning, unlike parameter learning, makes structural changes to a generative model by altering its causal connections or adding or removing parameters. Whilst these two types of learning have recently been formally differentiated, they have not been empirically distinguished. The aim of this research was to empirically differentiate between parameter learning and structure learning on the basis of how they affect pupil dilation. Participants took part in a within-subject computer-based learning experiment with two phases. In the first phase, participants had to learn the relationship between cues and target stimuli. In the second phase, they had to learn a conditional change in this relationship. Our results show that the learning dynamics were indeed qualitatively different between the two experimental phases, but in the opposite direction as we originally expected. Participants were learning more gradually in the second phase compared to the first phase. This might imply that participants built multiple models from scratch in the first phase (structure learning) before settling on one of these models. In the second phase, participants possibly just needed to update the probability distribution over the model parameters (parameter learning).

Human voices escape the auditory attentional blink: Evidence from detections and pupil responses

Attentional selection of a second target in a rapid stream of stimuli embedding two targets tends to be briefly impaired when two targets are presented in close temporal proximity, an effect known as an attentional blink (AB). Two target sounds (T1 and T2) were embedded in a rapid serial auditory presentation of environmental sounds with a short (Lag 3) or long lag (Lag 9). Participants were to first identify T1 (bell or sine tone) and then to detect T2 (present or absent). Individual stimuli had durations of either 30 or 90 ms, and were presented in streams of 20 sounds. The T2 varied in category: human voice, cello, or dog sound. Previous research has introduced pupillometry as a useful marker of the intensity of cognitive processing and attentional allocation in the visual AB paradigm. Results suggest that the interplay of stimulus factors is critical for target detection accuracy and provides support for the hypothesis that the human voice is the least likely to show an auditory AB (in the 90 ms condition). For the other stimuli, accuracy for T2 was significantly worse at Lag 3 than at Lag 9 in the 90 ms condition, suggesting the presence of an auditory AB. When AB occurred (at Lag 3), we observed smaller pupil dilations, time-locked to the onset of T2, compared to Lag 9, reflecting lower attentional processing when 'blinking' during target detection. Taken together, these findings support the conclusion that human voices escape the AB and that the pupillary changes are consistent with the so-called T2 attentional deficit. In addition, we found some indication that salient stimuli like human voices could require a less intense allocation of attention, or noradrenergic potentiation, compared to other auditory stimuli.

Radiology blues: Comparing occupational blue-light exposure to recommended safety standards

Background

The blue-light hazard is a well-documented entity addressing the detrimental health effects of high-energy visible light photons in the range of 305 nm - 450 nm. Radiologists spend long hours in front of multiple light-emitting diode (LED)-based diagnostic monitors emitting blue light, predisposing them to potentially higher blue-light dosages than other health professionals.

Objectives

The authors aimed to quantify the blue light that radiology registrars are exposed to in daily viewing of diagnostic monitors and compared this with international occupational safety standards.

Method

A limited cross-sectional observational study was conducted. Four radiology registrars at two academic hospitals in Bloemfontein from 01 October 2021 to 30 November 2021 participated. Diagnostic monitor viewing times on a standard workday were determined. Different image modalities obtained from 01 June 2019 to 30 November 2019 were assessed, and blue-light radiance was determined using a spectroscope and image analysis software. Blue-light radiance values were compared with international safety standards.

Results

Radiology registrars spent on average 380 min in front of a diagnostic display unit daily. Blue-light radiance from diagnostic monitors was elevated in higher-intensity images such as chest radiographs and lower for darker images like MRI brain studies. The total blue-light radiance from diagnostic display units was more than 10 000 times below the recommended threshold value for blue-light exposure.

Conclusion

Blue-light radiance from diagnostic displays measured well below the recommended values for occupational safety. Hence, blue-light exposure from diagnostic monitors does not significantly add to the occupational health burden of radiologists.

Contribution

Despite spending long hours in front of diagnostic monitors, radiologists' exposure to effective blue-light radiance from monitors was far below hazardous values. This suggests that blue-light exposure from diagnostic monitors does not increase the occupational health burden of radiologists.

Pupillometry to Assess Auditory Sensation in Guinea Pigs

Noise exposure is a leading cause of sensorineural hearing loss. Animal models of noise-induced hearing loss have generated mechanistic insight into the underlying anatomical and physiological pathologies of hearing loss. However, relating behavioral deficits observed in humans with hearing loss to behavioral deficits in animal models remains challenging. Here, pupillometry is proposed as a method that will enable the direct comparison of animal and human behavioral data. The method is based on a modified oddball paradigm - habituating the subject to the repeated presentation of a stimulus and intermittently presenting a deviant stimulus that varies in some parametric fashion from the repeated stimulus. The fundamental premise is that if the change between the repeated and deviant stimulus is detected by the subject, it will trigger a pupil dilation response that is larger than that elicited by the repeated stimulus. This approach is demonstrated using a vocalization categorization task in guinea pigs, an animal model widely used in auditory research, including in hearing loss studies. By presenting vocalizations from one vocalization category as standard stimuli and a second category as oddball stimuli embedded in noise at various signal-to-noise ratios, it is demonstrated that the magnitude of pupil dilation in response to the oddball category varies monotonically with the signal-to-noise ratio. Growth curve analyses can then be used to characterize the time course and statistical significance of these pupil dilation responses. In this protocol, detailed procedures for acclimating guinea pigs to the setup, conducting pupillometry, and evaluating/analyzing data are described. Although this technique is demonstrated in normal-hearing guinea pigs in this protocol, the method may be used to assess the sensory effects of various forms of hearing loss within each subject. These effects may then be correlated with concurrent electrophysiological measures and post-hoc anatomical observations.

Salient omissions-pupil dilation in response to unexpected omissions of sound and touch

Introduction

Recent theories describe perception as an inferential process based on internal predictive models adjusted by means of prediction violations (prediction error). To study and demonstrate predictive processing in the brain the use of unexpected stimulus omissions has been suggested as a promising approach as the evoked brain responses are uncontaminated by responses to stimuli. Here, we aimed to investigate the pupil's response to unexpected stimulus omissions in order to better understand surprise and orienting of attention resulting from prediction violation. So far only few studies have used omission in pupillometry research and results have been inconsistent.

Methods

This study adapted an EEG paradigm that has been shown to elicit omission responses in auditory and somatosensory modalities. Healthy adults pressed a button at their own pace, which resulted in the presentation of sounds or tactile stimuli in either 88%, 50% or 0% (motor-control) of cases. Pupil size was recorded continuously and averaged to analyze the pupil dilation response associated with each condition.

Results

Results revealed that omission responses were observed in both modalities in the 88%-condition compared to motor-control. Similar pupil omission responses were observed between modalities, suggesting modality-unspecific activation of the underlying brain circuits.

Discussion

In combination with previous omission studies using EEG, the findings demonstrate predictive models in brain processing and point to the involvement of subcortical structures in the omission response. Our pupillometry approach is especially suitable to study sensory prediction in vulnerable populations within the psychiatric field.

The Involvement of Listening Effort in Explaining Bilingual Listening Under Adverse Listening Conditions

The current review examines listening effort to uncover how it is implicated in bilingual performance under adverse listening conditions. Various measures of listening effort, including physiological, behavioral, and subjective measures, have been employed to examine listening effort in bilingual children and adults. Adverse listening conditions, stemming from environmental factors, as well as factors related to the speaker or listener, have been examined. The existing literature, although relatively limited to date, points to increased listening effort among bilinguals in their nondominant second language (L2) compared to their dominant first language (L1) and relative to monolinguals. Interestingly, increased effort is often observed even when speech intelligibility remains unaffected. These findings emphasize the importance of considering listening effort alongside speech intelligibility. Building upon the insights gained from the current review, we propose that various factors may modulate the observed effects. These include the particular measure selected to examine listening effort, the characteristics of the adverse condition, as well as factors related to the particular linguistic background of the bilingual speaker. Critically, further research is needed to better understand the impact of these factors on listening effort. The review outlines avenues for future research that would promote a comprehensive understanding of listening effort in bilingual individuals.

Strategic Pauses Relieve Listeners from the Effort of Listening to Fast Speech: Data Limited and Resource Limited Processes in Narrative Recall by Adult Users of Cochlear Implants

Speech that has been artificially accelerated through time compression produces a notable deficit in recall of the speech content. This is especially so for adults with cochlear implants (CI). At the perceptual level, this deficit may be due to the sharply degraded CI signal, combined with the reduced richness of compressed speech. At the cognitive level, the rapidity of time-compressed speech can deprive the listener of the ordinarily available processing time present when speech is delivered at a normal speech rate. Two experiments are reported. Experiment 1 was conducted with 27 normal-hearing young adults as a proof-of-concept demonstration that restoring lost processing time by inserting silent pauses at linguistically salient points within a time-compressed narrative ("time-restoration") returns recall accuracy to a level approximating that for a normal speech rate. Noise vocoder conditions with 10 and 6 channels reduced the effectiveness of time-restoration. Pupil dilation indicated that additional effort was expended by participants while attempting to process the time-compressed narratives, with the effortful demand on resources reduced with time restoration. In Experiment 2, 15 adult CI users tested with the same (unvocoded) materials showed a similar pattern of behavioral and pupillary responses, but with the notable exception that meaningful recovery of recall accuracy with time-restoration was limited to a subgroup of CI users identified by better working memory spans, and better word and sentence recognition scores. Results are discussed in terms of sensory-cognitive interactions in data-limited and resource-limited processes among adult users of cochlear implants.

Time-specific Components of Pupil Responses Reveal Alternations in Effort Allocation Caused by Memory Task Demands During Speech Identification in Noise

Daily communication may be effortful due to poor acoustic quality. In addition, memory demands can induce effort, especially for long or complex sentences. In the current study, we tested the impact of memory task demands and speech-to-noise ratio on the time-specific components of effort allocation during speech identification in noise. Thirty normally hearing adults (15 females, mean age 42.2 years) participated. In an established auditory memory test, listeners had to listen to a list of seven sentences in noise, and repeat the sentence-final word after presentation, and, if instructed, recall the repeated words. We tested the effects of speech-to-noise ratio (SNR; -4 dB, +1 dB) and recall (Recall; Yes, No), on the time-specific components of pupil responses, trial baseline pupil size, and their dynamics (change) along the list. We found three components in the pupil responses (early, middle, and late). While the additional memory task (recall versus no recall) lowered all components' values, SNR (-4 dB versus +1 dB SNR) increased the middle and late component values. Increasing memory demands (Recall) progressively increased trial baseline and steepened decrease of the late component's values. Trial baseline increased most steeply in the condition of +1 dB SNR with recall. The findings suggest that adding a recall to the auditory task alters effort allocation for listening. Listeners are dynamically re-allocating effort from listening to memorizing under changing memory and acoustic demands. The pupil baseline and the time-specific components of pupil responses provide a comprehensive picture of the interplay of SNR and recall on effort.

Luminance effects on pupil dilation in speech-in-noise recognition

There is an increasing interest in the field of audiology and speech communication to measure the effort that it takes to listen in noisy environments, with obvious implications for populations suffering from hearing loss. Pupillometry offers one avenue to make progress in this enterprise but important methodological questions remain to be addressed before such tools can serve practical applications. Typically, cocktail-party situations may occur in less-than-ideal lighting conditions, e.g. a pub or a restaurant, and it is unclear how robust pupil dynamics are to luminance changes. In this study, we first used a well-known paradigm where sentences were presented at different signal-to-noise ratios (SNR), all conducive of good intelligibility. This enabled us to replicate findings, e.g. a larger and later peak pupil dilation (PPD) at adverse SNR, or when the sentences were misunderstood, and to investigate the dependency of the PPD on sentence duration. A second experiment reiterated two of the SNR levels, 0 and +14 dB, but measured at 0, 75, and 220 lux. The results showed that the impact of luminance on the SNR effect was non-monotonic (sub-optimal in darkness or in bright light), and as such, there is no trivial way to derive pupillary metrics that are robust to differences in background light, posing considerable constraints for applications of pupillometry in daily life. Our findings raise an under-examined but crucial issue when designing and understanding listening effort studies using pupillometry, and offer important insights to future clinical application of pupillometry across sites.

Baseline pupil size encodes task-related information and modulates the task-evoked response in a speech-in-noise task

Pupillometry data are commonly reported relative to a baseline value recorded in a controlled pre-task condition. In this study, the influence of the experimental design and the preparatory processing related to task difficulty on the baseline pupil size was investigated during a speech intelligibility in noise paradigm. Furthermore, the relationship between the baseline pupil size and the temporal dynamics of the pupil response was assessed. The analysis revealed strong effects of block presentation order, within-block sentence order and task difficulty on the baseline values. An interaction between signal-to-noise ratio and block order was found, indicating that baseline values reflect listener expectations arising from the order in which the different blocks were presented. Furthermore, the baseline pupil size was found to affect the slope, delay and curvature of the pupillary response as well as the peak pupil dilation. This suggests that baseline correction might be sufficient when reporting pupillometry results in terms of mean pupil dilation only, but not when a more complex characterization of the temporal dynamics of the response is considered. By clarifying which factors affect baseline pupil size and how baseline values interact with the task-evoked response, the results from the present study can contribute to a better interpretation of the pupillary response as a marker of cognitive processing.

A Glimpse of Memory Through the Eyes: Pupillary Responses Measured During Encoding Reflect the Likelihood of Subsequent Memory Recall in an Auditory Free Recall Test

The aim of the current study was to investigate whether task-evoked pupillary responses measured during encoding, individual working memory capacity and noise reduction in hearing aids were associated with the likelihood of subsequently recalling an item in an auditory free recall test combined with pupillometry. Participants with mild to moderately severe symmetrical sensorineural hearing loss (n = 21) were included. The Sentence-final Word Identification and Recall (SWIR) test was administered in a background noise composed of sixteen talkers with noise reduction in hearing aids activated and deactivated. The task-evoked peak pupil dilation (PPD) was measured. The Reading Span (RS) test was used as a measure of individual working memory capacity. Larger PPD at a single trial level was significantly associated with higher likelihood of subsequently recalling a word, presumably reflecting the intensity of attention devoted during encoding. There was no clear evidence of a significant relationship between working memory capacity and subsequent memory recall, which may be attributed to the SWIR test and RS test being administered in different modalities, as well as differences in task characteristics. Noise reduction did not have a significant effect on subsequent memory recall. This may be due to the background noise not having a detrimental effect on attentional processing at the favorable signal-to-noise ratio levels at which the test was conducted.