The human task-evoked pupillary response function is linear: Implications for baseline response scaling in pupillometry

Communication Under Sharply Degraded Auditory Input and the "2-Sentence" Problem

OBJECTIVES

Despite performing well in standard clinical assessments of speech perception, many cochlear implant (CI) users report experiencing significant difficulties when listening in real-world environments. We hypothesize that this disconnect may be related, in part, to the limited ecological validity of tests that are currently used clinically and in research laboratories. The challenges that arise from degraded auditory information provided by a CI, combined with the listener's finite cognitive resources, may lead to difficulties when processing speech material that is more demanding than the single words or single sentences that are used in clinical tests.

DESIGN

Here, we investigate whether speech identification performance and processing effort (indexed by pupil dilation measures) are affected when CI users or normal-hearing control subjects are asked to repeat two sentences presented sequentially instead of just one sentence.

RESULTS

Response accuracy was minimally affected in normal-hearing listeners, but CI users showed a wide range of outcomes, from no change to decrements of up to 45 percentage points. The amount of decrement was not predictable from the CI users' performance in standard clinical tests. Pupillometry measures tracked closely with task difficulty in both the CI group and the normal-hearing group, even though the latter had speech perception scores near ceiling levels for all conditions.

CONCLUSIONS

Speech identification performance is significantly degraded in many (but not all) CI users in response to input that is only slightly more challenging than standard clinical tests; specifically, when two sentences are presented sequentially before requesting a response, instead of presenting just a single sentence at a time. This potential "2-sentence problem" represents one of the simplest possible scenarios that go beyond presentation of the single words or sentences used in most clinical tests of speech perception, and it raises the possibility that even good performers in single-sentence tests may be seriously impaired by other ecologically relevant manipulations. The present findings also raise the possibility that a clinical version of a 2-sentence test may provide actionable information for counseling and rehabilitating CI users, and for people who interact with them closely.

The Effects of Light Level and Signal-to-Noise Ratio on the Task-Evoked Pupil Response in a Speech-in-Noise Task

PURPOSE

There is increasing interest in the measurement of cognitive effort during listening tasks, for both research and clinical purposes. Quantification of task-evoked pupil responses (TEPRs) is a psychophysiological method that can be used to study cognitive effort. However, light level during cognitively demanding listening tasks may affect TEPRs, complicating interpretation of listening-related changes. The objective of this study was to examine the effects of light level on TEPRs during effortful listening across a range of signal-to-noise ratios (SNRs).

METHOD

Thirty-six adults without hearing loss were asked to repeat target sentences presented in background babble noise while their pupil diameter was recorded. Light level and SNRs were manipulated in a 4 × 4 repeated-measures design. Repeated-measures analyses of variance were used to measure the effects.

RESULTS

Peak and mean dilation were typically larger in more adverse SNR conditions (except for SNR -6 dB) and smaller in higher light levels. Differences in mean and peak dilation between SNR conditions were larger in dim light than in brighter light.

CONCLUSIONS

Brighter light conditions make TEPRs less sensitive to variations in listening effort across levels of SNR. Therefore, light level must be considered and reported in detail to ensure sensitivity of TEPRs and for comparisons of findings across different studies. It is recommended that TEPR testing be conducted in relatively low light conditions, considering both background illumination and screen luminance.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.25676538.

Machine learning approach for ambient-light-corrected parameters and the Pupil Reactivity (PuRe) score in smartphone-based pupillometry

Introduction

The pupillary light reflex (PLR) is the constriction of the pupil in response to light. The PLR in response to a pulse of light follows a complex waveform that can be characterized by several parameters. It is a sensitive marker of acute neurological deterioration, but is also sensitive to the background illumination in the environment in which it is measured. To detect a pathological change in the PLR, it is therefore necessary to separate the contributions of neuro-ophthalmic factors from ambient illumination. Illumination varies over several orders of magnitude and is difficult to control due to diurnal, seasonal, and location variations.

Methods and results

We assessed the sensitivity of seven PLR parameters to differences in ambient light, using a smartphone-based pupillometer (AI Pupillometer, Solvemed Inc.). Nine subjects underwent 345 measurements in ambient conditions ranging from complete darkness (<5 lx) to bright lighting (≲10,000 lx). Lighting most strongly affected the initial pupil size, constriction amplitude, and velocity. Nonlinear models were fitted to find the correction function that maximally stabilized PLR parameters across different ambient light levels. Next, we demonstrated that the lighting-corrected parameters still discriminated reactive from unreactive pupils. Ten patients underwent PLR testing in an ophthalmology outpatient clinic setting following the administration of tropicamide eye drops, which rendered the pupils unreactive. The parameters corrected for lighting were combined as predictors in a machine learning model to produce a scalar value, the Pupil Reactivity (PuRe) score, which quantifies Pupil Reactivity on a scale 0-5 (0, non-reactive pupil; 0-3, abnormal/"sluggish" response; 3-5, normal/brisk response). The score discriminated unreactive pupils with 100% accuracy and was stable under changes in ambient illumination across four orders of magnitude.

Discussion

This is the first time that a correction method has been proposed to effectively mitigate the confounding influence of ambient light on PLR measurements, which could improve the reliability of pupillometric parameters both in pre-hospital and inpatient care settings. In particular, the PuRe score offers a robust measure of Pupil Reactivity directly applicable to clinical practice. Importantly, the formulae behind the score are openly available for the benefit of the clinical research community.

Pupil Dilation Reflects Perceptual Priorities During a Receptive Speech Task

OBJECTIVES

The listening demand incurred by speech perception fluctuates in normal conversation. At the acoustic-phonetic level, natural variation in pronunciation acts as speedbumps to accurate lexical selection. Any given utterance may be more or less phonetically ambiguous-a problem that must be resolved by the listener to choose the correct word. This becomes especially apparent when considering two common speech registers-clear and casual-that have characteristically different levels of phonetic ambiguity. Clear speech prioritizes intelligibility through hyperarticulation which results in less ambiguity at the phonetic level, while casual speech tends to have a more collapsed acoustic space. We hypothesized that listeners would invest greater cognitive resources while listening to casual speech to resolve the increased amount of phonetic ambiguity, as compared with clear speech. To this end, we used pupillometry as an online measure of listening effort during perception of clear and casual continuous speech in two background conditions: quiet and noise.

DESIGN

Forty-eight participants performed a probe detection task while listening to spoken, nonsensical sentences (masked and unmasked) while recording pupil size. Pupil size was modeled using growth curve analysis to capture the dynamics of the pupil response as the sentence unfolded.

RESULTS

Pupil size during listening was sensitive to the presence of noise and speech register (clear/casual). Unsurprisingly, listeners had overall larger pupil dilations during speech perception in noise, replicating earlier work. The pupil dilation pattern for clear and casual sentences was considerably more complex. Pupil dilation during clear speech trials was slightly larger than for casual speech, across quiet and noisy backgrounds.

CONCLUSIONS

We suggest that listener motivation could explain the larger pupil dilations to clearly spoken speech. We propose that, bounded by the context of this task, listeners devoted more resources to perceiving the speech signal with the greatest acoustic/phonetic fidelity. Further, we unexpectedly found systematic differences in pupil dilation preceding the onset of the spoken sentences. Together, these data demonstrate that the pupillary system is not merely reactive but also adaptive-sensitive to both task structure and listener motivation to maximize accurate perception in a limited resource system.

From pre-processing to advanced dynamic modeling of pupil data

The pupil of the eye provides a rich source of information for cognitive scientists, as it can index a variety of bodily states (e.g., arousal, fatigue) and cognitive processes (e.g., attention, decision-making). As pupillometry becomes a more accessible and popular methodology, researchers have proposed a variety of techniques for analyzing pupil data. Here, we focus on time series-based, signal-to-signal approaches that enable one to relate dynamic changes in pupil size over time with dynamic changes in a stimulus time series, continuous behavioral outcome measures, or other participants' pupil traces. We first introduce pupillometry, its neural underpinnings, and the relation between pupil measurements and other oculomotor behaviors (e.g., blinks, saccades), to stress the importance of understanding what is being measured and what can be inferred from changes in pupillary activity. Next, we discuss possible pre-processing steps, and the contexts in which they may be necessary. Finally, we turn to signal-to-signal analytic techniques, including regression-based approaches, dynamic time-warping, phase clustering, detrended fluctuation analysis, and recurrence quantification analysis. Assumptions of these techniques, and examples of the scientific questions each can address, are outlined, with references to key papers and software packages. Additionally, we provide a detailed code tutorial that steps through the key examples and figures in this paper. Ultimately, we contend that the insights gained from pupillometry are constrained by the analysis techniques used, and that signal-to-signal approaches offer a means to generate novel scientific insights by taking into account understudied spectro-temporal relationships between the pupil signal and other signals of interest.

Contributions of listening effort and intelligibility to cortical tracking of speech in adverse listening conditions

Cortical tracking of speech is vital for speech segmentation and is linked to speech intelligibility. However, there is no clear consensus as to whether reduced intelligibility leads to a decrease or an increase in cortical speech tracking, warranting further investigation of the factors influencing this relationship. One such factor is listening effort, defined as the cognitive resources necessary for speech comprehension, and reported to have a strong negative correlation with speech intelligibility. Yet, no studies have examined the relationship between speech intelligibility, listening effort, and cortical tracking of speech. The aim of the present study was thus to examine these factors in quiet and distinct adverse listening conditions. Forty-nine normal hearing adults listened to sentences produced casually, presented in quiet and two adverse listening conditions: cafeteria noise and reverberant speech. Electrophysiological responses were registered with electroencephalogram, and listening effort was estimated subjectively using self-reported scores and objectively using pupillometry. Results indicated varying impacts of adverse conditions on intelligibility, listening effort, and cortical tracking of speech, depending on the preservation of the speech temporal envelope. The more distorted envelope in the reverberant condition led to higher listening effort, as reflected in higher subjective scores, increased pupil diameter, and stronger cortical tracking of speech in the delta band. These findings suggest that using measures of listening effort in addition to those of intelligibility is useful for interpreting cortical tracking of speech results. Moreover, reading and phonological skills of participants were positively correlated with listening effort in the cafeteria condition, suggesting a special role of expert language skills in processing speech in this noisy condition. Implications for future research and theories linking atypical cortical tracking of speech and reading disorders are further discussed.

Sequence effects and speech processing: cognitive load for speaker-switching within and across accents

Prior work in speech processing indicates that listening tasks with multiple speakers (as opposed to a single speaker) result in slower and less accurate processing. Notably, the trial-to-trial cognitive demands of switching between speakers or switching between accents have yet to be examined. We used pupillometry, a physiological index of cognitive load, to examine the demands of processing first (L1) and second (L2) language-accented speech when listening to sentences produced by the same speaker consecutively (no switch), a novel speaker of the same accent (within-accent switch), and a novel speaker with a different accent (across-accent switch). Inspired by research on sequential adjustments in cognitive control, we aimed to identify the cognitive demands of accommodating a novel speaker and accent by examining the trial-to-trial changes in pupil dilation during speech processing. Our results indicate that switching between speakers was more cognitively demanding than listening to the same speaker consecutively. Additionally, switching to a novel speaker with a different accent was more cognitively demanding than switching between speakers of the same accent. However, there was an asymmetry for across-accent switches, such that switching from an L1 to an L2 accent was more demanding than vice versa. Findings from the present study align with work examining multi-talker processing costs, and provide novel evidence that listeners dynamically adjust cognitive processing to accommodate speaker and accent variability. We discuss these novel findings in the context of an active control model and auditory streaming framework of speech processing.

Decoupling of the pupillary light response during internal attention: The modulating effect of luminance intensity

In a world full of sensory stimuli, attention guides us between the external environment and our internal thoughts. While external attention involves processing sensory stimuli, internal attention is devoted to self-generated representations such as planning or spontaneous mind wandering. They both draw from common cognitive resources, thus simultaneous engagement in both often leads to interference between processes. In order to maintain internal focus, an attentional mechanism known as perceptual decoupling takes effect. This mechanism supports internal cognition by decoupling attention from the perception of sensory information. Two previous studies of our lab investigated to what extent perceptual decoupling is evident in voluntary eye movements. Findings showed that the effect is mediated by the internal task modality and workload (visuospatial > arithmetic and high > low, respectively). However, it remains unclear whether it extends to involuntary eye behavior, which may not share cognitive resources with internal activities. Therefore, the present experiment aimed to further elucidate attentional dynamics by examining whether internal attention affects the pupillary light response (PLR). Specifically, we consistently observed that workload and task modality of the internal task reduced the PLR to luminance changes of medium intensity. However, the PLR to strong luminance changes was less or not at all affected by the internal task. These results suggest that perceptual decoupling effects may be less consistent in involuntary eye behavior, particularly in the context of a salient visual stimulus.

Nonlinear changes in pupillary attentional orienting responses across the lifespan

The cognitive aging process is not necessarily linear. Central task-evoked pupillary responses, representing a brainstem-pupil relationship, may vary across the lifespan. Thus we examined, in 75 adults ranging in age from 19 to 86, whether task-evoked pupillary responses to an attention task may serve in as an index of cognitive aging. This is because the locus coeruleus (LC), located in the brainstem, is not only among the earliest sites of degeneration in pathological aging, but also supports both attentional and pupillary behaviors. We assessed brief, task-evoked phasic attentional orienting to behaviorally relevant and irrelevant auditory tones, stimuli known specifically to recruit the LC in the brainstem and evoke pupillary responses. Due to potential nonlinear changes across the lifespan, we used a novel data-driven analysis on 6 dynamic pupillary behaviors on 10% of the data to reveal cut off points that best characterized the three age bands: young (19-41 years old), middle aged (42-68 years old), and older adults (69 + years old). Follow-up analyses on independent data, the remaining 90%, revealed age-related changes such as monotonic decreases in tonic pupillary diameter and dynamic range, along with curvilinear phasic pupillary responses to the behaviorally relevant target events, increasing in the middle-aged group and then decreasing in the older group. Additionally, the older group showed decreased differentiation of pupillary responses between target and distractor events. This pattern is consistent with potential compensatory LC activity in midlife that is diminished in old age, resulting in decreased adaptive gain. Beyond regulating responses to light, pupillary dynamics reveal a nonlinear capacity for neurally mediated gain across the lifespan, thus providing evidence in support of the LC adaptive gain hypothesis.

Attention Mobilization as a Modulator of Listening Effort: Evidence From Pupillometry

Listening to speech in noise can require substantial mental effort, even among younger normal-hearing adults. The task-evoked pupil response (TEPR) has been shown to track the increased effort exerted to recognize words or sentences in increasing noise. However, few studies have examined the trajectory of listening effort across longer, more natural, stretches of speech, or the extent to which expectations about upcoming listening difficulty modulate the TEPR. Seventeen younger normal-hearing adults listened to 60-s-long audiobook passages, repeated three times in a row, at two different signal-to-noise ratios (SNRs) while pupil size was recorded. There was a significant interaction between SNR, repetition, and baseline pupil size on sustained listening effort. At lower baseline pupil sizes, potentially reflecting lower attention mobilization, TEPRs were more sustained in the harder SNR condition, particularly when attention mobilization remained low by the third presentation. At intermediate baseline pupil sizes, differences between conditions were largely absent, suggesting these listeners had optimally mobilized their attention for both SNRs. Lastly, at higher baseline pupil sizes, potentially reflecting overmobilization of attention, the effect of SNR was initially reversed for the second and third presentations: participants initially appeared to disengage in the harder SNR condition, resulting in reduced TEPRs that recovered in the second half of the story. Together, these findings suggest that the unfolding of listening effort over time depends critically on the extent to which individuals have successfully mobilized their attention in anticipation of difficult listening conditions.

Self-regulating arousal via pupil-based biofeedback

The brain's arousal state is controlled by several neuromodulatory nuclei known to substantially influence cognition and mental well-being. Here we investigate whether human participants can gain volitional control of their arousal state using a pupil-based biofeedback approach. Our approach inverts a mechanism suggested by previous literature that links activity of the locus coeruleus, one of the key regulators of central arousal and pupil dynamics. We show that pupil-based biofeedback enables participants to acquire volitional control of pupil size. Applying pupil self-regulation systematically modulates activity of the locus coeruleus and other brainstem structures involved in arousal control. Furthermore, it modulates cardiovascular measures such as heart rate, and behavioural and psychophysiological responses during an oddball task. We provide evidence that pupil-based biofeedback makes the brain's arousal system accessible to volitional control, a finding that has tremendous potential for translation to behavioural and clinical applications across various domains, including stress-related and anxiety disorders.

Pupillometry reveals differences in cognitive demands of listening to face mask-attenuated speech

Face masks offer essential protection but also interfere with speech communication. Here, audio-only sentences spoken through four types of masks were presented in noise to young adult listeners. Pupil dilation (an index of cognitive demand), intelligibility, and subjective effort and performance ratings were collected. Dilation increased in response to each mask relative to the no-mask condition and differed significantly where acoustic attenuation was most prominent. These results suggest that the acoustic impact of the mask drives not only the intelligibility of speech, but also the cognitive demands of listening. Subjective effort ratings reflected the same trends as the pupil data.

The Influence of Hearing Loss on the Pupil Response to Degraded Speech

PURPOSE

Current evidence regarding the influence of hearing loss on the pupil response elicited by speech perception is inconsistent. This might be partially due to confounding effects of age. This study aimed to compare pupil responses in age-matched groups of normal-hearing (NH) and hard of hearing (HH) listeners during listening to speech.

METHOD

We tested the baseline pupil size and mean and peak pupil dilation response of 17 NH participants (Mage = 46 years; age range: 20-62 years) and 17 HH participants (Mage = 45 years; age range: 20-63 years) who were pairwise matched on age and educational level. Participants performed three speech perception tasks at a 50% intelligibility level: noise-vocoded speech and speech masked with either stationary noise or interfering speech. They also listened to speech presented in quiet.

RESULTS

Hearing loss was associated with poorer speech perception, except for noise-vocoded speech. In contrast to NH participants, performance of HH participants did not improve across trials for the interfering speech condition, and it decreased for speech in stationary noise. HH participants had a smaller mean pupil dilation in degraded speech conditions compared to NH participants, but not for speech in quiet. They also had a steeper decline in the baseline pupil size across trials. The baseline pupil size was smaller for noise-vocoded speech as compared to the other conditions. The normalized data showed an additional group effect on the baseline pupil response.

CONCLUSIONS

Hearing loss is associated with a smaller pupil response and steeper decline in baseline pupil size during the perception of degraded speech. This suggests difficulties of the HH participants to sustain their effort investment and performance across the test session.

Pupil Size Is Sensitive to Low-Level Stimulus Features, Independent of Arousal-Related Modulation

Similar to a camera aperture, pupil size adjusts to the surrounding luminance. Unlike a camera, pupil size is additionally modulated both by stimulus properties and by cognitive processes, including attention and arousal, though the interdependence of these factors is unclear. We hypothesized that different stimulus properties interact to jointly modulate pupil size while remaining independent from the impact of arousal. We measured pupil responses from human observers to equiluminant stimuli during a demanding rapid serial visual presentation (RSVP) task at fixation and tested how response amplitude depends on contrast, spatial frequency, and reward level. We found that under constant luminance, unattended stimuli evoke responses that are separable from changes caused by general arousal or attention. We further uncovered a double-dissociation between task-related responses and stimulus-evoked responses, suggesting that different sources of pupil size modulation are independent of one another. Our results shed light on neural pathways underlying pupillary response.

Copresence Was Found to Be Related to Some Pupil Measures in Persons With Hearing Loss While They Performed a Speech-in-Noise Task

OBJECTIVES

To assess if a manipulation of copresence was related to speech-in-noise task performance, arousal, and effort of persons with hearing loss. Task-related arousal and effort were measured by means of pupillometry.

DESIGN

Twenty-nine participants (mean age: 64.6 years) with hearing loss (4-frequency pure-tone average [4F-PTA] of 50.2 dB HL [SD = 8.9 dB] in the right ear and 51.3 dB HL [SD = 8.7 dB] in the left ear; averaged across 0.5, 1, 2, and 4 kHz) listened to and repeated spoken Danish sentences that were masked by four streams of continuous speech. Participants were presented with blocks of 20 sentences, during which copresence was manipulated by having participants do the task either alone or accompanied by two observers who were recruited from a similar age group. The task was presented at two difficulty levels, which was accomplished by fixing the signal-to-noise ratio of the speech and masker to match the thresholds at which participants were estimated to correctly repeat 50% (difficult) or 80% (easy) of the sentences in a block. Performance was assessed based on whether or not sentences were repeated correctly. Measures of pupil size (baseline pupil size [BPS], peak pupil dilation [PPD], and mean pupil dilation [MPD]) were used to index arousal and effort. Participants also completed ratings of subjective effort and stress after each block of sentences and a self-efficacy for listening-questionnaire.

RESULTS

Task performance was not associated with copresence, but was found to be related to 4F-PTA. An increase in BPS was found for copresence conditions, compared to alone conditions. Furthermore, a post-hoc exploratory analysis revealed that the copresence conditions were associated with a significantly larger pupil size in the second half of the task-evoked pupil response (TEPR). No change in PPD or MPD did was detected between copresence and alone conditions. Self-efficacy, 4F-PTA, and age were not found to be related to the pupil data. Subjective ratings were sensitive to task difficulty but not copresence.

CONCLUSION

Copresence was not found to be related to speech-in-noise performance, PPD, or MPD in persons with HL but was associated with an increase in arousal (as indicated by a larger BPS). This could be related to premobilization of effort and/or discomfort in response to the observers' presence. Furthermore, an exploratory analysis of the pupil data showed that copresence was associated with greater pupil dilations in the second half of the TEPR. This may indicate that participants invested more effort during the speech-in-noise task while in the presence of the observers, but that this increase in effort may not necessarily have been related to listening itself. Instead, other speech-in-noise task-related processes, such as preparing to respond, could have been influenced by copresence.

Methods in cognitive pupillometry: Design, preprocessing, and statistical analysis

Cognitive pupillometry is the measurement of pupil size to investigate cognitive processes such as attention, mental effort, working memory, and many others. Currently, there is no commonly agreed-upon methodology for conducting cognitive-pupillometry experiments, and approaches vary widely between research groups and even between different experiments from the same group. This lack of consensus makes it difficult to know which factors to consider when conducting a cognitive-pupillometry experiment. Here we provide a comprehensive, hands-on guide to methods in cognitive pupillometry, with a focus on trial-based experiments in which the measure of interest is the task-evoked pupil response to a stimulus. We cover all methodological aspects of cognitive pupillometry: experimental design, preprocessing of pupil-size data, and statistical techniques to deal with multiple comparisons when testing pupil-size data. In addition, we provide code and toolboxes (in Python) for preprocessing and statistical analysis, and we illustrate all aspects of the proposed workflow through an example experiment and example scripts.

The effects of type and workload of internal tasks on voluntary saccades in a target-distractor saccade task

When we engage in internally directed cognition, like doing mental arithmetic or mind wandering, fewer cognitive resources are assigned for other activities like reacting to perceptual input-an effect termed perceptual decoupling. However, the exact conditions under which perceptual decoupling occurs and its underlying cognitive mechanisms are still unclear. Hence, the present study systematically manipulated the task type (arithmetic, visuospatial) and workload (control, low, high) of the internal task in a within-subject design and tested its effects on voluntary saccades in a target-distractor saccade task. As expected, engagement in internal tasks delayed saccades to the target. This effect was moderated by time, task, and workload: The delay was largest right after internal task onset and then decreased, potentially reflecting the intensity of internal task demands. Saccades were also more delayed for the high compared to the low workload condition in the arithmetic task, whereas workload conditions had similarly high effects in the visuospatial task. Findings suggests that perceptual decoupling of eye behavior gradually increases with internal demands on general resources and that perceptual decoupling is specifically sensitive to internal demands on visuospatial resources. The latter may be mediated by interference due to eye behavior elicited by the internal task itself. Internal tasks did not affect the saccade latency-deviation trade-off, indicating that while the internal tasks delayed the execution of the saccade, the perception of the saccade stimuli and spatial planning of the saccade continued unaffected in parallel to the internal tasks. Together, these findings shed further light on the specific mechanisms underlying perceptual decoupling by suggesting that perceptual decoupling of eye behavior increases as internal demands on cognitive resources overlap more strongly with demands of the external task.

Eye-movement indices of arousal predict ADHD and comorbid externalizing symptoms over a 2-year period

Attention-deficit/hyperactivity disorder (ADHD) follows a variable course across childhood. Disrupted arousal has been hypothesized to underlie core symptoms as well as comorbid internalizing and externalizing conditions. The current study examined eye-movement and pupil-dilation metrics indexing arousal as longitudinal predictors of ADHD, externalizing, and internalizing symptoms over a 2-year period. Participants aged 8-13 years (N = 54, 30% with a diagnosis of ADHD) completed a modified version of the gap-overlap task including arousal-inducing auditory warning signals. Parents rated symptoms at the time of testing and at 2 years follow-up. Phasic alerting (reaction-time reduction after alerting cues) is an index of arousal. Here, larger phasic alerting effects predicted higher ADHD-symptom levels 2 years later. Blunted pupil-dilation responses predicted externalizing symptoms at T2, controlling for ADHD and externalizing at T1. Our results support the theory that ADHD is associated with altered arousal. Blunted arousal reactivity may be a longitudinal risk factor for externalizing problems in children with ADHD symptoms.

Investigating the Reliability of Pupillometry as a Measure of Individualized Listening Effort

Recordings of the pupillary response have been used in numerous studies to assess listening effort during a speech-in-noise task. Most studies focused on averaged responses across listeners, whereas less is known about pupil dilation as an indicator of the individuals’ listening effort. The present study investigated the reliability of several pupil features as potential indicators of individual listening effort and the impact of different normalization procedures on the reliability. The pupil diameters of 31 normal-hearing listeners were recorded during multiple visits while performing a speech-in-noise task. The signal-to-noise ratios (SNRs) of the stimuli ranged from −12 dB to +4 dB. All listeners were measured twice at separate visits, and 11 were re-tested at a third visit. To examine the reliability of the pupil responses across visits, the intraclass correlation coefficient was applied to the peak and mean pupil dilation and to the temporal features of the pupil response, extracted using growth curve analysis. The reliability of the pupillary response was assessed in relation to SNR and different normalization procedures over multiple visits. The most reliable pupil features were the traditional mean and peak pupil dilation. The highest reliability results were obtained when the data were baseline-corrected and normalized to the individual pupil response range across all visits. Moreover, the present study results showed only a minor impact of the SNR and the number of visits on the reliability of the pupil response. Overall, the results may provide an important basis for developing a standardized test for pupillometry in the clinic.

Strategic Pauses Relieve Listeners from the Effort of Listening to Fast Speech: Data Limited and Resource Limited Processes in Narrative Recall by Adult Users of Cochlear Implants

Speech that has been artificially accelerated through time compression produces a notable deficit in recall of the speech content. This is especially so for adults with cochlear implants (CI). At the perceptual level, this deficit may be due to the sharply degraded CI signal, combined with the reduced richness of compressed speech. At the cognitive level, the rapidity of time-compressed speech can deprive the listener of the ordinarily available processing time present when speech is delivered at a normal speech rate. Two experiments are reported. Experiment 1 was conducted with 27 normal-hearing young adults as a proof-of-concept demonstration that restoring lost processing time by inserting silent pauses at linguistically salient points within a time-compressed narrative ("time-restoration") returns recall accuracy to a level approximating that for a normal speech rate. Noise vocoder conditions with 10 and 6 channels reduced the effectiveness of time-restoration. Pupil dilation indicated that additional effort was expended by participants while attempting to process the time-compressed narratives, with the effortful demand on resources reduced with time restoration. In Experiment 2, 15 adult CI users tested with the same (unvocoded) materials showed a similar pattern of behavioral and pupillary responses, but with the notable exception that meaningful recovery of recall accuracy with time-restoration was limited to a subgroup of CI users identified by better working memory spans, and better word and sentence recognition scores. Results are discussed in terms of sensory-cognitive interactions in data-limited and resource-limited processes among adult users of cochlear implants.

Luminance effects on pupil dilation in speech-in-noise recognition

There is an increasing interest in the field of audiology and speech communication to measure the effort that it takes to listen in noisy environments, with obvious implications for populations suffering from hearing loss. Pupillometry offers one avenue to make progress in this enterprise but important methodological questions remain to be addressed before such tools can serve practical applications. Typically, cocktail-party situations may occur in less-than-ideal lighting conditions, e.g. a pub or a restaurant, and it is unclear how robust pupil dynamics are to luminance changes. In this study, we first used a well-known paradigm where sentences were presented at different signal-to-noise ratios (SNR), all conducive of good intelligibility. This enabled us to replicate findings, e.g. a larger and later peak pupil dilation (PPD) at adverse SNR, or when the sentences were misunderstood, and to investigate the dependency of the PPD on sentence duration. A second experiment reiterated two of the SNR levels, 0 and +14 dB, but measured at 0, 75, and 220 lux. The results showed that the impact of luminance on the SNR effect was non-monotonic (sub-optimal in darkness or in bright light), and as such, there is no trivial way to derive pupillary metrics that are robust to differences in background light, posing considerable constraints for applications of pupillometry in daily life. Our findings raise an under-examined but crucial issue when designing and understanding listening effort studies using pupillometry, and offer important insights to future clinical application of pupillometry across sites.

Baseline pupil size encodes task-related information and modulates the task-evoked response in a speech-in-noise task

Pupillometry data are commonly reported relative to a baseline value recorded in a controlled pre-task condition. In this study, the influence of the experimental design and the preparatory processing related to task difficulty on the baseline pupil size was investigated during a speech intelligibility in noise paradigm. Furthermore, the relationship between the baseline pupil size and the temporal dynamics of the pupil response was assessed. The analysis revealed strong effects of block presentation order, within-block sentence order and task difficulty on the baseline values. An interaction between signal-to-noise ratio and block order was found, indicating that baseline values reflect listener expectations arising from the order in which the different blocks were presented. Furthermore, the baseline pupil size was found to affect the slope, delay and curvature of the pupillary response as well as the peak pupil dilation. This suggests that baseline correction might be sufficient when reporting pupillometry results in terms of mean pupil dilation only, but not when a more complex characterization of the temporal dynamics of the response is considered. By clarifying which factors affect baseline pupil size and how baseline values interact with the task-evoked response, the results from the present study can contribute to a better interpretation of the pupillary response as a marker of cognitive processing.

Interactions between acoustic challenges and processing depth in speech perception as measured by task-evoked pupil response

Speech perception under adverse conditions is a multistage process involving a dynamic interplay among acoustic, cognitive, and linguistic factors. Nevertheless, prior research has primarily focused on factors within this complex system in isolation. The primary goal of the present study was to examine the interaction between processing depth and the acoustic challenge of noise and its effect on processing effort during speech perception in noise. Two tasks were used to represent different depths of processing. The speech recognition task involved repeating back a sentence after auditory presentation (higher-level processing), while the tiredness judgment task entailed a subjective judgment of whether the speaker sounded tired (lower-level processing). The secondary goal of the study was to investigate whether pupil response to alteration of dynamic pitch cues stems from difficult linguistic processing of speech content in noise or a perceptual novelty effect due to the unnatural pitch contours. Task-evoked peak pupil response from two groups of younger adult participants with typical hearing was measured in two experiments. Both tasks (speech recognition and tiredness judgment) were implemented in both experiments, and stimuli were presented with background noise in Experiment 1 and without noise in Experiment 2. Increased peak pupil dilation was associated with deeper processing (i.e., the speech recognition task), particularly in the presence of background noise. Importantly, there is a non-additive interaction between noise and task, as demonstrated by the heightened peak pupil dilation to noise in the speech recognition task as compared to in the tiredness judgment task. Additionally, peak pupil dilation data suggest dynamic pitch alteration induced an increased perceptual novelty effect rather than reflecting effortful linguistic processing of the speech content in noise. These findings extend current theories of speech perception under adverse conditions by demonstrating that the level of processing effort expended by a listener is influenced by the interaction between acoustic challenges and depth of linguistic processing. The study also provides a foundation for future work to investigate the effects of this complex interaction in clinical populations who experience both hearing and cognitive challenges.

Trait anxiety affects attentional bias to emotional stimuli across time: A growth curve analysis

Many studies have illustrated the close relationship between anxiety disorders and attentional functioning, but the relationship between trait anxiety and attentional bias remains controversial. This study examines the effect of trait anxiety on the time course of attention to emotional stimuli using materials from the International Affective Picture System. Participants with high vs. low trait anxiety (HTA vs. LTA) viewed four categories of pictures simultaneously: dysphoric, threatening, positive, and neutral. Their eye-movements for each emotional stimulus were recorded for static and dynamic analysis. Data were analyzed using a mixed linear model and growth curve analysis. Specifically, the HTA group showed a greater tendency to avoid threatening stimuli and more pupil diameter variation in the early period of stimulus presentation (0–7.9 s). The HTA group also showed a stronger attentional bias toward positive and dysphoric stimuli in the middle and late period of stimulus presentation (7.9–30 s). These results suggest that trait anxiety has a significant temporal effect on attention to emotional stimuli, and that this effect mainly manifests after 7 s. In finding stronger attentional avoidance of threatening stimuli and more changes in neural activity, as well as a stronger attentional bias toward positive stimuli, this study provides novel insights on the relationship between trait anxiety and selective attention.

Pupillometry as an integrated readout of distinct attentional networks

The course of pupillary constriction and dilation provides an easy-to-access, inexpensive, and noninvasive readout of brain activity. We propose a new taxonomy of factors affecting the pupil and link these to associated neural underpinnings in an ascending hierarchy. In addition to two well-established low-level factors (light level and focal distance), we suggest two further intermediate-level factors, alerting and orienting, and a higher-level factor, executive functioning. Alerting, orienting, and executive functioning - including their respective underlying neural circuitries - overlap with the three principal attentional networks, making pupil size an integrated readout of distinct states of attention. As a now widespread technique, pupillometry is ready to provide meaningful applications and constitutes a viable part of the psychophysiological toolbox.

The relationship between pupillary baseline manipulated by mental effort or luminance and subsequent pupillary responses

Measuring pupillary response is a prevalent technique to evaluate mental states. It is indispensable to conduct a correction procedure for the pupillary baseline to get a meaningful conclusion from the pupillary response. However, the relationship between pupillary baseline and subsequent stimulus-evoked pupillary response varies among studies. In this study, we used the subtractive and proportional baseline corrections to analyze the results. Furthermore, we manipulated the pupillary baseline through mental effort or luminance in the baseline period and investigated whether the subsequent stimulus-evoked pupillary responses were affected. We found that the mental effort–evoked pupillary response was attenuated with a larger pupillary baseline manipulated by a higher mental effort, whereas it was unaffected with the baseline manipulated by luminance. Also, the luminance-evoked pupillary response was attenuated with a smaller pupillary baseline manipulated by a brighter disk, whereas it was unaffected with the baseline manipulated by mental effort. The results could be obtained from subtractive and proportional baseline corrections. Our results suggest that mental effort manipulated pupillary baseline interacts with the subsequent mental effort elicited pupillary response, but not with the luminance elicited pupillary response; the luminance manipulated pupillary baseline interacts with the subsequent luminance elicited pupillary response, but not with the mental effort elicited pupillary response. It is important to consider the ways of controlling the pupillary baseline and subsequent pupillary response simultaneously.

The Eye Pupil Adjusts to Illusorily Expanding Holes

Some static patterns evoke the perception of an illusory expanding central region or “hole.” We asked observers to rate the magnitudes of illusory motion or expansion of black holes, and these predicted the degree of dilation of the pupil, measured with an eye tracker. In contrast, when the “holes” were colored (including white), i.e., emitted light, these patterns constricted the pupils, but the subjective expansions were also weaker compared with the black holes. The change rates of pupil diameters were significantly related to the illusory motion phenomenology only with the black holes. These findings can be accounted for within a perceiving-the-present account of visual illusions, where both the illusory motion and the pupillary adjustments represent compensatory mechanisms to the perception of the next moment, based on shared experiences with the ecological regularities of light.

A unified model of the task-evoked pupil response

The pupil dilates and reconstricts following task events. It is popular to model this task-evoked pupil response as a linear transformation of event-locked impulses, whose amplitudes are used as estimates of arousal. We show that this model is incorrect and propose an alternative model based on the physiological finding that a common neural input drives saccades and pupil size. The estimates of arousal from our model agreed with key predictions: Arousal scaled with task difficulty and behavioral performance but was invariant to small differences in trial duration. Moreover, the model offers a unified explanation for a wide range of phenomena: entrainment of pupil size and saccades to task timing, modulation of pupil response amplitude and noise with task difficulty, reaction time-dependent modulation of pupil response timing and amplitude, a constrictory pupil response time-locked to saccades, and task-dependent distortion of this saccade-locked pupil response.

Publication guidelines and recommendations for pupillary measurement in psychophysiological studies

A variety of psychological and physical phenomena elicit variations in the diameter of pupil of the eye. Changes in pupil size are mediated by the relative activation of the sphincter pupillae muscle (decrease pupil diameter) and the dilator pupillae muscle (increase pupil diameter), innervated by the parasympathetic and sympathetic branches, respectively, of the autonomic nervous system. The current guidelines are intended to inform and guide psychophysiological research involving pupil measurement by (1) summarizing important aspects concerning the physiology of the pupil, (2) providing methodological and data-analytic guidelines and recommendations, and (3) briefly reviewing psychological phenomena that modulate pupillary reactivity. Because of the increased ease and tractability of pupil measurement, the goal of these guidelines is to promote accurate recording, analysis, and reporting of pupillary data in psychophysiological research.

Adults with cochlear implants can use prosody to determine the clausal structure of spoken sentences

Speech prosody, including pitch contour, word stress, pauses, and vowel lengthening, can aid the detection of the clausal structure of a multi-clause sentence and this, in turn, can help listeners determine the meaning. However, for cochlear implant (CI) users, the reduced acoustic richness of the signal raises the question of whether CI users may have difficulty using sentence prosody to detect syntactic clause boundaries within sentences or whether this ability is rescued by the redundancy of the prosodic features that normally co-occur at clause boundaries. Twenty-two CI users, ranging in age from 19 to 77 years old, recalled three types of sentences: sentences in which the prosodic pattern was appropriate to the location of a clause boundary within the sentence (congruent prosody), sentences with reduced prosodic information, or sentences in which the location of the clause boundary and the prosodic marking of a clause boundary were placed in conflict. The results showed the presence of congruent prosody to be associated with superior sentence recall and a reduced processing effort as indexed by the pupil dilation. The individual differences in a standard test of word recognition (consonant-nucleus-consonant score) were related to the recall accuracy as well as the processing effort. The outcomes are discussed in terms of the redundancy of the prosodic features, which normally accompany a clause boundary and processing effort.

Infants show pupil dilatory responses to happy and angry facial expressions

Facial expressions are one way in which infants and adults communicate emotion. Infants scan expressions similarly to adults, yet it remains unclear whether they are receptive to the affective information they convey. The current study investigates 6-, 9- and 12-month infants' (N = 146) pupillary responses to the six "basic" emotional expressions (happy, sad, surprise, fear, anger, and disgust). To do this we use dynamic stimuli and gaze-contingent eye-tracking to simulate brief interactive exchanges, alongside a static control condition. Infants' arousal responses were stronger for dynamic compared to static stimuli. And for dynamic stimuli we found that, compared to neutral, infants showed dilatory responses for happy and angry expressions only. Although previous work has shown infants can discriminate perceptually between facial expressions, our data suggest that sensitivity to the affective content of all six basic emotional expressions may not fully emerge until later in ontogeny.

Pupillometry reveals cognitive demands of lexical competition during spoken word recognition in young and older adults

In most contemporary activation-competition frameworks for spoken word recognition, candidate words compete against phonological "neighbors" with similar acoustic properties (e.g., "cap" vs. "cat"). Thus, recognizing words with more competitors should come at a greater cognitive cost relative to recognizing words with fewer competitors, due to increased demands for selecting the correct item and inhibiting incorrect candidates. Importantly, these processes should operate even in the absence of differences in accuracy. In the present study, we tested this proposal by examining differences in processing costs associated with neighborhood density for highly intelligible items presented in quiet. A second goal was to examine whether the cognitive demands associated with increased neighborhood density were greater for older adults compared with young adults. Using pupillometry as an index of cognitive processing load, we compared the cognitive demands associated with spoken word recognition for words with many or fewer neighbors, presented in quiet, for young (n = 67) and older (n = 69) adult listeners. Growth curve analysis of the pupil data indicated that older adults showed a greater evoked pupil response for spoken words than did young adults, consistent with increased cognitive load during spoken word recognition. Words from dense neighborhoods were marginally more demanding to process than words from sparse neighborhoods. There was also an interaction between age and neighborhood density, indicating larger effects of density in young adult listeners. These results highlight the importance of assessing both cognitive demands and accuracy when investigating the mechanisms underlying spoken word recognition.

Listening Effort Is Not the Same as Speech Intelligibility Score

Listening effort is a valuable and important notion to measure because it is among the primary complaints of people with hearing loss. It is tempting and intuitive to accept speech intelligibility scores as a proxy for listening effort, but this link is likely oversimplified and lacks actionable explanatory power. This study was conducted to explain the mechanisms of listening effort that are not captured by intelligibility scores, using sentence-repetition tasks where specific kinds of mistakes were prospectively planned or analyzed retrospectively. Effort measured as changes in pupil size among 20 listeners with normal hearing and 19 listeners with cochlear implants. Experiment 1 demonstrates that mental correction of misperceived words increases effort even when responses are correct. Experiment 2 shows that for incorrect responses, listening effort is not a function of the proportion of words correct but is rather driven by the types of errors, position of errors within a sentence, and the need to resolve ambiguity, reflecting how easily the listener can make sense of a perception. A simple taxonomy of error types is provided that is both intuitive and consistent with data from these two experiments. The diversity of errors in these experiments implies that speech perception tasks can be designed prospectively to elicit the mistakes that are more closely linked with effort. Although mental corrective action and number of mistakes can scale together in many experiments, it is possible to dissociate them to advance toward a more explanatory (rather than correlational) account of listening effort.

Electrical stimulation of the external ear acutely activates noradrenergic mechanisms in humans

BACKGROUND

Transcutaneous stimulation of the external ear is thought to recruit afferents of the auricular vagus nerve, providing a means to activate noradrenergic pathways in the central nervous system. Findings from human studies examining the effects of auricular stimulation on noradrenergic biomarkers have been mixed, possibly relating to the limited and variable parameter space explored to date.

OBJECTIVE

We tested the extent to which brief pulse trains applied to locations of auricular innervation (canal and concha) elicit acute pupillary responses (PRs) compared to a sham location (lobe). Pulse amplitude and frequency were varied systematically to examine effects on PR features.

METHODS

Participants (n = 19) underwent testing in three separate experiments, each with stimulation applied to a different external ear location. Perceptual threshold (PT) was measured at the beginning of each experiment. Pulse trains (∼600 ms) consisting of different amplitude (0.0xPT, 0.8xPT, 1.0xPT, 1.5xPT, 2.0xPT) and frequency (25 Hz, 300 Hz) combinations were administered during eye tracking procedures.

RESULTS

Stimulation to all locations elicited PRs which began approximately halfway through the pulse train and peaked shortly after the final pulse (≤1 s). PR size and incidence increased with pulse amplitude and tended to be greatest with canal stimulation. Higher pulse frequency shortened the latency of PR onset and peak dilation. Changes in pupil diameter elicited by pulse trains were weakly associated with baseline pupil diameter.

CONCLUSION

(s): Auricular stimulation elicits acute PRs, providing a basis to synchronize neuromodulator release with task-related neural spiking which preclinical studies show is a critical determinant of therapeutic effects. Further work is needed to dissociate contributions from vagal and non-vagal afferents mediating activation of the biomarker.

The Presence of Another Individual Influences Listening Effort, But Not Performance

OBJECTIVES

The aim of this study was to modify a speech perception in noise test to assess whether the presence of another individual (copresence), relative to being alone, affected listening performance and effort expenditure. Furthermore, this study assessed if the effect of the other individual's presence on listening effort was influenced by the difficulty of the task and whether participants had to repeat the sentences they listened to or not.

DESIGN

Thirty-four young, normal-hearing participants (mean age: 24.7 years) listened to spoken Dutch sentences that were masked with a stationary noise masker and presented through a loudspeaker. The participants alternated between repeating sentences (active condition) and not repeating sentences (passive condition). They did this either alone or together with another participant in the booth. When together, participants took turns repeating sentences. The speech-in-noise test was performed adaptively at three intelligibility levels (20%, 50%, and 80% sentences correct) in a block-wise fashion. During testing, pupil size was recorded as an objective outcome measure of listening effort.

RESULTS

Lower speech intelligibility levels were associated with increased peak pupil dilation (PPDs) and doing the task in the presence of another individual (compared with doing it alone) significantly increased PPD. No interaction effect between intelligibility and copresence on PPD was found. The results suggested that the change of PPD between doing the task alone or together was especially apparent for people who started the experiment in the presence of another individual. Furthermore, PPD was significantly lower during passive listening, compared with active listening. Finally, it seemed that performance was unaffected by copresence.

CONCLUSION

The increased PPDs during listening in the presence of another participant suggest that more effort was invested during the task. However, it seems that the additional effort did not result in a change of performance. This study showed that at least one aspect of the social context of a listening situation (in this case copresence) can affect listening effort, indicating that social context might be important to consider in future cognitive hearing research.

Replicating five pupillometry studies of Eckhard Hess

Several papers by Eckhard Hess from the 1960s and 1970s report that the pupils dilate or constrict according to the interest value, arousing content, or mental demands of visual stimuli. However, Hess mostly used small sample sizes and undocumented luminance control. In a first experiment (N = 182) and a second preregistered experiment (N = 147), we replicated five studies of Hess using modern equipment. Our experiments (1) did not support the hypothesis of gender differences in pupil diameter change with respect to baseline (PC) when viewing stimuli of different interest value, (2) showed that solving more difficult multiplications yields a larger PC in the seconds before providing an answer and a larger maximum PC, but a smaller PC at a fixed time after the onset of the multiplication, (3) did not support the hypothesis that participants' PC mimics the pupil diameter in a pair of schematic eyes but not in single-eyed or three-eyed stimuli, (4) did not support the hypothesis of gender differences in PC when watching a video of a male trying to escape a mob, and (5) supported the hypothesis that arousing words yield a higher PC than non-arousing words. Although we did not observe consistent gender differences in PC, additional analyses showed gender differences in eye movements towards erogenous zones. Furthermore, PC strongly correlated with the luminance of the locations where participants looked. Overall, our replications confirm Hess's findings that pupils dilate in response to mental demands and stimuli of an arousing nature. Hess's hypotheses regarding pupil mimicry and gender differences in pupil dilation did not replicate.

Is baseline pupil size related to cognitive ability? Yes (under proper lighting conditions)

There has been some controversy as to whether baseline pupil size is related to individual differences in cognitive ability. Previously, we had shown that a larger baseline pupil size was associated with higher cognitive ability and that the correlation to fluid intelligence was larger than that to working memory capacity (Tsukahara, Harrison, & Engle, 2016). However, other researchers have not been able to replicate our findings - though they only measured working memory capacity and not fluid intelligence. Many of the studies showing no relationship had major methodological issues, namely small baseline pupil size values - down to the physiological minimum - that resulted in reduced variability on baseline pupil size. We conducted two large-scale studies to investigate how different lighting conditions affect baseline pupil size values and the correlation with cognitive abilities. We found that fluid intelligence, working memory capacity, and attention control did correlate with baseline pupil size except in the brightest lighting conditions. We showed that a reduced variability in baseline pupil size values is due to the monitor settings being too bright. Overall, our findings demonstrated that the baseline pupil size - working memory capacity relationship was not as strong or robust as that with fluid intelligence or attention control. Our findings have strong methodological implications for researchers investigating individual differences in task-free or task-evoked pupil size. We conclude that fluid intelligence does correlate with baseline pupil size and that this is related to the functional organization of the resting-state brain through the locus coeruleus-norepinephrine system.

Effects of Noise and Serial Position on Free Recall of Spoken Words and Pupil Dilation during Encoding in Normal-Hearing Adults

This preliminary study assessed the effects of noise and stimulus presentation order on recall of spoken words and recorded pupil sizes while normal-hearing listeners were trying to encode a series of words for a subsequent recall task. In three listening conditions (stationary noise in Experiment 1; quiet versus four-talker babble in Experiment 2), participants were assigned to remember as many words as possible to recall them in any order after each list of seven sentences. In the two noise conditions, lists of sentences fixed at 65 dB SPL were presented at an easily audible level via a loudspeaker. Reading span (RS) scores were used as a grouping variable, based on a median split. The primacy effect was present apart from the noise interference, and the high-RS group significantly outperformed the low-RS group at free recall measured in the quiet and four-talker babble noise conditions. RS scores were positively correlated with free-recall scores. In both quiet and four-talker babble noise conditions, sentence baselines after correction to the initial stimulus baseline increased significantly with increasing memory load. Larger sentence baselines but smaller peak pupil dilations seemed to be associated with noise interruption. The analysis method of pupil dilation used in this study is likely to provide a more thorough understanding of how listeners respond to a later recall task in comparison with previously used methods. Further studies are needed to confirm the applicability of our method in people with impaired hearing using multiple repetitions to estimate the allocation of relevant cognitive resources.

Is accommodation a confounder in pupillometry research?

Much psychological research uses pupil diameter measurements to investigate the cognitive and emotional effects of visual stimuli. A potential problem is that accommodating at a nearby point causes the pupil to constrict. This study examined to what extent accommodation is a confounder in pupillometry research. Participants solved multiplication problems at different distances (Experiment 1) and looked at line drawings with different monocular depth cues (Experiment 2) while their pupil diameter, refraction, and vergence angle were recorded using a photorefractor. Experiment 1 showed that the pupils dilated while performing the multiplications, for all presentation distances. Pupillary constriction due to accommodation was not strong enough to override pupil dilation due to cognitive load. Experiment 2 showed that monocular depth cues caused a small shift in refraction in the expected direction. We conclude that, for the young student sample we used, pupil diameter measurements are not substantially affected by accommodation.

Pupillary Responses for Cognitive Load Measurement to Classify Difficulty Levels in an Educational Video Game: Empirical Study

Background

A learning task recurrently perceived as easy (or hard) may cause poor learning results. Gamer data such as errors, attempts, or time to finish a challenge are widely used to estimate the perceived difficulty level. In other contexts, pupillometry is widely used to measure cognitive load (mental effort); hence, this may describe the perceived task difficulty.

Objective

This study aims to assess the use of task-evoked pupillary responses to measure the cognitive load measure for describing the difficulty levels in a video game. In addition, it proposes an image filter to better estimate baseline pupil size and to reduce the screen luminescence effect.

Methods

We conducted an experiment that compares the baseline estimated from our filter against that estimated from common approaches. Then, a classifier with different pupil features was used to classify the difficulty of a data set containing information from students playing a video game for practicing math fractions.

Results

We observed that the proposed filter better estimates a baseline. Mauchly’s test of sphericity indicated that the assumption of sphericity had been violated (χ²₁₄=0.05; P=.001); therefore, a Greenhouse-Geisser correction was used (ε=0.47). There was a significant difference in mean pupil diameter change (MPDC) estimated from different baseline images with the scramble filter (F_5,78=30.965; P<.001). Moreover, according to the Wilcoxon signed rank test, pupillary response features that better describe the difficulty level were MPDC (z=−2.15; P=.03) and peak dilation (z=−3.58; P<.001). A random forest classifier for easy and hard levels of difficulty showed an accuracy of 75% when the gamer data were used, but the accuracy increased to 87.5% when pupillary measurements were included.

Conclusions

The screen luminescence effect on pupil size is reduced with a scrambled filter on the background video game image. Finally, pupillary response data can improve classifier accuracy for the perceived difficulty of levels in educational video games.

GazeR: A Package for Processing Gaze Position and Pupil Size Data

Eye-tracking is widely used throughout the scientific community, from vision science and psycholinguistics to marketing and human-computer interaction. Surprisingly, there is little consistency and transparency in preprocessing steps, making replicability and reproducibility difficult. To increase replicability, reproducibility, and transparency, a package in R (a free and widely used statistical programming environment) called gazeR was created to read and preprocess two types of data: gaze position and pupil size. For gaze position data, gazeR has functions for reading in raw eye-tracking data, formatting it for analysis, converting from gaze coordinates to areas of interest, and binning and aggregating data. For data from pupillometry studies, the gazeR package has functions for reading in and merging multiple raw pupil data files, removing observations with too much missing data, eliminating artifacts, blink identification and interpolation, subtractive baseline correction, and binning and aggregating data. The package is open-source and freely available for download and installation: https://github.com/dmirman/gazer . We provide step-by-step analyses of data from two tasks exemplifying the package's capabilities.

Modeling pupil responses to rapid sequential events

Pupil size is an easily accessible, noninvasive online indicator of various perceptual and cognitive processes. Pupil measurements have the potential to reveal continuous processing dynamics throughout an experimental trial, including anticipatory responses. However, the relatively sluggish (~2 s) response dynamics of pupil dilation make it challenging to connect changes in pupil size to events occurring close together in time. Researchers have used models to link changes in pupil size to specific trial events, but such methods have not been systematically evaluated. Here we developed and evaluated a general linear model (GLM) pipeline that estimates pupillary responses to multiple rapid events within an experimental trial. We evaluated the modeling approach using a sample dataset in which multiple sequential stimuli were presented within 2-s trials. We found: (1) Model fits improved when the pupil impulse response function (PuRF) was fit for each observer. PuRFs varied substantially across individuals but were consistent for each individual. (2) Model fits also improved when pupil responses were not assumed to occur simultaneously with their associated trial events, but could have non-zero latencies. For example, pupil responses could anticipate predictable trial events. (3) Parameter recovery confirmed the validity of the fitting procedures, and we quantified the reliability of the parameter estimates for our sample dataset. (4) A cognitive task manipulation modulated pupil response amplitude. We provide our pupil analysis pipeline as open-source software (Pupil Response Estimation Toolbox: PRET) to facilitate the estimation of pupil responses and the evaluation of the estimates in other datasets.

Effect of Speech-to-Noise Ratio and Luminance on a Range of Current and Potential Pupil Response Measures to Assess Listening Effort

In hearing research, pupillometry is an established method of studying listening effort. The focus of this study was to evaluate several pupil measures extracted from the Task-Evoked Pupil Responses (TEPRs) in speech-in-noise test. A range of analysis approaches was applied to extract these pupil measures, namely (a) pupil peak dilation (PPD); (b) mean pupil dilation (MPD); (c) index of pupillary activity; (d) growth curve analysis (GCA); and (e) principal component analysis (PCA). The effect of signal-to-noise ratio (SNR; Data Set A: -20 dB, -10 dB, +5 dB SNR) and luminance (Data Set B: 0.1 cd/m2, 360 cd/m2) on the TEPRs were investigated. Data Sets A and B were recorded during a speech-in-noise test and included TEPRs from 33 and 27 normal-hearing native Dutch speakers, respectively. The main results were as follows: (a) A significant effect of SNR was revealed for all pupil measures extracted in the time domain (PPD, MPD, GCA, PCA); (b) Two time series analysis approaches (GCA, PCA) provided modeled temporal profiles of TEPRs (GCA); and time windows spanning subtasks performed in a speech-in-noise test (PCA); and (c) All pupil measures revealed a significant effect of luminance. In conclusion, multiple pupil measures showed similar effects of SNR, suggesting that effort may be reflected in multiple aspects of TEPR. Moreover, a direct analysis of the pupil time course seems to provide a more holistic view of TEPRs, yet further research is needed to understand and interpret its measures. Further research is also required to find pupil measures less sensitive to changes in luminance.

Individual differences in baseline oculometrics: Examining variation in baseline pupil diameter, spontaneous eye blink rate, and fixation stability

Individual differences in baseline oculometrics (baseline pupil diameter, spontaneous eye blink rate, fixation stability), and their relation with cognitive abilities, personality traits, and self-report assessments were examined. Participants performed a baseline eye measure in which they were instructed to stare at a fixation point onscreen for 5 min. Following the baseline eye measure, participants completed a questionnaire asking what they were thinking about during the baseline eye measure. Participants also completed various cognitive ability measures assessing working memory capacity, attention control, and off-task thinking. Finally, participants completed a number of questionnaires assessing personality, Attention Deficit/Hyperactivity Disorder symptomology, mind wandering, and morningness-eveningness. Overall, the vast majority of correlations with the baseline eye measures were weak and nonsignificant, suggesting that these associations may not be very robust. The results also demonstrated the importance of examining what participants are thinking about during the baseline measure. These results add to the growing body of findings suggesting inconsistent relations between different baseline eye measures and various individual differences constructs.

Best practices in eye tracking research

This guide describes best practices in using eye tracking technology for research in a variety of disciplines. A basic outline of the anatomy and physiology of the eyes and of eye movements is provided, along with a description of the sorts of research questions eye tracking can address. We then explain how eye tracking technology works and what sorts of data it generates, and provide guidance on how to select and use an eye tracker as well as selecting appropriate eye tracking measures. Challenges to the validity of eye tracking studies are described, along with recommendations for overcoming these challenges. We then outline correct reporting standards for eye tracking studies.

Rapid adaptation to fully intelligible nonnative-accented speech reduces listening effort

In noisy settings or when listening to an unfamiliar talker or accent, it can be difficult to understand spoken language. This difficulty typically results in reductions in speech intelligibility, but may also increase the effort necessary to process the speech even when intelligibility is unaffected. In this study, we used a dual-task paradigm and pupillometry to assess the cognitive costs associated with processing fully intelligible accented speech, predicting that rapid perceptual adaptation to an accent would result in decreased listening effort over time. The behavioural and physiological paradigms provided converging evidence that listeners expend greater effort when processing nonnative- relative to native-accented speech, and both experiments also revealed an overall reduction in listening effort over the course of the experiment. Only the pupillometry experiment, however, revealed greater adaptation to nonnative- relative to native-accented speech. An exploratory analysis of the dual-task data that attempted to minimise practice effects revealed weak evidence for greater adaptation to the nonnative accent. These results suggest that even when speech is fully intelligible, resolving deviations between the acoustic input and stored lexical representations incurs a processing cost, and adaptation may attenuate this cost.

Anticipatory Baseline Pupil Diameter Is Sensitive to Differences in Hearing Thresholds

Task-evoked changes in pupil dilation have long been used as a physiological index of cognitive effort. Unlike this response, that is measured during or after an experimental trial, the baseline pupil dilation (BPD) is a measure taken prior to an experimental trial. As such, it is considered to reflect an individual’s arousal level in anticipation of an experimental trial. We report data for 68 participants, ages 18 to 89, whose hearing acuity ranged from normal hearing to a moderate hearing loss, tested over a series 160 trials on an auditory sentence comprehension task. Results showed that BPDs progressively declined over the course of the experimental trials, with participants with poorer pure tone detection thresholds showing a steeper rate of decline than those with better thresholds. Data showed this slope difference to be due to participants with poorer hearing having larger BPDs than those with better hearing at the start of the experiment, but with their BPDs approaching that of the better hearing participants by the end of the 160 trials. A finding of increasing response accuracy over trials was seen as inconsistent with a fatigue or reduced task engagement account of the diminishing BPDs. Rather, the present results imply BPD as reflecting a heightened arousal level in poorer-hearing participants in anticipation of a task that demands accurate speech perception, a concern that dissipates over trials with task success. These data taken with others suggest that the baseline pupillary response may not reflect a single construct.

Does goal congruence dilate the pupil over and above goal relevance?

To better understand the mechanisms underlying preferential processing of affective stimuli (i.e., affective attention), it is useful to ask which appraisal dimensions characterize stimuli that capture affective attention. Viewed from such appraisal perspective, most accounts of affective attention predict that the appraisal of goal relevance (i.e. the overall relevance of a stimulus/event to one's current goals) should attract attention whereas some accounts also implicate the appraisal of goal congruence (i.e. the value of the stimulus with respect to these goals). We compared these accounts by investigating independent effects of goal relevance and goal congruence on attention capture operationalized through pupil dilation. Forty-seven participants completed a modified Monetary Incentive Delay task (MID) where they sought to maximize the amount of chocolate received at the end of the experiment, pupil data from forty-two participants were used in the final analysis. Implicating attention capture by goal relevance, we found that pupil size was increased by all win- and loss-related compared to neutral stimuli and by materialized wins and losses compared to non-materialized ones. By contrast, we did not find pupil dilation effects of goal congruence across three different operationalizations of this appraisal dimension. These findings suggest that goal congruence appraisal may not contribute to the aspects of affective attention reflected in pupil dilation over and above goal relevance appraisal.

Cortical modulation of pupillary function: systematic review

BACKGROUND

The pupillary light reflex is the main mechanism that regulates the pupillary diameter; it is controlled by the autonomic system and mediated by subcortical pathways. In addition, cognitive and emotional processes influence pupillary function due to input from cortical innervation, but the exact circuits remain poorly understood. We performed a systematic review to evaluate the mechanisms behind pupillary changes associated with cognitive efforts and processing of emotions and to investigate the cerebral areas involved in cortical modulation of the pupillary light reflex.

METHODOLOGY

We searched multiple databases until November 2018 for studies on cortical modulation of pupillary function in humans and non-human primates. Of 8,809 papers screened, 258 studies were included.

RESULTS

Most investigators focused on pupillary dilatation and/or constriction as an index of cognitive and emotional processing, evaluating how changes in pupillary diameter reflect levels of attention and arousal. Only few tried to correlate specific cerebral areas to pupillary changes, using either cortical activation models (employing micro-stimulation of cortical structures in non-human primates) or cortical lesion models (e.g., investigating patients with stroke and damage to salient cortical and/or subcortical areas). Results suggest the involvement of several cortical regions, including the insular cortex (Brodmann areas 13 and 16), the frontal eye field (Brodmann area 8) and the prefrontal cortex (Brodmann areas 11 and 25), and of subcortical structures such as the locus coeruleus and the superior colliculus.

CONCLUSIONS

Pupillary dilatation occurs with many kinds of mental or emotional processes, following sympathetic activation or parasympathetic inhibition. Conversely, pupillary constriction may occur with anticipation of a bright stimulus (even in its absence) and relies on a parasympathetic activation. All these reactions are controlled by subcortical and cortical structures that are directly or indirectly connected to the brainstem pupillary innervation system.