Methods in cognitive pupillometry: Design, preprocessing, and statistical analysis

Neurobehavioral meaning of pupil size

Pupil size is a widely used metric of brain state. It is one of the few signals originating from the brain that can be readily monitored with low-cost devices in basic science, clinical, and home settings. It is, therefore, important to investigate and generate well-defined theories related to specific interpretations of this metric. What exactly does it tell us about the brain? Pupils constrict in response to light and dilate during darkness, but the brain also controls pupil size irrespective of luminosity. Pupil size fluctuations resulting from ongoing "brain states" are used as a metric of arousal, but what is pupil-linked arousal and how should it be interpreted in neural, cognitive, and computational terms? Here, we discuss some recent findings related to these issues. We identify open questions and propose how to answer them through a combination of well-defined tasks, neurocomputational models, and neurophysiological probing of the interconnected loops of causes and consequences of pupil size.

Auditory Challenges and Listening Effort in School-Age Children With Autism: Insights From Pupillary Dynamics During Speech-in-Noise Perception

PURPOSE

This study aimed to investigate challenges in speech-in-noise (SiN) processing faced by school-age children with autism spectrum conditions (ASCs) and their impact on listening effort.

METHOD

Participants, including 23 Mandarin-speaking children with ASCs and 19 age-matched neurotypical (NT) peers, underwent sentence recognition tests in both quiet and noisy conditions, with a speech-shaped steady-state noise masker presented at 0-dB signal-to-noise ratio in the noisy condition. Recognition accuracy rates and task-evoked pupil responses were compared to assess behavioral performance and listening effort during auditory tasks.

RESULTS

No main effect of group was found on accuracy rates. Instead, significant effects emerged for autistic trait scores, listening conditions, and their interaction, indicating that higher trait scores were associated with poorer performance in noise. Pupillometric data revealed significantly larger and earlier peak dilations, along with more varied pupillary dynamics in the ASC group relative to the NT group, especially under noisy conditions. Importantly, the ASC group's peak dilation in quiet mirrored that of the NT group in noise. However, the ASC group consistently exhibited reduced mean dilations than the NT group.

CONCLUSIONS

Pupillary responses suggest a different resource allocation pattern in ASCs: An initial sharper and larger dilation may signal an intense, narrowed resource allocation, likely linked to heightened arousal, engagement, and cognitive load, whereas a subsequent faster tail-off may indicate a greater decrease in resource availability and engagement, or a quicker release of arousal and cognitive load. The presence of noise further accentuates this pattern. This highlights the unique SiN processing challenges children with ASCs may face, underscoring the importance of a nuanced, individual-centric approach for interventions and support.

The impact of presentation modes on mental rotation processing: a comparative analysis of eye movements and performance

Mental rotation is the ability to rotate mental representations of objects in space. Shepard and Metzler's shape-matching tasks, frequently used to test mental rotation, involve presenting pictorial representations of 3D objects. This stimulus material has raised questions regarding the ecological validity of the test for mental rotation with actual visual 3D objects. To systematically investigate differences in mental rotation with pictorial and visual stimuli, we compared data of N = 54 university students from a virtual reality experiment. Comparing both conditions within subjects, we found higher accuracy and faster reaction times for 3D visual figures. We expected eye tracking to reveal differences in participants' stimulus processing and mental rotation strategies induced by the visual differences. We statistically compared fixations (locations), saccades (directions), pupil changes, and head movements. Supplementary Shapley values of a Gradient Boosting Decision Tree algorithm were analyzed, which correctly classified the two conditions using eye and head movements. The results indicated that with visual 3D figures, the encoding of spatial information was less demanding, and participants may have used egocentric transformations and perspective changes. Moreover, participants showed eye movements associated with more holistic processing for visual 3D figures and more piecemeal processing for pictorial 2D figures.

No pupillometric evidence for effortful proactive control in the proportion-congruent Stroop paradigm

Cognitive control is the ability to allocate attention away from stimuli that are irrelevant to achieving a goal, towards stimuli that are. When conflict is anticipated, attention is biased in a global, top-down manner called proactive control and this effortful type of cognitive control is engaged before stimulus onset. The list-wise congruency proportion (LWPC) effect, where the Stroop congruency effect is reduced when there are more incongruent than congruent trials compared to vice versa, has been viewed as one of the prime signatures of this type of cognitive control. However, there has been recent debate about the extent to which this effect should be attributed to proactive control instead of alternative explanations such as simpler associative learning or reactive control. Thus, by using pupillometry (i.e., an indicator of cognitive effort), the present study investigated the extent to which LWPC effects result from effortful proactive control. Experiment 1 employed a classic proportion congruency manipulation, while Experiment 2 replaced congruent trials with neutral trials to control for potential effects of associative learning. While in line with past findings, proportion congruency effects were obtained in response times of both experiments and pupillometry showed both proportion congruency and Stroop effects after stimulus onset, no differences in pupil sizes were found during the preparatory phase. Therefore, these results do not support the idea that the observed LWPC effects are due to participants engaging in effortful proactive control.

Applicability and usefulness of pupillometry in the study of lexical access. A scoping review of primary research

Pupil dilation has been associated with the effort required to perform various cognitive tasks. At the lexical level, some studies suggest that this neurophysiological measure would provide objective, real-time information during word processing and lexical access. However, due to the scarcity and incipient advancement of this line of research, its applicability, use, and sensitivity are not entirely clear. This scoping review aims to determine the applicability and usefulness of pupillometry in the study of lexical access by providing an up-to-date overview of research in this area. Following the PRISMA protocol, 16 articles were included in this review. The results show that pupillometry is a highly applicable, useful, and sensitive method for assessing lexical skills of word recognition, word retrieval, and semantic activation. Moreover, it easily fits into traditional research paradigms and methods in the field. Because it is a non-invasive, objective, and automated procedure, it can be applied to any population or age group. However, the emerging development of this specific area of research and the methodological diversity observed in the included studies do not yet allow for definitive conclusions in this area, which in turn does not allow for meta-analyses or fully conclusive statements about what the pupil response actually reflects when processing words. Standardized pupillary recording and analysis methods need to be defined to generate more accurate, replicable research designs with more reliable results to strengthen this line of research.

Revealing visual working memory operations with pupillometry: Encoding, maintenance, and prioritization

Pupillary dynamics reflect effects of distinct and important operations of visual working memory: encoding, maintenance, and prioritization. Here, we review how pupil size predicts memory performance and how it provides novel insights into the mechanisms of each operation. Visual information must first be encoded into working memory with sufficient precision. The depth of this encoding process couples to arousal-linked baseline pupil size as well as a pupil constriction response before and after stimulus onset, respectively. Subsequently, the encoded information is maintained over time to ensure it is not lost. Pupil dilation reflects the effortful maintenance of information, wherein storing more items is accompanied by larger dilations. Lastly, the most task-relevant information is prioritized to guide upcoming behavior, which is reflected in yet another dilatory component. Moreover, activated content in memory can be pupillometrically probed directly by tagging visual information with distinct luminance levels. Through this luminance-tagging mechanism, pupil light responses reveal whether dark or bright items receive more attention during encoding and prioritization. Together, conceptualizing pupil responses as a sum of distinct components over time reveals insights into operations of visual working memory. From this viewpoint, pupillometry is a promising avenue to study the most vital operations through which visual working memory works. This article is categorized under: Psychology > Attention Psychology > Memory Psychology > Theory and Methods.

Non-image-forming vision as measured through ipRGC-mediated pupil constriction is not modulated by covert visual attention

In brightness, the pupil constricts, while in darkness, the pupil dilates; this is known as the pupillary light response (PLR). The PLR is driven by all photoreceptors: rods and cones, which contribute to image-forming vision, and intrinsically photosensitive retinal ganglion cells (ipRGCs), which mainly contribute to non-image-forming vision. Rods and cones cause immediate pupil constriction upon light exposure, whereas ipRGCs cause sustained constriction throughout light exposure. Recent studies have shown that covert attention modulated the initial PLR; however, it remains unclear whether the same holds for the sustained PLR. We tested this by leveraging ipRGCs' responsiveness to blue light, causing the most prominent sustained constriction. While replicating previous studies by showing that pupils constricted more when either directly looking at, or covertly attending to, bright as compared to dim stimuli (with the same color), we also found that the pupil constricted more when directly looking at blue as compared to red stimuli (with the same luminosity). Crucially, however, in two high-powered studies (n = 60), we did not find any pupil-size difference when covertly attending to blue as compared to red stimuli. This suggests that ipRGC-mediated pupil constriction, and possibly non-image-forming vision more generally, is not modulated by covert attention.

Changes in pupil size track self-control failure

People are more likely to perform poorly on a self-control task following a previous task requiring self-control (ego-depletion), but the mechanism for this effect remains unclear. We used pupillometry to test the role of attentional effort in ego-depletion. We hypothesized that an elevated pupil diameter (PD)-a common physiological measure of effort-during an initial task requiring self-control should be negatively associated with performance on a subsequent control task. To test this hypothesis, participants were first assigned to either a high- or low-demand attention task (manipulation; a standard ego-depletion paradigm), after which all participants completed the same Stroop task. We then separately extracted both sustained (low-frequency) and phasic (high-frequency) changes in PD from both tasks to evaluate possible associations with lapses of cognitive control on the Stroop task. We first show that in the initial task, sustained PD was larger among participants who were assigned to the demanding attention condition. Furthermore, ego-depletion effects were serially mediated by PD: an elevated PD response emerged rapidly among the experimental group during the manipulation, persisted as an elevated baseline response during the Stroop task, and predicted worse accuracy on incongruent trials, revealing a potential indirect pathway to ego-depletion via sustained attention. Secondary analyses revealed another, independent and direct pathway via high levels of transient attentional control: participants who exhibited large phasic responses during the manipulation tended to perform worse on the subsequent Stroop task. We conclude by exploring the neuroscientific implications of these results within the context of current theories of self-control.

Emotional response in babies' pupil contagion

In this study, it was investigated whether an emotional response would occur in pupil contagion by using skin conductance response (SCR) in 5- and 6-month-old infants. In the experiment, emotional responses to pupil diameter change (dilating/constricting) between the face and eyes regions were compared by using pupil diameter response and SCR. The results showed that pupil diameter responses to pupil diameter changes did not differ between face and eyes regions. The emotional response indicated by the SCR significantly increased when participants looked at dilating pupils of face stimuli compared with when participants looked at constricted pupils of face stimuli. In addition, we found a significant correlation between SCR and pupil dilation in the face. This means that pupil diameter expansion significantly increases emotional response in pupil dilation of the face region.

Changes in attentional breadth scale with the demands of Kanizsa-figure object completion-evidence from pupillometry

The present study investigated whether the integration of separate parts into a whole-object representation varies with the amount of available attentional resources. To this end, two experiments were performed, which required observers to maintain central fixation while searching in peripheral vision for a target among various distractor configurations. The target could either be a "grouped" whole-object Kanizsa figure, or an "ungrouped" configuration of identical figural parts, but which do not support object completion processes to the same extent. In the experiments, accuracies and changes in pupil size were assessed, with the latter reflecting a marker of the covert allocation of attention in the periphery. Experiment 1 revealed a performance benefit for grouped (relative to ungrouped) targets, which increased with decreasing distance from fixation. By contrast, search for ungrouped targets was comparably poor in accuracy without revealing any eccentricity-dependent variation. Moreover, measures of pupillary dilation mirrored this eccentricity-dependent advantage in localizing grouped targets. Next, in Experiment 2, an additional attention-demanding foveal task was introduced in order to further reduce the availability of attentional resources for the peripheral detection task. This additional task hampered performance overall, alongside with corresponding pupil size changes. However, there was still a substantial benefit for grouped over ungrouped targets in both the behavioral and the pupillometric data. This shows that perceptual grouping scales with the allocation of attention even when only residual attentional resources are available to trigger the representation of a complete (target) object, thus illustrating that object completion operates in the "near absence" of attention.

Manipulation of phasic arousal by auditory cues is associated with subsequent changes in visual orienting to faces in infancy

This eye-tracking study investigated the effect of sound-induced arousal on social orienting under different auditory cue conditions in 5-month-old (n = 25; n = 13 males) and 10-month-old infants (n = 21; n = 14 males) participating in a spontaneous visual search task. Results showed: (1) larger pupil dilation discriminating between high and low volume (b = 0.02, p = 0.007), but not between social and non-social sounds (b = 0.004, p = 0.64); (2) faster visual orienting (b =  - 0.09, p < 0.001) and better social orienting at older age (b = 0.94, p < 0.001); (3) a fast habituation effect on social orienting after high-volume sounds (χ2(2) = 7.39, p = 0.025); (4) a quadratic association between baseline pupil size and target selection (b =  - 1.0, SE = 0.5, χ2(1) = 4.04, p = 0.045); (5) a positive linear association between pupil dilation and social orienting (b = 0.09, p = 0.039). Findings support adaptive gain theories of arousal, extending the link between phasic pupil dilation and task performance to spontaneous social orienting in infancy.

Effects of pupil size as manipulated through ipRGC activation on visual processing

The size of the eyes' pupils determines how much light enters the eye and also how well this light is focused. Through this route, pupil size shapes the earliest stages of visual processing. Yet causal effects of pupil size on vision are poorly understood and rarely studied. Here we introduce a new way to manipulate pupil size, which relies on activation of intrinsically photosensitive retinal ganglion cells (ipRGCs) to induce sustained pupil constriction. We report the effects of both experimentally induced and spontaneous changes in pupil size on visual processing as measured through EEG. We compare these to the effects of stimulus intensity and covert visual attention, because previous studies have shown that these factors all have comparable effects on some common measures of early visual processing, such as detection performance and steady-state visual evoked potentials; yet it is still unclear whether these are superficial similarities, or rather whether they reflect similar underlying processes. Using a mix of neural-network decoding, ERP analyses, and time-frequency analyses, we find that induced pupil size, spontaneous pupil size, stimulus intensity, and covert visual attention all affect EEG responses, mainly over occipital and parietal electrodes, but-crucially-that they do so in qualitatively different ways. Induced and spontaneous pupil-size changes mainly modulate activity patterns (but not overall power or intertrial coherence) in the high-frequency beta range; this may reflect an effect of pupil size on oculomotor activity and/ or visual processing. In addition, spontaneous (but not induced) pupil size tends to correlate positively with intertrial coherence in the alpha band; this may reflect a non-causal relationship, mediated by arousal. Taken together, our findings suggest that pupil size has qualitatively different effects on visual processing from stimulus intensity and covert visual attention. This shows that pupil size as manipulated through ipRGC activation strongly affects visual processing, and provides concrete starting points for further study of this important yet understudied earliest stage of visual processing.

Noradrenaline tracks emotional modulation of attention in human amygdala

The noradrenaline (NA) system is one of the brain's major neuromodulatory systems; it originates in a small midbrain nucleus, the locus coeruleus (LC), and projects widely throughout the brain.1,2 The LC-NA system is believed to regulate arousal and attention3,4 and is a pharmacological target in multiple clinical conditions.5,6,7 Yet our understanding of its role in health and disease has been impeded by a lack of direct recordings in humans. Here, we address this problem by showing that electrochemical estimates of sub-second NA dynamics can be obtained using clinical depth electrodes implanted for epilepsy monitoring. We made these recordings in the amygdala, an evolutionarily ancient structure that supports emotional processing8,9 and receives dense LC-NA projections,10 while patients (n = 3) performed a visual affective oddball task. The task was designed to induce different cognitive states, with the oddball stimuli involving emotionally evocative images,11 which varied in terms of arousal (low versus high) and valence (negative versus positive). Consistent with theory, the NA estimates tracked the emotional modulation of attention, with a stronger oddball response in a high-arousal state. Parallel estimates of pupil dilation, a common behavioral proxy for LC-NA activity,12 supported a hypothesis that pupil-NA coupling changes with cognitive state,13,14 with the pupil and NA estimates being positively correlated for oddball stimuli in a high-arousal but not a low-arousal state. Our study provides proof of concept that neuromodulator monitoring is now possible using depth electrodes in standard clinical use.

The pupil collaboration: A multi-lab, multi-method analysis of goal attribution in infants

The rise of pupillometry in infant research over the last decade is associated with a variety of methods for data preprocessing and analysis. Although pupil diameter is increasingly recognized as an alternative measure of the popular cumulative looking time approach used in many studies (Jackson & Sirois, 2022), an open question is whether the many approaches used to analyse this variable converge. To this end, we proposed a crowdsourced approach to pupillometry analysis. A dataset from 30 9-month-old infants (15 girls; Mage = 282.9 days, SD = 8.10) was provided to 7 distinct teams for analysis. The data were obtained from infants watching video sequences showing a hand, initially resting between two toys, grabbing one of them (after Woodward, 1998). After habituation, infants were shown (in random order) a sequence of four test events that varied target position and target toy. Results show that looking times reflect primarily the familiar path of the hand, regardless of target toy. Gaze data similarly show this familiarity effect of path. The pupil dilation analyses show that features of pupil baseline measures (duration and temporal location) as well as data retention variation (trial and/or participant) due to different inclusion criteria from the various analysis methods are linked to divergences in findings. Two of the seven teams found no significant findings, whereas the remaining five teams differ in the pattern of findings for main and interaction effects. The discussion proposes guidelines for best practice in the analysis of pupillometry data.

Phonological discrimination and contrast detection in pupillometry

Introduction

The perception of phonemes is guided by both low-level acoustic cues and high-level linguistic context. However, differentiating between these two types of processing can be challenging. In this study, we explore the utility of pupillometry as a tool to investigate both low- and high-level processing of phonological stimuli, with a particular focus on its ability to capture novelty detection and cognitive processing during speech perception.

Methods

Pupillometric traces were recorded from a sample of 22 Danish-speaking adults, with self-reported normal hearing, while performing two phonological-contrast perception tasks: a nonword discrimination task, which included minimal-pair combinations specific to the Danish language, and a nonword detection task involving the detection of phonologically modified words within sentences. The study explored the perception of contrasts in both unprocessed speech and degraded speech input, processed with a vocoder.

Results

No difference in peak pupil dilation was observed when the contrast occurred between two isolated nonwords in the nonword discrimination task. For unprocessed speech, higher peak pupil dilations were measured when phonologically modified words were detected within a sentence compared to sentences without the nonwords. For vocoded speech, higher peak pupil dilation was observed for sentence stimuli, but not for the isolated nonwords, although performance decreased similarly for both tasks.

Conclusion

Our findings demonstrate the complexity of pupil dynamics in the presence of acoustic and phonological manipulation. Pupil responses seemed to reflect higher-level cognitive and lexical processing related to phonological perception rather than low-level perception of acoustic cues. However, the incorporation of multiple talkers in the stimuli, coupled with the relatively low task complexity, may have affected the pupil dilation.

The next frontier: Moving human fear conditioning research online

Fear conditioning is a significant area of research that has featured prominently among the topics published in Biological Psychology over the last 50 years. This work has greatly contributed to our understanding of human anxiety and stressor-related disorders. While mainly conducted in the laboratory, recently, there have been initial attempts to conduct fear conditioning experiments online, with around 10 studies published on the subject, primarily in the last two years. These studies have demonstrated the potential of online fear conditioning research, although challenges to ensure that this research meets the same methodological standards as in-person experimentation remain, despite recent progress. We expect that in the coming years new outcome measures will become available online including the measurement of eye-tracking, pupillometry and probe reaction time and that compliance monitoring will be improved. This exciting new approach opens new possibilities for large-scale data collection among hard-to-reach populations and has the potential to transform the future of fear conditioning research.

Emphasis on peripheral vision is accompanied by pupil dilation

People are best able to detect stimuli in peripheral vision when their pupils are large, and best able to discriminate stimuli in central vision when their pupils are small. However, it is unclear whether our visual system makes use of this by dilating the pupils when attention is directed towards peripheral vision. Therefore, throughout three experiments (N = 100), we tested whether pupil size adapts to the "breadth" of attention. We found that pupils dilate with increasing attentional breadth, both when attention is diffusely spread and when attention is directed at specific locations in peripheral vision. Based on our results and others, we propose that cognitively driven pupil dilation is not an epiphenomenal marker of locus coeruleus activity, as is often assumed, but rather is an adaptive response that reflects an emphasis on peripheral vision.

Time Scales and Moments of Listening Effort Revealed in Pupillometry

This article offers a collection of observations that highlight the value of time course data in pupillometry and points out ways in which these observations create deeper understanding of listening effort. The main message is that listening effort should be considered on a moment-to-moment basis rather than as a singular amount. A review of various studies and the reanalysis of data reveal distinct signatures of effort before a stimulus, during a stimulus, in the moments after a stimulus, and changes over whole experimental testing sessions. Collectively these observations motivate questions that extend beyond the "amount" of effort, toward understanding how long the effort lasts, and how precisely someone can allocate effort at specific points in time or reduce effort at other times. Apparent disagreements between studies are reconsidered as informative lessons about stimulus selection and the nature of pupil dilation as a reflection of decision making rather than the difficulty of sensory encoding.

Testing locus coeruleus-norepinephrine accounts of working memory, attention control, and fluid intelligence

The current set of studies examined the relationship among working memory capacity, attention control, fluid intelligence, and pupillary correlates of tonic arousal regulation and phasic responsiveness in a combined sample of more than 1,000 participants in two different age ranges (young adults and adolescents). Each study was designed to test predictions made by two recent theories regarding the role of the locus coeruleus-norepinephrine (LC-NE) system in determining individual differences in cognitive ability. The first theory, proposed by Unsworth and Robison (2017a), posits two important individual differences: the moment-to-moment regulation of tonic arousal, and the phasic responsiveness of the system to goal-relevant stimuli. The second theory, proposed by Tsukahara and Engle (2021a), argues that people with higher cognitive abilities have greater functional connectivity between the LC-NE system and cortical networks at rest. These two theories are not mutually exclusive, but they make different predictions. Overall, we found no evidence consistent with a resting-state theory. However, phasic responsiveness was consistently correlated with working memory capacity, attention control, and fluid intelligence, supporting a prediction made by Unsworth and Robison (2017a). Tonic arousal regulation was not correlated with working memory or fluid intelligence and was inconsistently correlated with attention control, which offers only partial support for Unsworth and Robison's (2017a) second prediction.