Analyzing the pupil response due to increased cognitive demand: An independent component analysis study

Classification of Hearing Status Based on Pupil Measures During Sentence Perception

PURPOSE

Speech understanding in noise can be effortful, especially for people with hearing impairment. To compensate for reduced acuity, hearing-impaired (HI) listeners may be allocating listening effort differently than normal-hearing (NH) peers. We expected that this might influence measures derived from the pupil dilation response. To investigate this in more detail, we assessed the sensitivity of pupil measures to hearing-related changes in effort allocation. We used a machine learning-based classification framework capable of combining and ranking measures to examine hearing-related, stimulus-related (signal-to-noise ratio [SNR]), and task response-related changes in pupil measures.

METHOD

Pupil data from 32 NH (40-70 years old, M = 51.3 years, six males) and 32 HI (31-76 years old, M = 59 years, 13 males) listeners were recorded during an adaptive speech reception threshold test. Peak pupil dilation (PPD), mean pupil dilation (MPD), principal pupil components (rotated principal components [RPCs]), and baseline pupil size (BPS) were calculated. As a precondition for ranking pupil measures, the ability to classify hearing status (NH/HI), SNR (high/low), and task response (correct/incorrect) above random prediction level was assessed. This precondition was met when classifying hearing status in subsets of data with varying SNR and task response, SNR in the NH group, and task response in the HI group.

RESULTS

A combination of pupil measures was necessary to classify the dependent factors. Hearing status, SNR, and task response were predicted primarily by the established measures-PPD (maximum effort), RPC2 (speech processing), and BPS (task anticipation)-and by the novel measures RPC1 (listening) and RPC3 (response preparation) in tasks involving SNR as an outcome or sometimes difficulty criterion.

CONCLUSIONS

A machine learning-based classification framework can assess sensitivity of, and rank the importance of, pupil measures in relation to three effort modulators (factors) during speech perception in noise. This indicates that the effects of these factors on the pupil measures allow for reasonable classification performance. Moreover, the varying contributions of each measure to the classification models suggest they are not equally affected by these factors. Thus, this study enhances our understanding of pupil responses and their sensitivity to relevant factors.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.28225199.

Pupillary manifolds: uncovering the latent geometrical structures behind phasic changes in pupil size

The size of the pupils reflects directly the balance of different branches of the autonomic nervous system. This measure is inexpensive, non-invasive, and has provided invaluable insights on a wide range of mental processes, from attention to emotion and executive functions. Two outstanding limitations of current pupillometry research are the lack of consensus in the analytical approaches, which vary wildly across research groups and disciplines, and the fact that, unlike other neuroimaging techniques, pupillometry lacks the dimensionality to shed light on the different sources of the observed effects. In other words, pupillometry provides an integrated readout of several distinct networks, but it is unclear whether each has a specific fingerprint, stemming from its function or physiological substrate. Here we show that phasic changes in pupil size are inherently low-dimensional, with modes that are highly consistent across behavioral tasks of very different nature, suggesting that these changes occur along pupillary manifolds that are highly constrained by the underlying physiological structures rather than functions. These results provide not only a unified approach to analyze pupillary data, but also the opportunity for physiology and psychology to refer to the same processes by tracing the sources of the reported changes in pupil size in the underlying biology.

Evaluation of mental load using EEG and eye movement characteristics

Mental load is a major cause of human-induced accidents. In this study, an explosive impact sensitivity experiment was used to induce mental load. A combination of subjective questionnaires and objective prospective time-distance tests were used to judge whether subjects experienced mental load. Four indicators, namely, β, γ, mean pupil diameter, and fixation time were selected by statistical analysis and PCA for the construction of a mental load assessment model. The study found that the occipital lobe was the most sensitive to mental load, especially β and γ bands. Lastly, it was found that subjects showed different degrees of mental load for the same mental load induction task. The results of the study are applicable to the evaluation and monitoring of the mental characteristics of workers and provide a scientific basis for adjusting the mental load of workers over time to reduce the rate of accidents and enhance production efficiency.

Pupil old/new effect as an objective measure of recognition memory: a meta-analysis of 17 eye-tracking experiments

Recognition memory, the ability to recognise previously encountered information, correlates with pupil diameter changes during the recognition period. This physiological response, known as the pupil old/new effect, generally reflects the variation in pupil dilation when encountering previously studied (old) stimuli compared to new stimuli. To develop a more precise understanding of the pupil old/new effect, we conducted a meta-analysis of 17 eye-tracking experiments (across 12 articles spanning from 2008 to 2023) involving 560 healthy adults with a mean age of 22.31 years. Analysis of publication bias showed a rather low risk of bias in the selected articles. The main meta-analysis revealed a significant and large pooled pupil old/new effect (Cohen's dz = 0.73, 95% CI [0.50, 0.95]). Further analysis of moderators showed that the number of participants included in the experiments and the criteria for selecting trials (only correct trials vs. all trials) had a significant impact on the meta-analytic results. In general, the analyses revealed a robust pupil old/new effect across all selected articles. This finding underscores its potential utility as a marker of recognition memory across different stimuli type, and various experimental designs.

Visual search in infantile nystagmus syndrome

CLINICAL RELEVANCE

Research on infantile nystagmus syndrome (INS) and visual search is limited. Conducting this research could assist practitioners in understanding how INS affects the real-life visual activities of patients and aid in developing new clinical visual function assessments for INS.

BACKGROUND

The aim of this work is to investigate how subjects with INS perform visual search tasks, and, particularly, to assess how INS subjects perform when targets are located at their null position or away from it, and when under additional cognitive demands.

METHODS

INS subjects (N = 15) and controls (N = 20) performed conjunction and feature search tasks, both with and without mental arithmetic. Search performance was assessed using log-transformed total search time, gaze-dependent search time, and accuracy. Cognitive demand was quantified by pupil size and the NASA task-load index score.

RESULTS

INS subjects showed longer search times compared to controls in conjunction search (P < 0.01), but not in feature search. Within INS and control subjects, the total search times were significantly increased by the addition of mental arithmetic (P < 0.0001). There was no difference in gaze-dependent search times between null target position and 15° away from null target position of subjects in conjunction search (P > 0.05). Accuracies were 100% for both control and INS subjects in both conjunction and feature search.

CONCLUSION

Conjunction visual search was impaired in adult INS subjects, and further worsened under increased cognitive demand. The null position did not affect the visual search performance in INS.

Investigating social comparison behaviour in an immersive virtual reality classroom based on eye-movement data

Higher-achieving peers have repeatedly been found to negatively impact students' evaluations of their own academic abilities (i.e., Big-Fish-Little-Pond Effect). Building on social comparison theory, this pattern is assumed to result from students comparing themselves to their classmates; however, based on existing research designs, it remains unclear how exactly students make use of social comparison information in the classroom. To determine the extent to which students (N = 353 sixth graders) actively attend and respond to social comparison information in the form of peers' achievement-related behaviour, we used eye-tracking data from an immersive virtual reality (IVR) classroom. IVR classrooms offer unprecedented opportunities for psychological classroom research as they allow to integrate authentic classroom scenarios with maximum experimental control. In the present study, we experimentally varied virtual classmates' achievement-related behaviour (i.e., their hand-raising in response to the teacher's questions) during instruction, and students' eye and gaze data showed that they actively processed this social comparison information. Students who attended more to social comparison information (as indicated by more frequent and longer gaze durations at peer learners) had less favourable self-evaluations. We discuss implications for the future use of IVR environments to study behaviours in the classroom and beyond.

Applying functional near-infrared spectroscopy and eye-tracking in a naturalistic educational environment to investigate physiological aspects that underlie the cognitive effort of children during mental rotation tests

Spatial cognition is related to academic achievement in science, technology, engineering, and mathematics (STEM) domains. Neuroimaging studies suggest that brain regions' activation might be related to the general cognitive effort while solving mental rotation tasks (MRT). In this study, we evaluate the mental effort of children performing MRT tasks by measuring brain activation and pupil dilation. We use functional near-infrared spectroscopy (fNIRS) concurrently to collect brain hemodynamic responses from children's prefrontal cortex (PFC) and an Eye-tracking system to measure pupil dilation during MRT. Thirty-two healthy students aged 9-11 participated in this experiment. Behavioral measurements such as task performance on geometry problem-solving tests and MRT scores were also collected. The results were significant positive correlations between the children's MRT and geometry problem-solving test scores. There are also significant positive correlations between dorsolateral PFC (dlPFC) hemodynamic signals and visuospatial task performances (MRT and geometry problem-solving scores). Moreover, we found significant activation in the amplitude of deoxy-Hb variation on the dlPFC and that pupil diameter increased during the MRT, suggesting that both physiological responses are related to mental effort processes during the visuospatial task. Our findings indicate that children with more mental effort under the task performed better. The multimodal approach to monitoring students' mental effort can be of great interest in providing objective feedback on cognitive resource conditions and advancing our comprehension of the neural mechanisms that underlie cognitive effort. Hence, the ability to detect two distinct mental states of rest or activation of children during the MRT could eventually lead to an application for investigating the visuospatial skills of young students using naturalistic educational paradigms.

Covariations between pupil diameter and supplementary eye field activity suggest a role in cognitive effort implementation

In both human and nonhuman primates (NHP), the medial prefrontal region, defined as the supplementary eye field (SEF), can indirectly influence behavior selection through modulation of the primary selection process in the oculomotor structures. To perform this oculomotor control, SEF integrates multiple cognitive signals such as attention, memory, reward, and error. As changes in pupil responses can assess these cognitive efforts, a better understanding of the precise dynamics by which pupil diameter and medial prefrontal cortex activity interact requires thorough investigations before, during, and after changes in pupil diameter. We tested whether SEF activity is related to pupil dynamics during a mixed pro/antisaccade oculomotor task in 2 macaque monkeys. We used functional ultrasound (fUS) imaging to examine temporal changes in brain activity at the 0.1-s time scale and 0.1-mm spatial resolution concerning behavioral performance and pupil dynamics. By combining the pupil signals and real-time imaging of NHP during cognitive tasks, we were able to infer localized cerebral blood volume (CBV) responses within a restricted part of the dorsomedial prefrontal cortex, referred to as the SEF, an area in which antisaccade preparation activity is also recorded. Inversely, SEF neurovascular activity measured by fUS imaging was found to be a robust predictor of specific variations in pupil diameter over short and long-time scales. Furthermore, we directly manipulated pupil diameter and CBV in the SEF using reward modulations. These results bring a novel understanding of the physiological links between pupil and SEF, but it also raises questions about the role of anterior cingulate cortex (ACC), as CBV variations in the ACC seems to be negligible compared to CBV variations in the SEF.

Ultrafast functional imaging reveals short- and long-term covariations between pupil diameter and activity in the Supplementary Eye Field (SEF) of awake behaving non-human primates, yielding a novel understanding of the physiological links between the pupil and SEF.

Toward the Use of Pupillary Responses for Pilot Selection

OBJECTIVE

For selection practitioners, it seems important to assess the level of mental resources invested in order to perform a demanding task. In this study, we investigated the potential of pupil size measurement to discriminate the most proficient pilot students from the less proficient.

BACKGROUND

Cognitive workload is known to influence learning outcome. More specifically, cognitive difficulties observed during pilot training are often related to a lack of efficient mental workload management.

METHOD

Twenty pilot students performed a laboratory multitasking scenario, composed of several stages with increasing workload, while their pupil size was recorded. Two levels of pilot students were compared according to the outcome after 2 years of training: high success and medium success.

RESULTS

Our findings suggested that task-evoked pupil size measurements could be a promising predictor of flight training difficulties during the 2-year training. Indeed, high-level pilot students showed greater pupil size changes from low-load to high-load stages of the multitasking scenario than medium-level pilot students. Moreover, average pupil diameters at the low-load stage were smallest for the high-level pilot students.

CONCLUSION

Following the neural efficiency hypothesis framework, the most proficient pilot students supposedly used their mental resources more efficiently than the least proficient while performing the multitasking scenario.

APPLICATION

These findings might introduce a new way of managing selection processes complemented with ocular measurements. More specifically, pupil size measurement could enable identification of applicants with greater chances of success during pilot training.

Publication guidelines and recommendations for pupillary measurement in psychophysiological studies

A variety of psychological and physical phenomena elicit variations in the diameter of pupil of the eye. Changes in pupil size are mediated by the relative activation of the sphincter pupillae muscle (decrease pupil diameter) and the dilator pupillae muscle (increase pupil diameter), innervated by the parasympathetic and sympathetic branches, respectively, of the autonomic nervous system. The current guidelines are intended to inform and guide psychophysiological research involving pupil measurement by (1) summarizing important aspects concerning the physiology of the pupil, (2) providing methodological and data-analytic guidelines and recommendations, and (3) briefly reviewing psychological phenomena that modulate pupillary reactivity. Because of the increased ease and tractability of pupil measurement, the goal of these guidelines is to promote accurate recording, analysis, and reporting of pupillary data in psychophysiological research.

Comparing methods for the analysis of pupillary response

Changes in eye-pupil size index a range of cognitive processes. However, variations in the protocols used to analyze such data exist in the psychological literature. This raises the question of whether different approaches to pupillary response data influence the outcome of the analysis. To address this question, four methods of analysis were compared, using pupillary responses to sexually appetitive visual content as example data. These methods comprised analysis of the unadjusted (raw) pupillary response data, z-scored data, percentage-change data, and data transformed by a prestimulus baseline correction. Across two experiments, these methods yielded near-identical outcomes, leading to similar conclusions. This suggests that the range of approaches that are employed in the psychological literature to analyze pupillary response data do not fundamentally influence the outcome of the analysis. However, some systematic carryover effects were observed when a prestimulus baseline correction was applied, whereby dilation effects from an arousing target on one trial still influenced pupil size on the next trial. This indicates that the appropriate application of this analysis might require additional information, such as prior knowledge of the duration of carryover effects.

Pupil dilation as an index of effort in cognitive control tasks: A review

Pupillometry research has experienced an enormous revival in the last two decades. Here we briefly review the surge of recent studies on task-evoked pupil dilation in the context of cognitive control tasks with the primary aim being to evaluate the feasibility of using pupil dilation as an index of effort exertion, rather than task demand or difficulty. Our review shows that across the three cognitive control domains of updating, switching, and inhibition, increases in task demands typically leads to increases in pupil dilation. Studies show a diverging pattern with respect to the relationship between pupil dilation and performance and we show how an effort account of pupil dilation can provide an explanation of these findings. We also discuss future directions to further corroborate this account in the context of recent theories on cognitive control and effort and their potential neurobiological substrates.

How pupil responses track value-based decision-making during and after reinforcement learning

Cognition can reveal itself in the pupil, as latent cognitive processes map onto specific pupil responses. For instance, the pupil dilates when we make decisions and these pupil size fluctuations reflect decision-making computations during and after a choice. Surprisingly little is known, however, about how pupil responses relate to decisions driven by the learned value of stimuli. This understanding is important, as most real-life decisions are guided by the outcomes of earlier choices. The goal of this study was to investigate which cognitive processes the pupil reflects during value-based decision-making. We used a reinforcement learning task to study pupil responses during value-based decisions and subsequent decision evaluations, employing computational modeling to quantitatively describe the underlying cognitive processes. We found that the pupil closely tracks reinforcement learning processes independently across participants and across trials. Prior to choice, the pupil dilated as a function of trial-by-trial fluctuations in value beliefs about the to-be chosen option and predicted an individual's tendency to exploit high value options. After feedback a biphasic pupil response was observed, the amplitude of which correlated with participants' learning rates. Furthermore, across trials, early feedback-related dilation scaled with value uncertainty, whereas later constriction scaled with signed reward prediction errors. These findings show that pupil size fluctuations can provide detailed information about the computations underlying value-based decisions and the subsequent updating of value beliefs. As these processes are affected in a host of psychiatric disorders, our results indicate that pupillometry can be used as an accessible tool to non-invasively study the processes underlying ongoing reinforcement learning in the clinic.

Recognition memory and featural similarity between concepts: The pupil's point of view

Differences in pupil dilation are observed for studied compared to new items in recognition memory. According to cognitive load theory, this effect reflects the greater cognitive demands of retrieving contextual information from study phase. Pupil dilation can also occur when new items conceptually related to old ones are erroneously recognized as old, but the aspects of similarity that modulate false memory and related pupil responses remain unclear. We investigated this issue by manipulating the degree of featural similarity between new (unstudied) and old (studied) concepts in an old/new recognition task. We found that new concepts with high similarity were mistakenly identified as old and had greater pupil dilation than those with low similarity, suggesting that pupil dilation reflects the strength of evidence on which recognition judgments are based and, importantly, greater locus coeruleus and prefrontal activity determined by the higher degree of retrieval monitoring involved in recognizing these items.

A pilot study of disparity vergence and near dissociated phoria in convergence insufficiency patients before vs. after vergence therapy

Purpose: This study examined the relationship between the near dissociated phoria and disparity vergence eye movements. Convergence insufficiency (CI) patients before vergence therapy were compared to: (1) the same patients after vergence therapy; and (2) binocularly normal controls (BNC).

Methods: Sixteen subjects were studied—twelve BNC and four with CI. Measurements from the CI subjects were obtained before and after 18 h of vergence eye movement therapy. The near dissociated phoria was measured using the flashed Maddox rod technique. Vergence responses were stimulated from 4° symmetrical disparity vergence step stimuli. The peak velocity of the vergence response and the magnitude of the fusion initiating component (FIC) from an independent component analysis (ICA) were calculated. A linear regression analysis was conducted studying the vergence peak velocity as a function of the near dissociated phoria where the Pearson correlation coefficient was computed.

Results: Before vergence therapy, the average with one standard deviation FIC magnitude of convergence responses from CI subjects was 0.29° ± 0.82 and significantly less than the FIC magnitude of 1.85° ± 0.84 for BNC (p < 0.02). A paired t-test reported that the FIC and near dissociated phoria before vergence therapy for CI subjects significantly increased to 1.49° ± 0.57 (p < 0.04) and became less exophoric to 3.5Δ ± 1.9 exo (p < 0.02) after vergence therapy. A significant correlation (r = 0.87; p < 0.01) was observed between the near dissociated phoria and the vergence ratio of convergence peak velocity divided by divergence peak velocity.

Conclusion: The results have clinical translational impact in understanding the mechanism by which vergence therapy may be changing the vergence system leading to a sustained reduction in visual symptoms.

Frequency analysis of a task-evoked pupillary response: Luminance-independent measure of mental effort

Pupil diameter is a widely-studied cognitive load measure, which, despite its convenience for non-intrusive operator state monitoring in complex environments, is still not available for in situ measurements because of numerous methodological limitations. The most important of these limitations is the influence of pupillary light reflex. Hence, there is the need of providing a pupil-based cognitive load measure that is independent of light conditions. In this paper, we present a promising technique of pupillary signal analysis resulting in luminance-independent measure of mental effort that could be used in real-time without a priori on luminous conditions. Twenty-two participants performed a short-term memory task under different screen luminance conditions. Our results showed that the amplitude of pupillary dilation due to load on memory was luminance-dependent with higher amplitude corresponding to lower-luminance condition. Furthermore, our experimentation showed that load on memory and luminance factors express themselves differently according to frequency. Therefore, as our statistical analysis revealed, the ratio between low (0-1.6 Hz) and high frequency (1.6-4 Hz) bands (LF/HF ratio) of power spectral densities of pupillary signal is sensitive to the cognitive load but not to luminance. Our results are promising for the measurement of load on memory in ecological settings.

What does germane load mean? An empirical contribution to the cognitive load theory

While over the last decades, much attention has been paid to the mental workload in the field of human computer interactions, there is still a lack of consensus concerning the factors that generate it as well as the measurement methods that could reflect workload variations. Based on the multifactorial Cognitive Load Theory (CLT), our study aims to provide some food for thought about the subjective and objective measurement that can be used to disentangle the intrinsic, extraneous, and germane load. The purpose is to provide insight into the way cognitive load can explain how users' cognitive resources are allocated in the use of hypermedia, such as an online newspaper. A two-phase experiment has been conducted on the information retention from online news stories. Phase 1 (92 participants) examined the influence of multimedia content on performance as well as the relationships between cognitive loads and cognitive absorption. In Phase 2 (36 participants), eye-tracking data were collected in order to provide reliable and objective measures. Results confirmed that performance in information retention was impacted by the presence of multimedia content such as animations and pictures. The higher number of fixations on these animations suggests that users' attention could have been attracted by them. Results showed the expected opposite relationships between germane and extraneous load, a positive association between germane load and cognitive absorption and a non-linear association between intrinsic and germane load. The trends based on eye-tracking data analysis provide some interesting findings about the relationship between longer fixations, shorter saccades and cognitive load. Some issues are raised about the respective contribution of mean pupil diameter and Index of Cognitive Activity.

Pupillometry

Pupillometry is the study of changes in the diameter of the pupil as a function of cognitive processing. This review paper provides a brief historical overview of the study of pupillometry in cognitive science. The physiology of pupillary responses is introduced, leading to an outline of early pupillometry work, which began with the seminal work of Hess and Polt in the 1960s. The paper then presents a broad review of contemporary research in cognitive sciences that relies on pupillometry. This review is organized around five general domains, namely perception, language processing, memory and decision making, emotion and cognition, and cognitive development. In order to illustrate the nature of the method, and the challenges of analysis, the next section of the review details the process of compiling, processing, and analyzing data from a simple, typical pupillometry study. WIREs Cogn Sci 2014, 5:679-692. doi: 10.1002/wcs.1323 For further resources related to this article, please visit the WIREs website.

CONFLICT OF INTEREST

The authors have declared no conflicts of interest for this article.

Pupil dilation co-varies with memory strength of individual traces in a delayed response paired-associate task

Studies on cognitive effort have shown that pupil dilation is a reliable indicator of memory load. However, it is conceivable that there are other sources of effort involved in memory that also affect pupil dilation. One of these is the ease with which an item can be retrieved from memory. Here, we present the results of an experiment in which we studied the way in which pupil dilation acts as an online marker for memory processing during the retrieval of paired associates while reducing confounds associated with motor responses. Paired associates were categorized into sets containing either 4 or 7 items. After learning the paired associates once, pupil dilation was measured during the presentation of the retrieval cue during four repetitions of each set. Memory strength was operationalized as the number of repetitions (frequency) and set-size, since having more items per set results in a lower average recency. Dilation decreased with increased memory strength, supporting the hypothesis that the amplitude of the evoked pupillary response correlates positively with retrieval effort. Thus, while many studies have shown that “memory load” influences pupil dilation, our results indicate that the task-evoked pupillary response is also sensitive to the experimentally manipulated memory strength of individual items. As these effects were observed well before the response had been given, this study also suggests that pupil dilation can be used to assess an item’s memory strength without requiring an overt response.