Predicting eye-movement characteristics across multiple tasks from working memory and executive control

EM-COGLOAD: An investigation into age and cognitive load detection using eye tracking and deep learning

Alzheimer's Disease is the most prevalent neurodegenerative disease, and is a leading cause of disability among the elderly. Eye movement behaviour demonstrates potential as a non-invasive biomarker for Alzheimer's Disease, with changes detectable at an early stage after initial onset. This paper introduces a new publicly available dataset: EM-COGLOAD (available at https://osf.io/zjtdq/, DOI: 10.17605/OSF.IO/ZJTDQ). A dual-task paradigm was used to create effects of declined cognitive performance in 75 healthy adults as they carried out visual tracking tasks. Their eye movement was recorded, and time series classification of the extracted eye movement traces was explored using a range of deep learning techniques. The results of this showed that convolutional neural networks were able to achieve an accuracy of 87.5% when distinguishing between eye movement under low and high cognitive load, and 76% when distinguishing between the oldest and youngest age groups.

Individuals vary in their overt attention preference for positive images consistently across time and stimulus types

What humans look at strongly determines what they see. We show that individual differences in the tendency to look at positive stimuli are stable across time and across contents, establishing gaze positivity preference as a perceptual trait that determines the amount of positively valence stimuli individuals select for visual processing. Furthermore, we show that patients with major depressive disorder exhibit consistently low positivity preference before treatment. In a subset of patients, we also assessed the positivity preference after two months of treatment in which positivity gaze preference increased to levels similar to healthy individuals. We discuss the possible practical diagnostic applications of these findings, as well as how this general gaze-related trait may influence other behavioral and psychological aspects.

An updating-based working memory load alters the dynamics of eye movements but not their spatial extent during free viewing of natural scenes

The relationship between spatial deployments of attention and working memory load is an important topic of study, with clear implications for real-world tasks such as driving. Previous research has generally shown that attentional breadth broadens under higher load, while exploratory eye-movement behaviour also appears to change with increasing load. However, relatively little research has compared the effects of working memory load on different kinds of spatial deployment, especially in conditions that require updating of the contents of working memory rather than simple retrieval. The present study undertook such a comparison by measuring participants' attentional breadth (via an undirected Navon task) and their exploratory eye-movement behaviour (a free-viewing recall task) under low and high updating working memory loads. While spatial aspects of task performance (attentional breadth, and peripheral extent of image exploration in the free-viewing task) were unaffected by the load manipulation, the exploratory dynamics of the free-viewing task (including fixation durations and scan-path lengths) changed under increasing load. These findings suggest that temporal dynamics, rather than the spatial extent of exploration, are the primary mechanism affected by working memory load during the spatial deployment of attention. Further, individual differences in exploratory behaviour were observed on the free-viewing task: all metrics were highly correlated across working memory load blocks. These findings suggest a need for further investigation of individual differences in eye-movement behaviour; potential factors associated with these individual differences, including working memory capacity and persistence versus flexibility orientations, are discussed.

Test-retest reliability of eye tracking measures in a computerized Trail Making Test

The Trail Making Test (TMT) is a frequently applied neuropsychological test that evaluates participants' executive functions based on their time to connect a sequence of numbers (TMT-A) or alternating numbers and letters (TMT-B). Test performance is associated with various cognitive functions ranging from visuomotor speed to working memory capabilities. However, although the test can screen for impaired executive functioning in a variety of neuropsychiatric disorders, it provides only little information about which specific cognitive impairments underlie performance detriments. To resolve this lack of specificity, recent cognitive research combined the TMT with eye tracking so that eye movements could help uncover reasons for performance impairments. However, using eye-tracking-based test scores to examine differences between persons, and ultimately apply the scores for diagnostics, presupposes that the reliability of the scores is established. Therefore, we investigated the test-retest reliabilities of scores in an eye-tracking version of the TMT recently introduced by Recker et al. (2022). We examined two healthy samples performing an initial test and then a retest 3 days (n = 31) or 10 to 30 days (n = 34) later. Results reveal that, although reliabilities of classic completion times were overall good, comparable with earlier versions, reliabilities of eye-tracking-based scores ranged from excellent (e.g., durations of fixations) to poor (e.g., number of fixations guiding manual responses). These findings indicate that some eye-tracking measures offer a strong basis for assessing interindividual differences beyond classic behavioral measures when examining processes related to information accumulation processes but are less suitable to diagnose differences in eye-hand coordination.

Working memory control predicts fixation duration in scene-viewing

When viewing scenes, observers differ in how long they linger at each fixation location and how far they move their eyes between fixations. What factors drive these differences in eye-movement behaviors? Previous work suggests individual differences in working memory capacity may influence fixation durations and saccade amplitudes. In the present study, participants (N = 98) performed two scene-viewing tasks, aesthetic judgment and memorization, while viewing 100 photographs of real-world scenes. Working memory capacity, working memory processing ability, and fluid intelligence were assessed with an operation span task, a memory updating task, and Raven's Advanced Progressive Matrices, respectively. Across participants, we found significant effects of task on both fixation durations and saccade amplitudes. At the level of each individual participant, we also found a significant relationship between memory updating task performance and participants' fixation duration distributions. However, we found no effect of fluid intelligence and no effect of working memory capacity on fixation duration or saccade amplitude distributions, inconsistent with previous findings. These results suggest that the ability to flexibly maintain and update working memory is strongly related to fixation duration behavior.

Computational cross-species views of the hippocampal formation

The discovery of place cells and head direction cells in the hippocampal formation of freely foraging rodents has led to an emphasis of its role in encoding allocentric spatial relationships. In contrast, studies in head-fixed primates have additionally found representations of spatial views. We review recent experiments in freely moving monkeys that expand upon these findings and show that postural variables such as eye/head movements strongly influence neural activity in the hippocampal formation, suggesting that the function of the hippocampus depends on where the animal looks. We interpret these results in the light of recent studies in humans performing challenging navigation tasks which suggest that depending on the context, eye/head movements serve one of two roles-gathering information about the structure of the environment (active sensing) or externalizing the contents of internal beliefs/deliberation (embodied cognition). These findings prompt future experimental investigations into the information carried by signals flowing between the hippocampal formation and the brain regions controlling postural variables, and constitute a basis for updating computational theories of the hippocampal system to accommodate the influence of eye/head movements.

Eye Gaze Patterns during Reasoning Provide Insights Regarding Individual Differences in Underlying Cognitive Abilities

Sequences of eye movements during performance of a reasoning task has provided insights into the strategies individuals use to solve that specific task; however, prior studies have not examined whether eye gaze metrics reflect cognitive abilities in a way that transcends a specific task. Thus, our study aimed to explore the relationship between eye movement sequences and other behavioral measures. Here, we present two studies that related different eye gaze metrics in a matrix reasoning task with performance on a different test of fluid reasoning and tests of planning, working memory, and cognitive flexibility. Additionally, we related gaze metrics to self-reported executive functioning in daily life, as measured by BRIEF-A. To perform this, we classified the participants' eye gaze in each item of the matrices test using an algorithm and then used LASSO regression models with the cognitive abilities as the dependent variable to select eye-tracking metrics to predict it. Overall, specific and distinct eye gaze metrics predicted 57% variance in the fluid reasoning scores; 17% variance in the planning scores; and 18% variance in the working memory scores. Taken together, these results support the hypothesis that the selected eye-tracking metrics reflect cognitive abilities that transcend specific tasks.

Searching for meaning: Local scene semantics guide attention during natural visual search in scenes

Models of visual search in scenes include image salience as a source of attentional guidance. However, because scene meaning is correlated with image salience, it could be that the salience predictor in these models is driven by meaning. To test this proposal, we generated meaning maps that represented the spatial distribution of semantic informativeness in scenes, and salience maps which represented the spatial distribution of conspicuous image features and tested their influence on fixation densities from two object search tasks in real-world scenes. The results showed that meaning accounted for significantly greater variance in fixation densities than image salience, both overall and in early attention across both studies. Here, meaning explained 58% and 63% of the theoretical ceiling of variance in attention across both studies, respectively. Furthermore, both studies demonstrated that fast initial saccades were not more likely to be directed to higher salience regions than slower initial saccades, and initial saccades of all latencies were directed to regions containing higher meaning than salience. Together, these results demonstrated that even though meaning was task-neutral, the visual system still selected meaningful over salient scene regions for attention during search.

Individual differences in working memory capacity and visual search while reading

Individual differences in working memory capacity are related to variations in a wide range of cognitive tasks. Surprisingly, effects of individual differences in working memory capacity are somewhat limited in visual search tasks. Here we tested the hypothesis that such an effect would be robust when search was one component of a dual task. Participants were presented strings of letters using rapid serial visual presentation and were required to detect all instances of a particular target letter. In Experiment 1, participants performed the letter search task in three contexts, while: (a) reading a prose passage, (b) processing a stream of random words, or (c) processing a random stream of non-words. In the absence of the dual task of reading prose, and in line with much of the literature on individual differences in working memory capacity and visual search, search performance was unaffected by working memory capacity. As hypothesized, however, higher working memory capacity participants detected more target letters than lower capacity participants in the "true" dual task (searching while reading prose). The hypothesized results from the prose passage were replicated in Experiment 2. These results show that visual search efficiency is dramatically affected by working memory capacity when searching is combined with another cognitive task but not when it is performed in isolation. Our findings are consistent with recent suggestions that visual search efficiency will be affected by working memory capacity so long as searching is embedded in a context that entails managing resource allocation between concurrent tasks.

Impact of Cognitive Demand on Eye Movement Pattern in Patients with Alzheimer’s Disease

Background: Eye-movement behavior has been used as a reliable tool to identify cognitive and behavioral patterns in individuals with different neuropsychiatric disorders including Alzheimer’s disease (AD). Most studies in the field have been dedicated to evaluating eye-movement behavior during cognitive tasks in different protocols using multiple parameters. Objective: We aimed to evaluate the differences of eye-movement behavior in healthy subjects, subjects with mild cognitive impairment (MCI), and those with AD in a simple color task with and without cognitive demand. Methods: 91 subjects: 18 AD, 47 MCI, and 26 healthy controls had their oculomotor parameters assessed during baseline (no cognitive demand involved) and during a simple computational color memory task using an eye-tracker. Results: Baseline showed statistically different and heterogeneous results between normal cognition and MCI groups. Familiarization phase of the task could not discriminate between groups in any of the analyzed parameters. AD subjects made longer fixations and visits on distractors, and more frequent fixations and visits on the target areas than other groups during the response phase. Conclusion: Eye-tracking time-related parameters differentiate AD subjects from other groups under cognitive demand even in a simple color memory task.

EXPRESS: The Effect of Sudden-Onset Distractors on Reading Efficiency and Comprehension

Reading is an essential skill that requires focused attention. However, much reading is done in non-optimal environments. These days, much reading is done on digital devices or with a digital device nearby. These devices often introduce momentary distractions during reading, interrupting with alerts, notifications, and pop-ups. In two eye-tracking experiments, we investigated how such momentary distractions impact reading. Participants read paragraphs while their eye movements were monitored. During half of the paragraphs, distractions appeared periodically on the screen that required a response from the participants. In experiment 1, the distractions were arrows that the participant had to respond to and then could immediately forget. In experiment 2, the participants performed a 1-back task that required them to remember the identity of the last distractor. Compared to the no-distraction condition, the respond-and-forget distractors of experiment 1 had minimal impact on reading behavior and comprehension, but the working-memory-load distractors of experiment 2 led to increased re-reading and decreased reading comprehension. It seems a simple pop-up does not disrupt reading, but a message you must remember will.

Consumer Neuroscience: Attentional Preferences for Wine Labeling Reflected in the Posterior Contralateral Negativity

During the decision-making process, consumers notice, inspect, and visually scan different products. External characteristics of a product, such as design, packaging, label, and logo, have been shown to strongly influence how customers perceive, assess, and select a product. Marketers have put a lot of effort into determining the factors that trigger consumers' visual attention toward products, using traditional research methods, self-reports, or observations. The use of neuroscientific tools to study consumer behavior may improve our understanding of how external characteristics influence consumers' visual attention. Consumer neuroscience research shows that preferences for a product may already be reflected in brain activity before customers make a final decision. Using electroencephalography (EEG), we investigated whether the design of different wine labeling influences individual preferences, reflected in the neural activity related to visual attention. More specifically, we examined whether the posterior contralateral negativity (PCN) can be used to assess and predict consumers' preferences for a specific product based on its external characteristics. The PCN is commonly used to estimate attentional selection by focusing on stimulus-side dependent EEG lateralization above parieto-occipital areas. We computed the PCN to assess whether a certain wine label caught participants' visual attention and additionally by comparing the PCN with behavioral data (wine preferences and reaction times) to determine whether early effects of visual attention could predict participants' final preferences for a specific label. Our findings indicate that the PCN provides relevant information on visual attention mechanisms for external characteristics, as the view of the four labels modulated PCN amplitude. We hope this study can help researchers and practitioners in examining the effects of external product characteristics on consumer choice by estimating the changes in the EEG that are related to visual attention.

Individual differences in first-pass fixation duration in reading are related to resting-state functional connectivity

Although there are considerable individual differences in eye movements during text reading, their neural correlates remain unclear. In this study, we investigated the relationship between the first-pass fixation duration (FPFD) in natural reading and resting-state functional connectivity (RSFC) in the brain. We defined the brain regions associated with early visual processing, word identification, attention shifts, and oculomotor control as seed regions. The results showed that individual FPFDs were positively correlated with individual RSFCs between the early visual network, visual word form area, and eye movement control/dorsal attention network. Our findings provide new evidence on the neural correlates of eye movements in text reading and indicate that individual differences in fixation time may shape the RSFC differences in the brain through the time-on-task effect and the mechanism of Hebbian learning.

Not all fixations are created equal: The benefits of using ex-Gaussian modeling of fixation durations

Various cognitive and perceptual factors have been shown to modulate the duration of fixations during visual exploration of complex scenes. The majority of these studies have only considered the mean of the distribution of fixation durations. However, this distribution is skewed to the right, so that an increase in the mean may be driven by a lengthening of all fixations (i.e., a right shift of the whole distribution) or only the relatively longer ones (i.e., a longer right tail of the distribution). To determine which factor is at play, the distribution can be modeled with an ex-Gaussian distribution, which is a convolution of a Gaussian and an exponential distribution. Here we demonstrate the usefulness of applying the ex-Gaussian model to empirical distributions of fixation durations and the reliability of its parameters across time. We demonstrate how the ex-Gaussian model had advantages over exclusive consideration of the mean, by showing that an increase in the mean can stem from specific changes in the components of the ex-Gaussian distribution. Specifically, the type of image leads to a change in the Gaussian component alone, indicating a right shift of the main mass of the distribution. By contrast, familiarity with the inspected image modifies the exponential component, and results in a more specific modulation of a subset of relatively long fixations. Hence, estimating the ex-Gaussian parameters may provide novel insights into the underlying processes that determine fixation duration and can contribute to the future development of process-based computational models of gaze behavior.

Tonic pupil size and its variability are associated with fluid intelligence in adolescents aged 11-14 years

Tonic pupil size and its variability are sensitive to cognitive abilities (such as fluid intelligence [Gf]) among individuals. The present study aimed to examine this relationship in a new sample set (i.e., adolescents aged 11-14 years) with several important factors considered. We conducted two task-free tasks (the blank-screen viewing task and the scene viewing task) to measure tonic pupil size and its variability in 11-14-year-old adolescents with different Gf levels and preliminarily tested the role of task type and stimuli's luminance on this relationship. The results found that high-Gf adolescents showed smaller tonic pupil size in both tasks but showed larger variability of tonic pupil size in the blank-screen viewing task. Task type and stimuli's luminance could influence tonic pupil size and its variability in different ways. Cognitive and underlying neural mechanisms of these results are discussed to provide an explanation and suggestions for future studies.

Literacy skill and intra-individual variability in eye-fixation durations during reading: Evidence from a diverse community-based adult sample

To understand the effects of literacy on fundamental processes involved in reading, we report a secondary data analysis examining individual differences in global eye-movement measures and first-pass eye-movement distributions in a diverse sample of community-dwelling adults aged 16 to 64. Participants (n = 80) completed an assessment battery probing verbal and non-verbal cognitive abilities and read simple two-sentence passages while their eye movements were recorded. Analyses were focused on characterising the effects of literacy skill on both global indices of eye-fixation distributions and distributional differences in the sensitivity to lexical features. Global reading measures showed that lower literate adults read more slowly on average. However, distributional analyses of fixation durations revealed that the first-pass fixation durations of adults with lower literacy skill were not slower in general (i.e., there was no shift in the fixation duration distribution among lower literate adults). Instead, lower literacy was associated with greater intra-individual variability in first-pass fixation durations, including an increased proportion of extremely long fixations, differentially skewing the distribution of both first-fixation and gaze durations. Exploratory repeated-measures quantile regression analyses of gaze duration revealed differentially greater influences of word length among lower literate readers and greater activation of phonological and orthographic neighbours among higher literate readers, particularly in the tail of the distribution. Collectively, these findings suggest that literacy skill in adulthood is associated with systematic differences in both global and lexically driven eye-movement control during reading.