Switch from ambient to focal processing mode explains the dynamics of free viewing eye movements

The time-course of visual scanning behaviour of paramedicine students upon arrival at a simulated emergency call

BACKGROUND

Limited knowledge exists regarding how paramedics acquire an understanding of the scene they encounter upon arrival, despite their need to quickly gather information for effective clinical decision-making. This study examined visual scanning behaviour during the early stages of simulated emergency calls.

METHODS

Eye movements of 10 paramedicine students were recorded during simulated calls conducted in both a high-fidelity classroom setting and a full sensory immersion setting.

RESULTS

Students focused on similar areas in both settings, with most time spent looking at the patient rather than distractors such as room features or other people. Analysis of gaze behaviour across the first five minutes revealed a more nuanced pattern: attention initially gravitated towards distractors but decreased as students became familiar with their surroundings and focused on the task at hand. This pattern was consistent across both simulation settings, indicating that information-seeking strategies may be independent of scene complexity.

CONCLUSIONS

Expertise relies on the ability to differentiate between relevant and irrelevant information. Given the unpredictable nature of their work, paramedics must continuously adapt their understanding of a scene from the moment they enter it. Understanding how this skill develops may help identify expert strategies to inform training of novice paramedics.

SARS-CoV-2 Infection Impairs Oculomotor Functions: A Longitudinal Eye-tracking Study

Although Severe Acute Respiratory Syndrome Coronavirus 2 infection (SARS-CoV-2) is primarily recognized as a respiratory disease, mounting evidence suggests that it may lead to neurological and cognitive impairments. The current study used three eye-tracking tasks (free-viewing, fixation, and smooth pursuit) to assess the oculomotor functions of mild infected cases over six months with symptomatic SARS-CoV-2 infected volunteers. Fifty symptomatic SARS-CoV-2 infected, and 24 self-reported healthy controls completed the eye-tracking tasks in an initial assessment. Then, 45, and 40 symptomatic SARS-CoV-2 infected completed the tasks at 2- and 6-months post-infection, respectively. In the initial assessment, symptomatic SARS-CoV-2 infected exhibited impairments in diverse eye movement metrics. Over the six months following infection, the infected reported overall improvement in health condition, except for self-perceived mental health. The eye movement patterns in the free-viewing task shifted toward a more focal processing mode and there was no significant improvement in fixation stability among the infected. A linear discriminant analysis shows that eye movement metrics could differentiate the infected from healthy controls with an accuracy of approximately 62%, even 6 months post-infection. These findings suggest that symptomatic SARSCoV- 2 infection may result in persistent impairments in oculomotor functions, and the employment of eye-tracking technology can offer valuable insights into both the immediate and long-term effects of SARS-CoV-2 infections. Future studies should employ a more balanced research design and leverage advanced machine-learning methods to comprehensively investigate the impact of SARSCoV- 2 infection on oculomotor functions.

Eye Movement Characteristics for Predicting a Transition to Psychosis: Longitudinal Changes and Implications

BACKGROUND AND HYPOTHESIS

Substantive inquiry into the predictive power of eye movement (EM) features for clinical high-risk (CHR) conversion and their longitudinal trajectories is currently sparse. This study aimed to investigate the efficiency of machine learning predictive models relying on EM indices and examine the longitudinal alterations of these indices across the temporal continuum.

STUDY DESIGN

EM assessments (fixation stability, free-viewing, and smooth pursuit tasks) were performed on 140 CHR and 98 healthy control participants at baseline, followed by a 1-year longitudinal observational study. We adopted Cox regression analysis and constructed random forest prediction models. We also employed linear mixed-effects models (LMMs) to analyze longitudinal changes of indices while stratifying by group and time.

STUDY RESULTS

Of the 123 CHR participants who underwent a 1-year clinical follow-up, 25 progressed to full-blown psychosis, while 98 remained non-converters. Compared with the non-converters, the converters exhibited prolonged fixation durations, decreased saccade amplitudes during the free-viewing task; larger saccades, and reduced velocity gain during the smooth pursuit task. Furthermore, based on 4 baseline EM measures, a random forest model classified converters and non-converters with an accuracy of 0.776 (95% CI: 0.633, 0.882). Finally, LMMs demonstrated no significant longitudinal alterations in the aforementioned indices among converters after 1 year.

CONCLUSIONS

Aberrant EMs may precede psychosis onset and remain stable after 1 year, and applying eye-tracking technology combined with a modeling approach could potentially aid in predicting CHRs evolution into overt psychosis.

Unraveling the developmental dynamic of visual exploration of social interactions in autism

Atypical deployment of social gaze is present early on in toddlers with autism spectrum disorders (ASDs). Yet, studies characterizing the developmental dynamic behind it are scarce. Here, we used a data-driven method to delineate the developmental change in visual exploration of social interaction over childhood years in autism. Longitudinal eye-tracking data were acquired as children with ASD and their typically developing (TD) peers freely explored a short cartoon movie. We found divergent moment-to-moment gaze patterns in children with ASD compared to their TD peers. This divergence was particularly evident in sequences that displayed social interactions between characters and even more so in children with lower developmental and functional levels. The basic visual properties of the animated scene did not account for the enhanced divergence. Over childhood years, these differences dramatically increased to become more idiosyncratic. These findings suggest that social attention should be targeted early in clinical treatments.

Autogenous cerebral processes: an invitation to look at the brain from inside out

While external stimulation can reliably trigger neuronal activity, cerebral processes can operate independently from the environment. In this study, we conceptualize autogenous cerebral processes (ACPs) as intrinsic operations of the brain that exist on multiple scales and can influence or shape stimulus responses, behavior, homeostasis, and the physiological state of an organism. We further propose that the field should consider exploring to what extent perception, arousal, behavior, or movement, as well as other cognitive functions previously investigated mainly regarding their stimulus-response dynamics, are ACP-driven.

Neuronal Population Activity in Macaque Visual Cortices Dynamically Changes through Repeated Fixations in Active Free Viewing

During free viewing, we move our eyes and fixate on objects to recognize the visual scene of our surroundings. To investigate the neural representation of objects in this process, we studied individual and population neuronal activity in three different visual regions of the brains of macaque monkeys (Macaca fuscata): the primary and secondary visual cortices (V1, V2) and the inferotemporal cortex (IT). We designed a task where the animal freely selected objects in a stimulus image to fixate on while we examined the relationship between spiking activity, the order of fixations, and the fixated objects. We found that activity changed across repeated fixations on the same object in all three recorded areas, with observed reductions in firing rates. Furthermore, the responses of individual neurons became sparser and more selective with individual objects. The population activity for individual objects also became distinct. These results suggest that visual neurons respond dynamically to repeated input stimuli through a smaller number of spikes, thereby allowing for discrimination between individual objects with smaller energy.

The policy of testing hypotheses in Chilean science. The role of a hypothesis-driven research funding programme in the installation of a hypothesis-driven experimental system in visual neuroscience

O'Malley et al. (2009) and Haufe (2013) suggest that the philosophical idea of science as hypothesis testing generates a pernicious bias towards hypothesis-driven research and against exploratory research in the review process of research proposals and the allocation of resources. This paper addresses a conceptual objection to the argument by O'Malley et al. (2009) and Haufe (2013). We argue that the funding agencies' concepts of good science do not belong to epistemological or philosophical contexts but to political and institutional contexts. This means that correcting (potential) biases in research funding does not entail correcting funding agencies' (supposed) philosophies of science. To illustrate this point, we provide an in-depth historical case study: the granting of funds to neuroscientist Pedro Maldonado by the Chilean funding programme FONDECYT. This is a relevant comparison as FONDECYT's guidelines explicitly promote hypothesis-driven research and endorse a view of "good science" as hypothesis testing. However, we will see that the overall influence of the philosophical idea of science as hypothesis testing over this funding programme, the research project, and the actual practice of hypothesis testing is somewhat limited. The concept of science as hypothesis testing seems to play a crucial institutional or political (not philosophical) role in allowing the conceptual articulation of social expectations and researchers' expectations.

How central and peripheral vision influence focal and ambient processing during scene viewing

Central and peripheral vision carry out different functions during scene processing. The ambient mode of visual processing is more likely to involve peripheral visual processes, whereas the focal mode of visual processing is more likely to involve central visual processes. Although the ambient mode is responsible for navigating space and comprehending scene layout, the focal mode gathers detailed information as central vision is oriented to salient areas of the visual field. Previous work suggests that during the time course of scene viewing, there is a transition from ambient processing during the first few seconds to focal processing during later time intervals, characterized by longer fixations and shorter saccades. In this study, we identify the influence of central and peripheral vision on changes in eye movements and the transition from ambient to focal processing during the time course of scene processing. Using a gaze-contingent protocol, we restricted the visual field to central or peripheral vision while participants freely viewed scenes for 20 seconds. Results indicated that fixation durations are shorter when vision is restricted to central vision compared to normal vision. During late visual processing, fixations in peripheral vision were longer than those in central vision. We show that a transition from more ambient to more focal processing during scene viewing will occur even when vision is restricted to only central vision or peripheral vision.

Modeling eye movement in dynamic interactive tasks for maximizing situation awareness based on Markov decision process

For complex dynamic interactive tasks (such as aviating), operators need to continuously extract information from areas of interest (AOIs) through eye movement to maintain high level of situation awareness (SA), as failures of SA may cause task performance degradation, even system accident. Most of the current eye movement models focus on either static tasks (such as image viewing) or simple dynamic tasks (such as video watching), without considering SA. In this study, an eye movement model with the goal of maximizing SA is proposed based on Markov decision process (MDP), which is designed to describe the dynamic eye movement of experienced operators in dynamic interactive tasks. Two top-down factors, expectancy and value, are introduced into this model to represent the update probability and the importance of information in AOIs, respectively. In particular, the model regards sequence of eye fixations to different AOIs as sequential decisions to maximize the SA-related reward (value) in the context of uncertain information update (expectancy). Further, this model was validated with a flight simulation experiment. Results show that the predicted probabilities of fixation on and shift between AOIs are highly correlated (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$R = 0.928$$\end{document}R=0.928 and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$R = 0.951$$\end{document}R=0.951, respectively) with those of the experiment data.

Eye movement indices as predictors of conversion to psychosis in individuals at clinical high risk

Eye movement abnormalities have been established as an "endophenotype" of schizophrenia. However, less is known about the possibility of these abnormalities as biomarkers for psychosis conversion among clinical high risk (CHR) populations. In the present study, 108 CHR individuals and 70 healthy controls (HC) underwent clinical assessments and eye-tracking tests, comprising fixation stability and free-viewing tasks. According to three-year follow-up outcomes, CHR participants were further stratified into CHR-converter (CHR-C; n = 21) and CHR-nonconverter (CHR-NC; n = 87) subgroups. Prediction models were constructed using Cox regression and logistic regression. The CHR-C group showed more saccades of the fixation stability test (no distractor) and a reduced saccade amplitude of the free-viewing test than HC. Moreover, the CHR-NC group exhibited excessive saccades and an increased saccade amplitude of the fixation stability test (no distractor; with distractor) compared with HC. Furthermore, two indices could effectively discriminate CHR-C from CHR-NC with an area under the receiver-operating characteristic (ROC) curve of 0.80, including the saccade number of the fixation stability test (no distractor) and the saccade amplitude of the free-viewing test. Combined with negative symptom scores of the Scale of Prodromal Symptoms, the area was 0.81. These findings support that eye movement alterations might emerge before the onset of clinically overt psychosis and could assist in predicting psychosis transition among CHR populations.

Effective differentiation between depressed patients and controls using discriminative eye movement features

BACKGROUND

Depression is a common debilitating mental disorder caused by various factors. Identifying and diagnosing depression are challenging because the clinical evaluation of depression is mainly subjective, lacking objective and quantitative indicators. The present study investigated the value and significance of eye movement measurements in distinguishing depressed patients from controls.

METHODS

Ninety-five depressed patients and sixty-nine healthy controls performed three eye movement tests, including fixation stability, free-viewing, and anti-saccade tests, and eleven eye movement indexes were obtained from these tests. The independent t-test was adopted for group comparisons, and multiple logistic regression analysis was employed to identify diagnostic biomarkers. Support vector machine (SVM), quadratic discriminant analysis (QDA), and Bayesian (BYS) algorithms were applied to build the classification models.

RESULTS

Depressed patients exhibited eye movement anomalies, characterized by increased saccade amplitude in the fixation stability test; diminished saccade velocity in the anti-saccade test; and reduced saccade amplitude, shorter scan path length, lower saccade velocity, decreased dynamic range of pupil size, and lower pupil size ratio in the free-viewing test. Four features mentioned above entered the logistic regression equation. The classification accuracies of SVM, QDA, and BYS models reached 86.0%, 81.1%, and 83.5%, respectively.

CONCLUSIONS

Depressed patients exhibited abnormalities across multiple tests of eye movements, assisting in differentiating depressed patients from healthy controls in a cost-effective and non-invasive manner.

Latency shortening with enhanced sparseness and responsiveness in V1 during active visual sensing

In natural vision, neuronal responses to visual stimuli occur due to self-initiated eye movements. Here, we compare single-unit activity in the primary visual cortex (V1) of non-human primates to flashed natural scenes (passive vision condition) to when they freely explore the images by self-initiated eye movements (active vision condition). Active vision enhances the number of neurons responding, and the response latencies become shorter and less variable across neurons. The increased responsiveness and shortened latency during active vision were not explained by increased visual contrast. While the neuronal activities in all layers of V1 show enhanced responsiveness and shortened latency, a significant increase in lifetime sparseness during active vision is observed only in the supragranular layer. These findings demonstrate that the neuronal responses become more distinct in active vision than passive vision, interpreted as consequences of top-down predictive mechanisms.

What are the visuo-motor tendencies of omnidirectional scene free-viewing in virtual reality?

Central and peripheral vision during visual tasks have been extensively studied on two-dimensional screens, highlighting their perceptual and functional disparities. This study has two objectives: replicating on-screen gaze-contingent experiments removing central or peripheral field of view in virtual reality, and identifying visuo-motor biases specific to the exploration of 360 scenes with a wide field of view. Our results are useful for vision modelling, with applications in gaze position prediction (e.g., content compression and streaming). We ask how previous on-screen findings translate to conditions where observers can use their head to explore stimuli. We implemented a gaze-contingent paradigm to simulate loss of vision in virtual reality, participants could freely view omnidirectional natural scenes. This protocol allows the simulation of vision loss with an extended field of view (\(\gt \)80°) and studying the head's contributions to visual attention. The time-course of visuo-motor variables in our pure free-viewing task reveals long fixations and short saccades during first seconds of exploration, contrary to literature in visual tasks guided by instructions. We show that the effect of vision loss is reflected primarily on eye movements, in a manner consistent with two-dimensional screens literature. We hypothesize that head movements mainly serve to explore the scenes during free-viewing, the presence of masks did not significantly impact head scanning behaviours. We present new fixational and saccadic visuo-motor tendencies in a 360° context that we hope will help in the creation of gaze prediction models dedicated to virtual reality.

From Social Gaze to Indirect Speech Constructions: How to Induce the Impression That Your Companion Robot Is a Conscious Creature

We implemented different modes of social gaze behavior in our companion robot, F-2, to evaluate the impression of the gaze behaviors on humans in three symmetric communicative situations: (a) the robot telling a story, (b) the person telling a story to the robot, and (c) both parties communicating about objects in the real world while solving a Tangram puzzle. In all the situations the robot localized the human’s eyes and directed its gaze between the human, the environment, and the object of interest in the problem space (if it existed). We examined the balance between different gaze directions as the novel key element to maintaining a feeling of social connection with the robot in humans. We extended the computer model of the robot in order to simulate realistic gaze behavior in the robot and create the impression of the robot changing its internal cognitive states. Other novel results include the implicit, rather than explicit, character of the robot gaze perception for many of our subjects and the role of individual differences, especially the level of emotional intelligence, in terms of human sensitivity to the robotic gaze. Therefore, in this study, we used an iterative approach, extending the applied cognitive architecture in order to simulate the balance between different behavioral reactions and to test it in the experiments. In such a way, we came to a description of the key behavioral cues that suggest to a person that the particular robot can be perceived as an emotional and even conscious creature.

The extrafoveal preview paradigm as a measure of predictive, active sampling in visual perception

A key feature of visual processing in humans is the use of saccadic eye movements to look around the environment. Saccades are typically used to bring relevant information, which is glimpsed with extrafoveal vision, into the high-resolution fovea for further processing. With the exception of some unusual circumstances, such as the first fixation when walking into a room, our saccades are mainly guided based on this extrafoveal preview. In contrast, the majority of experimental studies in vision science have investigated "passive" behavioral and neural responses to suddenly appearing and often temporally or spatially unpredictable stimuli. As reviewed here, a growing number of studies have investigated visual processing of objects under more natural viewing conditions in which observers move their eyes to a stationary stimulus, visible previously in extrafoveal vision, during each trial. These studies demonstrate that the extrafoveal preview has a profound influence on visual processing of objects, both for behavior and neural activity. Starting from the preview effect in reading research we follow subsequent developments in vision research more generally and finally argue that taking such evidence seriously leads to a reconceptualization of the nature of human visual perception that incorporates the strong influence of prediction and action on sensory processing. We review theoretical perspectives on visual perception under naturalistic viewing conditions, including theories of active vision, active sensing, and sampling. Although the extrafoveal preview paradigm has already provided useful information about the timing of, and potential mechanisms for, the close interaction of the oculomotor and visual systems while reading and in natural scenes, the findings thus far also raise many new questions for future research.

Eye movement characteristics in schizophrenia: A recent update with clinical implications

Eye movements are indispensable for the collection of visual information in everyday life. Many findings regarding the neural basis of eye movements have been accumulated from neurophysiological and psychophysical studies. In the field of psychiatry, studies on eye movement characteristics in mental illnesses have been conducted since the early 1900s. Participants with schizophrenia are known to have characteristic eye movements during smooth pursuit, saccade control, and visual search. Recently, studies evaluating eye movement characteristics as biomarkers for schizophrenia have attracted considerable attention. In this article, we review the neurophysiological basis of eye movement control and eye movement characteristics in schizophrenia. Furthermore, we discuss the prospects for eye movements as biomarkers for mental illnesses.

Eye movement abnormalities and their association with cognitive impairments in schizophrenia

BACKGROUND

Eye movement abnormalities have been identified in schizophrenia; however, their relevance to cognition is still unknown. In this study, we explored the general relationship between eye movements and cognitive function.

METHODS

The three eye movement measures (scanpath length, horizontal position gain, and duration of fixations) that were previously reported to be useful in distinguishing subjects with schizophrenia from healthy subjects, as well as Wechsler Adult Intelligence Scale-III (WAIS-III) scores, were collected and tested for association in 113 subjects with schizophrenia and 404 healthy subjects.

RESULTS

Scanpath length was positively correlated with matrix reasoning and digit symbol coding in subjects with schizophrenia and correlated with vocabulary and symbol search in healthy subjects. Upon testing for interaction effects of diagnosis and scanpath length on correlated WAIS-III scores, a significant interaction effect was only observed for matrix reasoning. The positive correlation between scanpath length and matrix reasoning, which was specific to subjects with schizophrenia, remained significant after controlling for demographic confounders such as medication and negative symptoms. No correlation was observed between the two other eye movement measures and any of the WAIS-III scores.

CONCLUSIONS

Herein, we reveal novel findings on the association between eye-movement-based measures of visual exploration and cognitive scores requiring visual search in subjects with schizophrenia and in healthy subjects. The association between scanpath length and matrix reasoning, a measure of perceptual organization in subjects with schizophrenia, implies the existence of common cognitive processes, and subjects with longer scanpath length may be advantageous in performance of perceptual organization tasks.

Consciousness in a multilevel architecture: What causes the lateralization of effective connectivity under resting state?

Here we present our answers to a critical commentary by Elkhonon Goldberg on our recent publication (Velichkovsky et al., 2018). To avoid discussions about novelty effects in the human brain activity and memory processes, we narrowed down this response to a reanalysis of our data along the lines proposed in the commentary, namely to comparing the effective links between symmetrical brain structures during the first and the last parts of a prolonged resting-state fMRI experiment. We also tested for sex differences in our results and checked for a stability of top-down interactions during the course of experiment because learning is often expressed in the weakening of upper level control over low-level mechanisms. Our attempts to test the predictions based on the novelty hypothesis has led to mixed results suggesting that the discovered right-to-left dominance of causal connections at rest may have a deeper origin than supposed in the Goldberg's commentary.

Distinct modes of top-down cognitive processing in the ventral visual cortex

Top-down cognitive control leads to changes in the sensory processing of the brain. In visual perception such changes can take place in the ventral visual cortex altering the functional asymmetry in forward and backward connections. Here we used fixation-related evoked responses of EEG measurement and dynamic causal modeling to examine hierarchical forward-backward asymmetry, while twenty-six healthy adults performed cognitive tasks that require different types of top-down cognitive control (memorizing or searching visual objects embedded in a natural scene image). The generative model revealed an enhanced asymmetry toward forward connections during memorizing, whereas enhanced backward connections were found during searching. This task-dependent modulation of forward and backward connections suggests two distinct modes of top-down cognitive processing in cortical networks. The alteration in forward-backward asymmetry might underlie the functional role in the cognitive control of visual information processing.

Emotional Content Modulates Attentional Visual Orientation During Free Viewing of Natural Images

Visual attention is the process that enables us to select relevant visual stimuli in our environment to achieve a goal or perform adaptive behaviors. In this process, bottom-up mechanisms interact with top-down mechanisms underlying the automatic and voluntary orienting of attention. Cognitive functions, such as emotional processing, can influence visual attention by increasing or decreasing the resources destined for processing stimuli. The relationship between attention and emotion has been explored mainly in the field of automatic attentional capturing; especially, emotional stimuli are suddenly presented and detection rates or reaction times are recorded. Unlike these paradigms, natural visual scenes may be comprised in multiple stimuli with different emotional valences. In this setting, the mechanisms supporting voluntary visual orientation, under the influence of the emotional components of stimuli, are unknown. We employed a mosaic of pictures with different emotional valences (positive, negative, and neutral) and explored the dynamics of attentional visual orientation, assessed by eye tracking and measurements of pupil diameter. We found that pictures with affective content display increased dwelling times when compared to neutral pictures with a larger effect for negative pictures. The valence, regardless of the arousal levels, was the main factor driving the behavioral modulation of visual orientation. On the other hand, the visual exploration was accompanied by a systematic pupillary response, with the pupil contraction and dilation influenced by the arousal levels, with minor effects driven by the valence. Our results emphasize that arousal and valence should be considered different dimensions of emotional processing both interacting with cognitive processes such as visual attention.

Consciousness in a multilevel architecture: Evidence from the right side of the brain

By taking into account Bruce Bridgeman's interest in an evolutionary framing of human cognition, we examine effective (cause-and-effect) connectivity among cortical structures related to different parts of the triune phylogenetic stratification: archicortex, paleocortex and neocortex. Using resting-state functional magnetic resonance imaging data from 25 healthy subjects and spectral Dynamic Causal Modeling, we report interactions among 10 symmetrical left and right brain areas. Our results testify to general rightward and top-down biases in excitatory interactions of these structures during resting state, when self-related contemplation prevails over more objectified conceptual thinking. The right hippocampus is the only structure that shows bottom-up excitatory influences extending to the frontopolar cortex. The right ventrolateral cortex also plays a prominent role as it interacts with the majority of nodes within and between evolutionary distinct brain subdivisions. These results suggest the existence of several levels of cognitive-affective organization in the human brain and their profound lateralization.