Assessing Saccadic Eye Movements With Head-Mounted Display Virtual Reality Technology

Visually induced vertical vergence as a motion processing biomarker associated with postural instability

The present study explored visually induced vertical vergence (VIVV) as non-specific motion processing response. Healthy participants (7 male, mean age 28.57 ± 2.30; 9 female, mean age 27.67 ± 3.65) were exposed to optokinetic stimuli in an HTC VIVE virtual reality headset while VIVV, pupil-size, and postural sway was recorded. The methodology was shown to produce VIVV in the roll plane at 30 deg/s. Subsequent trials consisted of 40 s optokinetic motion in yaw, pitch, and roll directions at 60 deg/s, and radial optic flow; optokinetic directions were inverted after 20 s of motion. Median VIVV amplitude changes were normalized to the clockwise roll rotation, analysed, and correlated with changes in pupil-size and body sway. VIVV, pupil-size, and body sway were all affected by changes in optokinetic direction. Post-hoc analyses showed significant VIVV responses during optokinetic yaw and pitch rotations, as well as during radial optic flow stimulations. VIVV magnitudes were universally correlated with pupil-size and body sway. In conclusion, VIVV was expressed in all tested dimensions and may consequently serve as a visual motion processing biomarker. Failing to support binocularity while responding to optokinetic directionality, VIVV may reflect an eye-movement response associated with increased postural instability and stress, similar to a dorsal light reflex.

Color constancy mechanisms in virtual reality environments

Prior research has demonstrated high levels of color constancy in real-world scenarios featuring single light sources, extensive fields of view, and prolonged adaptation periods. However, exploring the specific cues humans rely on becomes challenging, if not unfeasible, with actual objects and lighting conditions. To circumvent these obstacles, we employed virtual reality technology to craft immersive, realistic settings that can be manipulated in real time. We designed forest and office scenes illuminated by five colors. Participants selected a test object most resembling a previously shown achromatic reference. To study color constancy mechanisms, we modified scenes to neutralize three contributors: local surround (placing a uniform-colored leaf under test objects), maximum flux (keeping the brightest object constant), and spatial mean (maintaining a neutral average light reflectance), employing two methods for the latter: changing object reflectances or introducing new elements. We found that color constancy was high in conditions with all cues present, aligning with past research. However, removing individual cues led to varied impacts on constancy. Local surrounds significantly reduced performance, especially under green illumination, showing strong interaction between greenish light and rose-colored contexts. In contrast, the maximum flux mechanism barely affected performance, challenging assumptions used in white balancing algorithms. The spatial mean experiment showed disparate effects: Adding objects slightly impacted performance, while changing reflectances nearly eliminated constancy, suggesting human color constancy relies more on scene interpretation than pixel-based calculations.

Assessing REALTER simulator: analysis of ocular movements in simulated low-vision conditions with extended reality technology

Immersive technology, such as extended reality, holds promise as a tool for educating ophthalmologists about the effects of low vision and for enhancing visual rehabilitation protocols. However, immersive simulators have not been evaluated for their ability to induce changes in the oculomotor system, which is crucial for understanding the visual experiences of visually impaired individuals. This study aimed to assess the REALTER (Wearable Egocentric Altered Reality Simulator) system's capacity to induce specific alterations in healthy individuals' oculomotor systems under simulated low-vision conditions. We examined task performance, eye movements, and head movements in healthy participants across various simulated scenarios. Our findings suggest that REALTER can effectively elicit behaviors in healthy individuals resembling those observed in individuals with low vision. Participants with simulated binocular maculopathy demonstrated unstable fixations and a high frequency of wide saccades. Individuals with simulated homonymous hemianopsia showed a tendency to maintain a fixed head position while executing wide saccades to survey their surroundings. Simulation of tubular vision resulted in a significant reduction in saccade amplitudes. REALTER holds promise as both a training tool for ophthalmologists and a research instrument for studying low vision conditions. The simulator has the potential to enhance ophthalmologists' comprehension of the limitations imposed by visual disabilities, thereby facilitating the development of new rehabilitation protocols.

Diagnosing and tracking depression based on eye movement in response to virtual reality

Introduction

Depression is a prevalent mental illness that is primarily diagnosed using psychological and behavioral assessments. However, these assessments lack objective and quantitative indices, making rapid and objective detection challenging. In this study, we propose a novel method for depression detection based on eye movement data captured in response to virtual reality (VR).

Methods

Eye movement data was collected and used to establish high-performance classification and prediction models. Four machine learning algorithms, namely eXtreme Gradient Boosting (XGBoost), multilayer perceptron (MLP), Support Vector Machine (SVM), and Random Forest, were employed. The models were evaluated using five-fold cross-validation, and performance metrics including accuracy, precision, recall, area under the curve (AUC), and F1-score were assessed. The predicted error for the Patient Health Questionnaire-9 (PHQ-9) score was also determined.

Results

The XGBoost model achieved a mean accuracy of 76%, precision of 94%, recall of 73%, and AUC of 82%, with an F1-score of 78%. The MLP model achieved a classification accuracy of 86%, precision of 96%, recall of 91%, and AUC of 86%, with an F1-score of 92%. The predicted error for the PHQ-9 score ranged from -0.6 to 0.6.To investigate the role of computerized cognitive behavioral therapy (CCBT) in treating depression, participants were divided into intervention and control groups. The intervention group received CCBT, while the control group received no treatment. After five CCBT sessions, significant changes were observed in the eye movement indices of fixation and saccade, as well as in the PHQ-9 scores. These two indices played significant roles in the predictive model, indicating their potential as biomarkers for detecting depression symptoms.

Discussion

The results suggest that eye movement indices obtained using a VR eye tracker can serve as useful biomarkers for detecting depression symptoms. Specifically, the fixation and saccade indices showed promise in predicting depression. Furthermore, CCBT demonstrated effectiveness in treating depression, as evidenced by the observed changes in eye movement indices and PHQ-9 scores. In conclusion, this study presents a novel approach for depression detection using eye movement data captured in VR. The findings highlight the potential of eye movement indices as biomarkers and underscore the effectiveness of CCBT in treating depression.

Implication of Corneal Refractive Surgery in Duchenne Muscular Dystrophy

Duchenne muscular dystrophy (DMD) is an X-linked disorder due to a dystrophin mutation and is the leading cause of muscular dystrophy. DMD presents with characteristic systemic effects, including severe muscular atrophy, cardiomyopathy, and ocular manifestations. Performing corneal refractive surgeries in patients with DMD raises concerns regarding patient positioning, risk of cataracts, and other comorbid conditions. Published reports of photorefractive keratectomy, laser-assisted in situ keratomileuses, and small incision lenticule extraction are lacking in this population. Here, we discuss a patient being evaluated for a corneal refractive surgery. This article also discusses the current understanding of DMD, known ocular manifestations, and factors to consider when evaluating a patient for potential corrective vision laser surgery.

Assessment of oculomotor function after prolonged computer use

To analyze the specific effects of prolonged computer use on oculomotor function, we propose an oculomotor function evaluation system to analyze changes in oculomotor movement function by using an eye tracker to record eye movement data when performing gaze, smooth pursuit, and saccade under normal condition, after one hour and one and a half hours of continuous working at a computer. The captured eye movement data is pre-processed, and then data features are calculated and analyzed to understand the specific effects of continuously using the computer on the oculomotor function. The results show that the oculomotor function decreases as we gaze at the computer screen for longer periods, as evidenced by a decrease in the stability of the gaze function, a reduction in the gaze focus, a reduction in the speed of eye saccades, and a decrease in the smooth pursuit function. In short, the oculomotor function worsens after prolonged working at a computer. This paper presents the effects of continuously using the computer quantificationally for the first time. The proposed oculomotor function evaluation system could also be used to assess patients who have a disability in oculomotor function and specific individuals, e.g. pilots.

Virtual reality for assessing stereopsis performance and eye characteristics in Post-COVID

In 2019, we faced a pandemic due to the coronavirus disease (COVID-19), with millions of confirmed cases and reported deaths. Even in recovered patients, symptoms can be persistent over weeks, termed Post-COVID. In addition to common symptoms of fatigue, muscle weakness, and cognitive impairments, visual impairments have been reported. Automatic classification of COVID and Post-COVID is researched based on blood samples and radiation-based procedures, among others. However, a symptom-oriented assessment for visual impairments is still missing. Thus, we propose a Virtual Reality environment in which stereoscopic stimuli are displayed to test the patient's stereopsis performance. While performing the visual tasks, the eyes' gaze and pupil diameter are recorded. We collected data from 15 controls and 20 Post-COVID patients in a study. Therefrom, we extracted features of three main data groups, stereopsis performance, pupil diameter, and gaze behavior, and trained various classifiers. The Random Forest classifier achieved the best result with 71% accuracy. The recorded data support the classification result showing worse stereopsis performance and eye movement alterations in Post-COVID. There are limitations in the study design, comprising a small sample size and the use of an eye tracking system.

Targeting displacement as an indicator of binocular fixation in normal Chinese adults

Purpose

The stability of fixation is crucial for the development of visual function. In this study, we quantify the deviation of visual target during fixational and saccadic tasks using eye-tracking technology, reflecting the control ability and characteristics of fixational displacement among healthy adults in a convenient method.

Methods

One hundred healthy participants aged between 18 and 55 years were recruited in the study. All participants underwent a complete ophthalmic assessment. The eye positions in the fixational and saccadic tasks were documented and analyzed by the Tobii eye-tracking system. Participants were grouped by age and gender. Targeting displacement (TD), defined as the average displacement between visual target and the mean of fixation points corresponding to that stimuli, was used to quantitatively observe fixational displacement in the horizontal and vertical directions.

Result

There was a strong reproducibility of TD as an indicator of fixation (ICC 0.812 to 0.891, p < 0.001). The TD in fixational task was significantly smaller than that of the saccadic task (3.884 ± 0.525 vs. 4.484 ± 0.509, p < 0.001) among normal people. Moreover, the difference of TD in the horizontal and vertical meridians was related to the nature of the task: In the fixational task, the TD in horizontal was smaller than that in the vertical (p < 0.001), whereas the TD in horizontal was larger than that in vertical in the saccadic task (p = 0.003). In the different age and gender groups: There was no significant difference between different gender and age groups in fixational task. However, during the saccadic task, males had smaller TD in the vertical direction than females (4.061 ± 0.495 vs. 4.404 ± 0.484, p = 0.002), and the average TD increased with age, mainly in the vertical direction (all p < 0.05). The fixation stability decreased significantly in the group over 50-years-old.

Conclusion

By reporting the fixational displacement of different genders and ages in fixational and saccadic tasks, as well as different longitude lines among normal people, our study might provide an objective, quantitative and convenient reference index for the evaluation of fixation stability in visual impairment diseases and aging phenomenon of visual function.

Effects of different virtual reality technology driven dual-tasking paradigms on posture and saccadic eye movements in healthy older adults

Postural sway and eye movements are potential biomarkers for dementia screening. Assessing the two movements comprehensively could improve the understanding of complicated syndrome for more accurate screening. The purpose of this research is to evaluate the effects of comprehensive assessment in healthy older adults (OA), using a novel concurrent comprehensive assessment system consisting of stabilometer and virtual reality headset. 20 healthy OA (70.4 ± 4.9 years) were recruited. Using a cross-sectional study design, this study investigated the effects of various dual-tasking paradigms with integrated tasks of visuospatial memory (VM), spatial orientation (SO), and visual challenge on posture and saccades. Dual-task paradigms with VM and SO affected the saccadic eye movements significantly. Two highly intensive tests of anti-saccade with VM task and pro-saccade with SO task also influenced postural sway significantly. Strong associations were seen between postural sway and eye movements for the conditions where the two movements theoretically shared common neural pathways in the brain, and vice versa. This study suggests that assessing posture and saccades with the integrated tasks comprehensively and simultaneously could be useful to explain different functions of the brain. The results warrant a cross-sectional study in OA with and without dementia to explore differences between these groups.

Biosensors toward behavior detection in diagnosis of alzheimer’s disease

In recent years, a huge number of individuals all over the world, elderly people, in particular, have been suffering from Alzheimer’s disease (AD), which has had a significant negative impact on their quality of life. To intervene early in the progression of the disease, accurate, convenient, and low-cost detection technologies are gaining increased attention. As a result of their multiple merits in the detection and assessment of AD, biosensors are being frequently utilized in this field. Behavioral detection is a prospective way to diagnose AD at an early stage, which is a more objective and quantitative approach than conventional neuropsychological scales. Furthermore, it provides a safer and more comfortable environment than those invasive methods (such as blood and cerebrospinal fluid tests) and is more economical than neuroimaging tests. Behavior detection is gaining increasing attention in AD diagnosis. In this review, cutting-edge biosensor-based devices for AD diagnosis together with their measurement parameters and diagnostic effectiveness have been discussed in four application subtopics: body movement behavior detection, eye movement behavior detection, speech behavior detection, and multi-behavior detection. Finally, the characteristics of behavior detection sensors in various application scenarios are summarized and the prospects of their application in AD diagnostics are presented as well.

Context matters during pick-and-place in VR: Impact on search and transport phases

When considering external assistive systems for people with motor impairments, gaze has been shown to be a powerful tool as it is anticipatory to motor actions and is promising for understanding intentions of an individual even before the action. Up until now, the vast majority of studies investigating the coordinated eye and hand movement in a grasping task focused on single objects manipulation without placing them in a meaningful scene. Very little is known about the impact of the scene context on how we manipulate objects in an interactive task. In the present study, it was investigated how the scene context affects human object manipulation in a pick-and-place task in a realistic scenario implemented in VR. During the experiment, participants were instructed to find the target object in a room, pick it up, and transport it to a predefined final location. Thereafter, the impact of the scene context on different stages of the task was examined using head and hand movement, as well as eye tracking. As the main result, the scene context had a significant effect on the search and transport phases, but not on the reach phase of the task. The present work provides insights into the development of potential supporting intention predicting systems, revealing the dynamics of the pick-and-place task behavior once it is realized in a realistic context-rich scenario.

Linking cognitive functioning and postural balance control through virtual reality environmental manipulations

Background

Dementia is becoming a relevant problem worldwide. A simple screening at an early stage will be important to detect the risk of developing dementia. Vestibular dysfunction is likely to be associated with cognitive impairment. Since head-mounted display (HMD) virtual reality (VR) technology has the potential to activate the vestibular function, assessing postural sway with visual stimulation using HMD VR technology could be potentially useful for dementia screening.

Objective

The purpose of this study is to evaluate the effect of HMD-based VR visual stimuli on posture in older adults and the relationship between the stimulated body sway behaviors and cognitive performance.

Method

Using a cross-sectional study design, we investigated the effect of an optokinetic design-based room with stripes (OKR) VR environment oscillating forwards and backwards at 23/60Hz. Center of pressure (COP) displacement was measured in older adults aged 65 years and over in the OKR VR environment. The frequency response of COP was compared to the cognitive performance of the Montreal Cognitive Assessment (MoCA).

Results

20 healthy older adults (70.4 ± 4.9 years; 27.2 ± 1.6 MoCA score) and 3 people with mild cognitive impairment (74.7 ± 4.0 years; 20.3 ± 2.1 MoCA score) were assessed. The results reveal that the oscillating OKR VR environment induced different postural sway in the anterior-posterior direction in the real world. Correlation analysis shows that the cognitive test score was associated with the frequency response of stimulated postural sway in the anterior-posterior direction (frequency Band 1 of 0−0.5Hz related to the visual and vestibular systems: r_s = 0.45, P = 0.03).

Conclusion

Outcomes would suggest that a potential link may emerge between cognition and posture when the HMD-based VR visual stimuli are applied. The simple screening of stimulated postural sway could explain cognitive functioning. Further studies are warranted to clarify the vestibular system and spatial cognitive function more specifically in the proposed assessment system.

Assessment methods for determining small changes in hearing performance over time

Although the behavioral pure-tone threshold audiogram is considered the gold standard for quantifying hearing loss, assessment of speech understanding, especially in noise, is more relevant to quality of life but is only partly related to the audiogram. Metrics of speech understanding in noise are therefore an attractive target for assessing hearing over time. However, speech-in-noise assessments have more potential sources of variability than pure-tone threshold measures, making it a challenge to obtain results reliable enough to detect small changes in performance. This review examines the benefits and limitations of speech-understanding metrics and their application to longitudinal hearing assessment, and identifies potential sources of variability, including learning effects, differences in item difficulty, and between- and within-individual variations in effort and motivation. We conclude by recommending the integration of non-speech auditory tests, which provide information about aspects of auditory health that have reduced variability and fewer central influences than speech tests, in parallel with the traditional audiogram and speech-based assessments.

A survey of challenges and methods for Quality of Experience assessment of interactive VR applications

User acceptance of virtual reality (VR) applications is dependent on multiple aspects, such as usability, enjoyment, and cybersickness. To fully realize the disruptive potential of VR technology in light of recent technological advancements (e.g., advanced headsets, immersive graphics), gaining a deeper understanding of underlying factors and dimensions impacting and contributing to the overall end-user experience is of great benefit to hardware manufacturers, software and content developers, and service providers. To provide insight into user behaviour and preferences, researchers conduct user studies exploring the influence of various user-, system-, and context-related factors on the overall Quality of Experience (QoE) and its dimensions. When planning and executing such studies, researchers are faced with numerous methodological challenges related to study design aspects, such as specification of dependant and independent variables, subjective and objective assessment methods, preparation of test materials, test environment, and participant recruitment. Approaching these challenges from a multidisciplinary perspective, this paper reviews different aspects of performing perception-based QoE assessment for interactive VR applications and presents options and recommendations for research methodology design. We provide an overview of different influence factors and dimensions that may affect the overall QoE, with a focus on presence, immersion, and discomfort. Furthermore, we address ethical and practical issues regarding participant choice and test material, present different assessment methods and measures commonly used in VR research, and discuss approaches to choosing study duration and location. Lastly, we provide a concise analysis of key challenges that need to be addressed in future studies centered around VR QoE.

Anti-saccade as a Tool to Evaluate Neurocognitive Impairment in Alcohol Use Disorder

It has been widely shown that chronic alcohol use leads to cognitive dysfunctions, especially inhibitory control. In an extension of the traditional approach, this research field has benefited from the emergence of innovative measures, among which is an anti-saccade, allowing direct and sensitive measure of the eye movements indexing attention bias to alcohol-related cues and the capability of inhibiting the reflexive saccades to the cues. During the past decade, there are numerous reports showing that drinkers make more unwanted reflexive saccades and longer latency in the anti-saccade task. These increased errors are usually explained by the deficits in inhibitory control. It has been demonstrated that inhibitory control on eye movement may be one of the earliest biomarkers of the onset of alcohol-related cognitive impairments. This review summarizes how an anti-saccade task can be used as a tool to investigate and assess the cognitive dysfunctions and the early detection of relapsing risk of alcohol dependence.

How Free-Viewing Eye Movements Can Be Used to Detect the Presence of Visual Field Defects in Glaucoma Patients

Purpose: There is a need for more intuitive perimetric screening methods, which can also be performed by elderly people and children currently unable to perform standard automated perimetry (SAP). Ideally, these methods should also be easier to administer, such that they may be used outside of a regular clinical environment. We evaluated the suitability of various methodological and analytical approaches for detecting and localizing VFD in glaucoma patients, based on eye movement recordings. Methods: The present study consisted of two experiments. In experiment 1, we collected data from 20 glaucoma patients and 20 age-matched controls, who monocularly viewed 28 1-min video clips while their eyes were being tracked. In experiment 2, we re-analyzed a published dataset, that contained data of 44 glaucoma patients and 32 age-matched controls who had binocularly viewed three longer-duration (3, 5, and 7 min) video clips. For both experiments, we first examined if the two groups differed in the basic properties of their fixations and saccades. In addition, we computed the viewing priority (VP) of each participant. Following a previously reported approach, for each participant, we mapped their fixation locations and used kernel Principal Component Analysis (kPCA) to distinguish patients from controls. Finally, we attempted to reconstruct the location of a patient's VFD by mapping the relative fixation frequency and the VP across their visual field. Results: We found direction dependent saccade amplitudes in glaucoma patients that often differed from those of the controls. Moreover, the kPCA indicated that the fixation maps of the two groups separated into two clusters based on the first two principal components. On average, glaucoma patients had a significantly lower VP than the controls, with this decrease depending on the specific video viewed. Conclusions: It is possible to detect the presence of VFD in glaucoma patients based on their gaze behavior made during video viewing. While this corroborates earlier conclusions, we show that it requires participants to view the videos monocularly. Nevertheless, we could not reconstruct the VFD with any of the evaluated methods, possibly due to compensatory eye movements made by the glaucoma patients.

Sensorimotor serial dependencies in head movements

On average, we redirect our gaze with a frequency at about 3 Hz. In real life, gaze shifts consist of eye and head movements. Much research has focused on how the accuracy of eye movements is monitored and calibrated. By contrast, little is known about how head movements remain accurate. I wondered whether serial dependencies between artificially induced errors in head movement targeting and the immediately following head movement might recalibrate movement accuracy. I also asked whether head movement targeting errors would influence visual localization. To this end, participants wore a head mounted display and performed head movements to targets, which were displaced as soon as the start of the head movement was detected. I found that target displacements influenced head movement amplitudes in the same trial, indicating that participants could adjust their movement online to reach the new target location. However, I also found serial dependencies between the target displacement in trial n-1 and head movements amplitudes in the following trial n. I did not find serial dependencies between target displacements and visuomotor localization. The results reveal that serial dependencies recalibrate head movement accuracy.

Saliency-Aware Subtle Augmentation Improves Human Visual Search Performance in VR

Visual search becomes challenging when the time to find the target is limited. Here we focus on how performance in visual search can be improved via a subtle saliency-aware modulation of the scene. Specifically, we investigate whether blurring salient regions of the scene can improve participant's ability to find the target faster when the target is located in non-salient areas. A set of real-world omnidirectional images were displayed in virtual reality with a search target overlaid on the visual scene at a pseudorandom location. Participants performed a visual search task in three conditions defined by blur strength, where the task was to find the target as fast as possible. The mean search time, and the proportion of trials where participants failed to find the target, were compared across different conditions. Furthermore, the number and duration of fixations were evaluated. A significant effect of blur on behavioral and fixation metrics was found using linear mixed models. This study shows that it is possible to improve the performance by a saliency-aware subtle scene modulation in a challenging realistic visual search scenario. The current work provides an insight into potential visual augmentation designs aiming to improve user's performance in everyday visual search tasks.