Using biomechanics to investigate the effect of VR on eye vergence system

The Virtual Reality Radiology Workstation: Current Technology and Future Applications

Virtual reality (VR) and augmented reality (AR) technology hold potential across many disciplines in medicine to expand the delivery of education and healthcare. VR-AR applications in radiology, in particular, have gained prominence and have demonstrated advantages in many areas within the field. Recently, VR software has emerged to redesign the physical radiology workstation (ie, reading room) to expand the possibilities of diagnostic interpretation. Given the novelty of this technology, there is limited research investigating the potential applications of a simulated radiology workstation. In this review article, we explore VR-simulated reading room technology in its current form and illustrate the practical applications this technology will bring to future radiologists and learners. We also discuss the limitations and barriers to adopting this technology that must be overcome to truly understand its potential benefits. VR reading room technology offers great potential in radiology, but further research is needed to appreciate its benefits and identify areas for improvement. The findings and insights presented in this review contribute to the ongoing discourse on future technological advancements in radiology and healthcare, offering valuable recommendations for further research and practical implementation.

Identification of Language-Induced Mental Load from Eye Behaviors in Virtual Reality

Experiences of virtual reality (VR) can easily break if the method of evaluating subjective user states is intrusive. Behavioral measures are increasingly used to avoid this problem. One such measure is eye tracking, which recently became more standard in VR and is often used for content-dependent analyses. This research is an endeavor to utilize content-independent eye metrics, such as pupil size and blinks, for identifying mental load in VR users. We generated mental load independently from visuals through auditory stimuli. We also defined and measured a new eye metric, focus offset, which seeks to measure the phenomenon of "staring into the distance" without focusing on a specific surface. In the experiment, VR-experienced participants listened to two native and two foreign language stimuli inside a virtual phone booth. The results show that with increasing mental load, relative pupil size on average increased 0.512 SDs (0.118 mm), with 57% reduced variance. To a lesser extent, mental load led to fewer fixations, less voluntary gazing at distracting content, and a larger focus offset as if looking through surfaces (about 0.343 SDs, 5.10 cm). These results are in agreement with previous studies. Overall, we encourage further research on content-independent eye metrics, and we hope that hardware and algorithms will be developed in the future to further increase tracking stability.

Design guidelines for limiting and eliminating virtual reality-induced symptoms and effects at work: a comprehensive, factor-oriented review

Virtual reality (VR) can induce side effects known as virtual reality-induced symptoms and effects (VRISE). To address this concern, we identify a literature-based listing of these factors thought to influence VRISE with a focus on office work use. Using those, we recommend guidelines for VRISE amelioration intended for virtual environment creators and users. We identify five VRISE risks, focusing on short-term symptoms with their short-term effects. Three overall factor categories are considered: individual, hardware, and software. Over 90 factors may influence VRISE frequency and severity. We identify guidelines for each factor to help reduce VR side effects. To better reflect our confidence in those guidelines, we graded each with a level of evidence rating. Common factors occasionally influence different forms of VRISE. This can lead to confusion in the literature. General guidelines for using VR at work involve worker adaptation, such as limiting immersion times to between 20 and 30 min. These regimens involve taking regular breaks. Extra care is required for workers with special needs, neurodiversity, and gerontechnological concerns. In addition to following our guidelines, stakeholders should be aware that current head-mounted displays and virtual environments can continue to induce VRISE. While no single existing method fully alleviates VRISE, workers' health and safety must be monitored and safeguarded when VR is used at work.

Eye movement characteristics and visual fatigue assessment of virtual reality games with different interaction modes

This study aimed to investigate the eye movement characteristics and visual fatigue of virtual reality games with different interaction modes. Eye movement data were recorded using the built-in eye tracker of the VR device and eye movement parameters were calculated from the recorded raw data. The Visual Fatigue Scales and Simulator Sickness Questionnaire were used to subjectively assess visual fatigue and overall discomfort of the VR experience. Sixteen male and 17 female students were recruited for this study. Results showed that both the primary and 360 mode of VR could cause visual fatigue after 30 min of gameplay, with significant differences observed in eye movement behavior between the two modes. The primary mode was more likely to cause visual fatigue, as shown by objective measurements of blinking and pupil diameter. Fixation and saccade parameters also showed significant differences between the two modes, possibly due to the different interaction modes employed in the 360 mode. Further research is required to examine the effects of different content and interactive modes of VR on visual fatigue, as well as to develop more objective measures for assessing it.

Effects of virtual target size, position, and parallax on vergence-accommodation conflict as estimated by actual gaze

Due to the increased utilization of stereoscopic displays, the scope of the vergence-accommodation conflict has been studied extensively to reveal how the human visual system operates. The purpose of this work was to study the phenomenon of vergence-accommodation conflict by comparing the theoretical eye vergence angle (vergence response) and gaze-based eye vergence angle (vergence stimulus) based on eye tracker gaze data. The results indicated that the gaze-based eye vergence angle was largest at the greatest parallax. The result also revealed that the eye vergence angle accuracy was significantly highest at the nearest parallax. Generally, accuracy improves when virtual objects are put in the middle and close to participants' positions. Moreover, the signed error decreases significantly when the virtual object is in the middle. Based on the results of this study, we can gain a greater understanding of the vergence-accommodation conflict in the stereoscopic environment.

Changes to Visual Parameters Following Virtual Reality Gameplay

Introduction:

Virtual reality (VR) gameplay is popular with a range of games and educational resources available. However, it puts high demands on the visual system. Current evidence shows conflicting impacts on visual parameters. Therefore, this study explores the changes to vision following VR gameplay.

Methods:

The study was conducted at the School of Health Sciences, University of Liverpool. All participants had binocular vision with good visual acuity and no manifest strabismus. Participants were assessed before and after playing 15 minutes of the VR game Beat Saber, which incorporated convergence and divergence movements. Clinical assessments including near point of convergence (NPC) and near point of accommodation (NPA) using the RAF rule; accommodative convergence to accommodation (AC/A) ratio; motor fusion using the prism fusion range (at 33cm), accommodation facility using +2.00/–2.00DS flipper lenses, and stereoacuity using the Frisby stereo test were assessed before and after playing.

Results:

Seventy-eight participants (19–25 years old) were included in the study, with 16 males and 41 females respectively. The breakpoint of convergence reduced by 0.5 cm (p = 0.001). The binocular accommodative facility improved by 2 cycles per minute (cpm); p = 0.004. The mean, near horizontal prism fusion range (PFR) base break and recovery points both worsened by of 5.0 dioptres (p = 0.003), whereas the mean near horizontal PFR base in recovery point improved by of 4.0 dioptres (p = 0.003).

Discussion:

The study validated previous findings as VR gameplay over-exercised and fatigued convergence muscles, but to a small degree. The VR experience improved the participants’ ability to change focus quickly and improve accommodation, as well as the divergence function of the eye. However, as the participants were retested directly after the VR gameplay, the findings were limited to short term effects on vision.

Variations in intraocular pressure and visual parameters before and after using mobile virtual reality glasses and their effects on the eyes

We examined the effects of using mobile devices with immersive virtual reality for a short period on the physiological parameters of both eyes. The average age of the 50 participants (23 men and 27 women) was 17.72 ± 1.48 years, and refractive error ranged from 0 D to − 5.00 D. All the participants wore + 3.00 D glasses and underwent a 5-min relaxation adjustment through the atomization method. The participants wore immersive virtual reality (VR) glasses to watch a movie on a roller coaster for 10 min. Their relevant physiological parameters of the eyes were measured both before and after using VR glasses. Compared with before VR use, no significant difference (P > 0.05) was observed in the near-horizontal vergence and refractive error but a significant difference (P < 0.05) was observed in the amplitude of accommodation, intraocular pressure, divergence/convergence, and stereopsis after VR use. The corneal elastic coefficient was > 0.2 MPa, and we used Friedenwald’s eye rigidity relationship to obtain the K value (0.065–0.09). Approximately 10% of the participants experienced cybersickness symptoms such as nausea and dizziness. The use of VR to watch three-dimensional movies reduced intraocular pressure, which may help prevent or treat glaucoma. Moreover, the binocular convergence was higher when viewing near-field objects in VR than in the real world. Therefore, individuals with convergence excess may experience symptoms. Binocular parallax is the most likely cause of cybersickness symptoms. Thus, mobile VR devices with higher quality and comfort are necessary.

Refined Continuous Control of DDPG Actors via Parametrised Activation

Continuous action spaces impose a serious challenge for reinforcement learning agents. While several off-policy reinforcement learning algorithms provide a universal solution to continuous control problems, the real challenge lies in the fact that different actuators feature different response functions due to wear and tear (in mechanical systems) and fatigue (in biomechanical systems). In this paper, we propose enhancing the actor-critic reinforcement learning agents by parameterising the final layer in the actor network. This layer produces the actions to accommodate the behaviour discrepancy of different actuators under different load conditions during interaction with the environment. To achieve this, the actor is trained to learn the tuning parameter controlling the activation layer (e.g., Tanh and Sigmoid). The learned parameters are then used to create tailored activation functions for each actuator. We ran experiments on three OpenAI Gym environments, i.e., Pendulum-v0, LunarLanderContinuous-v2, and BipedalWalker-v2. Results showed an average of 23.15% and 33.80% increase in total episode reward of the LunarLanderContinuous-v2 and BipedalWalker-v2 environments, respectively. There was no apparent improvement in Pendulum-v0 environment but the proposed method produces a more stable actuation signal compared to the state-of-the-art method. The proposed method allows the reinforcement learning actor to produce more robust actions that accommodate the discrepancy in the actuators’ response functions. This is particularly useful for real life scenarios where actuators exhibit different response functions depending on the load and the interaction with the environment. This also simplifies the transfer learning problem by fine-tuning the parameterised activation layers instead of retraining the entire policy every time an actuator is replaced. Finally, the proposed method would allow better accommodation to biological actuators (e.g., muscles) in biomechanical systems.

To measure the amount of ocular deviation in strabismus patients with an eye-tracking virtual reality headset

PURPOSE

To investigate the accuracy of a newly developed, eye-tracking virtual reality (VR)-based ocular deviation measurement system in strabismus patients.

METHODS

A VR-based ocular deviation measurement system was designed to simulate the alternative prism cover test (APCT). A fixation target was made to alternate between two screens, one in front of each eye, to simulate the steps of a normal prism cover test. Patient's eye movements were recorded by built-in eye tracking. The angle of ocular deviation was compared between the APCT and the VR-based system.

RESULTS

This study included 38 patients with strabismus. The angle of ocular deviation measured by the VR-based system and the APCT showed good to excellent correlation (intraclass correlation coefficient, ICC = 0.897 (range: 0.810-0.945)). The 95% limits of agreement was 11.32 PD. Subgroup analysis revealed a significant difference between esotropia and exotropia (p < 0.001). In the esotropia group, the amount of ocular deviation measured by the VR-based system was greater than that measured by the APCT (mean = 4.65 PD), while in the exotropia group, the amount of ocular deviation measured by the VR-based system was less than that of the APCT (mean = - 3.01 PD). The ICC was 0.962 (range: 0.902-0.986) in the esotropia group and 0.862 (range: 0.651-0.950) in the exotropia group. The 95% limits of agreement were 6.62 PD and 11.25 PD in the esotropia and exotropia groups, respectively.

CONCLUSIONS

This study reports the first application of a consumer-grade and commercial-grade VR-based device for assessing angle of ocular deviation in strabismus patients. This device could provide measurements with near excellent correlation with the APCT. The system also provides the first step to digitize the strabismus examination, as well as the possibility for its application in telemedicine.

Gaze During Locomotion in Virtual Reality and the Real World

How vision guides gaze in realistic settings has been researched for decades. Human gaze behavior is typically measured in laboratory settings that are well controlled but feature-reduced and movement-constrained, in sharp contrast to real-life gaze control that combines eye, head, and body movements. Previous real-world research has shown environmental factors such as terrain difficulty to affect gaze; however, real-world settings are difficult to control or replicate. Virtual reality (VR) offers the experimental control of a laboratory, yet approximates freedom and visual complexity of the real world (RW). We measured gaze data in 8 healthy young adults during walking in the RW and simulated locomotion in VR. Participants walked along a pre-defined path inside an office building, which included different terrains such as long corridors and flights of stairs. In VR, participants followed the same path in a detailed virtual reconstruction of the building. We devised a novel hybrid control strategy for movement in VR: participants did not actually translate: forward movements were controlled by a hand-held device, rotational movements were executed physically and transferred to the VR. We found significant effects of terrain type (flat corridor, staircase up, and staircase down) on gaze direction, on the spatial spread of gaze direction, and on the angular distribution of gaze-direction changes. The factor world (RW and VR) affected the angular distribution of gaze-direction changes, saccade frequency, and head-centered vertical gaze direction. The latter effect vanished when referencing gaze to a world-fixed coordinate system, and was likely due to specifics of headset placement, which cannot confound any other analyzed measure. Importantly, we did not observe a significant interaction between the factors world and terrain for any of the tested measures. This indicates that differences between terrain types are not modulated by the world. The overall dwell time on navigational markers did not differ between worlds. The similar dependence of gaze behavior on terrain in the RW and in VR indicates that our VR captures real-world constraints remarkably well. High-fidelity VR combined with naturalistic movement control therefore has the potential to narrow the gap between the experimental control of a lab and ecologically valid settings.

Just Don’t Fall: An AI Agent’s Learning Journey Towards Posture Stabilisation

Learning to maintain postural balance while standing requires a significant, fine coordination effort between the neuromuscular system and the sensory system. It is one of the key contributing factors towards fall prevention, especially in the older population. Using artificial intelligence (AI), we can similarly teach an agent to maintain a standing posture, and thus teach the agent not to fall. In this paper, we investigate the learning progress of an AI agent and how it maintains a stable standing posture through reinforcement learning. We used the Deep Deterministic Policy Gradient method (DDPG) and the OpenSim musculoskeletal simulation environment based on OpenAI Gym. During training, the AI agent learnt three policies. First, it learnt to maintain the Centre-of-Gravity and Zero-Moment-Point in front of the body. Then, it learnt to shift the load of the entire body on one leg while using the other leg for fine tuning the balancing action. Finally, it started to learn the coordination between the two pre-trained policies. This study shows the potentials of using deep reinforcement learning in human movement studies. The learnt AI behaviour also exhibited attempts to achieve an unplanned goal because it correlated with the set goal (e.g., walking in order to prevent falling). The failed attempts to maintain a standing posture is an interesting by-product which can enrich the fall detection and prevention research efforts.

Target Eccentricity and Form Influences Disparity Vergence Eye Movements Responses: A Temporal and Dynamic Analysis

This study sought to investigate whether stimulation to the fovea or the parafovea with different color combinations influenced the temporal and dynamic features of 4° disparity vergence step responses. Twelve unique types of stimuli were displayed within a haploscope presented along the participant's midsagittal plane. Vergence eye movement responses from fifteen naïve participants were recorded using video-based infrared eye tracking instrumentation. Latency and peak velocity from left and right eye movement responses were quantified. Results show that the type of stimulus projection (foveal versus parafoveal) significantly (p<0.001) influences the vergence response latency but did not impact peak velocity. Vergence responses to eccentric circles with 6° eccentricity targeting the parafovea resulted in a significantly faster response latency compared to vergence responses to a cross with 2° eccentricity stimuli targeting the fovea. Results have implications for the stimulus design of a variety of applications from virtual reality to vision therapy interventions.