Seeing other perspectives: evaluating the use of virtual and augmented reality to simulate visual impairments (OpenVisSim)

Assessing REALTER simulator: analysis of ocular movements in simulated low-vision conditions with extended reality technology

Immersive technology, such as extended reality, holds promise as a tool for educating ophthalmologists about the effects of low vision and for enhancing visual rehabilitation protocols. However, immersive simulators have not been evaluated for their ability to induce changes in the oculomotor system, which is crucial for understanding the visual experiences of visually impaired individuals. This study aimed to assess the REALTER (Wearable Egocentric Altered Reality Simulator) system's capacity to induce specific alterations in healthy individuals' oculomotor systems under simulated low-vision conditions. We examined task performance, eye movements, and head movements in healthy participants across various simulated scenarios. Our findings suggest that REALTER can effectively elicit behaviors in healthy individuals resembling those observed in individuals with low vision. Participants with simulated binocular maculopathy demonstrated unstable fixations and a high frequency of wide saccades. Individuals with simulated homonymous hemianopsia showed a tendency to maintain a fixed head position while executing wide saccades to survey their surroundings. Simulation of tubular vision resulted in a significant reduction in saccade amplitudes. REALTER holds promise as both a training tool for ophthalmologists and a research instrument for studying low vision conditions. The simulator has the potential to enhance ophthalmologists' comprehension of the limitations imposed by visual disabilities, thereby facilitating the development of new rehabilitation protocols.

Advanced Visualization Engineering for Vision Disorders: A Clinically Focused Guide to Current Technology and Future Applications

Head-mounted visualization technology, often in the form of virtual, augmented, and mixed reality (VAMR), has revolutionized how visual disorders may be approached clinically. In this manuscript, we review the available literature on VAMR for visual disorders and provide a clinically oriented guide to how VAMR technology has been deployed for visual impairments. The chief areas of clinical investigation with VAMR are divided include (1) vision assessment, (2) vision simulation, and (3) vision rehabilitation. We discuss in-depth the current literature of these areas in VAMR and upcoming/future applications to combat the detrimental impact of visual impairment worldwide.

The Invisibility of Scotomas I: The Carving Hypothesis

SIGNIFICANCE

Veridical depictions of scene appearance with scotomas allow better understanding of the impact of field loss and may improve the development and implementation of rehabilitation. Explanation and depiction of the invisibility of scotoma may lead to patients' understanding and thus better compliance with related treatments.

PURPOSE

Simulations of perception with scotomas guide training, patient education, and rehabilitation research. Most simulations incorrectly depict scotomas as black patches, although the scotomas and the missing contents are usually invisible to patients. We present a novel approach to capture the reported appearance of scenes with scotomas.

METHODS

We applied a content-aware image resizing algorithm to carve out the content elided under the scotomas. With video sequences, we show how and why eye movements fail to increase the visibility of the carved scotomas.

RESULTS

Numerous effects, reported by patients, emerge naturally from the scotoma carving. Carving-eliminated scotomas over natural images are barely visible, despite causing substantial distortions. Low resolution and contrast sensitivity at farther eccentricities and saccadic blur reduce the visibility of the distortions. In a walking scenario, static objects moving smoothly to the periphery disappear into and then reemerge out of peripheral scotomas, invisibly.

CONCLUSIONS

Scotoma carving provides a viable hypothetical simulation of vision with scotomas due to loss of neurons at the retinal ganglion cell level and higher. As a hypothesis, it generates predictions that lend themselves to future clinical testing. The different effects of scotomas due to loss of photoreceptors are left for follow-up work.

Virtual reality as a means to explore assistive technologies for the visually impaired

Visual impairment represents a significant health and economic burden affecting 596 million globally. The incidence of visual impairment is expected to double by 2050 as our population ages. Independent navigation is challenging for persons with visual impairment, as they often rely on non-visual sensory signals to find the optimal route. In this context, electronic travel aids are promising solutions that can be used for obstacle detection and/or route guidance. However, electronic travel aids have limitations such as low uptake and limited training that restrict their widespread use. Here, we present a virtual reality platform for testing, refining, and training with electronic travel aids. We demonstrate the viability on an electronic travel aid developed in-house, consist of a wearable haptic feedback device. We designed an experiment in which participants donned the electronic travel aid and performed a virtual task while experiencing a simulation of three different visual impairments: age-related macular degeneration, diabetic retinopathy, and glaucoma. Our experiments indicate that our electronic travel aid significantly improves the completion time for all the three visual impairments and reduces the number of collisions for diabetic retinopathy and glaucoma. Overall, the combination of virtual reality and electronic travel aid may have a beneficial role on mobility rehabilitation of persons with visual impairment, by allowing early-phase testing of electronic travel aid prototypes in safe, realistic, and controllable settings.

A systematic review of extended reality (XR) for understanding and augmenting vision loss

Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. A defining quality of these XR technologies is their ability to update the stimulus based on the user's eye, head, or body movements. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. Here we present a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology to further visual accessibility. In contrast to other reviews, we sample studies from multiple scientific disciplines, focus on technology that augments a person's residual vision, and require studies to feature a quantitative evaluation with appropriate end users. We summarize prominent findings from different XR research areas, show how the landscape has changed over the past decade, and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the usability of different XR-based accessibility aids.

A Review of Design and Evaluation Practices in Mobile Text Entry for Visually Impaired and Blind Persons

Millions of people with vision impairment or vision loss face considerable barriers in using mobile technology and services due to the difficulty of text entry. In this paper, we review related studies involving the design and evaluation of novel prototypes for mobile text entry for persons with vision loss or impairment. We identify the practices and standards of the research community and compare them against the practices in research for non-impaired persons. We find that there are significant shortcomings in the methodological and result-reporting practices in both population types. In highlighting these issues, we hope to inspire more and better quality research in the domain of mobile text entry for persons with and without vision impairment.

Metaverse and Virtual Health Care in Ophthalmology: Opportunities and Challenges

ABSTRACT

The outbreak of the coronavirus disease 2019 has further increased the urgent need for digital transformation within the health care settings, with the use of artificial intelligence/deep learning, internet of things, telecommunication network/virtual platform, and blockchain. The recent advent of metaverse, an interconnected online universe, with the synergistic combination of augmented, virtual, and mixed reality described several years ago, presents a new era of immersive and real-time experiences to enhance human-to-human social interaction and connection. In health care and ophthalmology, the creation of virtual environment with three-dimensional (3D) space and avatar, could be particularly useful in patient-fronting platforms (eg, telemedicine platforms), operational uses (eg, meeting organization), digital education (eg, simulated medical and surgical education), diagnostics, and therapeutics. On the other hand, the implementation and adoption of these emerging virtual health care technologies will require multipronged approaches to ensure interoperability with real-world virtual clinical settings, user-friendliness of the technologies and clinical efficiencies while complying to the clinical, health economics, regulatory, and cybersecurity standards. To serve the urgent need, it is important for the eye community to continue to innovate, invent, adapt, and harness the unique abilities of virtual health care technology to provide better eye care worldwide.

Augmented Reality in Ophthalmology: Applications and Challenges

Augmented reality (AR) has been developed rapidly and implemented in many fields such as medicine, maintenance, and cultural heritage. Unlike other specialties, ophthalmology connects closely with AR since most AR systems are based on vision systems. Here we summarize the applications and challenges of AR in ophthalmology and provide insights for further research. Firstly, we illustrate the structure of the standard AR system and present essential hardware. Secondly, we systematically introduce applications of AR in ophthalmology, including therapy, education, and clinical assistance. To conclude, there is still a large room for development, which needs researchers to pay more effort. Applications in diagnosis and protection might be worth exploring. Although the obstacles of hardware restrict the development of AR in ophthalmology at present, the AR will realize its potential and play an important role in ophthalmology in the future with the rapidly developing technology and more in-depth research.

How wide should you view to fight? Establishing the size of the visual field necessary for grip fighting in judo

Peripheral vision is often considered vital in (combat) sports, yet most experimental paradigms (e.g., eye tracking) ignore peripheral information or struggle to make inferences about the role of peripheral vision in an in-situ performance environment. This study aimed to determine where visual information is located in the peripheral field during an in-situ combat sports task. Eight advanced judokas competed in grip-fighting exchanges while wearing a mobile eye-tracker to locate gaze direction. Three-dimensional position data of the head and hands were tracked using a VICON motion capture system. Gaze analysis through automatic feature detection showed that participants predominantly fixated on their opponent's chest. Kinematic data were used to calculate the angles between the opponent's hands and the gaze-anchor point on the chest of the opponent. Results revealed a nonlinear relationship between visual field (VF) size and visibility of the hands, with athletes needing a VF of at least 30-40 degrees radius to simultaneously monitor both hands of the opponent most of the time. These findings hold implications for the regulation of Paralympic judo for athletes with vision impairment, suggesting that a less severe degree of impairment should be required to qualify than the current criterion of 20 degrees radius.

Simulating Macular Degeneration to Investigate Activities of Daily Living: A Systematic Review

Purpose: Investigating difficulties during activities of daily living is a fundamental first step for the development of vision-related intervention and rehabilitation strategies. One way to do this is through visual impairment simulations. The aim of this review is to synthesize and assess the types of simulation methods that have been used to simulate age-related macular degeneration (AMD) in normally sighted participants, during activities of daily living (e.g., reading, cleaning, and cooking).

Methods: We conducted a systematic literature search in five databases and a critical analysis of the advantages and disadvantages of various AMD simulation methods (following PRISMA guidelines). The review focuses on the suitability of each method for investigating activities of daily living, an assessment of clinical validation procedures, and an evaluation of the adaptation periods for participants.

Results: Nineteen studies met the criteria for inclusion. Contact lenses, computer manipulations, gaze contingent displays, and simulation glasses were the main forms of AMD simulation identified. The use of validation and adaptation procedures were reported in approximately two-thirds and half of studies, respectively.

Conclusions: Synthesis of the methodology demonstrated that the choice of simulation has been, and should continue to be, guided by the nature of the study. While simulations may never completely replicate vision loss experienced during AMD, consistency in simulation methodology is critical for generating realistic behavioral responses under vision impairment simulation and limiting the influence of confounding factors. Researchers could also come to a consensus regarding the length and form of adaptation by exploring what is an adequate amount of time and type of training required to acclimatize participants to vision impairment simulations.

Applications of Extended Reality in Ophthalmology: Systematic Review

BACKGROUND

Virtual reality, augmented reality, and mixed reality make use of a variety of different software and hardware, but they share three main characteristics: immersion, presence, and interaction. The umbrella term for technologies with these characteristics is extended reality. The ability of extended reality to create environments that are otherwise impossible in the real world has practical implications in the medical discipline. In ophthalmology, virtual reality simulators have become increasingly popular as tools for surgical education. Recent developments have also explored diagnostic and therapeutic uses in ophthalmology.

OBJECTIVE

This systematic review aims to identify and investigate the utility of extended reality in ophthalmic education, diagnostics, and therapeutics.

METHODS

A literature search was conducted using PubMed, Embase, and Cochrane Register of Controlled Trials. Publications from January 1, 1956 to April 15, 2020 were included. Inclusion criteria were studies evaluating the use of extended reality in ophthalmic education, diagnostics, and therapeutics. Eligible studies were evaluated using the Oxford Centre for Evidence-Based Medicine levels of evidence. Relevant studies were also evaluated using a validity framework. Findings and relevant data from the studies were extracted, evaluated, and compared to determine the utility of extended reality in ophthalmology.

RESULTS

We identified 12,490 unique records in our literature search; 87 met final eligibility criteria, comprising studies that evaluated the use of extended reality in education (n=54), diagnostics (n=5), and therapeutics (n=28). Of these, 79 studies (91%) achieved evidence levels in the range 2b to 4, indicating poor quality. Only 2 (9%) out of 22 relevant studies addressed all 5 sources of validity evidence. In education, we found that ophthalmic surgical simulators demonstrated efficacy and validity in improving surgical performance and reducing complication rates. Ophthalmoscopy simulators demonstrated efficacy and validity evidence in improving ophthalmoscopy skills in the clinical setting. In diagnostics, studies demonstrated proof-of-concept in presenting ocular imaging data on extended reality platforms and validity in assessing the function of patients with ophthalmic diseases. In therapeutics, heads-up surgical systems had similar complication rates, procedural success rates, and outcomes in comparison with conventional ophthalmic surgery.

CONCLUSIONS

Extended reality has promising areas of application in ophthalmology, but additional high-quality comparative studies are needed to assess their roles among incumbent methods of ophthalmic education, diagnostics, and therapeutics.

Simulating Visibility and Reading Performance in Low Vision

PURPOSE

Low vision reduces text visibility and causes difficulties in reading. A valid low-vision simulation could be used to evaluate the accessibility of digital text for readers with low vision. We examined the validity of a digital simulation for replicating the text visibility and reading performance of low-vision individuals.

METHODS

Low-vision visibility was modeled with contrast sensitivity functions (CSFs) with parameters to represent reduced acuity and contrast sensitivity. Digital filtering incorporating these CSFs were applied to digital versions of the Lighthouse Letter Acuity Chart and the Pelli-Robson Contrast Sensitivity Chart. Reading performance (reading acuity, critical print size, and maximum reading speed) was assessed with filtered versions of the MNREAD reading acuity Chart. Thirty-six normally sighted young adults completed chart testing under normal and simulated low-vision conditions. Fifty-eight low-vision subjects (thirty with macular pathology and twenty-eight with non-macular pathology) and fifteen normally sighted older subjects completed chart testing with their habitual viewing. We hypothesized that the performance of the normally sighted young adults under simulated low-vision conditions would match the corresponding performance of actual low-vision subjects.

RESULTS

When simulating low-vision conditions with visual acuity better than 1.50 logMAR (Snellen 20/630) and contrast sensitivity better than 0.15 log unit, the simulation adequately reduced the acuity and contrast sensitivity in normally sighted young subjects to the desired low-vision levels. When performing the MNREAD test with simulated low vision, the normally sighted young adults had faster maximum reading speed than both the Non-macular and Macular groups, by an average of 0.07 and 0.12 log word per minute, respectively. However, they adequately replicated the reading acuity as well as the critical print size, up to 2.00 logMAR of both low-vision groups.

CONCLUSION

A low-vision simulation based on clinical measures of visual acuity and contrast sensitivity can provide good estimates of reading performance and the accessibility of digital text for a broad range of low-vision conditions.

Gaze During Locomotion in Virtual Reality and the Real World

How vision guides gaze in realistic settings has been researched for decades. Human gaze behavior is typically measured in laboratory settings that are well controlled but feature-reduced and movement-constrained, in sharp contrast to real-life gaze control that combines eye, head, and body movements. Previous real-world research has shown environmental factors such as terrain difficulty to affect gaze; however, real-world settings are difficult to control or replicate. Virtual reality (VR) offers the experimental control of a laboratory, yet approximates freedom and visual complexity of the real world (RW). We measured gaze data in 8 healthy young adults during walking in the RW and simulated locomotion in VR. Participants walked along a pre-defined path inside an office building, which included different terrains such as long corridors and flights of stairs. In VR, participants followed the same path in a detailed virtual reconstruction of the building. We devised a novel hybrid control strategy for movement in VR: participants did not actually translate: forward movements were controlled by a hand-held device, rotational movements were executed physically and transferred to the VR. We found significant effects of terrain type (flat corridor, staircase up, and staircase down) on gaze direction, on the spatial spread of gaze direction, and on the angular distribution of gaze-direction changes. The factor world (RW and VR) affected the angular distribution of gaze-direction changes, saccade frequency, and head-centered vertical gaze direction. The latter effect vanished when referencing gaze to a world-fixed coordinate system, and was likely due to specifics of headset placement, which cannot confound any other analyzed measure. Importantly, we did not observe a significant interaction between the factors world and terrain for any of the tested measures. This indicates that differences between terrain types are not modulated by the world. The overall dwell time on navigational markers did not differ between worlds. The similar dependence of gaze behavior on terrain in the RW and in VR indicates that our VR captures real-world constraints remarkably well. High-fidelity VR combined with naturalistic movement control therefore has the potential to narrow the gap between the experimental control of a lab and ecologically valid settings.

Virtual Reality and Augmented Reality in Ophthalmology: A Contemporary Prospective

PURPOSE

Most published systematic reviews have focused on the use of virtual reality (VR)/augmented reality (AR) technology in ophthalmology as it relates to surgical training. To date, this is the first review that investigates the current state of VR/AR technology applied more broadly to the entire field of ophthalmology.

METHODS

PubMed, Embase, and CINAHL databases were searched systematically from January 2014 through December 1, 2020. Studies that discussed VR and/or AR as it relates to the field of ophthalmology and provided information on the technology used were considered. Abstracts, non-peer-reviewed literature, review articles, studies that reported only qualitative data, and studies without English translations were excluded.

RESULTS

A total of 77 studies were included in this review. Of these, 28 evaluated the use of VR/AR in ophthalmic surgical training/assessment and guidance, 7 in clinical training, 23 in diagnosis/screening, and 19 in treatment/therapy. 15 studies used AR, 61 used VR, and 1 used both. Most studies focused on the validity and usability of novel technologies.

CONCLUSIONS

Ophthalmology is a field of medicine that is well suited for the use of VR/AR. However, further longitudinal studies examining the practical feasibility, efficacy, and safety of such novel technologies, the cost-effectiveness, and medical/legal considerations are still needed. We believe that time will indeed foster further technological advances and lead to widespread use of VR/AR in routine ophthalmic practice.

A Virtual Reality Orientation and Mobility Test for Inherited Retinal Degenerations: Testing a Proof-of-Concept After Gene Therapy

Purpose

To test the ability of a virtual reality (VR) orientation and mobility (O&M) protocol to serve a measure of functional vision for patients with inherited retinal degenerations (IRDs).

Methods

A VR-O&M protocol designed using a commercially available VR hardware was tested in normally sighted control subjects (n=7; ages 10–35yo; Average 22.5yo) and patients with RPE65-associated Leber Congenital Amaurosis (n=3; ages 7–18yo; Average 12.7yo), in two of them before and after gene therapy. Patients underwent perimetry and full-field sensitivity testing. VR-O&M parameters correlated with the visual dysfunction.

Results

Visual acuities in RPE65 patients were on average worse than 20/200, dark-adapted sensitivity losses >5 log units, and fields constricted between 20° and 40°. Before treatment, patients required ~1000-fold brighter environment to navigate, had at least x4 more collisions, and were slower both to orient and navigate compared to control subjects. Improvements in cone- (by 1–2 L.u.) and rod-mediated (by >4 L.u.) sensitivities post-treatment led to fewer collisions (at least by half) at ~100-fold dimmer luminances, and to x4 times faster navigation times.

Conclusion

This study provides proof-of-concept data in support for the use of VR-O&M systems to quantify the impact that the visual dysfunction and improvement of vision following treatments has on functional vision in IRDs. The VR-O&M was useful in potentially challenging scenarios such as in pediatric patients with severe IRDs.

Translational Relevance

A VR-O&M test will provide much needed flexibility, both in its deployment as well as in the possibility to test various attributes of vision that may be impacted by gene therapy in the setting of translational studies.

Precis

This study provides proof-of-concept data in support for the use of a virtual reality orientation and mobility test to quantify the impact of the disease and of treatments thereof on functional vision in inherited retinal degenerations.

Influence of Systematic Gaze Patterns in Navigation and Search Tasks with Simulated Retinitis Pigmentosa

People living with a degenerative retinal disease such as retinitis pigmentosa are oftentimes faced with difficulties navigating in crowded places and avoiding obstacles due to their severely limited field of view. The study aimed to assess the potential of different patterns of eye movement (scanning patterns) to (i) increase the effective area of perception of participants with simulated retinitis pigmentosa scotoma and (ii) maintain or improve performance in visual tasks. Using a virtual reality headset with eye tracking, we simulated tunnel vision of 20° in diameter in visually healthy participants (n = 9). Employing this setup, we investigated how different scanning patterns influence the dynamic field of view—the average area over time covered by the field of view—of the participants in an obstacle avoidance task and in a search task. One of the two tested scanning patterns showed a significant improvement in both dynamic field of view (navigation 11%, search 7%) and collision avoidance (33%) when compared to trials without the suggested scanning pattern. However, participants took significantly longer (31%) to finish the navigation task when applying this scanning pattern. No significant improvements in search task performance were found when applying scanning patterns.

Applications of augmented reality in ophthalmology [Invited]

Throughout the last decade, augmented reality (AR) head-mounted displays (HMDs) have gradually become a substantial part of modern life, with increasing applications ranging from gaming and driver assistance to medical training. Owing to the tremendous progress in miniaturized displays, cameras, and sensors, HMDs are now used for the diagnosis, treatment, and follow-up of several eye diseases. In this review, we discuss the current state-of-the-art as well as potential uses of AR in ophthalmology. This review includes the following topics: (i) underlying optical technologies, displays and trackers, holography, and adaptive optics; (ii) accommodation, 3D vision, and related problems such as presbyopia, amblyopia, strabismus, and refractive errors; (iii) AR technologies in lens and corneal disorders, in particular cataract and keratoconus; (iv) AR technologies in retinal disorders including age-related macular degeneration (AMD), glaucoma, color blindness, and vision simulators developed for other types of low-vision patients.

Assessing Functional Disability in Glaucoma: The Relative Importance of Central Versus Far Peripheral Visual Fields

Purpose

To evaluate the importance of central versus far peripheral visual field (VF) loss in assessing disability in glaucoma.

Methods

In total, 231 patients with glaucoma or suspected glaucoma completed 24-2 VF testing and automated peripheral VFs using the suprathreshold 30- to 60-degree pattern. Questionnaires assessed fear of falling (FoF), quality of life (QOL), instrumental activities of daily living (IADLs), and driving habits; nonsedentary time, reading speed, and gait were objectively measured. Multivariable regression models analyzed the effect of central VF and/or peripheral VF damage on each outcome.

Results

In models including both central and peripheral VF damage (independent effects), greater central, but not peripheral, VF damage was associated with greater FoF, worse QOL, fewer daily steps, and difficulty with IADLs (P < 0.02 for central; P > 0.5 for peripheral). For gait measures, greater peripheral, but not central, damage was associated with shorter steps and shorter strides, as well as greater variability in step length (P < 0.03 for peripheral; P > 0.14 for central). Model R² values were not substantially higher (less than 5% additional explained variability) for models including both central and peripheral VF damage as compared to the best models incorporating only one region of VF damage (i.e., central or peripheral).

Conclusions

The relative importance of central 24 degrees versus more peripheral VF damage differs across functional domains in patients with glaucoma. Central damage is more strongly associated with most disability outcomes, although peripheral damage is more associated with specific gait measures. Studies examining the relative importance of various VF regions should assess functional domain separately and eschew integrated measures of quality of life/activity limitation.

The worse eye revisited: Evaluating the impact of asymmetric peripheral vision loss on everyday function

In instances of asymmetric peripheral vision loss (e.g., glaucoma), binocular performance on simple psychophysical tasks (e.g., static threshold perimetry) is well-predicted by the better seeing eye alone. This suggests that peripheral vision is largely 'better-eye limited'. In the present study, we examine whether this also holds true for real-world tasks, or whether even a degraded fellow eye contributes important information for tasks of daily living. Twelve normally-sighted adults performed an everyday visually-guided action (finding a mobile phone) in a virtual-reality domestic environment, while levels of peripheral vision loss were independently manipulated in each eye (gaze-contingent blur). The results showed that even when vision in the better eye was held constant, participants were significantly slower to locate the target, and made significantly more head- and eye-movements, as peripheral vision loss in the worse eye increased. A purely unilateral peripheral impairment increased response times by up to 25%, although the effect of bilateral vision loss was much greater (>200%). These findings indicate that even a degraded visual field still contributes important information for performing everyday visually-guided actions. This may have clinical implications for how patients with visual field loss are managed or prioritized, and for our understanding of how binocular information in the periphery is integrated.