Enhanced Depth Navigation Through Augmented Reality Depth Mapping in Patients with Low Vision

Efficacy of a wearable night-vision aid in patients with concentric peripheral visual field loss: a randomized, crossover trial

PURPOSE

To investigate the efficacy of our wearable night-vision aid in patients with concentric peripheral visual field loss.

STUDY DESIGN

Prospective, single blind, three-group, and three-period crossover clinical study.

METHODS

The study included patients with concentric peripheral visual field loss, a best-corrected visual acuity (decimal visual acuity) of 0.1 or higher in the better eye, and the presence of a central visual field. HOYA MW10 HiKARI® (HOYA Corporation), our original wearable night-vision aid, was used as the test device with three types of camera lenses (standard-, middle-, and wide-angle lenses). Under both bright and dark conditions, the angle of the horizontal visual field was measured using each of the three lens types for each group. The baseline angle was measured when each participant wore the night-vision aid (powered off).

RESULTS

The study included 21 participants. Under bright condition, the perceived horizontal visual field was significantly wider than the baseline setup when using the standard-angle lens ("the standard lens"); the middle-angle lens ("the middle lens") was significantly wider than both the baseline setup and the standard lens; and the wide-angle lens ("the wide lens") was significantly wider than the other lenses. Under dark condition, the perceived horizontal visual field was again significantly wider when using the middle lens than the baseline setup and the standard lens, and when using the wide lens, the perceived horizontal visual field was again wider than when using the other lenses. The control in the bright condition was significantly wider (p < 0.001) than when used in the dark condition, while the standard-angle lens in the dark condition was significantly wider (p = 0.05) than when used in the bright condition. In regards to the middle and wide lenses, there was no statistically significant result emerging from either of the illumination conditions.

CONCLUSION

Our wearable night-vision aid with a middle-angle or wide-angle lens appears to provide wider visual field images in patients with concentric peripheral visual field loss, regardless of whether the illumination conditions are bright or dark.

Spatial perception in stereoscopic augmented reality based on multifocus sensing

In many areas ranging from medical imaging to visual entertainment, 3D information acquisition and display is a key task. In this regard, in multifocus computational imaging, stacks of images of a certain 3D scene are acquired under different focus configurations and are later combined by means of post-capture algorithms based on image formation model in order to synthesize images with novel viewpoints of the scene. Stereoscopic augmented reality devices, through which is possible to simultaneously visualize the three dimensional real world along with overlaid digital stereoscopic image pair, could benefit from the binocular content allowed by multifocus computational imaging. Spatial perception of the displayed stereo pairs can be controlled by synthesizing the desired point of view of each image of the stereo-pair along with their parallax setting. The proposed method has the potential to alleviate the accommodation-convergence conflict and make augmented reality stereoscopic devices less vulnerable to visual fatigue.

A systematic review of extended reality (XR) for understanding and augmenting vision loss

Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. A defining quality of these XR technologies is their ability to update the stimulus based on the user's eye, head, or body movements. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. Here we present a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology to further visual accessibility. In contrast to other reviews, we sample studies from multiple scientific disciplines, focus on technology that augments a person's residual vision, and require studies to feature a quantitative evaluation with appropriate end users. We summarize prominent findings from different XR research areas, show how the landscape has changed over the past decade, and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the usability of different XR-based accessibility aids.

The HoloLens in medicine: A systematic review and taxonomy

The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality (AR) display, is the main player in the recent boost in medical AR research. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021. We identified 217 relevant publications through a systematic search of the PubMed, Scopus, IEEE Xplore and SpringerLink databases. We propose a new taxonomy including use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation, and analyze the retrieved publications accordingly. We find that the bulk of research focuses on supporting physicians during interventions, where the HoloLens is promising for procedures usually performed without image guidance. However, the consensus is that accuracy and reliability are still too low to replace conventional guidance systems. Medical students are the second most common target group, where AR-enhanced medical simulators emerge as a promising technology. While concerns about human-computer interactions, usability and perception are frequently mentioned, hardly any concepts to overcome these issues have been proposed. Instead, registration and tracking lie at the core of most reviewed publications, nevertheless only few of them propose innovative concepts in this direction. Finally, we find that the validation of HoloLens applications suffers from a lack of standardized and rigorous evaluation protocols. We hope that this review can advance medical AR research by identifying gaps in the current literature, to pave the way for novel, innovative directions and translation into the medical routine.

Using augmented reality to cue obstacles for people with low vision

Detecting and avoiding obstacles while navigating can pose a challenge for people with low vision, but augmented reality (AR) has the potential to assist by enhancing obstacle visibility. Perceptual and user experience research is needed to understand how to craft effective AR visuals for this purpose. We developed a prototype AR application capable of displaying multiple kinds of visual cues for obstacles on an optical see-through head-mounted display. We assessed the usability of these cues via a study in which participants with low vision navigated an obstacle course. The results suggest that 3D world-locked AR cues were superior to directional heads-up cues for most participants during this activity.

The use of augmented reality and virtual reality for visual field expansion and visual acuity improvement in low vision rehabilitation: a systematic review

INTRODUCTION

Developments in image processing techniques and display technology have led to the emergence of augmented reality (AR) and virtual reality (VR)-based low vision devices (LVDs). However, their promise and limitations in low vision rehabilitation are poorly understood. The objective of this systematic review is to appraise the application of AR/VR LVDs aimed at visual field expansion and visual acuity improvement in low vision rehabilitation.

METHODS

A systematic search of the literature was performed using MEDLINE, Embase, PsychInfo, HealthStar, and National Library of Medicine (PubMed) from inception to March 6, 2022. Articles were eligible if they included an AR or VR LVD tested on a sample of individuals with low vision and provided visual outcomes such as visual acuity, visual fields, and object recognition.

RESULTS

Of the 652 articles identified, 16 studies comprising 382 individuals with a mean age of 52.17 (SD = 18.30) years, and with heterogeneous low vision etiologies (i.e., glaucoma, age-related macular degeneration, retinitis pigmentosa) were included in this systematic review. Most articles used AR (53%), VR (40%), and one article used both AR and VR. The main visual outcomes evaluated were visual fields (67%), visual acuity (65%), and contrast sensitivity (27%). Various visual enhancement techniques were employed including variable magnification using digital zoom (67%), contrast enhancements (53%), and minification (27%). AR LVDs were reported to expand the visual field from threefold to ninefold. On average, individuals using AR/VR LVDs experienced an improved in visual acuity from 0.9 to 0.2 logMAR. Ten articles were classified as high or moderate risk of bias.

CONCLUSION

AR/VR LVDs were found to afford visual field expansion and visual acuity improvement in low vision populations. Even though the results of this review are promising, the lack of controlled studies with well-defined populations, use of small, convenience samples, and incomplete reporting of inclusion and exclusion criteria among included studies makes it challenging to judge the true impact of these devices. Future studies should address these limitations and compare various AR/LVDs to determine what is the ideal LVD type and vision enhancement combination based on the user's level of visual ability and lifestyle.

Design of Smart Head–Mounted Display Technology: A Convergent Mixed-Methods Study

Introduction: The purpose of this study was to characterize functional impairments and human factor considerations that affect perceptions and preferences for head-mounted display (HMD) technology for adults with low vision and chronic eye disease. Methods: Through a convergent mixed-methods design, participants with visual impairments (age-related macular degeneration, diabetic retinopathy, glaucoma, or retinitis pigmentosa) were recruited. Participants completed the Impact of Vision Impairment (IVI) questionnaire, used commercially available HMDs (eSight, NuEyes, and Epson Moverio), and were interviewed. The IVI was used to identify groups with low, moderate, and high vision–related quality of life (VRQOL). Transcribed interviews were analyzed using a thematic approach. The survey and qualitative findings were integrated using mixed-methods joint display analysis. Results: Twenty-one participants were enrolled (mean age of 58.2 years, 57% male, median Snellen acuity of 20/40 [range: 20/20–hand movement]). An equal number ( n = 9) expressed a preference for eSight and NuEyes, while ( n = 3) preferred the Moverio. Participants emphasized ease of use, including HMD controls and screen, as common reasons for preference. Those with lower IVI well-being scores preferred eSight due to vision improvement. Those with moderate IVI well-being scores preferred NuEyes due to comfort and size. Those with high IVI well-being scores cited usability as the most important feature. Discussion: User preferences for HMD features were associated with VRQOL. A mixed-methods approach explained how varying degrees of visual impairment and HMD preferences were qualitatively related to usability at the individual level. Implications for Practitioners: To increase acceptance, new HMD development for low vision should focus on performance, usability, and human factors engineering. Although HMD technology can benefit individuals with low vision, device features and functions vary in meaningful ways based on vision parameters. Practitioners should be aware of how patient and device variations influence preferences when they recommend wearable systems and optimize training to harness these systems.

Longitudinal Trends in Case Histories and Rehabilitative Device Assessments at Low Vision Examinations

SIGNIFICANCE

Understanding longitudinal changes in why individuals frequent low-vision clinics is crucial for ensuring that patient care keeps current with changing technology and changing lifestyles. Among other findings, our results suggest that reading remains a prevailing patient complaint, with shifting priorities toward technology-related topics.

PURPOSE

This study aimed to understand changes in patient priorities and patient care in low vision over the past decade.

METHODS

We conducted a retrospective study of examination records (2009 to 2019, 3470 examinations) from two U.S. low-vision clinics. Automated word searches summarized two properties of the records: topics discussed during the case history and types of rehabilitative devices assessed. Logistic regression was used to model the effects of examination year, patient age, patient sex, and level of visual impairment.

RESULTS

Collapsing across all years, the most common topic discussed was reading (78%), followed by light-related topics (71%) and technology (59%). Whereas the odds of discussing reading trended downward over the decade (odds ratio, 0.57; P = .03), technology, social interaction, mobility, and driving trended upward (odds ratios, 4.53, 3.31, 2.71, and 1.95; all P 's < 0.001). The most frequently assessed devices were tinted lenses (95%). Over time, video magnifier and spectacle assessments trended downward (odds ratios, 0.64 and 0.72; P = .004, 0.04), whereas assessments of other optical aids increased. The data indicate several consistent differences among patient demographics.

CONCLUSIONS

Reading is likely to remain a prevailing patient complaint, but an increase in technology-related topics suggests shifting priorities, particularly in younger demographics. "Low-tech" optical aids have remained prominent in low-vision care even as "high-tech" assistive devices in the marketplace continue to advance.

A Scoping Review of Assistance and Therapy with Head-Mounted Displays for People Who Are Visually Impaired

Given the inherent visual affordances of Head-Mounted Displays (HMDs) used for Virtual and Augmented Reality (VR/AR), they have been actively used over many years as assistive and therapeutic devices for the people who are visually impaired. In this paper, we report on a scoping review of literature describing the use of HMDs in these areas. Our high-level objectives included detailed reviews and quantitative analyses of the literature, and the development of insights related to emerging trends and future research directions.

Our review began with a pool of 1251 papers collected through a variety of mechanisms. Through a structured screening process, we identified 61 English research papers employing HMDs to enhance the visual sense of people with visual impairments for more detailed analyses. Our analyses reveal that there is an increasing amount of HMD-based research on visual assistance and therapy, and there are trends in the approaches associated with the research objectives. For example, AR is most often used for visual assistive purposes, whereas VR is used for therapeutic purposes. We report on eight existing survey papers, and present detailed analyses of the 61 research papers, looking at the mitigation objectives of the researchers (assistive versus therapeutic), the approaches used, the types of HMDs, the targeted visual conditions, and the inclusion of user studies. In addition to our detailed reviews and analyses of the various characteristics, we present observations related to apparent emerging trends and future research directions.

Photon-efficient 3D reconstruction employing a edge enhancement method

Photon-efficient 3D reconstruction under sparse photon conditions remains challenges. Especially for scene edge locations, the light scattering results in a weaker echo signal than non-edge locations. Depth images can be viewed as smooth regions stitched together by edge segmentation, yet none of the existing methods focus on how to improve the accuracy of edge reconstruction when performing 3D reconstruction. Moreover, the impact of edge reconstruction to overall depth reconstruction hasn't been investigated. In this paper, we explore how to improve the edge reconstruction accuracy from various aspects such as improving the network structure, employing hybrid loss functions and taking advantages of the non-local correlation of SPAD measurements. Meanwhile, we investigate the correlation between the edge reconstruction accuracy and the reconstruction accuracy of overall depth based on quantitative metrics. The experimental results show that the proposed method achieves superior performance in both edge reconstruction and overall depth reconstruction compared with other state-of-the-art methods. Besides, it proves that the improvement of edge reconstruction accuracy promotes the reconstruction accuracy of depth map.

Augmented Reality in Ophthalmology: Applications and Challenges

Augmented reality (AR) has been developed rapidly and implemented in many fields such as medicine, maintenance, and cultural heritage. Unlike other specialties, ophthalmology connects closely with AR since most AR systems are based on vision systems. Here we summarize the applications and challenges of AR in ophthalmology and provide insights for further research. Firstly, we illustrate the structure of the standard AR system and present essential hardware. Secondly, we systematically introduce applications of AR in ophthalmology, including therapy, education, and clinical assistance. To conclude, there is still a large room for development, which needs researchers to pay more effort. Applications in diagnosis and protection might be worth exploring. Although the obstacles of hardware restrict the development of AR in ophthalmology at present, the AR will realize its potential and play an important role in ophthalmology in the future with the rapidly developing technology and more in-depth research.

Machine Learning Method for Functional Assessment of Retinal Models

Challenges in the field of retinal prostheses motivate the development of retinal models to accurately simulate Retinal Ganglion Cells (RGCs) responses. The goal of retinal prostheses is to enable blind individuals to solve complex, reallife visual tasks. In this paper, we introduce the functional assessment (FA) of retinal models, which describes the concept of evaluating the performance of retinal models on visual understanding tasks. We present a machine learning method for FA: we feed traditional machine learning classifiers with RGC responses generated by retinal models, to solve object and digit recognition tasks (CIFAR-10, MNIST, Fashion MNIST, Imagenette). We examined critical FA aspects, including how the performance of FA depends on the task, how to optimally feed RGC responses to the classifiers and how the number of output neurons correlates with the model's accuracy. To increase the number of output neurons, we manipulated input images - by splitting and then feeding them to the retinal model and we found that image splitting does not significantly improve the model's accuracy. We also show that differences in the structure of datasets result in largely divergent performance of the retinal model (MNIST and Fashion MNIST exceeded 80% accuracy, while CIFAR-10 and Imagenette achieved ∼40%). Furthermore, retinal models which perform better in standard evaluation, i.e. more accurately predict RGC response, perform better in FA as well. However, unlike standard evaluation, FA results can be straightforwardly interpreted in the context of comparing the quality of visual perception.

Recent understanding of binocular vision in the natural environment with clinical implications

Technological advances in recent decades have allowed us to measure both the information available to the visual system in the natural environment and the rich array of behaviors that the visual system supports. This review highlights the tasks undertaken by the binocular visual system in particular and how, for much of human activity, these tasks differ from those considered when an observer fixates a static target on the midline. The everyday motor and perceptual challenges involved in generating a stable, useful binocular percept of the environment are discussed, together with how these challenges are but minimally addressed by much of current clinical interpretation of binocular function. The implications for new technology, such as virtual reality, are also highlighted in terms of clinical and basic research application.

Non-fusion time-resolved depth image reconstruction using a highly efficient neural network architecture

Single-photon avalanche diodes (SPAD) are powerful sensors for 3D light detection and ranging (LiDAR) in low light scenarios due to their single-photon sensitivity. However, accurately retrieving ranging information from noisy time-of-arrival (ToA) point clouds remains a challenge. This paper proposes a photon-efficient, non-fusion neural network architecture that can directly reconstruct high-fidelity depth images from ToA data without relying on other guiding images. Besides, the neural network architecture was compressed via a low-bit quantization scheme so that it is suitable to be implemented on embedded hardware platforms. The proposed quantized neural network architecture achieves superior reconstruction accuracy and fewer parameters than previously reported networks.

Influence of Systematic Gaze Patterns in Navigation and Search Tasks with Simulated Retinitis Pigmentosa

People living with a degenerative retinal disease such as retinitis pigmentosa are oftentimes faced with difficulties navigating in crowded places and avoiding obstacles due to their severely limited field of view. The study aimed to assess the potential of different patterns of eye movement (scanning patterns) to (i) increase the effective area of perception of participants with simulated retinitis pigmentosa scotoma and (ii) maintain or improve performance in visual tasks. Using a virtual reality headset with eye tracking, we simulated tunnel vision of 20° in diameter in visually healthy participants (n = 9). Employing this setup, we investigated how different scanning patterns influence the dynamic field of view—the average area over time covered by the field of view—of the participants in an obstacle avoidance task and in a search task. One of the two tested scanning patterns showed a significant improvement in both dynamic field of view (navigation 11%, search 7%) and collision avoidance (33%) when compared to trials without the suggested scanning pattern. However, participants took significantly longer (31%) to finish the navigation task when applying this scanning pattern. No significant improvements in search task performance were found when applying scanning patterns.

Applications of augmented reality in ophthalmology [Invited]

Throughout the last decade, augmented reality (AR) head-mounted displays (HMDs) have gradually become a substantial part of modern life, with increasing applications ranging from gaming and driver assistance to medical training. Owing to the tremendous progress in miniaturized displays, cameras, and sensors, HMDs are now used for the diagnosis, treatment, and follow-up of several eye diseases. In this review, we discuss the current state-of-the-art as well as potential uses of AR in ophthalmology. This review includes the following topics: (i) underlying optical technologies, displays and trackers, holography, and adaptive optics; (ii) accommodation, 3D vision, and related problems such as presbyopia, amblyopia, strabismus, and refractive errors; (iii) AR technologies in lens and corneal disorders, in particular cataract and keratoconus; (iv) AR technologies in retinal disorders including age-related macular degeneration (AMD), glaucoma, color blindness, and vision simulators developed for other types of low-vision patients.