Applications of augmented reality in ophthalmology [Invited]

Evaluating the Diagnostic Accuracy of a Portable, Motorized, and Remotely Controlled Slit Lamp Imaging Adaptor Prototype for Head-Mounted Displays

Purpose

The purpose of this study was to validate the performance of a portable and remotely controlled slit lamp imaging adaptor.

Methods

Twenty patients with anterior eye segment conditions participated in a randomized masked clinical trial. Imaging was performed using a Haag-Streit AG, BX 900 slit lamp biomicroscope and a new slit lamp prototype. Three ophthalmologists independently reviewed masked and randomized 2D images from both instruments and conducted physical eye examinations based on these images. Inter- and intra-grader reliability were assessed using kappa statistics, and sensitivity and specificity were determined with reference to the clinical eye examinations performed during the patients' visits.

Results

The sensitivity and specificity of the evaluations with the prototype were 47.8% and 64.1%. Similarly, the evaluations from the conventional system obtained a sensitivity and specificity of 49.5% and 66.2%. The differences in the sensitivity and specificity between imaging modalities were not statistically significant (P > 0.05). The intra-grader reliability showed moderate to substantial agreement between the systems (κ = 0.522-0.708). The inter-grader reliability also indicated moderate agreement for the evaluations with the conventional system (κ = 0.552) and the prototype (κ = 0.474).

Conclusions

This study presents a new prototype that exhibits diagnostic accuracy on par with conventional slit lamps and moderate reliability. Further studies with larger sample sizes are required to characterize the prototype's performance. However, its remote functionality and accessibility suggest the potential to extend eye care.

Translational Relevance

The development of portable and remotely controlled eye imaging systems will enhance teleophthalmology services and broaden access to eye care at the primary care level.

Feasibility and Usability of Augmented Reality Technology in the Orthopaedic Operating Room

PURPOSE OF REVIEW

Augmented reality (AR) has gained popularity in various sectors, including gaming, entertainment, and healthcare. The desire for improved surgical navigation within orthopaedic surgery has led to the evaluation of the feasibility and usability of AR in the operating room (OR). However, the safe and effective use of AR technology in the OR necessitates a proper understanding of its capabilities and limitations. This review aims to describe the fundamental elements of AR, highlight limitations for use within the field of orthopaedic surgery, and discuss potential areas for development.

RECENT FINDINGS

To date, studies have demonstrated evidence that AR technology can be used to enhance navigation and performance in orthopaedic procedures. General hardware and software limitations of the technology include the registration process, ergonomics, and battery life. Other limitations are related to the human response factors such as inattentional blindness, which may lead to the inability to see complications within the surgical field. Furthermore, the prolonged use of AR can cause eye strain and headache due to phenomena such as the vergence-convergence conflict. AR technology may prove to be a better alternative to current orthopaedic surgery navigation systems. However, the current limitations should be mitigated to further improve the feasibility and usability of AR in the OR setting. It is important for both non-clinicians and clinicians to work in conjunction to guide the development of future iterations of AR technology and its implementation into the OR workflow.

The photoreal new-age innovative pedagogical & counseling tool for glaucoma with 3D augmented reality (Eye MG AR)

In this manuscript, we have reported an augmented reality (AR) application named, 'Eye MG AR' innovated by us, to show different anatomical/pathological parts of the eyeball pertaining to glaucoma, from multiple customized angles of the user's choice to simplify glaucoma learning and clinical counseling. It is available free of cost from the Google Play Store for Android users. Procedures ranging from a simple outpatient department procedure (yttrium aluminium garnet peripheral iridotomy) to a complex surgical technique (trabeculectomy/tube surgery) can be explained and counseled with this Android application. Also, complex structures such as the angle of the anterior chamber and optic nerve head, are constructed in advanced real-time three-dimensional (3D) high-resolution confocal images. These 3D models are useful for glaucoma neophytes' immersive learning and 3D patient counseling experiences. This AR tool aims to reinvent the approach to glaucoma counseling with 'Unreal Engine' software and is created in a patient-friendly approach. Incepting 3D pedagogy and counseling with AR in glaucoma with real-time and high-resolution TrueColor confocal images has never been reported in the literature according to our knowledge.

Spatial perception in stereoscopic augmented reality based on multifocus sensing

In many areas ranging from medical imaging to visual entertainment, 3D information acquisition and display is a key task. In this regard, in multifocus computational imaging, stacks of images of a certain 3D scene are acquired under different focus configurations and are later combined by means of post-capture algorithms based on image formation model in order to synthesize images with novel viewpoints of the scene. Stereoscopic augmented reality devices, through which is possible to simultaneously visualize the three dimensional real world along with overlaid digital stereoscopic image pair, could benefit from the binocular content allowed by multifocus computational imaging. Spatial perception of the displayed stereo pairs can be controlled by synthesizing the desired point of view of each image of the stereo-pair along with their parallax setting. The proposed method has the potential to alleviate the accommodation-convergence conflict and make augmented reality stereoscopic devices less vulnerable to visual fatigue.

Visual Field Dependence Persists in Age-Related Central Visual Field Loss

Purpose

To examine whether the age-related increase in visual field dependence persists in older adults with central field loss (CFL).

Methods

Twenty individuals with CFL were grouped into participants with age-related binocular CFL (CFL, n = 9), age-related monocular CFL/relative scotomata (mCFL, n = 8), and CFL occurring at a young age (yCFL, n = 3). Seventeen controls were age-matched to the older CFL groups (OA) and three to the yCFL group (yOA). Participants judged the tilt direction of a rod presented at various orientations under conditions with and without a visual reference. Visual field dependence was determined as the difference in judgment bias between trials with and without the visual reference. Visual field dependence was examined between groups and relative to visual acuity and contrast sensitivity.

Results

All older groups performed similarly without the visual reference. The CFL group showed greater visual field dependence than the OA group (Mann-Whitney U test; U = 39, P = 0.045). However, there was no group difference when considering all three older groups (Kruskal-Wallis ANOVA; H(2, N = 34) = 4.31, P = 0.116). Poorer contrast sensitivity correlated with greater visual field dependence (P = 0.017; ρ = -0.43).

Conclusions

Visual field dependence persists in older adults with CFL and seems exacerbated in those with dense binocular scotomata. This could be attributed to the sensitivity of the spared peripheral retina to orientation and motion cues. The relationship with contrast sensitivity further suggests that a decline in visual function is associated with an increase in visual field dependence beyond the effects of normal aging. These observations can guide tailored care and rehabilitation in older adults with CFL.

A systematic review of extended reality (XR) for understanding and augmenting vision loss

Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. A defining quality of these XR technologies is their ability to update the stimulus based on the user's eye, head, or body movements. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. Here we present a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology to further visual accessibility. In contrast to other reviews, we sample studies from multiple scientific disciplines, focus on technology that augments a person's residual vision, and require studies to feature a quantitative evaluation with appropriate end users. We summarize prominent findings from different XR research areas, show how the landscape has changed over the past decade, and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the usability of different XR-based accessibility aids.

ChromaCorrect: prescription correction in virtual reality headsets through perceptual guidance

A large portion of today's world population suffers from vision impairments and wears prescription eyeglasses. However, prescription glasses cause additional bulk and discomfort when used with virtual reality (VR) headsets, negatively impacting the viewer's visual experience. In this work, we remedy the usage of prescription eyeglasses with screens by shifting the optical complexity into the software. Our proposal is a prescription-aware rendering approach for providing sharper and more immersive imagery for screens, including VR headsets. To this end, we develop a differentiable display and visual perception model encapsulating the human visual system's display-specific parameters, color, visual acuity, and user-specific refractive errors. Using this differentiable visual perception model, we optimize the rendered imagery in the display using gradient-descent solvers. This way, we provide prescription glasses-free sharper images for a person with vision impairments. We evaluate our approach and show significant quality and contrast improvements for users with vision impairments.

Rendering algorithms for aberrated human vision simulation

Vision-simulated imagery-the process of generating images that mimic the human visual system-is a valuable tool with a wide spectrum of possible applications, including visual acuity measurements, personalized planning of corrective lenses and surgeries, vision-correcting displays, vision-related hardware development, and extended reality discomfort reduction. A critical property of human vision is that it is imperfect because of the highly influential wavefront aberrations that vary from person to person. This study provides an overview of the existing computational image generation techniques that properly simulate human vision in the presence of wavefront aberrations. These algorithms typically apply ray tracing with a detailed description of the simulated eye or utilize the point-spread function of the eye to perform convolution on the input image. Based on the description of the vision simulation techniques, several of their characteristic features have been evaluated and some potential application areas and research directions have been outlined.

Stereopsis Only: Validation of a Monocular Depth Cues Reduced Gamified Virtual Reality with Reaction Time Measurement

The visual depth perception is composed of monocular and binocular depth cues. Studies show that in absence of binocular depth cues the performance of visuomotor tasks like pointing to or grasping objects is limited. Thus, binocular depth cues are of great importance for motor control required in everyday life. However, binocular depth cues like retinal disparity (basis for stereopsis) might be influenced due to developmental disorders of the visual system. For example, amblyopia in which one eye's visual input is not processed leads to loss of stereopsis. The primary amblyopia treatment is occlusion of the healthy eye to force the amblyopic eye to train. However, improvements in stereopsis are poor. Therefore, binocular treatments arose that equilibrate both eyes' visual input to enable binocular vision. However, most approaches rely on divided stimuli which do not account for loss of stereopsis. We created a Virtual Reality (VR) with reduced monocular depth cues in which a stereoscopic task is shown to both eyes simultaneously, consisting of two balls jumping towards the user. One ball appears closer to the user which must be identified. To evaluate the task performance the reaction time is measured. We validated our approach with 18 participants with stereopsis under three contrast settings including one leading to monocular vision. The number of correct responses reduces from 90% under binocular vision to 52% under monocular vision corresponding to random guessing. Our results indicate that it is possible to disable monocular depth cues and create a dynamic stereoscopic task inside a VR.

Augmented Reality, Virtual Reality, and Game Technologies in Ophthalmology Training

Ophthalmology is a medical profession with a tradition in teaching that has developed throughout history. Although ophthalmologists are generally considered to only prescribe contact lenses, and they handle more than half of eye-related enhancements, diagnoses, and treatments. The training of qualified ophthalmologists is generally carried out under the traditional settings, where there is a supervisor and a student, and training is based on the use of animal eyes or artificial eye models. These models have significant disadvantages, as they are not immersive and are extremely expensive and difficult to acquire. Therefore, technologies related to Augmented Reality (AR) and Virtual Reality (VR) are rapidly and prominently positioning themselves in the medical sector, and the field of ophthalmology is growing exponentially both in terms of the training of professionals and in the assistance and recovery of patients. At the same time, it is necessary to highlight and analyze the developments that have made use of game technologies for the teaching of ophthalmology and the results that have been obtained. This systematic review aims to investigate software and hardware applications developed exclusively for educational environments related to ophthalmology and provide an analysis of other related tools. In addition, the advantages and disadvantages, limitations, and challenges involved in the use of virtual reality, augmented reality, and game technologies in this field are also presented.

Intraocular scatter compensation with spatial light amplitude modulation for improved vision in simulated cataractous eyes

Cataract is one of the common causes of visual impairment due to opacification of the crystalline lens. Increased intraocular scattering affects the vision of cataract patients by reducing the quality of the retinal image. In this study, an amplitude modulation-based scatter compensation (AM-SC) method is developed to minimize the impact of straylight on the retinal image. The performance of the AM-SC method was quantified by numerical simulations of point spread function and retinal images in the presence of different amounts of straylight. The approach was also experimentally realized in a single-pass system with a digital micro-mirror device used as a spatial amplitude modulator. We showed that the AM-SC method allows to enhance contrast sensitivity in the human eyes in vivo with induced scattering.

Liquid crystal microlens array with positive and negative focal lengths based on a patterned electrode

A liquid crystal microlens array (LCMLA) with positive and negative focal lengths based on a ring-array patterned electrodes is demonstrated. By carefully designing patterned electrodes with a circular electrode array area and outer ring electrode region area, the switching of the positive and negative lens effect can be easily achieved in a single cell. A positive lens effect appeared when the voltage was applied to the outer ring electrode region and the top substrate. The focal length changed from infinity to 1 mm as the voltage varied from 0 to 3V_rms. A negative lens effect occurred when the voltage was applied to the circular electrode array and the top substrate. The focal length varied from infinity to -1mm when the voltage changed from 0 to 2V_rms. The imaging properties of the LCMLA at different voltages are evaluated. Our LCMLA, with simple structure, low driving voltage, and good stability, has potential applications in optical communication, imaging processing, and displays.

Virtual Reality and Augmented Reality- Emerging Screening and Diagnostic Techniques in Ophthalmology: a Systematic Review

In healthcare, virtual reality (VR) and augmented reality (AR) have been applied extensively for many purposes. Similar to other technologies such as telemedicine and artificial intelligence, VR and AR may improve clinical diagnosis and screening services in ophthalmology by alleviating current problems, including workforce shortage, diagnostic error, and underdiagnosis. In the past decade a number of studies and products have used VR and AR concepts to build clinical tests for ophthalmology, but comprehensive reviews on these studies are limited. Therefore, we conducted a systematic review on the use of VR and AR as a diagnostic and screening tool in ophthalmology. We identified 26 studies that implemented a variety of VR and AR tests on different conditions, including VR cover tests for binocular vision disorder, VR perimetry for glaucoma, and AR slit lamp biomicroscopy for retinal diseases. In general, while VR and AR tools can become standardized, automated, and cost-effective tests with good user experience, several weaknesses, including unsatisfactory accuracy, weak validation, and hardware limitations, have prevented these VR and AR tools from having wider clinical application. Also, a comparison between VR and AR is made to explain why studies have predominantly used VR rather than AR.

Learned holographic light transport: invited

Computer-generated holography algorithms often fall short in matching simulations with results from a physical holographic display. Our work addresses this mismatch by learning the holographic light transport in holographic displays. Using a camera and a holographic display, we capture the image reconstructions of optimized holograms that rely on ideal simulations to generate a dataset. Inspired by the ideal simulations, we learn a complex-valued convolution kernel that can propagate given holograms to captured photographs in our dataset. Our method can dramatically improve simulation accuracy and image quality in holographic displays while paving the way for physically informed learning approaches.

Augmented Reality in Ophthalmology: Applications and Challenges

Augmented reality (AR) has been developed rapidly and implemented in many fields such as medicine, maintenance, and cultural heritage. Unlike other specialties, ophthalmology connects closely with AR since most AR systems are based on vision systems. Here we summarize the applications and challenges of AR in ophthalmology and provide insights for further research. Firstly, we illustrate the structure of the standard AR system and present essential hardware. Secondly, we systematically introduce applications of AR in ophthalmology, including therapy, education, and clinical assistance. To conclude, there is still a large room for development, which needs researchers to pay more effort. Applications in diagnosis and protection might be worth exploring. Although the obstacles of hardware restrict the development of AR in ophthalmology at present, the AR will realize its potential and play an important role in ophthalmology in the future with the rapidly developing technology and more in-depth research.

Pupil steering holographic display for pre-operative vision screening of cataracts

Cataract is the most common cause of preventable blindness and vision loss where the only treatment is surgical replacement of the natural lens with an intraocular lens. Computer-generated holography (CGH) enables to control phase, size, and shape of the light beam entering through the eye-pupil. We developed a holographic vision simulator to assess visual acuity for patients to experience the postoperative corrected vision before going through surgery. A holographically shaped light beam is directed onto the retina using small non-cataractous regions of the lens with the help of a pupil tracker. A Snellen chart hologram is shown to subjects at desired depth with myopia and hyperopia correction. Tests with 13 patients demonstrated substantial improvements in visual acuity and the simulator results are consistent with the post-operative vision tests. Holographic simulator overperforms the existing vision simulators, which are limited to static pinhole exit pupils and incapable of correcting aberrations.