Extraspectral Imaging for Improving the Perceived Information Presented in Retinal Prosthesis

Pre-processing visual scenes for retinal prosthesis systems: A comprehensive review

BACKGROUND

Retinal prostheses offer hope for individuals with degenerative retinal diseases by stimulating the remaining retinal cells to partially restore their vision. This review delves into the current advancements in retinal prosthesis technology, with a special emphasis on the pivotal role that image processing and machine learning techniques play in this evolution.

METHODS

We provide a comprehensive analysis of the existing implantable devices and optogenetic strategies, delineating their advantages, limitations, and challenges in addressing complex visual tasks. The review extends to various image processing algorithms and deep learning architectures that have been implemented to enhance the functionality of retinal prosthetic devices. We also illustrate the testing results by demonstrating the clinical trials or using Simulated Prosthetic Vision (SPV) through phosphene simulations, which is a critical aspect of simulating visual perception for retinal prosthesis users.

RESULTS

Our review highlights the significant progress in retinal prosthesis technology, particularly its capacity to augment visual perception among the visually impaired. It discusses the integration between image processing and deep learning, illustrating their impact on individual interactions and navigations within the environment through applying clinical trials and also illustrating the limitations of some techniques to be used with current devices, as some approaches only use simulation even on sighted-normal individuals or rely on qualitative analysis, where some consider realistic perception models and others do not.

CONCLUSION

This interdisciplinary field holds promise for the future of retinal prostheses, with the potential to significantly enhance the quality of life for individuals with retinal prostheses. Future research directions should pivot towards optimizing phosphene simulations for SPV approaches, considering the distorted and confusing nature of phosphene perception, thereby enriching the visual perception provided by these prosthetic devices. This endeavor will not only improve navigational independence but also facilitate a more immersive interaction with the environment.

A systematic review of extended reality (XR) for understanding and augmenting vision loss

Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. A defining quality of these XR technologies is their ability to update the stimulus based on the user's eye, head, or body movements. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. Here we present a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology to further visual accessibility. In contrast to other reviews, we sample studies from multiple scientific disciplines, focus on technology that augments a person's residual vision, and require studies to feature a quantitative evaluation with appropriate end users. We summarize prominent findings from different XR research areas, show how the landscape has changed over the past decade, and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the usability of different XR-based accessibility aids.

Semantic and structural image segmentation for prosthetic vision

Prosthetic vision is being applied to partially recover the retinal stimulation of visually impaired people. However, the phosphenic images produced by the implants have very limited information bandwidth due to the poor resolution and lack of color or contrast. The ability of object recognition and scene understanding in real environments is severely restricted for prosthetic users. Computer vision can play a key role to overcome the limitations and to optimize the visual information in the prosthetic vision, improving the amount of information that is presented. We present a new approach to build a schematic representation of indoor environments for simulated phosphene images. The proposed method combines a variety of convolutional neural networks for extracting and conveying relevant information about the scene such as structural informative edges of the environment and silhouettes of segmented objects. Experiments were conducted with normal sighted subjects with a Simulated Prosthetic Vision system. The results show good accuracy for object recognition and room identification tasks for indoor scenes using the proposed approach, compared to other image processing methods.

A head mounted device stimulator for optogenetic retinal prosthesis

Objective. Our main objective is to demonstrate that compact high radiance gallium nitride displays can be used with conventional virtual reality optics to stimulate an optogenetic retina. Hence, we aim to introduce a non-invasive approach to restore vision for people with conditions such as retinitis pigmentosa where there is a remaining viable communication link between the retina and the visual cortex. Approach. We design and implement the headset using a high-density µLED matrix, Raspberry Pi, microcontroller from NXP and virtual reality lens. Then, a test platform is developed to evaluate the performance of the headset and the optical system. Furthermore, image simplification algorithms are used to simplify the scene to be sent to the retina. Moreover, in vivo evaluation of the genetically modified retina response at different light intensity is discussed to prove the reliability of the proposed system. Main results. We demonstrate that in keeping with regulatory guidance, the headset displays need to limit their luminance to 90 kcd m⁻². We demonstrate an optical system with 5.75% efficiency which allows for 0.16 mW mm⁻² irradiance on the retina within the regulatory guidance, but which is capable of an average peak irradiance of 1.35 mW mm⁻². As this is lower than the commonly accepted threshold for channelrhodopsin-2, we demonstrate efficacy through an optical model of an eye onto a biological retina. Significance. We demonstrate a fully functional 8100-pixel headset system including software/hardware which can operate on a standard consumer battery for periods exceeding a 24 h recharge cycle. The headset is capable of delivering enough light to stimulate the genetically modified retina cells and also keeping the amount of light below the regulation threshold for safety.