Visual prosthetics 2006: assessment and expectations

Benefits of thermal and distance-filtered imaging for wayfinding with prosthetic vision

Visual prostheses such as the Argus II provide partial vision for individuals with limited or no light perception. However, their effectiveness in daily life situations is limited by scene complexity and variability. We investigated whether additional image processing techniques could improve mobility performance in everyday indoor environments. A mobile system connected to the Argus II provided thermal or distance-filtered video stimulation. Four participants used the thermal camera to locate a person and the distance filter to navigate a hallway with obstacles. The thermal camera allowed for finding a target person in 99% of trials, while unfiltered video led to confusion with other objects and a success rate of only 55% ([Formula: see text]). Similarly, the distance filter enabled participants to detect and avoid 88% of obstacles by removing background clutter, whereas unfiltered video resulted in a detection rate of only 10% ([Formula: see text]). For any given elapsed time, the success rate with filtered video was higher than with unfiltered video. After 90 s, participants' success rate reached above 50% with filtered video and 24% and 3% with normal camera in the first and second tasks, respectively. Despite individual variations, all participants showed significant improvement when using the thermal and distance filters compared to unfiltered video. Adding a thermal and distance filter to a visual prosthesis system can enhance the performance of mobility activities by removing clutter in the background, showing people and warm objects with the thermal camera, or nearby obstacles with the distance filter.

Glow in the dark: Using a heat-sensitive camera for blind individuals with prosthetic vision

To date, retinal implants are the only available treatment for blind individuals with retinal degenerations such as retinitis pigmentosa. Argus II is the only visual implant with FDA approval, with more than 300 users worldwide. Argus II stimulation is based on a grayscale image coming from a head-mounted visible-light camera. Normally, the 11°×19° field of view of the Argus II user is full of objects that may elicit similar phosphenes. The prosthesis cannot meaningfully convey so much visual information, and the percept is reduced to an ambiguous impression of light. This study is aimed at investigating the efficacy of simplifying the video input in real-time using a heat-sensitive camera. Data were acquired from four Argus II users in 5 stationary tasks with either hot objects or human targets as stimuli. All tasks were of m-alternative forced choice design where precisely one of the m≥2 response alternatives was defined to be "correct" by the experimenter. To compare performance with heat-sensitive and normal cameras across all tasks, regardless of m, we used an extension of signal detection theory to latent variables, estimating person ability and item difficulty in d' units. Results demonstrate that subject performance was significantly better across all tasks with the thermal camera compared to the regular Argus II camera. The future addition of thermal imaging to devices with very poor spatial resolution may have significant real-life benefits for orientation, personal safety, and social interactions, thereby improving quality of life.

Neurolight: A Deep Learning Neural Interface for Cortical Visual Prostheses

Visual neuroprosthesis, that provide electrical stimulation along several sites of the human visual system, constitute a potential tool for vision restoration for the blind. Scientific and technological progress in the fields of neural engineering and artificial vision comes with new theories and tools that, along with the dawn of modern artificial intelligence, constitute a promising framework for the further development of neurotechnology. In the framework of the development of a Cortical Visual Neuroprosthesis for the blind (CORTIVIS), we are now facing the challenge of developing not only computationally powerful tools and flexible approaches that will allow us to provide some degree of functional vision to individuals who are profoundly blind. In this work, we propose a general neuroprosthesis framework composed of several task-oriented and visual encoding modules. We address the development and implementation of computational models of the firing rates of retinal ganglion cells and design a tool — Neurolight — that allows these models to be interfaced with intracortical microelectrodes in order to create electrical stimulation patterns that can evoke useful perceptions. In addition, the developed framework allows the deployment of a diverse array of state-of-the-art deep-learning techniques for task-oriented and general image pre-processing, such as semantic segmentation and object detection in our system’s pipeline. To the best of our knowledge, this constitutes the first deep-learning-based system designed to directly interface with the visual brain through an intracortical microelectrode array. We implement the complete pipeline, from obtaining a video stream to developing and deploying task-oriented deep-learning models and predictive models of retinal ganglion cells’ encoding of visual inputs under the control of a neurostimulation device able to send electrical train pulses to a microelectrode array implanted at the visual cortex.

Semantic and structural image segmentation for prosthetic vision

Prosthetic vision is being applied to partially recover the retinal stimulation of visually impaired people. However, the phosphenic images produced by the implants have very limited information bandwidth due to the poor resolution and lack of color or contrast. The ability of object recognition and scene understanding in real environments is severely restricted for prosthetic users. Computer vision can play a key role to overcome the limitations and to optimize the visual information in the prosthetic vision, improving the amount of information that is presented. We present a new approach to build a schematic representation of indoor environments for simulated phosphene images. The proposed method combines a variety of convolutional neural networks for extracting and conveying relevant information about the scene such as structural informative edges of the environment and silhouettes of segmented objects. Experiments were conducted with normal sighted subjects with a Simulated Prosthetic Vision system. The results show good accuracy for object recognition and room identification tasks for indoor scenes using the proposed approach, compared to other image processing methods.

Development of visual Neuroprostheses: trends and challenges

Visual prostheses are implantable medical devices that are able to provide some degree of vision to individuals who are blind. This research field is a challenging subject in both ophthalmology and basic science that has progressed to a point where there are already several commercially available devices. However, at present, these devices are only able to restore a very limited vision, with relatively low spatial resolution. Furthermore, there are still many other open scientific and technical challenges that need to be solved to achieve the therapeutic benefits envisioned by these new technologies. This paper provides a brief overview of significant developments in this field and introduces some of the technical and biological challenges that still need to be overcome to optimize their therapeutic success, including long-term viability and biocompatibility of stimulating electrodes, the selection of appropriate patients for each artificial vision approach, a better understanding of brain plasticity and the development of rehabilitative strategies specifically tailored for each patient.

Development of the Ultra-Low Vision Visual Functioning Questionnaire (ULV-VFQ)

Purpose

To develop and psychometrically evaluate a visual functioning questionnaire (VFQ) in an ultra-low vision (ULV) population.

Methods

Questionnaire items, based on visual activities self-reported by a ULV population, were categorized by functional visual domain (e.g., mobility) and visual aspect (e.g., contrast) to ensure a representative distribution. In Round 1, an initial set of 149 items was generated and administered to 90 participants with ULV (visual acuity [VA] ≤ 20/500; mean [SD] age 61 [15] years), including six patients with a retinal implant. Psychometric properties were evaluated through Rasch analysis and a revised set (150 items) was administered to 80 participants in Round 2.

Results

In Round 1, the person measure distribution (range, 8.6 logits) was centered at −1.50 logits relative to the item measures. In Round 2, the person measure distribution (range, 9.5 logits) was centered at −0.86 relative to the item mean. The reliability index in both rounds was 0.97 for Items and 0.99 for Persons. Infit analysis showed four underfit items in Round 1, five underfit items in Round 2 with a z-score greater than 4 cutoff. Principal component analysis on the residuals found 69.9% explained variance; the largest component in the unexplained variance was less than 3%.

Conclusions

The ULV-VFQ, developed with content generated from a ULV population, showed excellent psychometric properties as well as superior measurement validity in a ULV population.

Translational Relevance

The ULV-VFQ, part of the Prosthetic Low Vision Rehabilitation (PLoVR) development program, is a new VFQ developed for assessment of functional vision in ULV populations.

Retinal implants: emergence of a multidisciplinary field

PURPOSE OF REVIEW

This review summarizes the current status of retinal prostheses, recent accomplishments, and major remaining research, engineering, and rehabilitation challenges.

RECENT FINDINGS

Retinal research, materials and biocompatibility studies, and clinical trials in patients blind from retinitis pigmentosa are representative of an emerging field with considerable promise and sobering challenges. A summary of progress in dozens of laboratories, companies, and clinics around the world is presented through a synopsis of relevant studies, not only to summarize the progress but also to convey the remarkable increase in interest, effort, and outside funding this field has enjoyed.

SUMMARY

At present, clinical applications of retinal implant technology are dominated by one or two groups/companies, but the field is wide open for others to take the lead through novel approaches in technology, tissue interfacing, information transfer paradigms, and rehabilitation. Where the field will go in the next few years is almost anybody's guess, but that it will move forward is a certainty.

Angiographic findings following tack fixation of a wireless epiretinal retina implant device in blind RP patients

BACKGROUND

The fixation of polyimide stimulator foils as the basic substrate of epiretinal prostheses by using retinal tacks may cause retinal or choroidal alterations such as retinal proliferations or choroidal neovascularizations. During the prospective trial for the semichronical testing of a wireless intraocular retinal implant (EPIRET3) we investigated alterations in angiographic findings during implantation and after explantation of the device, to detect potential vascular pathologies at the fixation site or elsewhere.

METHODS

As the final step of the implantation surgery in six blind patients, the stimulator was placed on the retinal surface using retinal tacks. For the detection of possible morphological or vascular alterations committed by the implant fluorescein angiography (FA) was performed 1 day before and 4 weeks after implantation surgery, as well as at the final visit 5 months after explantation.

RESULTS

Following implantation surgery funduscopy and FA did not reveal any evidence of either vascular pathologies or choroidal neovascularisations (CNV), in addition, no cystoid macular edema (CME) occurred after 4 weeks. At the 6-month follow-up visit, we found a mild epiretinal gliosis formation in four patients. In one patient a retinal break occurred during explantation, requiring a temporary silicone-oil endotamponade. At the final visit, we observed a focal proliferative vitreoretinopathy (PVR) reaction without activity, while there was no evidence for a CNV formation in that area.

CONCLUSIONS

The FA findings confirm our previous results on the safety of the EPIRET3 system, which was tolerated in all patients but revealed a certain risk profile in regard to the stimulator fixation. While there was no evidence for newly occurred CME or CNV during the follow-up visits, nevertheless gliosis or even PVR reaction at the tack's fixation site suggests the need to develop alternative fixation procedures of epiretinal stimulators.

Building the bionic eye: an emerging reality and opportunity

Once the topic of folklore and science fiction, the notion of restoring vision to the blind is now approaching a tractable reality. Technological advances have inspired numerous multidisciplinary groups worldwide to develop visual neuroprosthetic devices that could potentially provide useful vision and improve the quality of life of profoundly blind individuals. While a variety of approaches and designs are being pursued, they all share a common principle of creating visual percepts through the stimulation of visual neural elements using appropriate patterns of electrical stimulation. Human clinical trials are now well underway and initial results have been met with a balance of excitement and cautious optimism. As remaining technical and surgical challenges continue to be solved and clinical trials move forward, we now enter a phase of development that requires careful consideration of a new set of issues. Establishing appropriate patient selection criteria, methods of evaluating long-term performance and effectiveness, and strategies to rehabilitate implanted patients will all need to be considered in order to achieve optimal outcomes and establish these devices as viable therapeutic options.

Virtual wayfinding using simulated prosthetic vision in gaze-locked viewing

PURPOSE

To assess virtual maze navigation performance with simulated prosthetic vision in gaze-locked viewing, under the conditions of varying luminance contrast, background noise, and phosphene dropout.

METHODS

Four normally sighted subjects performed virtual maze navigation using simulated prosthetic vision in gaze-locked viewing, under five conditions of luminance contrast, background noise, and phosphene dropout. Navigation performance was measured as the time required to traverse a 10-room maze using a game controller, and the number of errors made during the trip.

RESULTS

Navigation performance time (1) became stable after 6 to 10 trials, (2) remained similar on average at luminance contrast of 68% and 16% but had greater variation at 16%, (3) was not significantly affected by background noise, and (4) increased by 40% when 30% of phosphenes were removed. Navigation performance time and number of errors were significantly and positively correlated.

CONCLUSIONS

Assuming that the simulated gaze-locked viewing conditions are extended to implant wearers, such prosthetic vision can be helpful for wayfinding in simple mobility tasks, though phosphene dropout may interfere with performance.

Initiation and stability of pursuit eye movements in simulated retinal prosthesis at different implant locations

PURPOSE

To assess the possible effects of retinal prosthesis implant location on the initiation and stability of pursuit eye movements.

METHODS

Six normally sighted subjects visually tracked a horizontally moving target in natural vision and in simulated prosthetic vision. Subjects were instructed to press a key when the target jumped. Prosthetic vision was simulated with a 10 x 10 array of 1 degrees diameter phosphenes. Three implant locations in the retina were simulated: macular, 8 degrees superior, and 8 degrees nasal. Target motion had two speeds: 4 degrees /s and 8 degrees /s. Eye movement latency, horizontal stability, and vertical stability were assessed. Key-press behaviors responding to target jump were analyzed to evaluate functional eye movements.

RESULTS

Compared with natural vision, horizontal eye position with respect to target position was less stable in simulated prosthetic vision at macular, superior, and nasal implant locations, in ascending order of the degree of instability. Vertical eye position with respect to target position in simulated prosthetic vision with the superior implant location was less stable in tracking slow target motion than fast. Eye movement latency in simulated prosthetic vision was longer than in natural vision. Key-press performance was impaired in simulated prosthetic vision.

CONCLUSIONS

Pursuit eye movements in prosthetic vision, compared to natural vision, are significantly slower in initiation and less smooth in motion. They seem, however, still functional, even if the prosthesis is implanted in the peripheral retina. A superior implant locus may help the prosthesis wearer better control horizontal eye movements, which are more frequently used in the activities of daily living.

'Who is the ideal candidate?': decisions and issues relating to visual neuroprosthesis development, patient testing and neuroplasticity

Appropriate delivery of electrical stimulation to intact visual structures can evoke patterned sensations of light in individuals who have been blind for many years. This pivotal finding has lent credibility to the concept of restoring functional vision by artificial means. As numerous groups worldwide pursue human clinical testing with visual prosthetic devices, it is becoming increasingly clear that there remains a considerable gap between the challenges of prosthetic device development and the rehabilitative strategies needed to implement this new technology in patients. An important area of future work will be the development of appropriate pre- and post-implantation measures of performance and establishing candidate selection criteria in order to quantify technical advances, guide future device design and optimize therapeutic success. We propose that the selection of an 'ideal' candidate should also be considered within the context of the variable neuroplastic changes that follow vision loss. Specifically, an understanding of the adaptive and compensatory changes that occur within the brain could assist in guiding the development of post-implantation rehabilitative strategies and optimize behavioral outcomes.

Retinal charge sensitivity and spatial discrimination obtainable by subretinal implants: key lessons learned from isolated chicken retina

In order to obtain functional parameters relevant to the designing of a subretinal implant, we carried out electrical stimulation experiments with isolated chicken retina. The median threshold for network activation with planar disc electrodes (diameter 10 microm) was 0.5 nC (625 microC cm(-2)) for anodal voltage impulses and 1.6 nC (2 mC cm(-2)) for cathodal impulses. Above threshold, the number of spikes evoked by a single voltage impulse increased up to saturation within a range of injected charge from 0.1 nC to 1 nC for anodal impulses and from 1 nC to 10 nC for cathodal impulses. Using needle electrodes with a tip diameter of 1 microm, we determined the electrical point spread function (EPSF) for subretinal stimulation. It had a half width in the range of 100 microm, which corresponds to a visual angle of 21' and to a visual acuity of 20/417 in the human eye. It is reasonable to conclude that with subretinal implants the minimum separable will be of the same dimension.