Artificial vision: needs, functioning, and testing of a retinal electronic prosthesis

Benefits of thermal and distance-filtered imaging for wayfinding with prosthetic vision

Visual prostheses such as the Argus II provide partial vision for individuals with limited or no light perception. However, their effectiveness in daily life situations is limited by scene complexity and variability. We investigated whether additional image processing techniques could improve mobility performance in everyday indoor environments. A mobile system connected to the Argus II provided thermal or distance-filtered video stimulation. Four participants used the thermal camera to locate a person and the distance filter to navigate a hallway with obstacles. The thermal camera allowed for finding a target person in 99% of trials, while unfiltered video led to confusion with other objects and a success rate of only 55% ([Formula: see text]). Similarly, the distance filter enabled participants to detect and avoid 88% of obstacles by removing background clutter, whereas unfiltered video resulted in a detection rate of only 10% ([Formula: see text]). For any given elapsed time, the success rate with filtered video was higher than with unfiltered video. After 90 s, participants' success rate reached above 50% with filtered video and 24% and 3% with normal camera in the first and second tasks, respectively. Despite individual variations, all participants showed significant improvement when using the thermal and distance filters compared to unfiltered video. Adding a thermal and distance filter to a visual prosthesis system can enhance the performance of mobility activities by removing clutter in the background, showing people and warm objects with the thermal camera, or nearby obstacles with the distance filter.

New tricks and emerging applications from contemporary azobenzene research

Azobenzenes have many faces. They are well-known as dyes, but most of all, azobenzenes are versatile photoswitchable molecules with powerful photochemical properties. Azobenzene photochemistry has been extensively studied for decades, but only relatively recently research has taken a steer towards applications, ranging from photonics and robotics to photobiology. In this perspective, after an overview of the recent trends in the molecular design of azobenzenes, we highlight three research areas where the azobenzene photoswitches may bring about promising technological innovations: chemical sensing, organic transistors, and cell signaling. Ingenious molecular designs have enabled versatile control of azobenzene photochemical properties, which has in turn facilitated the development of chemical sensors and photoswitchable organic transistors. Finally, the power of azobenzenes in biology is exemplified by vision restoration and photactivation of neural signaling. Although the selected examples reveal only some of the faces of azobenzenes, we expect the fields presented to develop rapidly in the near future, and that azobenzenes will play a central role in this development.

Longitudinal Microstructure Changes of the Retina and Choroid in Retinitis Pigmentosa

PURPOSE

To investigate longitudinal changes in the retinal and choroidal microstructure of the macula in patients with retinitis pigmentosa (RP).

DESIGN

Retrospective, observational cohort study.

METHODS

A total of 69 patients with RP and 69 age- and sex-matched controls who underwent optical coherence tomography (OCT) over a 4-year follow-up period were included. The severity of RP was classified into 3 stages according to the integrity of the inner segment ellipsoid zone. The retinal and choroidal layers were segmented manually from OCT images. The areas of retinal pigment epithelium (RPE) atrophy and choroidal vascular index (CVI) were also analyzed. Longitudinal changes in the OCT parameters were compared among the groups.

RESULTS

Significant decreases (median [interquartile range]) in the thickness of the ganglion cell inner plexiform layer (GCIPL; -1.04 [-2.41 to -0.17]), outer nuclear layer (ONL; -1.44 [-1.86 to -0.28]), and inner segment ellipsoid (ISE; -0.74 [-1.33 to -0.49]) at the moderate stage and retinal nerve fiber layer (RNFL; -1.49 [-2.08 to -0.66]) and GCIPL (0.58 [-1.79 to 0.06]) at the advanced stage were observed. Choroidal thickness decreased significantly from -7.62 to -9.40 μm per year at all stages. RPE atrophy and CVI reduction were observed at the advanced stage. There was no change in the control group.

CONCLUSIONS

ONL and GCIPL thicknesses decreased at the moderate and advanced stages of RP; RNFL thickness decreased only at the advanced stage; and choroidal thickness decreased continuously. In addition, RPE atrophy and CVI reduction were prominent at the advanced stage. These results indicate that there is a temporal variation in the damage of each retinal layer and the choroid in RP patients.

Real-Time Optimization of Retinal Ganglion Cell Spatial Activity in Response to Epiretinal Stimulation

Retinal prostheses aim to improve visual perception in patients blinded by photoreceptor degeneration. However, shape and letter perception with these devices is currently limited due to low spatial resolution. Previous research has shown the retinal ganglion cell (RGC) spatial activity and phosphene shapes can vary due to the complexity of retina structure and electrode-retina interactions. Visual percepts elicited by single electrodes differ in size and shapes for different electrodes within the same subject, resulting in interference between phosphenes and an unclear image. Prior work has shown that better patient outcomes correlate with spatially separate phosphenes. In this study we use calcium imaging, in vitro retina, neural networks (NN), and an optimization algorithm to demonstrate a method to iteratively search for optimal stimulation parameters that create focal RGC activation. Our findings indicate that we can converge to stimulation parameters that result in focal RGC activation by sampling less than 1/3 of the parameter space. A similar process implemented clinically can reduce time required for optimizing implant operation and enable personalized fitting of retinal prostheses.

Perceptual Learning of Object Recognition in Simulated Retinal Implant Perception - The Effect of Video Training

Purpose

Retinal implants (RIs) provide new vision for patients suffering from photoreceptor degeneration in the retina. The limited vision gained by RI, however, leaves room for improvement by training regimes.

Methods

Two groups of normal-sighted participants were respectively trained with videos or still images of daily objects in a labeling task. Object appearance was simulated to resemble RI perception. In Experiment 1, the training effect was measured as the change in performance during the training, and the same labeling task was conducted after 1 week to test the retention. In Experiment 2 with a different pool of participants, a reverse labeling task was included before (pre-test) and after the training (post-test) to show if the training effect could be generalized into a different task context.

Results

Both groups showed improved object recognition through training that was maintained for a week, and the video group showed better improvement (Experiment 1). Both groups showed improved object recognition in a different task that was maintained for a week, but the video group did not show better retention than the image group (Experiment 2).

Conclusions

Training with video materials leads to more improvement than training with still images in simulated RI perception, but this better improvement was specific to the trained task.

Translational Relevance

We recommend videos as better training materials than still images for patients with RIs to improve object recognition when the task-goal is highly specific. We also propose here that achieving highly specific training goals runs the risk of limiting the generalization of the training effects.

Retinal prosthetic vision simulation: temporal aspects

Objective. The perception of individuals fitted with retinal prostheses is not fully understood, although several retinal implants have been tested and commercialized. Realistic simulations of perception with retinal implants would be useful for future development and evaluation of such systems.Approach.We implemented a retinal prosthetic vision simulation, including temporal features, which have not been previously simulated. In particular, the simulation included temporal aspects such as persistence and perceptual fading of phosphenes and the electrode activation rate.Main results.The simulated phosphene persistence showed an effective reduction in flickering at low electrode activation rates. Although persistence has a positive effect on static scenes, it smears dynamic scenes. Perceptual fading following continuous stimulation affects prosthetic vision of both static and dynamic scenes by making them disappear completely or partially. However, we showed that perceptual fading of a static stimulus might be countered by head-scanning motions, which together with the persistence revealed the contours of the faded object. We also showed that changing the image polarity may improve simulated prosthetic vision in the presence of persistence and perceptual fading.Significance.Temporal aspects have important roles in prosthetic vision, as illustrated by the simulations. Considering these aspects may improve the future design, the training with, and evaluation of retinal prostheses.

Human Eye Optics within a Non-Euclidian Geometrical Approach and Some Implications in Vision Prosthetics Design

An analogy with our previously published theory on the ionospheric auroral gyroscope provides a new perspective in human eye optics. Based on cone cells' real distribution, we model the human eye macula as a pseudospherical surface. This allows the rigorous description of the photoreceptor cell densities in the parafoveal zones modeled further by an optimized paving method. The hexagonal photoreceptors' distribution has been optimally projected on the elliptical pseudosphere, thus designing a prosthetic array counting almost 7000 pixel points. Thanks to the high morphological similarities to a normal human retina, the visual prosthesis performance in camera-free systems might be significantly improved.

Micro-Fabrication of Components for a High-Density Sub-Retinal Visual Prosthesis

We present a retrospective of unique micro-fabrication problems and solutions that were encountered through over 10 years of retinal prosthesis product development, first for the Boston Retinal Implant Project initiated at the Massachusetts Institute of Technology and at Harvard Medical School’s teaching hospital, the Massachusetts Eye and Ear—and later at the startup company Bionic Eye Technologies, by some of the same personnel. These efforts culminated in the fabrication and assembly of 256+ channel visual prosthesis devices having flexible multi-electrode arrays that were successfully implanted sub-retinally in mini-pig animal models as part of our pre-clinical testing program. We report on the processing of the flexible multi-layered, planar and penetrating high-density electrode arrays, surgical tools for sub-retinal implantation, and other parts such as coil supports that facilitated the implantation of the peri-ocular device components. We begin with an overview of the implantable portion of our visual prosthesis system design, and describe in detail the micro-fabrication methods for creating the parts of our system that were assembled outside of our hermetically-sealed electronics package. We also note the unique surgical challenges that sub-retinal implantation of our micro-fabricated components presented, and how some of those issues were addressed through design, materials selection, and fabrication approaches.

An update on retinal prostheses

Retinal prostheses are designed to restore a basic sense of sight to people with profound vision loss. They require a relatively intact posterior visual pathway (optic nerve, lateral geniculate nucleus and visual cortex). Retinal implants are options for people with severe stages of retinal degenerative disease such as retinitis pigmentosa and age-related macular degeneration. There have now been three regulatory-approved retinal prostheses. Over five hundred patients have been implanted globally over the past 15 years. Devices generally provide an improved ability to localize high-contrast objects, navigate, and perform basic orientation tasks. Adverse events have included conjunctival erosion, retinal detachment, loss of light perception, and the need for revision surgery, but are rare. There are also specific device risks, including overstimulation (which could cause damage to the retina) or delamination of implanted components, but these are very unlikely. Current challenges include how to improve visual acuity, enlarge the field-of-view, and reduce a complex visual scene to its most salient components through image processing. This review encompasses the work of over 40 individual research groups who have built devices, developed stimulation strategies, or investigated the basic physiology underpinning retinal prostheses. Current technologies are summarized, along with future challenges that face the field.

Vitreo-macular interface disorders in retinitis pigmentosa

PURPOSE

To investigate the prevalence and progression of vitreo-macular interface disorders (VMID) phenotypes and their natural history in retinitis pigmentosa (RP).

METHODS

A total of 257 eyes of 145 RP patients with VMID were retrospectively evaluated. Patients were divided according to the VMID subtypes into epiretinal membranes (ERMs), vitreo-macular traction (VMT) group, and macular hole (MH). Serial eye-tracked spectral-domain optical coherence tomography (SD-OCT) and best-corrected visual acuity (BCVA) changes were analyzed for a mean follow-up of 36.95 months. The status of posterior vitreous cortex was also considered. A control group of 65 eyes belonging to 65 RP patients with no macular changes was also recruited.

RESULTS

VMID and control groups had the same baseline BCVA (0.50 vs 0.44 LogMAR) and did not differ in terms of phakic status. Different VMID groups had similar BCVA at baseline (p = 0.98). ERM represented the most prevalent disorder (207/257 eyes, 80.5%), followed by 35/257 (13.6%) VMT, and 15/257 Lamellar MH (LMH) eyes (5.8%). There were no cases of full thickness MH. Throughout the 36.9 months of follow-up, BCVA decreased an average 0.09 LogMAR from 0.31 to 0.4 in VMID patients and 0.01 in controls. VMID subgroup analysis showed a significant BCVA decrease in ERM patients (- 20.29%, p < 0.001), while VMT and LMH did not change significantly. Foveal thickness also remained stable over time. Complete PVD was present in 11 eyes in ERM, VMT, and LMH.

CONCLUSIONS

Our study confirms the high prevalence of VMID in RP patients; however, only ERMs determined a significant loss of vision over 24 months. The high prevalence of VMID in RP patients suggests that macular alteration other than edema represents part of disease spectrum.

A bionic eye: performance of the Argus II retinal prosthesis in low-vision and social rehabilitation of patients with end-stage retinitis pigmentosa

The death of outer retinal layers occurring in retinitis pigmentosa causes severe visual impairment and often leads to total blindness. Inner retinal layers are spared, though, which provides a possibility of inducing visual perception by direct electrical stimulation of intact retinal cells. This article presents clinical outcomes of two patients who were the first in Russia to have received the Argus II Retinal Prosthesis System. Both implantations were successful. No complications were reported throughout the entire follow-up period. Upon completing 3 rehabilitation sessions, the patients were able to navigate indoors and outdoors, locate small high-contrast objects, discern contours of large objects and people’s silhouettes.

Prospects of Optogenetic Prosthesis of the Degenerative Retina of the Eye

The review discusses the prospects of using rhodopsin as an optogenetic tool for prosthetics of degenerative (blind) eye retina and the principles of optogenetic techniques. Retinal-containing proteins that depolarize/hyperpolarize the plasma membrane of nerve cells and, accordingly, excite/inhibit physiological activity of neurons, are described. The problem of what cells of the degenerative retina can be treated with what particular rhodopsins is discussed in detail. Viruses and promoters required for the rhodopsin gene delivery into the degenerative retina cells are described. In conclusion, main concepts and tasks associated with the optogenetic prosthetic treatment of degenerative retina employing rhodopsins are presented.

The Effect of Irrelevant Environmental Noise on the Performance of Visual-to-Auditory Sensory Substitution Devices Used by Blind Adults

Visual-to-auditory Sensory Substitution Devices (SSDs) are a family of non-invasive devices for visual rehabilitation aiming at conveying whole-scene visual information through the intact auditory modality. Although proven effective in lab environments, the use of SSDs has yet to be systematically tested in real-life situations. To start filling this gap, in the present work we tested the ability of expert SSD users to filter out irrelevant background noise while focusing on the relevant audio information. Specifically, nine blind expert users of the EyeMusic visual-to-auditory SSD performed a series of identification tasks via SSDs (i.e., shape, color, and conjunction of the two features). Their performance was compared in two separate conditions: silent baseline, and with irrelevant background sounds from real-life situations, using the same stimuli in a pseudo-random balanced design. Although the participants described the background noise as disturbing, no significant performance differences emerged between the two conditions (i.e., noisy; silent) for any of the tasks. In the conjunction task (shape and color) we found a non-significant trend for a disturbing effect of the background noise on performance. These findings suggest that visual-to-auditory SSDs can indeed be successfully used in noisy environments and that users can still focus on relevant auditory information while inhibiting irrelevant sounds. Our findings take a step towards the actual use of SSDs in real-life situations while potentially impacting rehabilitation of sensory deprived individuals.

Cross-task perceptual learning of object recognition in simulated retinal implant perception

The perception gained by retina implants (RI) is limited, which asks for a learning regime to improve patients' visual perception. Here we simulated RI vision and investigated if object recognition in RI patients can be improved and maintained through training. Importantly, we asked if the trained object recognition can be generalized to a new task context, and to new viewpoints of the trained objects. For this purpose, we adopted two training tasks, a labelling task where participants had to choose the correct label out of other distracting labels for the presented object, and a reverse labelling task where participants had to choose the correct object out of other distracting objects to match the presented label. Our results showed that, despite of the task order, recognition performance was improved in both tasks and lasted at least for a week. The improved object recognition, however, can be transferred only from the labelling task to the reverse labelling task but not vice versa. Additionally, the trained object recognition can be transferred to new viewpoints of the trained objects only in the labelling task but not in the reverse labelling task. Training with the labelling task is therefore recommended for RI patients to achieve persistent and flexible visual perception.

Occurrence of Phosphenes in Patients Undergoing Proton Beam Therapy for Ocular Tumor

PURPOSE

Phosphenes are frequently reported by patients irradiated in the head and neck area. The aim of the present study was to characterize and investigate potential mechanisms of proton beam therapy (PBT)-induced phosphenes in a large population of patients undergoing PBT for ocular tumors.

DESIGN

Prospective cohort study.

METHODS

Consecutive patients who underwent PBT in a single center were included. Immediately after the first session, all patients completed a questionnaire collecting information about the presence of phosphenes as well as their color, shape, and duration. Patient, tumor and treatment characteristics (dose volume histograms) were also collected.

RESULTS

Among the 474 patients included, 62.8% reported phosphenes during the first session of PBT. Reported colors were mainly blue-violet (70.5%) and white (14.1%). The prevalence of phosphenes was higher in younger patients (P = .003); other patient or ocular characteristics were not associated with the occurrence of phosphenes. Irradiation of the macula (P < .001) and/or optic disc (P < .001) were significantly associated with the presence of phosphenes, whereas blue-violet color was only associated with young age and irradiation of macular area (P = .04). Pupillary constriction was reported for 57.1% of patients with phosphenes vs 18.5% of patients without (P < .001). Blue-violet phosphenes (P < .001) and irradiation of macula (P = .001) were statistically associated with pupillary constriction.

CONCLUSIONS

The present study reported a high rate of phosphenes in patients irradiated by PBT for ocular tumor. Their blue-violet color and their association with a pupillary constriction probably indicates the stimulation of S-cones and retinal ganglion cells that reflects the activation of the afferent visual pathway.

Optoelectronic system for brain neuronal network stimulation

We propose an optoelectronic system for stimulation of living neurons. The system consists of an electronic circuit based on the FitzHugh–Nagumo model, an optical fiber, and a photoelectrical converter. We used this system for electrical stimulation of hippocampal living neurons in acute hippocampal brain slices (350-μm thick) obtained from a 20–28 days old C57BL/6 mouse or a Wistar rat. The main advantage of our system over other similar stimulators is that it contains an optical fiber for signal transmission instead of metallic wires. The fiber is placed between the electronic circuit and stimulated neurons and provides galvanic isolation from external electrical and magnetic fields. The use of the optical fiber allows avoiding electromagnetic noise and current flows which could affect metallic wires. Furthermore, it gives us the possibility to simulate “synaptic plasticity” by adaptive signal transfer through optical fiber. The proposed optoelectronic system (hybrid neural circuit) provides a very high efficiency in stimulating hippocampus neurons and can be used for restoring brain activity in particular regions or replacing brain parts (neuroprosthetics) damaged due to a trauma or neurodegenerative diseases.

Extraspectral Imaging for Improving the Perceived Information Presented in Retinal Prosthesis

Retinal prosthesis is steadily improving as a clinical treatment for blindness caused by retinitis pigmentosa. However, despite the continued exciting progress, the level of visual return is still very poor. It is also unlikely that those utilising these devices will stop being legally blind in the near future. Therefore, it is important to develop methods to maximise the transfer of useful information extracted from the visual scene. Such an approach can be achieved by digitally suppressing less important visual features and textures within the scene. The result can be interpreted as a cartoon-like image of the scene. Furthermore, utilising extravisual wavelengths such as infrared can be useful in the decision process to determine the optimal information to present. In this paper, we, therefore, present a processing methodology that utilises information extracted from the infrared spectrum to assist in the preprocessing of the visual image prior to conversion to retinal information. We demonstrate how this allows for enhanced recognition and how it could be implemented for optogenetic forms of retinal prosthesis. The new approach has been quantitatively evaluated on volunteers showing 112% enhancement in recognizing objects over normal approaches.

Preoperative Candidate Evaluations for Retinal Prosthesis Trials

With the development of retinal prostheses, the first clinical trials for these devices are being planned or are already underway. Yet the candidate selection criteria and the preparation of early recipients have received very little attention. As clinical trials grow in scope, the need for such issues will become increasingly important. Especially in preliminary stages of clinical trials, it is absolutely vital to develop the preoperative evaluations of candidates in identifying the suitable candidate for retinal prosthesis implants. This paper addresses the preoperative evaluations of candidates that may provide insight into selecting potential candidates for retinal prosthetic devices. Screening an ideal candidate is complex and, at the minimum, should include medical evaluations, visual function tests, assessments of psychological status, motivation, expectations, as well as rehabilitation capabilities. This will ensure that appropriate candidates are selected, candidate expectations are managed, and success with the retinal prosthetic devices is maximized.

Waist-up protection for blind individuals using the EyeCane as a primary and secondary mobility aid

Background: One of the most stirring statistics in relation to the mobility of blind individuals is the high rate of upper body injuries, even when using the white-cane.

Objective: We here addressed a rehabilitation- oriented challenge of providing a reliable tool for blind people to avoid waist-up obstacles, namely one of the impediments to their successful mobility using currently available methods (e.g., white-cane).

Methods: We used the EyeCane, a device we developed which translates distances from several angles to haptic and auditory cues in an intuitive and unobtrusive manner, serving both as a primary and secondary mobility aid. We investigated the rehabilitation potential of such a device in facilitating visionless waist-up body protection.

Results: After ∼5 minutes of training with the EyeCane blind participants were able to successfully detect and avoid obstacles waist-high and up. This was significantly higher than their success when using the white-cane alone. As avoidance of obstacles required participants to perform an additional cognitive process after their detection, the avoidance rate was significantly lower than the detection rate.

Conclusion: Our work has demonstrated that the EyeCane has the potential to extend the sensory world of blind individuals by expanding their currently accessible inputs, and has offered them a new practical rehabilitation tool.

Ganglion Cell Assessment in Rodents with Retinal Degeneration

Analysis of how retinal ganglion cells change in retinal degeneration is critical for evaluating the potential of photoreceptor restorative therapies. Immunocytochemistry in combination with image analysis provides a way for quantifying not only the density of ganglion cells during disease, but also information about their morphology and an evaluation of excitatory and inhibitory synaptic inputs. Here, we describe how indirect immunofluorescence can be used in retinal whole mounts to obtain information about ganglion cells in retinal degeneration.

An optimized content-aware image retargeting method: toward expanding the perceived visual field of the high-density retinal prosthesis recipients

OBJECTIVE

Retinal prosthesis devices have shown great value in restoring some sight for individuals with profoundly impaired vision, but the visual acuity and visual field provided by prostheses greatly limit recipients' visual experience. In this paper, we employ computer vision approaches to seek to expand the perceptible visual field in patients implanted potentially with a high-density retinal prosthesis while maintaining visual acuity as much as possible.

APPROACH

We propose an optimized content-aware image retargeting method, by introducing salient object detection based on color and intensity-difference contrast, aiming to remap important information of a scene into a small visual field and preserve their original scale as much as possible. It may improve prosthetic recipients' perceived visual field and aid in performing some visual tasks (e.g. object detection and object recognition). To verify our method, psychophysical experiments, detecting object number and recognizing objects, are conducted under simulated prosthetic vision. As control, we use three other image retargeting techniques, including Cropping, Scaling, and seam-assisted shrinkability.

MAIN RESULTS

Results show that our method outperforms in preserving more key features and has significantly higher recognition accuracy in comparison with other three image retargeting methods under the condition of small visual field and low-resolution.

SIGNIFICANCE

The proposed method is beneficial to expand the perceived visual field of prosthesis recipients and improve their object detection and recognition performance. It suggests that our method may provide an effective option for image processing module in future high-density retinal implants.

Radial Peripapillary Capillary Network in Patients with Retinitis Pigmentosa: An Optical Coherence Tomography Angiography Study

Purpose

To investigate radial peripapillary capillary (RPC) network in patients affected by retinitis pigmentosa (RP).

Methods

Eleven patients (22 eyes) with previous diagnosis of RP and 16 age-matched healthy subjects (16 eyes) were enrolled. The diagnosis of RP was made based on both clinical features and electrophysiological examination. All patients underwent a complete ophthalmologic examination, including optical coherence tomography angiography and visual field (VF). The primary outcomes were the RPC vessel density in the peripapillary and disk areas; the secondary outcomes were the peripapillary retinal nerve fiber layer (RNFL) thickness and the mean defect at VF.

Results

A total of 19 eyes of 11 RP patients (5 males, 6 females) and 16 eyes of 16 healthy subjects (10 males, 6 females) were included for the analysis. RPC vessel density in the disk area was 46.5 ± 7.1% in the RP group and 45.4 ± 10.6% in the control group (p = 0.754). RPC vessel density in the peripapillary area was significantly reduced in the RP group after the comparison with the control group (52.5 ± 5.0 and 57.2 ± 5.1%, respectively, p = 0.011). RNFL thickness was 85.9 ± 20.4 μm in the RP group and 104.0 ± 6.4 μm in the control group (p = 0.002). RPC vessel density was significantly correlated with RNFL thickness values in RP patients, both in the disk and in the peripapillary area (Rho = 0.599 and p = 0.007 in the disk area, Rho = 0.665 and p = 0.002 in the peripapillary area, respectively).

Conclusion

We showed that density of RPC is reduced in these patients in the peripapillary area. Moreover, the RPC vessel density correlates with the RNFL thickness.

Recognition of a Virtual Scene via Simulated Prosthetic Vision

In order to effectively aid the blind with optimal low-resolution vision and visual recovery training, pathfinding and recognition tests were performed using a simulated visual prosthetic scene. Simple and complex virtual scenes were built using 3DMAX and Unity, and pixelated to three different resolutions (32 × 32, 64 × 64, and 128 × 128) for real-time pixel processing. Twenty subjects were recruited to complete the pathfinding and object recognition tasks within the scene. The recognition accuracy and time required were recorded and analyzed after the trials. In the simple simulated prosthetic vision (SPV) scene, when the resolution was increased from 32 × 32 to 48 × 48, the object recognition time decreased from 92.19 ± 6.97 to 43.05 ± 6.08 s, and the recognition accuracy increased from 51.22 ± 8.53 to 85.52 ± 4.93%. Furthermore, the number of collisions decreased from 10.00 ± 2.31 to 3.00 ± 0.68. When the resolution was increased from 48 × 48 to 64 × 64, the object recognition time further decreased from 43.05 ± 6.08 to 19.46 ± 3.71 s, the recognition accuracy increased from 85.52 ± 4.93 to 96.89 ± 2.06%, and the number of collisions decreased from 3.00 ± 0.68 to 1.00 ± 0.29. In complex scenes, the time required to recognize the room type decreased from 115.00 ± 23.02 to 68.25 ± 17.23 s, and object recognition accuracy increased from 65.69 ± 9.61 to 80.42 ± 7.70% when the resolution increased from 48 × 48 to 64 × 64. When the resolution increased from 64 × 64 to 128 × 128, the time required to recognize the room type decreased from 68.25 ± 17.23 to 44.88 ± 9.94 s, and object recognition accuracy increased from 80.42 ± 7.71 to 85.69 ± 7.39%. Therefore, one can conclude that there are correlations between pathfinding and recognition. When the resolution increased, the time required for recognition decreased, the recognition accuracy increased, and the number of collisions decreased. Although the subjects could partially complete the recognition task at a resolution of 32 × 32, the recognition time was too long and recognition accuracy was not good enough to identify simple scenes. Complex scenes required a resolution of at least 48 × 48 for complete recognition. In addition, increasing the resolution shortened the time required to identify the type of room, and improved the recognition accuracy.

Developing a Very Low Vision Orientation and Mobility Test Battery (O&M-VLV)

PURPOSE

This study aimed to determine the feasibility of an assessment of vision-related orientation and mobility (O&M) tasks in persons with severe vision loss. These tasks may be used for future low vision rehabilitation clinical assessments or as outcome measures in vision restoration trials.

METHODS

Forty legally blind persons (mean visual acuity logMAR 2.3, or hand movements) with advanced retinitis pigmentosa participated in the Orientation & Mobility-Very Low Vision (O&M-VLV) subtests from the Low Vision Assessment of Daily Activities (LoVADA) protocol. Four categories of tasks were evaluated: route travel in three indoor hospital environments, a room orientation task (the "cafe"), a visual exploration task (the "gallery"), and a modified version of the Timed Up and Go (TUG) test, which assesses re-orientation and route travel. Spatial cognition was assessed using the Stuart Tactile Maps test. Visual acuity and visual fields were measured.

RESULTS

A generalized linear regression model showed that a number of measures in the O&M-VLV tasks were related to residual visual function. The percentage of preferred walking speed without an aid on three travel routes was associated with visual field (p < 0.01 for all routes) whereas the number of contacts with obstacles during route travel was associated with acuity (p = 0.001). TUG-LV task time was associated with acuity (p = 0.003), as was the cafe time and distance traveled (p = 0.006 and p < 0.001, respectively). The gallery score was the only measure that was significantly associated with both residual acuity and fields (p < 0.001 and p = 0.001, respectively).

CONCLUSIONS

The O&M-VLV was designed to capture key elements of O&M performance in persons with severe vision loss, which is a population not often studied previously. Performance on these tasks was associated with both binocular visual acuity and visual field. This new protocol includes assessments of orientation, which may be of benefit in vision restoration clinical trials.

Tuning electrical stimulation for thalamic visual prosthesis: An autoencoder-based approach

Visual prosthesis holds hope of vision restoration for millions with retinal degenerative diseases. Machine learning techniques such as artificial neural networks could help in improving prosthetic devices as they could learn how the brain encodes information and imitate that code. This paper introduces an autoencoder-based approach for tuning thalamic visual prostheses. The objective of the proposed approach is to estimate electrical stimuli that are equivalent to a given natural visual stimulus, in a way such that they both elicit responses that are as similar as possible when introduced to a Lateral Geniculate Nucleus (LGN) population. Applying the proposed method to a probabilistic model of LGN neurons, results demonstrate a significant similarity between both responses with a mean correlation of 0.672 for optimal electrodes placement and 0.354 for random electrodes placement. The results indicate the efficacy of the proposed approach in estimating an electrical stimulus equivalent to a specific visual stimulus.

Mechanisms of phosphenes in irradiated patients

Anomalous visual perceptions have been reported in various diseases of the retina and visual pathways or can be experienced under specific conditions in healthy individuals. Phosphenes are perceptions of light in the absence of ambient light, occurring independently of the physiological and classical photonic stimulation of the retina. They are a frequent symptom in patients irradiated in the region of the central nervous system (CNS), head and neck and the eyes. Phosphenes have historically been attributed to complex physical phenomena such as Cherenkov radiation. While phosphenes are related to Cherenkov radiation under high energy photon/electron irradiation conditions, physical phenomena are unlikely to be responsible for light flashes at energies used for ocular proton therapy. Phosphenes may involve a direct role for ocular photoreceptors and possible interactions between cones and rods. Other mechanisms involving the retinal ganglion cells or ultraweak biophoton emission and rhodopsin bleaching after exposure to free radicals are also likely to be involved. Despite their frequency as shown in our preliminary observations, phosphenes have been underreported probably because their mechanism and impact are poorly understood. Recently, phosphenes have been used to restore the vision and whether they might predict vision loss after therapeutic irradiation is a current field of investigation. We have reviewed and also investigated here the mechanisms related to the occurrence of phosphenes in irradiated patients and especially in patients irradiated by proton therapy for ocular tumors.

Learning to see again: biological constraints on cortical plasticity and the implications for sight restoration technologies

The 'bionic eye'-so long a dream of the future-is finally becoming a reality with retinal prostheses available to patients in both the US and Europe. However, clinical experience with these implants has made it apparent that the visual information provided by these devices differs substantially from normal sight. Consequently, the ability of patients to learn to make use of this abnormal retinal input plays a critical role in whether or not some functional vision is successfully regained. The goal of the present review is to summarize the vast basic science literature on developmental and adult cortical plasticity with an emphasis on how this literature might relate to the field of prosthetic vision. We begin with describing the distortion and information loss likely to be experienced by visual prosthesis users. We then define cortical plasticity and perceptual learning, and describe what is known, and what is unknown, about visual plasticity across the hierarchy of brain regions involved in visual processing, and across different stages of life. We close by discussing what is known about brain plasticity in sight restoration patients and discuss biological mechanisms that might eventually be harnessed to improve visual learning in these patients.

Development of the Ultra-Low Vision Visual Functioning Questionnaire (ULV-VFQ)

Purpose

To develop and psychometrically evaluate a visual functioning questionnaire (VFQ) in an ultra-low vision (ULV) population.

Methods

Questionnaire items, based on visual activities self-reported by a ULV population, were categorized by functional visual domain (e.g., mobility) and visual aspect (e.g., contrast) to ensure a representative distribution. In Round 1, an initial set of 149 items was generated and administered to 90 participants with ULV (visual acuity [VA] ≤ 20/500; mean [SD] age 61 [15] years), including six patients with a retinal implant. Psychometric properties were evaluated through Rasch analysis and a revised set (150 items) was administered to 80 participants in Round 2.

Results

In Round 1, the person measure distribution (range, 8.6 logits) was centered at −1.50 logits relative to the item measures. In Round 2, the person measure distribution (range, 9.5 logits) was centered at −0.86 relative to the item mean. The reliability index in both rounds was 0.97 for Items and 0.99 for Persons. Infit analysis showed four underfit items in Round 1, five underfit items in Round 2 with a z-score greater than 4 cutoff. Principal component analysis on the residuals found 69.9% explained variance; the largest component in the unexplained variance was less than 3%.

Conclusions

The ULV-VFQ, developed with content generated from a ULV population, showed excellent psychometric properties as well as superior measurement validity in a ULV population.

Translational Relevance

The ULV-VFQ, part of the Prosthetic Low Vision Rehabilitation (PLoVR) development program, is a new VFQ developed for assessment of functional vision in ULV populations.

Improvements in vision-related quality of life in blind patients implanted with the Argus II Epiretinal Prosthesis

BACKGROUND

The purpose of this analysis is to report the change in quality of life (QoL) after treatment with the Argus II Epiretinal Prosthesis in patients with end-stage retinitis pigmentosa.

METHODS

The Vision and Quality of Life Index (VisQoL) was used to assess changes in QoL dimensions and overall utility score in a prospective 30-patient single-arm clinical study. VisQoL is a multi-attribute instrument consisting of six dimensions (injury, life, roles, assistance, activity and friendship) that may be affected by visual impairment. Within each dimension, patients were divided into two groups based on how much their QoL was affected by their blindness at baseline (moderate/severe or minimal). Outcomes were compared within each dimension sub-group between baseline and the combined follow-up periods using the Friedman test. In addition, data from the six dimensions were combined into a single utility score, with baseline data compared to the combined follow-up periods.

RESULTS

Overall, 80 per cent of the patients reported difficulty in one or more dimensions pre-implant. Composite VisQoL utility scores at follow-up showed no statistically significant change from baseline; however, in three of the six VisQoL dimensions (injury, life and roles), patients with baseline deficits showed significant and lasting improvement after implantation with Argus II. In two of the three remaining dimensions (assistance and activity), data trended toward an improvement. In the final VisQoL dimension (friendship), none of the patients reported baseline deficits, suggesting that patients had largely adjusted to this attribute.

CONCLUSION

Patients whose vision negatively affected them with respect to three VisQoL dimensions (that is, getting injured, coping with the demands of their life and fulfilling their life roles) reported significant improvement in QoL after implantation of the Argus II retinal prosthesis. Furthermore, the benefit did not deteriorate at any point during the 36-month follow-up, suggesting a long-term, durable improvement.

Visual BOLD Response in Late Blind Subjects with Argus II Retinal Prosthesis

Retinal prosthesis technologies require that the visual system downstream of the retinal circuitry be capable of transmitting and elaborating visual signals. We studied the capability of plastic remodeling in late blind subjects implanted with the Argus II Retinal Prosthesis with psychophysics and functional MRI (fMRI). After surgery, six out of seven retinitis pigmentosa (RP) blind subjects were able to detect high-contrast stimuli using the prosthetic implant. However, direction discrimination to contrast modulated stimuli remained at chance level in all of them. No subject showed any improvement of contrast sensitivity in either eye when not using the Argus II. Before the implant, the Blood Oxygenation Level Dependent (BOLD) activity in V1 and the lateral geniculate nucleus (LGN) was very weak or absent. Surprisingly, after prolonged use of Argus II, BOLD responses to visual input were enhanced. This is, to our knowledge, the first study tracking the neural changes of visual areas in patients after retinal implant, revealing a capacity to respond to restored visual input even after years of deprivation.

Long-term Repeatability and Reproducibility of Phosphene Characteristics in Chronically Implanted Argus II Retinal Prosthesis Subjects

PURPOSE

Previously published literatures of acute studies on few subjects have shown contradictory evidence on the reproducibility and characteristics of the elicited phosphenes, despite using the same stimulating parameters with epiretinal electrode arrays. In this study, we set out to investigate the long-term repeatilibity and reproducibility of phosphenes in subjects chronically implanted with the Argus II retinal prosthesis (Second Sight Medical Products, Inc., Sylmar, CA, USA).

DESIGN

Retrospective interventional case series and reliability study.

METHODS

Six Argus II subjects of >5 years implantation from a single site participated. The 4-electrode cluster ("quad") closest to fovea was stimulated in each subject with a fixed biphasic current. Perceived phosphenes were depicted relative to subjective visual field center. The stimulus was applied at reducing time intervals from 20 minutes to 1 second. Two sets of stimulations were performed on the same day and 2 further sets repeated on a separate visit >1 week apart.

RESULTS

Each subject depicted phosphenes of consistent shapes and sizes, and reported seeing the same colors with the fixed stimulating parameters, irrespective of the interstimuli intervals. However, there is a wide intersubject variation in the phosphene characteristics. Four subjects drew phosphenes in the same visual field quadrant, as predicted by the quad-fovea location. Two subjects depicted phosphenes in the same hemifield as the expected locations.

CONCLUSION

Phosphenes for each subject were consistently reproducible in all our chronically implanted subjects. This has important implications in the development of long-term pixelated prosthetic vision for future devices.

Towards high-resolution retinal prostheses with direct optical addressing and inductive telemetry

OBJECTIVE

Despite considerable advances in retinal prostheses over the last two decades, the resolution of restored vision has remained severely limited, well below the 20/200 acuity threshold of blindness. Towards drastic improvements in spatial resolution, we present a scalable architecture for retinal prostheses in which each stimulation electrode is directly activated by incident light and powered by a common voltage pulse transferred over a single wireless inductive link.

APPROACH

The hybrid optical addressability and electronic powering scheme provides separate spatial and temporal control over stimulation, and further provides optoelectronic gain for substantially lower light intensity thresholds than other optically addressed retinal prostheses using passive microphotodiode arrays. The architecture permits the use of high-density electrode arrays with ultra-high photosensitive silicon nanowires, obviating the need for excessive wiring and high-throughput data telemetry. Instead, the single inductive link drives the entire array of electrodes through two wires and provides external control over waveform parameters for common voltage stimulation.

MAIN RESULTS

A complete system comprising inductive telemetry link, stimulation pulse demodulator, charge-balancing series capacitor, and nanowire-based electrode device is integrated and validated ex vivo on rat retina tissue.

SIGNIFICANCE

Measurements demonstrate control over retinal neural activity both by light and electrical bias, validating the feasibility of the proposed architecture and its system components as an important first step towards a high-resolution optically addressed retinal prosthesis.

Recent Advances in Flexible and Stretchable Bio-Electronic Devices Integrated with Nanomaterials

Flexible and stretchable electronics and optoelectronics configured in soft, water resistant formats uniquely address seminal challenges in biomedicine. Over the past decade, there has been enormous progress in the materials, designs, and manufacturing processes for flexible/stretchable system subcomponents, including transistors, amplifiers, bio-sensors, actuators, light emitting diodes, photodetector arrays, photovoltaics, energy storage elements, and bare die integrated circuits. Nanomaterials prepared using top-down processing approaches and synthesis-based bottom-up methods have helped resolve the intrinsic mechanical mismatch between rigid/planar devices and soft/curvilinear biological structures, thereby enabling a broad range of non-invasive, minimally invasive, and implantable systems to address challenges in biomedicine. Integration of therapeutic functional nanomaterials with soft bioelectronics demonstrates therapeutics in combination with unconventional diagnostics capabilities. Recent advances in soft materials, devices, and integrated systems are reviewes, with representative examples that highlight the utility of soft bioelectronics for advanced medical diagnostics and therapies.

Rasch Analysis of the Independent Mobility Questionnaire

PURPOSE

The Independent Mobility Questionnaire (IMQ) assesses participants' perceived ability for independent mobility. However, it has not been validated in a severely visually impaired population. The aim of this study was to explore the IMQ's psychometric properties in participants with severe visual impairment.

METHODS

This was a cross-sectional study of 40 participants with retinitis pigmentosa (better eye visual acuity <20/200 and/or visual field <10%). The key psychometric properties of the IMQ were examined using Rasch analysis, including precision, targeting, and item fit. Construct validity was assessed by testing the correlation between the IMQ and the Mobility and Independence subscale of the Impact of Vision Impairment questionnaire (Pearson correlation coefficient, r). Criterion validity was also assessed.

RESULTS

The IMQ had excellent precision (Person Separation Index, 3.01) with the capacity to distinguish at least four strata of participant ability, and item difficulty was well targeted to participant ability (difference between mean person and item measures, -0.21). Items 34, 35, 21, and 14 displayed misfit (infit MnSq >1.4); however, given our sample size restrictions, these items were not removed from the analysis. The IMQ had good construct validity (moderate correlation with the Impact of Vision Impairment Mobility subscale, r = 0.595, p < 0.05) but did not demonstrate criterion validity.

CONCLUSIONS

The psychometric properties of the IMQ were promising. Our findings are useful for researchers evaluating the effectiveness of novel treatment technologies on mobility in a severely visually impaired population from the participant's perspective. However, further validation studies in larger samples are required to confirm our results.

Electrophysiology Alterations in Primary Visual Cortex Neurons of Retinal Degeneration (S334ter-line-3) Rats

The dynamic nature of the brain is critical for the success of treatments aimed at restoring vision at the retinal level. The success of these treatments relies highly on the functionality of the surviving neurons along the entire visual pathway. Electrophysiological properties at the retina level have been investigated during the progression of retinal degeneration; however, little is known about the changes in electrophysiological properties that occur in the primary visual cortex (V1) during the course of retinal degeneration. By conducting extracellular recording, we examined the electrophysiological properties of V1 in S334ter-line-3 rats (a transgenic model of retinal degeneration developed to express a rhodopsin mutation similar to that found in human retinitis pigmentosa patients). We measured the orientation tuning, spatial and temporal frequency tunings and the receptive field (RF) size for 127 V1 neurons from 11 S334ter-3 rats and 10 Long-Evans (LE) rats. V1 neurons in the S334ter-3 rats showed weaker orientation selectivity, lower optimal spatial and temporal frequency values and a smaller receptive field size compared to the LE rats. These results suggest that the visual cognitive ability significantly changes during retinal degeneration.

Changes in ganglion cells during retinal degeneration

Inherited retinal degeneration such as retinitis pigmentosa (RP) is associated with photoreceptor loss and concomitant morphological and functional changes in the inner retina. It is not known whether these changes are associated with changes in the density and distribution of synaptic inputs to retinal ganglion cells (RGCs). We quantified changes in ganglion cell density in rd1 and age-matched C57BL/6J-(wildtype, WT) mice using the immunocytochemical marker, RBPMS. Our data revealed that following complete loss of photoreceptors, (∼3months of age), there was a reduction in ganglion cell density in the peripheral retina. We next examined changes in synaptic inputs to A type ganglion cells by performing double labeling experiments in mice with the ganglion cell reporter lines, rd1-Thy1 and age-matched wildtype-Thy1. Ribbon synapses were identified by co-labelling with CtBP2 (RIBEYE) and conventional synapses with the clustering molecule, gephyrin. ON RGCs showed a significant reduction in RIBEYE-immunoreactive synapse density while OFF RGCs showed a significant reduction in the gephyrin-immmunoreactive synapse density. Distribution patterns of both synaptic markers across the dendritic trees of RGCs were unchanged. The change in synaptic inputs to RGCs was associated with a reduction in the number of immunolabeled rod bipolar and ON cone bipolar cells. These results suggest that functional changes reported in ganglion cells during retinal degeneration could be attributed to loss of synaptic inputs.

Neural Substrate Expansion for the Restoration of Brain Function

Restoring neurological and cognitive function in individuals who have suffered brain damage is one of the principal objectives of modern translational neuroscience. Electrical stimulation approaches, such as deep-brain stimulation, have achieved the most clinical success, but they ultimately may be limited by the computational capacity of the residual cerebral circuitry. An alternative strategy is brain substrate expansion, in which the computational capacity of the brain is augmented through the addition of new processing units and the reconstitution of network connectivity. This latter approach has been explored to some degree using both biological and electronic means but thus far has not demonstrated the ability to reestablish the function of large-scale neuronal networks. In this review, we contend that fulfilling the potential of brain substrate expansion will require a significant shift from current methods that emphasize direct manipulations of the brain (e.g., injections of cellular suspensions and the implantation of multi-electrode arrays) to the generation of more sophisticated neural tissues and neural-electric hybrids in vitro that are subsequently transplanted into the brain. Drawing from neural tissue engineering, stem cell biology, and neural interface technologies, this strategy makes greater use of the manifold techniques available in the laboratory to create biocompatible constructs that recapitulate brain architecture and thus are more easily recognized and utilized by brain networks.

A Multimodal, SU-8 - Platinum - Polyimide Microelectrode Array for Chronic In Vivo Neurophysiology

Utilization of polymers as insulator and bulk materials of microelectrode arrays (MEAs) makes the realization of flexible, biocompatible sensors possible, which are suitable for various neurophysiological experiments such as in vivo detection of local field potential changes on the surface of the neocortex or unit activities within the brain tissue. In this paper the microfabrication of a novel, all-flexible, polymer-based MEA is presented. The device consists of a three dimensional sensor configuration with an implantable depth electrode array and brain surface electrodes, allowing the recording of electrocorticographic (ECoG) signals with laminar ones, simultaneously. In vivo recordings were performed in anesthetized rat brain to test the functionality of the device under both acute and chronic conditions. The ECoG electrodes recorded slow-wave thalamocortical oscillations, while the implanted component provided high quality depth recordings. The implants remained viable for detecting action potentials of individual neurons for at least 15 weeks.

Optogenetic Approaches to Restoring Vision

Severe loss of photoreceptor cells in inherited or acquired retinal degenerative diseases can result in partial loss of sight or complete blindness. The optogenetic strategy for restoration of vision utilizes optogenetic tools to convert surviving inner retinal neurons into photosensitive cells; thus, light sensitivity is imparted to the retina after the death of photoreceptor cells. Proof-of-concept studies, especially those using microbial rhodopsins, have demonstrated restoration of light responses in surviving retinal neurons and visually guided behaviors in animal models. Significant progress has also been made in improving microbial rhodopsin-based optogenetic tools, developing virus-mediated gene delivery, and targeting specific retinal neurons and subcellular compartments of retinal ganglion cells. In this article, we review the current status of the field and outline further directions and challenges to the advancement of this strategy toward clinical application and improvement in the outcomes of restored vision.