Full gaze contingency provides better reading performance than head steering alone in a simulation of prosthetic vision

Gaze-contingent processing improves mobility, scene recognition and visual search in simulated head-steered prosthetic vision

Objective.The enabling technology of visual prosthetics for the blind is making rapid progress. However, there are still uncertainties regarding the functional outcomes, which can depend on many design choices in the development. In visual prostheses with a head-mounted camera, a particularly challenging question is how to deal with the gaze-locked visual percept associated with spatial updating conflicts in the brain. The current study investigates a recently proposed compensation strategy based on gaze-contingent image processing with eye-tracking. Gaze-contingent processing is expected to reinforce natural-like visual scanning and reestablished spatial updating based on eye movements. The beneficial effects remain to be investigated for daily life activities in complex visual environments.Approach.The current study evaluates the benefits of gaze-contingent processing versus gaze-locked and gaze-ignored simulations in the context of mobility, scene recognition and visual search, using a virtual reality simulated prosthetic vision paradigm with sighted subjects.Main results.Compared to gaze-locked vision, gaze-contingent processing was consistently found to improve the speed in all experimental tasks, as well as the subjective quality of vision. Similar or further improvements were found in a control condition that ignores gaze-dependent effects, a simulation that is unattainable in the clinical reality.Significance.Our results suggest that gaze-locked vision and spatial updating conflicts can be debilitating for complex visually-guided activities of daily living such as mobility and orientation. Therefore, for prospective users of head-steered prostheses with an unimpaired oculomotor system, the inclusion of a compensatory eye-tracking system is strongly endorsed.

Towards biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses

Blindness affects millions of people around the world. A promising solution to restoring a form of vision for some individuals are cortical visual prostheses, which bypass part of the impaired visual pathway by converting camera input to electrical stimulation of the visual system. The artificially induced visual percept (a pattern of localized light flashes, or 'phosphenes') has limited resolution, and a great portion of the field's research is devoted to optimizing the efficacy, efficiency, and practical usefulness of the encoding of visual information. A commonly exploited method is non-invasive functional evaluation in sighted subjects or with computational models by using simulated prosthetic vision (SPV) pipelines. An important challenge in this approach is to balance enhanced perceptual realism, biologically plausibility, and real-time performance in the simulation of cortical prosthetic vision. We present a biologically plausible, PyTorch-based phosphene simulator that can run in real-time and uses differentiable operations to allow for gradient-based computational optimization of phosphene encoding models. The simulator integrates a wide range of clinical results with neurophysiological evidence in humans and non-human primates. The pipeline includes a model of the retinotopic organization and cortical magnification of the visual cortex. Moreover, the quantitative effects of stimulation parameters and temporal dynamics on phosphene characteristics are incorporated. Our results demonstrate the simulator's suitability for both computational applications such as end-to-end deep learning-based prosthetic vision optimization as well as behavioral experiments. The modular and open-source software provides a flexible simulation framework for computational, clinical, and behavioral neuroscientists working on visual neuroprosthetics.

The Influence of Phosphene Synchrony in Driving Object Binding in a Simulation of Artificial Vision

Purpose

Electrical microstimulation techniques used in visual prostheses are designed to restore visual function following acquired blindness. Patterns of induced focal percepts, known as phosphenes, are achieved by applying localized electrical pulses to the visual pathway to bypass the impaired site in order to convey images from the external world. Here, we use a simulation of artificial vision to manipulate relationships between individual phosphenes to observe the effects on object binding and perception. We hypothesize that synchronous phosphene presentation will facilitate object perception as compared to asynchronous presentation.

Methods

A model system that tracks gaze position of normal, sighted participants to present patterns of phosphenes on a computer screen was used to simulate prosthetic vision. Participants performed a reading task at varying font sizes (1.1-1.4 logMAR) and under varying levels of phosphene temporal noise while reading accuracy and speed were measured.

Results

Reading performance was significantly affected by temporal noise in phosphene presentation, with increasing desynchronization leading to lower reading scores. A drop in performance was also observed when the total latency between the gaze position and phosphene update was increased without adding temporal noise.

Conclusions

Object perception (here, text perception) is enhanced with synchronously presented phosphenes as compared to asynchronously presented ones. These results are fundamental for developing an efficient temporal pattern of stimulation and for the creation of high-fidelity prosthetic vision.

A novel simulation paradigm utilising MRI-derived phosphene maps for cortical prosthetic vision

Objective.We developed a realistic simulation paradigm for cortical prosthetic vision and investigated whether we can improve visual performance using a novel clustering algorithm.Approach.Cortical visual prostheses have been developed to restore sight by stimulating the visual cortex. To investigate the visual experience, previous studies have used uniform phosphene maps, which may not accurately capture generated phosphene map distributions of implant recipients. The current simulation paradigm was based on the Human Connectome Project retinotopy dataset and the placement of implants on the cortices from magnetic resonance imaging scans. Five unique retinotopic maps were derived using this method. To improve performance on these retinotopic maps, we enabled head scanning and a density-based clustering algorithm was then used to relocate centroids of visual stimuli. The impact of these improvements on visual detection performance was tested. Using spatially evenly distributed maps as a control, we recruited ten subjects and evaluated their performance across five sessions on the Berkeley Rudimentary Visual Acuity test and the object recognition task.Main results.Performance on control maps is significantly better than on retinotopic maps in both tasks. Both head scanning and the clustering algorithm showed the potential of improving visual ability across multiple sessions in the object recognition task.Significance.The current paradigm is the first that simulates the experience of cortical prosthetic vision based on brain scans and implant placement, which captures the spatial distribution of phosphenes more realistically. Utilisation of evenly distributed maps may overestimate the performance that visual prosthetics can restore. This simulation paradigm could be used in clinical practice when making plans for where best to implant cortical visual prostheses.

Attitudes of potential recipients toward emerging visual prosthesis technologies

With the advent of multiple visual prosthesis devices to treat blindness, the question of how potential patients view such interventions becomes important in order to understand the levels of expectation and acceptance, and the perceived risk-reward balance across the different device approaches. Building on previous work on single device approaches done with blind individuals in Chicago and Detroit, USA, Melbourne, Australia, and Bejing, China, we investigated attitudes in blind individuals in Athens, Greece with coverage expanded to three of the contemporary approaches, Retinal, Thalamic, and Cortical. We presented an informational lecture on the approaches, had potential participants fill out a preliminary Questionnaire 1, then organized selected subjects into focus groups for guided discussion on visual prostheses, and finally had these subjects fill out a more detailed Questionnaire 2. We report here the first quantitative data that compares multiple prosthesis approaches. Our primary findings are that for these potential patients, perceived risk continues to outweigh perceived benefits, with the Retinal approach having the least negative overall impression and the Cortical approach the most negative. Concerns about the quality of restored vision were primary. Factors that drove the choice of hypothetical participation in a clinical trial were age and years of blindness. Secondary factors focused on positive clinical outcomes. The focus groups served to swing the impressions of each approach from neutrality toward the extremes of a Likert scale, and shifted the overall willingness to participate in a clinical trial from neutral to negative. These results, coupled with informal assessment of audience questions after the informational lecture, suggest that a substantial improvement in performance over currently available devices will be necessary before visual prostheses gain wide acceptance.

A systematic review of extended reality (XR) for understanding and augmenting vision loss

Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. A defining quality of these XR technologies is their ability to update the stimulus based on the user's eye, head, or body movements. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. Here we present a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology to further visual accessibility. In contrast to other reviews, we sample studies from multiple scientific disciplines, focus on technology that augments a person's residual vision, and require studies to feature a quantitative evaluation with appropriate end users. We summarize prominent findings from different XR research areas, show how the landscape has changed over the past decade, and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the usability of different XR-based accessibility aids.

Reading text works better than watching videos to improve acuity in a simulation of artificial vision

Simulated artificial vision is used in visual prosthesis design to answer questions about device usability. We previously reported a striking increase in equivalent visual acuity with daily use of a simulation of artificial vision in an active task, reading sentences, that required high levels of subject engagement, but passive activities are more likely to dominate post-implant experience. Here, we investigated the longitudinal effects of a passive task, watching videos. Eight subjects used a simulation of a thalamic visual prosthesis with 1000 phosphenes to watch 23 episodes of classic American television in daily, 25-min sessions, for a period of 1 month with interspersed reading tests that quantified reading accuracy and reading speed. For reading accuracy, we found similar dynamics to the early part of the learning process in our previous report, here leading to an improvement in visual acuity of 0.15 ± 0.05 logMAR. For reading speed, however, no change was apparent by the end of training. We found that single reading sessions drove about twice the improvement in acuity of single video sessions despite being only half as long. We conclude that while passive viewing tasks may prove useful for post-implant rehabilitation, active tasks are likely to be preferable.