Image Processing Strategies Dedicated to Visual Cortical Stimulators: A Survey

Neural activity shaping utilizing a partitioned target pattern

Electrical stimulation of neural tissue is used in both clinical and experimental devices to evoke a desired spatiotemporal pattern of neural activity. These devices induce a local field that drives neural activation, referred to as an activating function or generator signal. In visual prostheses, the spread of generator signal from each electrode within the neural tissue results in a spread of visual perception, referred to as a phosphene. In cases where neighboring phosphenes overlap, it is desirable to use current steering or neural activity shaping strategies to manipulate the generator signal between the electrodes to provide greater control over the total pattern of neural activity. Applying opposite generator signal polarities in neighboring regions of the retina forces the generator signal to pass through zero at an intermediate point, thus inducing low neural activity that may be perceived as a high-contrast line. This approach provides a form of high contrast visual perception, but it requires partitioning of the target pattern into those regions that use positive or negative generator signals. This discrete optimization is an NP-hard problem that is subject to being trapped in detrimental local minima. This investigation proposes a new partitioning method using image segmentation to determine the most beneficial positive and negative generator signal regions. Utilizing a database of 1000 natural images, the method is compared to alternative approaches based upon the mean squared error of the outcome. Under nominal conditions and with a set computation limit, partitioning provided improvement for 32% of these images. This percentage increased to 89% when utilizing image pre-processing to emphasize perceptual features of the images. The percentage of images that were dealt with most effectively with image segmentation increased as lower computation limits were imposed on the algorithms.

Moving object detection and background enhancement for thalamic visual prostheses

Visual prostheses open the door of hope to restore functional vision for the blind. One of the main challenges facing their development is the limited number of electrodes used in the stimulation process which limits the resolution of the perceived images. To improve the perception, the useful features in the scene need to be enhanced while the other features should be suppressed to achieve better resolution. This paper introduces an image processing method to enhance three main features detectable by the natural visual pathway; namely the contrast, the motion and the edges. It then reduces the size of the image into an activity matrix used to generate the electric stimulation for the electrodes array. We compared the proposed method to four other image processing strategies in terms of the quality of the resulting image in addition to the perceived image using a simulation of prosthetic vision. Results demonstrate that the proposed method outperforms the other techniques in both aspects.

Phosphene object perception employs holistic processing during early visual processing stage

Psychophysical studies have verified the possibility of recovering the visual ability by the form of low-resolution format of images, that is, phosphene-based representations. Our previous study has found that early visual processing for phosphene patterns is configuration based. This study further investigated the configural processing mechanisms of prosthetic vision by analyzing the event-related potential components (P1 and N170) in response to phosphene face and non-face stimuli. The results reveal that the coarse processing of phosphenes involves phosphene-specific holistic processing that recovers separated phosphenes into a gestalt; low-level feature processing of phosphenes is also enhanced compared with that of normal stimuli due to increased contrast borders introduced by phosphenes; while fine processing of phosphene stimuli is impaired reflected by reduced N170 amplitude because of the degraded detailed features in the low-resolution format representations. Therefore, we suggest that strategies that can facilitate the specific holistic processing stages of prosthetic vision should be considered in order to improve the performance when designing the visual prosthesis system.

Object recognition under distorted prosthetic vision

Psychophysical studies have reported the efficacy of phosphene-based prosthetic vision in partly recovering the visual function of blind individuals. However, results by far have been based on evenly aligned phosphene arrays, which neglected the complicated visuotopy in the visual prosthesis system. In this study, we investigated how the objects were recognized under the stimuli with distorted phosphene arrays simulated by transformations of barrel distortion, rotation, or translation. The results revealed that distortions significantly decreased the accuracy of categorization (CA) and showed distinct interactive effects with the factors of object category and phosphene array density. Moreover, the CA changed differently with the increase of distortion levels. Regression analysis suggested a phosphene array of at least 10 × 10 be the essential for achieving a CA over the threshold value (CA(t)=62.5%) under distorted prosthetic vision. It is recommended that discriminative features be extracted to improve the performance of prosthetic vision.

A wearable real-time image processor for a vision prosthesis

Rapid progress in recent years has made implantable retinal prostheses a promising therapeutic option in the near future for patients with macular degeneration or retinitis pigmentosa. Yet little work on devices that encode visual images into electrical stimuli have been reported to date. This paper presents a wearable image processor for use as the external module of a vision prosthesis. It is based on a dual-core microprocessor architecture and runs the Linux operating system. A set of image-processing algorithms executes on the digital signal processor of the device, which may be controlled remotely via a standard desktop computer. The results indicate that a highly flexible and configurable image processor can be built with the dual-core architecture. Depending on the image-processing requirements, general-purpose embedded microprocessors alone may be inadequate for implementing image-processing strategies required by retinal prostheses.

Detection, eye-hand coordination and virtual mobility performance in simulated vision for a cortical visual prosthesis device

In order to assess visual performance using a future cortical prosthesis device, the ability of normally sighted and low vision subjects to adapt to a dotted 'phosphene' image was studied. Similar studies have been conduced in the past and adaptation to phosphene maps has been shown but the phosphene maps used have been square or hexagonal in pattern. The phosphene map implemented for this testing is what is expected from a cortical implantation of the arrays of intracortical electrodes, generating multiple phosphenes. The dotted image created depends upon the surgical location of electrodes decided for implantation and the expected cortical response. The subjects under tests were required to perform tasks requiring visual inspection, eye-hand coordination and way finding. The subjects did not have any tactile feedback and the visual information provided was live dotted images captured by a camera on a head-mounted low vision enhancing system and processed through a filter generating images similar to the images we expect the blind persons to perceive. The images were locked to the subject's gaze by means of video-based pupil tracking. In the detection and visual inspection task, the subject scanned a modified checkerboard and counted the number of square white fields on a square checkerboard, in the eye-hand coordination task, the subject placed black checkers on the white fields of the checkerboard, and in the way-finding task, the subjects maneuvered themselves through a virtual maze using a game controller. The accuracy and the time to complete the task were used as the measured outcome. As per the surgical studies by this research group, it might be possible to implant up to 650 electrodes; hence, 650 dots were used to create images and performance studied under 0% dropout (650 dots), 25% dropout (488 dots) and 50% dropout (325 dots) conditions. It was observed that all the subjects under test were able to learn the given tasks and showed improvement in performance with practice even with a dropout condition of 50% (325 dots). Hence, if a cortical prosthesis is implanted in human subjects, they might be able to perform similar tasks and with practice should be able to adapt to dotted images even with a low resolution of 325 dots of phosphene.

A highly flexible system for microstimulation of the visual cortex: design and implementation

This paper presents the design of a system intended to be used as a prosthesis allowing profoundly visually impaired patients to recover partial vision by means of microstimulation in the primary visual cortex area. The main component of the system is a bio-electronic device to be implanted inside the skull of the user, composed of a plurality of stimulation modules, whose actions are controlled via an interface module. Power and data are transmitted to the implant wirelessly through a bidirectional inductive link, allowing diagnosis of the stimulating device and its environment after implantation, as well as power delivery optimization. A high level of flexibility is supported in terms of stimulation parameters, but a configurable communication protocol allows the device to be used with maximum efficiency. The core of an external controller implemented in a system on a programmable chip is also presented, performing data conversion and timing management such that phosphene intensity can be modulated by any parameter defining stimulation, either at the pulse level or in the time domain. Measured performances achieved with a prototype using two types of custom ASICs implemented in a 0.18-mum CMOS process and commercial components fulfill the requirements for a complete visual prosthesis for humans. When on/off activation is used with predefined parameters, stimuli measured on an electronic test bench could attain a rate in excess of 500 k pulses/s.

Recognition of Pixelized Chinese Characters Using Simulated Prosthetic Vision

The rehabilitation of the reading ability of the blind with a limited number of stimulating electrodes is regarded as one of the major functions of the envisioned visual prosthesis. This article systematically studied how many pixels of individual Chinese characters should be needed for correct and economic recognition by blind Chinese subjects. In this study, 40 normal-sighted subjects were tested on a self-developed platform HanziConvertor (Institute for Laser Medicine & Bio-photonics, Shanghai Jiaotong University, China) with digital imaging processing capacities to convert images of printed text into various pixelized patterns made up of discrete dots, and present them orderly on a computer screen. It was found that various complicated factors such as pixel number, character typeface, stroke number, etc., can obviously affect the recognition accuracy. It was also found that optimal recognition accuracy occurs at a specific size of binary pixel array, due to a trade-off between a strictly limited number of stimulation electrodes and character sampling resolution. The results showed that (i) recognition accuracy of pixelized characters is optimal with at least 12 x 12 binary pixels, and therefore it is recommended to apply a minimum of 150 discrete and functioning electrodes for restoring the reading ability of blind Chinese individuals in the visual prosthesis; (ii) fonts of Song Ti and Hei Ti are clearer and more effective than other typefaces; and (iii) characters with fewer strokes lead to better accuracy.

Visual prosthetics 2006: assessment and expectations

This report provides a brief overview of blinding eye diseases for which prosthetic vision may hold promise as a treatment modality, and of current and near-term technological approaches towards the creation of prosthetic interfaces with the remaining visual system. Principal anatomical, physiological, technological and functional obstacles and possible solutions are outlined, and references are provided to pioneering work by over a dozen groups on four continents.