Prediction of cortical responses to simultaneous electrical stimulation of the retina

Neural activity shaping in visual prostheses with deep learning

Objective.The visual perception provided by retinal prostheses is limited by the overlapping current spread of adjacent electrodes. This reduces the spatial resolution attainable with unipolar stimulation. Conversely, simultaneous multipolar stimulation guided by the measured neural responses-neural activity shaping (NAS)-can attenuate excessive spread of excitation allowing for more precise control over the pattern of neural activation. However, defining effective multipolar stimulus patterns is a challenging task. Previous attempts focused on analytical solutions based on an assumed linear nonlinear model of retinal response; an analytical model inversion (AMI) approach. Here, we propose a model-free solution for NAS, using artificial neural networks (ANNs) that could be trained with data acquired from the implant.Approach.Our method consists of two ANNs trained sequentially. The measurement predictor network (MPN) is trained on data from the implant and is used to predict how the retina responds to multipolar stimulation. The stimulus generator network is trained on a large dataset of natural images and uses the trained MPN to determine efficient multipolar stimulus patterns by learning its inverse model. We validate our methodin silicousing a realistic model of retinal response to multipolar stimulation.Main results.We show that our ANN-based NAS approach produces sharper retinal activations than the conventional unipolar stimulation strategy. As a theoretical bench-mark of optimal NAS results, we implemented AMI stimulation by inverting the model used to simulate the retina. Our ANN strategy produced equivalent results to AMI, while not being restricted to any specific type of retina model and being three orders of magnitude more computationally efficient.Significance.Our novel protocol provides a method for efficient and personalized retinal stimulation, which may improve the visual experience and quality of life of retinal prosthesis users.

Quantifying visual acuity for pre-clinical testing of visual prostheses

Objective.Visual prostheses currently restore only limited vision. More research and pre-clinical work are required to improve the devices and stimulation strategies that are used to induce neural activity that results in visual perception. Evaluation of candidate strategies and devices requires an objective way to convert measured and modelled patterns of neural activity into a quantitative measure of visual acuity.Approach.This study presents an approach that compares evoked patterns of neural activation with target and reference patterns. A d-prime measure of discriminability determines whether the evoked neural activation pattern is sufficient to discriminate between the target and reference patterns and thus provides a quantified level of visual perception in the clinical Snellen and MAR scales. The magnitude of the resulting value was demonstrated using scaled standardized 'C' and 'E' optotypes.Main results.The approach was used to assess the visual acuity provided by two alternative stimulation strategies applied to simulated retinal implants with different electrode pitch configurations and differently sized spreads of neural activity. It was found that when there is substantial overlap in neural activity generated by different electrodes, an estimate of acuity based only upon electrode pitch is incorrect; our proposed method gives an accurate result in both circumstances.Significance.Quantification of visual acuity using this approach in pre-clinical development will allow for more rapid and accurate prototyping of improved devices and neural stimulation strategies.

Implications of Neural Plasticity in Retinal Prosthesis

Retinal degenerative diseases such as retinitis pigmentosa cause a progressive loss of photoreceptors that eventually prevents the affected person from perceiving visual sensations. The absence of a visual input produces a neural rewiring cascade that propagates along the visual system. This remodeling occurs first within the retina. Then, subsequent neuroplastic changes take place at higher visual centers in the brain, produced by either the abnormal neural encoding of the visual inputs delivered by the diseased retina or as the result of an adaptation to visual deprivation. While retinal implants can activate the surviving retinal neurons by delivering electric current, the unselective activation patterns of the different neural populations that exist in the retinal layers differ substantially from those in physiologic vision. Therefore, artificially induced neural patterns are being delivered to a brain that has already undergone important neural reconnections. Whether or not the modulation of this neural rewiring can improve the performance for retinal prostheses remains a critical question whose answer may be the enabler of improved functional artificial vision and more personalized neurorehabilitation strategies.

Neural activity shaping utilizing a partitioned target pattern

Electrical stimulation of neural tissue is used in both clinical and experimental devices to evoke a desired spatiotemporal pattern of neural activity. These devices induce a local field that drives neural activation, referred to as an activating function or generator signal. In visual prostheses, the spread of generator signal from each electrode within the neural tissue results in a spread of visual perception, referred to as a phosphene. In cases where neighboring phosphenes overlap, it is desirable to use current steering or neural activity shaping strategies to manipulate the generator signal between the electrodes to provide greater control over the total pattern of neural activity. Applying opposite generator signal polarities in neighboring regions of the retina forces the generator signal to pass through zero at an intermediate point, thus inducing low neural activity that may be perceived as a high-contrast line. This approach provides a form of high contrast visual perception, but it requires partitioning of the target pattern into those regions that use positive or negative generator signals. This discrete optimization is an NP-hard problem that is subject to being trapped in detrimental local minima. This investigation proposes a new partitioning method using image segmentation to determine the most beneficial positive and negative generator signal regions. Utilizing a database of 1000 natural images, the method is compared to alternative approaches based upon the mean squared error of the outcome. Under nominal conditions and with a set computation limit, partitioning provided improvement for 32% of these images. This percentage increased to 89% when utilizing image pre-processing to emphasize perceptual features of the images. The percentage of images that were dealt with most effectively with image segmentation increased as lower computation limits were imposed on the algorithms.

Computational challenges and opportunities for a bi-directional artificial retina

A future artificial retina that can restore high acuity vision in blind people will rely on the capability to both read (observe) and write (control) the spiking activity of neurons using an adaptive, bi-directional and high-resolution device. Although current research is focused on overcoming the technical challenges of building and implanting such a device, exploiting its capabilities to achieve more acute visual perception will also require substantial computational advances. Using high-density large-scale recording and stimulation in the primate retina with an ex vivo multi-electrode array lab prototype, we frame several of the major computational problems, and describe current progress and future opportunities in solving them. First, we identify cell types and locations from spontaneous activity in the blind retina, and then efficiently estimate their visual response properties by using a low-dimensional manifold of inter-retina variability learned from a large experimental dataset. Second, we estimate retinal responses to a large collection of relevant electrical stimuli by passing current patterns through an electrode array, spike sorting the resulting recordings and using the results to develop a model of evoked responses. Third, we reproduce the desired responses for a given visual target by temporally dithering a diverse collection of electrical stimuli within the integration time of the visual system. Together, these novel approaches may substantially enhance artificial vision in a next-generation device.

In vivo feasibility of epiretinal stimulation using ultrananocrystalline diamond electrodes

OBJECTIVE

Due to their increased proximity to retinal ganglion cells (RGCs), epiretinal visual prostheses present the opportunity for eliciting phosphenes with low thresholds through direct RGC activation. This study characterised the in vivo performance of a novel prototype monolithic epiretinal prosthesis, containing Nitrogen incorporated ultrananocrystalline (N-UNCD) diamond electrodes.

APPROACH

A prototype implant containing up to twenty-five 120 × 120 µm N-UNCD electrodes was implanted into 16 anaesthetised cats and attached to the retina either using a single tack or via magnetic coupling with a suprachoroidally placed magnet. Multiunit responses to retinal stimulation using charge-balanced biphasic current pulses were recorded acutely in the visual cortex using a multichannel planar array. Several stimulus parameters were varied including; the stimulating electrode, stimulus polarity, phase duration, return configuration and the number of electrodes stimulated simultaneously.

MAIN RESULTS

The rigid nature of the device and its form factor necessitated complex surgical procedures. Surgeries were considered successful in 10/16 animals and cortical responses to single electrode stimulation obtained in eight animals. Clinical imaging and histological outcomes showed severe retinal trauma caused by the device in situ in many instances. Cortical measures were found to significantly depend on the surgical outcomes of individual experiments, phase duration, return configuration and the number of electrodes stimulated simultaneously, but not stimulus polarity. Cortical thresholds were also found to increase over time within an experiment.

SIGNIFICANCE

The study successfully demonstrated that an epiretinal prosthesis containing diamond electrodes could produce cortical activity with high precision, albeit only in a small number of cases. Both surgical approaches were highly challenging in terms of reliable and consistent attachment to and stabilisation against the retina, and often resulted in severe retinal trauma. There are key challenges (device form factor and attachment technique) to be resolved for such a device to progress towards clinical application, as current surgical techniques are unable to address these issues.

Stimulation Strategies for Improving the Resolution of Retinal Prostheses

Electrical stimulation using implantable devices with arrays of stimulating electrodes is an emerging therapy for neurological diseases. The performance of these devices depends greatly on their ability to activate populations of neurons with high spatiotemporal resolution. To study electrical stimulation of populations of neurons, retina serves as a useful model because the neural network is arranged in a planar array that is easy to access. Moreover, retinal prostheses are under development to restore vision by replacing the function of damaged light sensitive photoreceptors, which makes retinal research directly relevant for curing blindness. Here we provide a progress review on stimulation strategies developed in recent years to improve the resolution of electrical stimulation in retinal prostheses. We focus on studies performed with explanted retinas, in which electrophysiological techniques are the most advanced. We summarize achievements in improving the spatial and temporal resolution of electrical stimulation of the retina and methods to selectively stimulate neurons with different visual functions. Future directions for retinal prostheses development are also discussed, which could provide insights for other types of neuromodulatory devices in which high-resolution electrical stimulation is required.

CMOS stimulating chips capable of wirelessly driving 473 electrodes for a cortical vision prosthesis

OBJECTIVE

Implantable neural stimulating and recording devices have the potential to restore capabilities such as vision or motor control to disabled patients, improving quality of life. Implants with a large number of stimulating electrodes typically utilize implanted batteries and/or subcutaneous wiring to deal with their high-power consumption and high data throughput needed to address all electrodes with low latency. The use of batteries places severe limitations on the implant's size, usable duty cycle, device longevity while subcutaneous wiring increases the risk of infection and mechanical damage due to device movement.

APPROACH

To overcome these limitations, we have designed and implemented a system that supports up to 473 implanted stimulating microelectrodes, all wirelessly powered and individually controlled by micropower application specific integrated circuits (ASICs).

MAIN RESULTS

Each ASIC controls 43 electrodes and draws 3.18 mW of power when stimulating through 24 channels. We measured the linearity of the digital-to-analog convertors (DACs) to be 0.21 LSB (integrated non-linearity) and the variability in timing of stimulation pulses across ASICs to be 172 ns.

SIGNIFICANCE

This work demonstrates the feasibility of a new low power ASIC designed to be implanted in the visual cortex of humans. The fully implantable device will greatly reduce the risks of infection and damage due to mechanical issues.

Electrical Field Shaping Techniques in a Feline Model of Retinal Degeneration

The majority of preclinical studies investigating multi-electrode field shaping stimulation strategies for retinal prostheses, have been conducted in normally-sighted animals. This study aimed to reassess the effectiveness of two electrical field shaping techniques that have been shown to work in healthy retinae, in a more clinically relevant animal model of photoreceptor degeneration. Four cats were unilaterally blinded via intravitreal injections of adenosine triphosphate. Cortical responses to traditional monopolar (MP) stimulation, focused multipolar (FMP) stimulation and two-dimensional current steering were recorded. Contrary to our previous work, we found no significant difference between the spread of cortical activation elicited by FMP and MP stimulation, and we were not able to reproduce cortical responses to singleelectrode retinal stimulation using two-dimensional current steering. These findings suggest that while shown to be effective in normally-sighted animals, these techniques may not be readily translatable to patients with retinal degeneration and require further optimization.

Feasibility of Nitrogen Doped Ultrananocrystalline Diamond Microelectrodes for Electrophysiological Recording From Neural Tissue

Neural prostheses that can monitor the physiological state of a subject are becoming clinically viable through improvements in the capacity to record from neural tissue. However, a significant limitation of current devices is that it is difficult to fabricate electrode arrays that have both high channel counts and the appropriate electrical properties required for neural recordings. In earlier work, we demonstrated nitrogen doped ultrananocrystalline diamond (N-UNCD) can provide efficacious electrical stimulation of neural tissue, with high charge injection capacity, surface stability and biocompatibility. In this work, we expand on this functionality to show that N-UNCD electrodes can also record from neural tissue owing to its low electrochemical impedance. We show that N-UNCD electrodes are highly flexible in their application, with successful recordings of action potentials from single neurons in an in vitro retina preparation, as well as local field potential responses from in vivo visual cortex tissue. Key properties of N-UNCD films, combined with scalability of electrode array fabrication with custom sizes for recording or stimulation along with integration through vertical interconnects to silicon based integrated circuits, may in future form the basis for the fabrication of versatile closed-loop neural prostheses that can both record and stimulate.

Electrical receptive fields of retinal ganglion cells: Influence of presynaptic neurons

Implantable retinal stimulators activate surviving neurons to restore a sense of vision in people who have lost their photoreceptors through degenerative diseases. Complex spatial and temporal interactions occur in the retina during multi-electrode stimulation. Due to these complexities, most existing implants activate only a few electrodes at a time, limiting the repertoire of available stimulation patterns. Measuring the spatiotemporal interactions between electrodes and retinal cells, and incorporating them into a model may lead to improved stimulation algorithms that exploit the interactions. Here, we present a computational model that accurately predicts both the spatial and temporal nonlinear interactions of multi-electrode stimulation of rat retinal ganglion cells (RGCs). The model was verified using in vitro recordings of ON, OFF, and ON-OFF RGCs in response to subretinal multi-electrode stimulation with biphasic pulses at three stimulation frequencies (10, 20, 30 Hz). The model gives an estimate of each cell’s spatiotemporal electrical receptive fields (ERFs); i.e., the pattern of stimulation leading to excitation or suppression in the neuron. All cells had excitatory ERFs and many also had suppressive sub-regions of their ERFs. We show that the nonlinearities in observed responses arise largely from activation of presynaptic interneurons. When synaptic transmission was blocked, the number of sub-regions of the ERF was reduced, usually to a single excitatory ERF. This suggests that direct cell activation can be modeled accurately by a one-dimensional model with linear interactions between electrodes, whereas indirect stimulation due to summated presynaptic responses is nonlinear.

Implantable neural stimulation devices are being widely used and clinically tested for the restoration of lost function (e.g. cochlear implants) and the treatment of neurological disorders. Smart devices that can combine sensing and stimulation will dramatically improve future patient outcomes. To this end, mathematical models that can accurately predict neural responses to electrical stimulation will be critical for the development of smart stimulation devices. Here, we demonstrate a model that predicts neural responses to simultaneous stimulation across multiple electrodes in the retina. We show that the activation of presynaptic neurons leads to nonlinearities in the responses of postsynaptic retinal ganglion cells. The model is accurate and is applicable to a wide range of neural stimulation devices.

A Mixed Integer Linear Programming Approach to Electrical Stimulation Optimization Problems

Electrical stimulation optimization is a challenging problem. Even when a single region is targeted for excitation, the problem remains a constrained multi-objective optimization problem. The constrained nature of the problem results from safety concerns while its multi-objectives originate from the requirement that non-targeted regions should remain unaffected. In this paper, we propose a mixed integer linear programming formulation that can successfully address the challenges facing this problem. Moreover, the proposed framework can conclusively check the feasibility of the stimulation goals. This helps researchers to avoid wasting time trying to achieve goals that are impossible under a chosen stimulation setup. The superiority of the proposed framework over alternative methods is demonstrated through simulation examples.