High-Fidelity Reproduction of Visual Signals by Electrical Stimulation in the Central Primate Retina

Understanding responses to multi-electrode epiretinal stimulation using a biophysical model

Objective

Neural interfaces are designed to evoke specific patterns of electrical activity in populations of neurons by stimulating with many electrodes. However, currents passed simultaneously through multiple electrodes often combine nonlinearly to drive neural responses, making evoked responses difficult to predict and control. This response nonlinearity could arise from the interaction of many excitable sites in each cell, any of which can produce a spike. However, this multi-site activation hypothesis is difficult to verify experimentally.

Approach

We developed a biophysical model to study retinal ganglion cell (RGC) responses to multi-electrode stimulation and validated it using data collected from ex vivo preparations of the macaque retina using a microelectrode array (512 electrodes; 30µm pitch; 10µm diameter).

Results

First, the model was validated by using it to reproduce essential empirical findings from single-electrode recording and stimulation, including recorded spike voltage waveforms at multiple locations and sigmoidal responses to injected current. Then, stimulation with two electrodes was modeled to test how the positioning of the electrodes relative to the cell affected the degree of response nonlinearity. Currents passed through pairs of electrodes positioned near the cell body or far from the axon (>40 µm) exhibited approximately linear summation in evoking spikes. Currents passed through pairs of electrodes close to the axon summed linearly when their locations along the axon were similar, and nonlinearly otherwise. Over a range of electrode placements, several distinct, localized spike initiation sites were observed, and the number of these sites covaried with the degree of response nonlinearity. Similar trends were observed for three-electrode stimuli. All of these trends in the simulation were consistent with experimental observations. Significance . These findings support the multi-site activation hypothesis for nonlinear activation of neurons, providing a biophysical interpretation of previous experimental results and potentially enabling more efficient use of multi-electrode stimuli in future neural implants.

Precise control of neural activity using dynamically optimized electrical stimulation

Neural implants have the potential to restore lost sensory function by electrically evoking the complex naturalistic activity patterns of neural populations. However, it can be difficult to predict and control evoked neural responses to simultaneous multi-electrode stimulation due to nonlinearity of the responses. We present a solution to this problem and demonstrate its utility in the context of a bidirectional retinal implant for restoring vision. A dynamically optimized stimulation approach encodes incoming visual stimuli into a rapid, greedily chosen, temporally dithered and spatially multiplexed sequence of simple stimulation patterns. Stimuli are selected to optimize the reconstruction of the visual stimulus from the evoked responses. Temporal dithering exploits the slow time scales of downstream neural processing, and spatial multiplexing exploits the independence of responses generated by distant electrodes. The approach was evaluated using an experimental laboratory prototype of a retinal implant: large-scale, high-resolution multi-electrode stimulation and recording of macaque and rat retinal ganglion cells ex vivo. The dynamically optimized stimulation approach substantially enhanced performance compared to existing approaches based on static mapping between visual stimulus intensity and current amplitude. The modular framework enabled parallel extensions to naturalistic viewing conditions, incorporation of perceptual similarity measures, and efficient implementation for an implantable device. A direct closed-loop test of the approach supported its potential use in vision restoration.

Decoding dynamic visual scenes across the brain hierarchy

Understanding the computational mechanisms that underlie the encoding and decoding of environmental stimuli is a crucial investigation in neuroscience. Central to this pursuit is the exploration of how the brain represents visual information across its hierarchical architecture. A prominent challenge resides in discerning the neural underpinnings of the processing of dynamic natural visual scenes. Although considerable research efforts have been made to characterize individual components of the visual pathway, a systematic understanding of the distinctive neural coding associated with visual stimuli, as they traverse this hierarchical landscape, remains elusive. In this study, we leverage the comprehensive Allen Visual Coding-Neuropixels dataset and utilize the capabilities of deep learning neural network models to study neural coding in response to dynamic natural visual scenes across an expansive array of brain regions. Our study reveals that our decoding model adeptly deciphers visual scenes from neural spiking patterns exhibited within each distinct brain area. A compelling observation arises from the comparative analysis of decoding performances, which manifests as a notable encoding proficiency within the visual cortex and subcortical nuclei, in contrast to a relatively reduced encoding activity within hippocampal neurons. Strikingly, our results unveil a robust correlation between our decoding metrics and well-established anatomical and functional hierarchy indexes. These findings corroborate existing knowledge in visual coding related to artificial visual stimuli and illuminate the functional role of these deeper brain regions using dynamic stimuli. Consequently, our results suggest a novel perspective on the utility of decoding neural network models as a metric for quantifying the encoding quality of dynamic natural visual scenes represented by neural responses, thereby advancing our comprehension of visual coding within the complex hierarchy of the brain.

Comparison of modulation efficiency between normal and degenerated primate retina

With electrical stimulation, retinal prostheses bypass dysfunctional photoreceptors and activate the surviving bipolar or retinal ganglion cells (RGCs). Therefore, the effective modulation of RGCs is crucial for developing retinal prostheses. Substantial research has been performed on the ability of an electrical stimulus to generate a reliable RGC response. However, different experimental conditions show varying levels of how well the electrical stimulation evokes RGC spikes. Therefore, in this study, we attempted to extract an indicator to understand how the electrical stimulation effectively evokes RGC spikes. Six cynomolgus monkeys were used: three as controls and three as an N-methyl-N-nitrosourea (MNU)-induced retinal degeneration model. The retinal recordings were performed using 8 × 8 multi-electrode arrays (MEAs). Electrical stimulation consisted of symmetrical biphasic pulses of varying amplitudes and durations. The number of stimulation conditions that resulted in significantly higher post-stimulation firing rates than pre-stimulus firing rates was defined as the modulation efficiency ratio (MER). The MER was significantly lower in degenerated retinas than in normal retinas. We investigated the relationship between the variables and the MER in normal and degenerated primate RGCs. External variables, such as duration and inter-electrode distance, and internal variables, such as average firing rates and statistics (mean, standard deviation, and coefficient of variation [CV]) of inter-spike intervals (ISIs) of spontaneous spikes, were used. External variables had similar effects on MER in normal and degenerated RGCs. In contrast, internal variables affected MER differently in normal and degenerated RGCs. While in normal RGCs, they were not related to MER, in degenerated RGCs, the mean ISIs were positively correlated with MER, and the CV of ISIs was negatively correlated with MER. The most important variable affecting MER was the mean ISI. A shorter ISI indicates hyperactive firing in the degenerated retina, which prevents electrical stimulation from evoking more RGCs. We believe that this hyperactivity in degenerated retinas results in a lower MER than that in the normal retina. Our findings can be used to optimize the selection of stimulation channels for in vitro MEA experiments and practical calibration methods to achieve higher efficiency when testing retinal prostheses.

A virtual patient simulation modeling the neural and perceptual effects of human visual cortical stimulation, from pulse trains to percepts

The field of cortical sight restoration prostheses is making rapid progress with three clinical trials of visual cortical prostheses underway. However, as yet, we have only limited insight into the perceptual experiences produced by these implants. Here we describe a computational model or 'virtual patient', based on the neurophysiological architecture of V1, which successfully predicts the perceptual experience of participants across a wide range of previously published human cortical stimulation studies describing the location, size, brightness and spatiotemporal shape of electrically induced percepts in humans. Our simulations suggest that, in the foreseeable future the perceptual quality of cortical prosthetic devices is likely to be limited by the neurophysiological organization of visual cortex, rather than engineering constraints.

Avoidance of axonal stimulation with sinusoidal epiretinal stimulation

Objective.Neuromodulation, particularly electrical stimulation, necessitates high spatial resolution to achieve artificial vision with high acuity. In epiretinal implants, this is hindered by the undesired activation of distal axons. Here, we investigate focal and axonal activation of retinal ganglion cells (RGCs) in epiretinal configuration for different sinusoidal stimulation frequencies.Approach.RGC responses to epiretinal sinusoidal stimulation at frequencies between 40 and 100 Hz were tested inex-vivophotoreceptor degenerated (rd10) isolated retinae. Experiments were conducted using a high-density CMOS-based microelectrode array, which allows to localize RGC cell bodies and axons at high spatial resolution.Main results.We report current and charge density thresholds for focal and distal axon activation at stimulation frequencies of 40, 60, 80, and 100 Hz for an electrode size with an effective area of 0.01 mm2. Activation of distal axons is avoided up to a stimulation amplitude of 0.23µA (corresponding to 17.3µC cm-2) at 40 Hz and up to a stimulation amplitude of 0.28µA (14.8µC cm-2) at 60 Hz. The threshold ratio between focal and axonal activation increases from 1.1 for 100 Hz up to 1.6 for 60 Hz, while at 40 Hz stimulation frequency, almost no axonal responses were detected in the tested intensity range. With the use of synaptic blockers, we demonstrate the underlying direct activation mechanism of the ganglion cells. Finally, using high-resolution electrical imaging and label-free electrophysiological axon tracking, we demonstrate the extent of activation in axon bundles.Significance.Our results can be exploited to define a spatially selective stimulation strategy avoiding axonal activation in future retinal implants, thereby solving one of the major limitations of artificial vision. The results may be extended to other fields of neuroprosthetics to achieve selective focal electrical stimulation.

Modeling responses of macaque and human retinal ganglion cells to natural images using a convolutional neural network

Linear-nonlinear (LN) cascade models provide a simple way to capture retinal ganglion cell (RGC) responses to artificial stimuli such as white noise, but their ability to model responses to natural images is limited. Recently, convolutional neural network (CNN) models have been shown to produce light response predictions that were substantially more accurate than those of a LN model. However, this modeling approach has not yet been applied to responses of macaque or human RGCs to natural images. Here, we train and test a CNN model on responses to natural images of the four numerically dominant RGC types in the macaque and human retina - ON parasol, OFF parasol, ON midget and OFF midget cells. Compared with the LN model, the CNN model provided substantially more accurate response predictions. Linear reconstructions of the visual stimulus were more accurate for CNN compared to LN model-generated responses, relative to reconstructions obtained from the recorded data. These findings demonstrate the effectiveness of a CNN model in capturing light responses of major RGC types in the macaque and human retinas in natural conditions.

Spike sorting in the presence of stimulation artifacts: a dynamical control systems approach

Objective. Bi-directional electronic neural interfaces, capable of both electrical recording and stimulation, communicate with the nervous system to permit precise calibration of electrical inputs by capturing the evoked neural responses. However, one significant challenge is that stimulation artifacts often mask the actual neural signals. To address this issue, we introduce a novel approach that employs dynamical control systems to detect and decipher electrically evoked neural activity despite the presence of electrical artifacts.Approach. Our proposed method leverages the unique spatiotemporal patterns of neural activity and electrical artifacts to distinguish and identify individual neural spikes. We designed distinctive dynamical models for both the stimulation artifact and each neuron observed during spontaneous neural activity. We can estimate which neurons were active by analyzing the recorded voltage responses across multiple electrodes post-stimulation. This technique also allows us to exclude signals from electrodes heavily affected by stimulation artifacts, such as the stimulating electrode itself, yet still accurately differentiate between evoked spikes and electrical artifacts.Main results. We applied our method to high-density multi-electrode recordings from the primate retina in anex vivosetup, using a grid of 512 electrodes. Through repeated electrical stimulations at varying amplitudes, we were able to construct activation curves for each neuron. The curves obtained with our method closely resembled those derived from manual spike sorting. Additionally, the stimulation thresholds we estimated strongly agreed with those determined through manual analysis, demonstrating high reliability (R2=0.951for human 1 andR2=0.944for human 2).Significance. Our method can effectively separate evoked neural spikes from stimulation artifacts by exploiting the distinct spatiotemporal propagation patterns captured by a dense, large-scale multi-electrode array. This technique holds promise for future applications in real-time closed-loop stimulation systems and for managing multi-channel stimulation strategies.