Assessing the efficacy of visual prostheses by decoding ms-LFPs: application to retinal implants

Nonlinear Decoding of Natural Images From Large-Scale Primate Retinal Ganglion Recordings

Decoding sensory stimuli from neural activity can provide insight into how the nervous system might interpret the physical environment, and facilitates the development of brain-machine interfaces. Nevertheless, the neural decoding problem remains a significant open challenge. Here, we present an efficient nonlinear decoding approach for inferring natural scene stimuli from the spiking activities of retinal ganglion cells (RGCs). Our approach uses neural networks to improve on existing decoders in both accuracy and scalability. Trained and validated on real retinal spike data from more than 1000 simultaneously recorded macaque RGC units, the decoder demonstrates the necessity of nonlinear computations for accurate decoding of the fine structures of visual stimuli. Specifically, high-pass spatial features of natural images can only be decoded using nonlinear techniques, while low-pass features can be extracted equally well by linear and nonlinear methods. Together, these results advance the state of the art in decoding natural stimuli from large populations of neurons.

Spectral features of cortical auditory evoked potentials inform hearing threshold and intensity percepts in acoustic and electric hearing

Objective. Stimulus-elicited changes in electroencephalography (EEG) recordings can be represented using Fourier magnitude and phase features (Makeiget al(2004Trends Cogn. Sci.8204-10)). The present study aimed to quantify how much information about hearing responses are contained in the magnitude, quantified by event-related spectral perturbations (ERSPs); and the phase, quantified by inter-trial coherence (ITC). By testing if one feature contained more information and whether this information was mutually exclusive to the features, we aimed to relate specific EEG magnitude and phase features to hearing perception.Approach.EEG responses were recorded from 20 adults who were presented with acoustic stimuli, and 20 adult cochlear implant users with electrical stimuli. Both groups were presented with short, 50 ms stimuli at varying intensity levels relative to their hearing thresholds. Extracted ERSP and ITC features were inputs for a linear discriminant analysis classifier (Wonget al(2016J. Neural. Eng.13036003)). The classifier then predicted whether the EEG signal contained information about the sound stimuli based on the input features. Classifier decoding accuracy was quantified with the mutual information measure (Cottaris and Elfar (2009J. Neural. Eng.6026007), Hawelleket al(2016Proc. Natl Acad. Sci.11313492-7)), and compared across the two feature sets, and to when both feature sets were combined.Main results. We found that classifiers using either ITC or ERSP feature sets were both able to decode hearing perception, but ITC-feature classifiers were able to decode responses to a lower but still audible stimulation intensity, making ITC more useful than ERSP for hearing threshold estimation. We also found that combining the information from both feature sets did not improve decoding significantly, implying that ERSP brain dynamics has a limited contribution to the EEG response, possibly due to the stimuli used in this study.Significance.We successfully related hearing perception to an EEG measure, which does not require behavioral feedback from the listener; an objective measure is important in both neuroscience research and clinical audiology.

Amplitude Modulation-based Electrical Stimulation for Encoding Multipixel Spatiotemporal Visual Information in Retinal Neural Activities

Retinal implants have been developed as a promising way to restore partial vision for the blind. The observation and analysis of neural activities can offer valuable insights for successful prosthetic electrical stimulation. Retinal ganglion cell (RGC) activities have been investigated to provide knowledge on the requirements for electrical stimulation, such as threshold current and the effect of stimulation waveforms. To develop a detailed 'stimulation strategy' for faithful delivery of spatiotemporal visual information to the brain, it is essential to examine both the temporal and spatial characteristics of RGC responses, whereas previous studies were mainly focused on one or the other. In this study, we investigate whether the spatiotemporal visual information can be decoded from the RGC network activity evoked by patterned electrical stimulation. Along with a thorough characterization of spatial spreading of stimulation current and temporal information encoding, we demonstrated that multipixel spatiotemporal visual information can be accurately decoded from the population activities of RGCs stimulated by amplitude-modulated pulse trains. We also found that the details of stimulation, such as pulse amplitude range and pulse rate, were crucial for accurate decoding. Overall, the results suggest that useful visual function may be restored by amplitude modulation-based retinal stimulation.

Second spatial derivative analysis of cortical surface potentials recorded in cat primary auditory cortex using thin film surface arrays: Comparisons with multi-unit data

BACKGROUND

Current source density analysis of recordings from penetrating electrode arrays has traditionally been used to examine the layer- specific cortical activation and plastic changes associated with changed afferent input. We report on a related analysis, the second spatial derivative (SSD) of surface local field potentials (LFPs) recorded using custom designed thin-film polyimide substrate arrays.

RESULTS

SSD analysis of tone- evoked LFPs generated from the auditory cortex under the recording array demonstrated a stereotypical single local minimum, often flanked by maxima on both the caudal and rostral sides. In contrast, tone-pips at frequencies not represented in the region under the array, but known (on the basis of normal tonotopic organization) to be represented caudal to the recording array, had a more complex pattern of many sources and sinks.

COMPARISON WITH EXISTING METHODS

Compared to traditional analysis of LFPs, SSD analysis produced a tonotopic map that was more similar to that obtained with multi-unit recordings in a normal-hearing animal. Additionally, the statistically significant decrease in the number of acoustically responsive cortical locations in partially deafened cats following 6 months of cochlear implant use compared to unstimulated cases observed with multi-unit data (p=0.04) was also observed with SSD analysis (p=0.02), but was not apparent using traditional analysis of LFPs (p=0.6).

CONCLUSIONS

SSD analysis of surface LFPs from the thin-film array provides a rapid and robust method for examining the spatial distribution of cortical activity with improved spatial resolution compared to more traditional LFP recordings.

Decoding brain responses to pixelized images in the primary visual cortex: implications for visual cortical prostheses

Visual cortical prostheses have the potential to restore partial vision. Still limited by the low-resolution visual percepts provided by visual cortical prostheses, implant wearers can currently only "see" pixelized images, and how to obtain the specific brain responses to different pixelized images in the primary visual cortex (the implant area) is still unknown. We conducted a functional magnetic resonance imaging experiment on normal human participants to investigate the brain activation patterns in response to 18 different pixelized images. There were 100 voxels in the brain activation pattern that were selected from the primary visual cortex, and voxel size was 4 mm × 4 mm × 4 mm. Multi-voxel pattern analysis was used to test if these 18 different brain activation patterns were specific. We chose a Linear Support Vector Machine (LSVM) as the classifier in this study. The results showed that the classification accuracies of different brain activation patterns were significantly above chance level, which suggests that the classifier can successfully distinguish the brain activation patterns. Our results suggest that the specific brain activation patterns to different pixelized images can be obtained in the primary visual cortex using a 4 mm × 4 mm × 4 mm voxel size and a 100-voxel pattern.

Parametric models to relate spike train and LFP dynamics with neural information processing

Spike trains and local field potentials (LFPs) resulting from extracellular current flows provide a substrate for neural information processing. Understanding the neural code from simultaneous spike-field recordings and subsequent decoding of information processing events will have widespread applications. One way to demonstrate an understanding of the neural code, with particular advantages for the development of applications, is to formulate a parametric statistical model of neural activity and its covariates. Here, we propose a set of parametric spike-field models (unified models) that can be used with existing decoding algorithms to reveal the timing of task or stimulus specific processing. Our proposed unified modeling framework captures the effects of two important features of information processing: time-varying stimulus-driven inputs and ongoing background activity that occurs even in the absence of environmental inputs. We have applied this framework for decoding neural latencies in simulated and experimentally recorded spike-field sessions obtained from the lateral intraparietal area (LIP) of awake, behaving monkeys performing cued look-and-reach movements to spatial targets. Using both simulated and experimental data, we find that estimates of trial-by-trial parameters are not significantly affected by the presence of ongoing background activity. However, including background activity in the unified model improves goodness of fit for predicting individual spiking events. Uncovering the relationship between the model parameters and the timing of movements offers new ways to test hypotheses about the relationship between neural activity and behavior. We obtained significant spike-field onset time correlations from single trials using a previously published data set where significantly strong correlation was only obtained through trial averaging. We also found that unified models extracted a stronger relationship between neural response latency and trial-by-trial behavioral performance than existing models of neural information processing. Our results highlight the utility of the unified modeling framework for characterizing spike-LFP recordings obtained during behavioral performance.

Comparison of LFP-based and spike-based spectro-temporal receptive fields and cross-correlation in cat primary auditory cortex

Multi-electrode array recordings of spike and local field potential (LFP) activity were made from primary auditory cortex of 12 normal hearing, ketamine-anesthetized cats. We evaluated 259 spectro-temporal receptive fields (STRFs) and 492 frequency-tuning curves (FTCs) based on LFPs and spikes simultaneously recorded on the same electrode. We compared their characteristic frequency (CF) gradients and their cross-correlation distances. The CF gradient for spike-based FTCs was about twice that for 2–40 Hz-filtered LFP-based FTCs, indicating greatly reduced frequency selectivity for LFPs. We also present comparisons for LFPs band-pass filtered between 4–8 Hz, 8–16 Hz and 16–40 Hz, with spike-based STRFs, on the basis of their marginal frequency distributions. We find on average a significantly larger correlation between the spike based marginal frequency distributions and those based on the 16–40 Hz filtered LFP, compared to those based on the 4–8 Hz, 8–16 Hz and 2–40 Hz filtered LFP. This suggests greater frequency specificity for the 16–40 Hz LFPs compared to those of lower frequency content. For spontaneous LFP and spike activity we evaluated 1373 pair correlations for pairs with >200 spikes in 900 s per electrode. Peak correlation-coefficient space constants were similar for the 2–40 Hz filtered LFP (5.5 mm) and the 16–40 Hz LFP (7.4 mm), whereas for spike-pair correlations it was about half that, at 3.2 mm. Comparing spike-pairs with 2–40 Hz (and 16–40 Hz) LFP-pair correlations showed that about 16% (9%) of the variance in the spike-pair correlations could be explained from LFP-pair correlations recorded on the same electrodes within the same electrode array. This larger correlation distance combined with the reduced CF gradient and much broader frequency selectivity suggests that LFPs are not a substitute for spike activity in primary auditory cortex.

[Vision restoration with implants in retinal degenerations]

Up until now there has been no available treatment for diseases causing the permanent impairment of retinal photoreceptors. Currently the development of the retinal prostheses is the earliest to promise a result that can be implemented in the clinical treatment of these patients. Implants with different operating principles and in various stages of progress are presented in details, highlighting the characteristics, as well as the Hungarian aspects of the development. This survey intends to provide an overview on retinal prostheses, implantable in case of degenerative diseases of the retina, by reviewing and assessing the papers published in relevant journals and based on personal experience. Developments in microelectronics in recent years made it possible and proved to be feasible to replace the degenerated elements in the retina with electrical stimulation. Multiple comparable approaches are running simultaneously. Two types of these implants are directly stimulating the remaining living cells in the retina. Hitherto the finest resolution has been achieved with the subretinal implants. Although the epiretinal implant offer lower resolution, but requires shorter surgery for implantation. Retinal implants in certain retinal diseases are proved to be capable of generating vision-like experiences. A number of types of retinal implants can be expected to appear in clinical practice a few years after the successful conclusion of clinical trials.