Development of visual Neuroprostheses: trends and challenges

Pre-processing visual scenes for retinal prosthesis systems: A comprehensive review

BACKGROUND

Retinal prostheses offer hope for individuals with degenerative retinal diseases by stimulating the remaining retinal cells to partially restore their vision. This review delves into the current advancements in retinal prosthesis technology, with a special emphasis on the pivotal role that image processing and machine learning techniques play in this evolution.

METHODS

We provide a comprehensive analysis of the existing implantable devices and optogenetic strategies, delineating their advantages, limitations, and challenges in addressing complex visual tasks. The review extends to various image processing algorithms and deep learning architectures that have been implemented to enhance the functionality of retinal prosthetic devices. We also illustrate the testing results by demonstrating the clinical trials or using Simulated Prosthetic Vision (SPV) through phosphene simulations, which is a critical aspect of simulating visual perception for retinal prosthesis users.

RESULTS

Our review highlights the significant progress in retinal prosthesis technology, particularly its capacity to augment visual perception among the visually impaired. It discusses the integration between image processing and deep learning, illustrating their impact on individual interactions and navigations within the environment through applying clinical trials and also illustrating the limitations of some techniques to be used with current devices, as some approaches only use simulation even on sighted-normal individuals or rely on qualitative analysis, where some consider realistic perception models and others do not.

CONCLUSION

This interdisciplinary field holds promise for the future of retinal prostheses, with the potential to significantly enhance the quality of life for individuals with retinal prostheses. Future research directions should pivot towards optimizing phosphene simulations for SPV approaches, considering the distorted and confusing nature of phosphene perception, thereby enriching the visual perception provided by these prosthetic devices. This endeavor will not only improve navigational independence but also facilitate a more immersive interaction with the environment.

The mechanism of human color vision and potential implanted devices for artificial color vision

Vision plays a major role in perceiving external stimuli and information in our daily lives. The neural mechanism of color vision is complicated, involving the co-ordinated functions of a variety of cells, such as retinal cells and lateral geniculate nucleus cells, as well as multiple levels of the visual cortex. In this work, we reviewed the history of experimental and theoretical studies on this issue, from the fundamental functions of the individual cells of the visual system to the coding in the transmission of neural signals and sophisticated brain processes at different levels. We discuss various hypotheses, models, and theories related to the color vision mechanism and present some suggestions for developing novel implanted devices that may help restore color vision in visually impaired people or introduce artificial color vision to those who need it.

Right to mental integrity and neurotechnologies: implications of the extended mind thesis

The possibility of neurotechnological interference with our brain and mind raises questions about the moral rights that would protect against the (mis)use of these technologies. One such moral right that has received recent attention is the right to mental integrity. Though the metaphysical boundaries of the mind are a matter of live debate, most defences of this moral right seem to assume an internalist (brain-based) view of the mind. In this article, we will examine what an extended account of the mind might imply for the right to mental integrity and the protection it provides against neurotechnologies. We argue that, on an extended account of the mind, the scope of the right to mental integrity would expand significantly, implying that neurotechnologies would no longer pose a uniquely serious threat to the right. In addition, some neurotechnologies may even be protected by the right to mental integrity, as the technologies would become part of the mind. We conclude that adopting an extended account of the mind has significant implications for the right to mental integrity in terms of its protective scope and capacity to protect against neurotechnologies, demonstrating that metaphysical assumptions about the mind play an important role in determining the moral protection provided by the right.

Great expectations: Aligning visual prosthetic development with implantee needs

Purpose

Visual prosthetics have emerged as a promising assistive technology for individuals with vision loss, yet research often overlooks the human aspects of this technology. While previous studies have concentrated on the perceptual experiences of implant recipients (implantees) or the attitudes of potential implantees towards near-future implants, a systematic account of how current implants are being used in everyday life is still lacking.

Methods

We interviewed six recipients of the most widely used visual implants (Argus II and Orion) and six leading researchers in the field. Through thematic and statistical analyses, we explored the daily usage of these implants by implantees and compared their responses to the expectations of researchers. We also sought implantees' input on desired features for future versions, aiming to inform the development of the next generation of implants.

Results

Although implants are designed to facilitate various daily activities, we found that implantees use them less frequently than researchers expected. This discrepancy primarily stems from issues with usability and reliability, with implantees finding alternative methods to accomplish tasks, reducing the need to rely on the implant. For future implants, implantees emphasized the desire for improved vision, smart integration, and increased independence.

Conclusions

Our study reveals a significant gap between researcher expectations and implantee experiences with visual prostheses, underscoring the importance of focusing future research on usability and real-world application.

Translational relevance

This work advocates for a better alignment between technology development and implantee needs to enhance clinical relevance and practical utility of visual prosthetics.

Towards biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses

Blindness affects millions of people around the world. A promising solution to restoring a form of vision for some individuals are cortical visual prostheses, which bypass part of the impaired visual pathway by converting camera input to electrical stimulation of the visual system. The artificially induced visual percept (a pattern of localized light flashes, or 'phosphenes') has limited resolution, and a great portion of the field's research is devoted to optimizing the efficacy, efficiency, and practical usefulness of the encoding of visual information. A commonly exploited method is non-invasive functional evaluation in sighted subjects or with computational models by using simulated prosthetic vision (SPV) pipelines. An important challenge in this approach is to balance enhanced perceptual realism, biologically plausibility, and real-time performance in the simulation of cortical prosthetic vision. We present a biologically plausible, PyTorch-based phosphene simulator that can run in real-time and uses differentiable operations to allow for gradient-based computational optimization of phosphene encoding models. The simulator integrates a wide range of clinical results with neurophysiological evidence in humans and non-human primates. The pipeline includes a model of the retinotopic organization and cortical magnification of the visual cortex. Moreover, the quantitative effects of stimulation parameters and temporal dynamics on phosphene characteristics are incorporated. Our results demonstrate the simulator's suitability for both computational applications such as end-to-end deep learning-based prosthetic vision optimization as well as behavioral experiments. The modular and open-source software provides a flexible simulation framework for computational, clinical, and behavioral neuroscientists working on visual neuroprosthetics.

Neuromorphic hardware for somatosensory neuroprostheses

In individuals with sensory-motor impairments, missing limb functions can be restored using neuroprosthetic devices that directly interface with the nervous system. However, restoring the natural tactile experience through electrical neural stimulation requires complex encoding strategies. Indeed, they are presently limited in effectively conveying or restoring tactile sensations by bandwidth constraints. Neuromorphic technology, which mimics the natural behavior of neurons and synapses, holds promise for replicating the encoding of natural touch, potentially informing neurostimulation design. In this perspective, we propose that incorporating neuromorphic technologies into neuroprostheses could be an effective approach for developing more natural human-machine interfaces, potentially leading to advancements in device performance, acceptability, and embeddability. We also highlight ongoing challenges and the required actions to facilitate the future integration of these advanced technologies.

Human-in-the-Loop Optimization for Deep Stimulus Encoding in Visual Prostheses

Neuroprostheses show potential in restoring lost sensory function and enhancing human capabilities, but the sensations produced by current devices often seem unnatural or distorted. Exact placement of implants and differences in individual perception lead to significant variations in stimulus response, making personalized stimulus optimization a key challenge. Bayesian optimization could be used to optimize patient-specific stimulation parameters with limited noisy observations, but is not feasible for high-dimensional stimuli. Alternatively, deep learning models can optimize stimulus encoding strategies, but typically assume perfect knowledge of patient-specific variations. Here we propose a novel, practically feasible approach that overcomes both of these fundamental limitations. First, a deep encoder network is trained to produce optimal stimuli for any individual patient by inverting a forward model mapping electrical stimuli to visual percepts. Second, a preferential Bayesian optimization strategy utilizes this encoder to optimize patient-specific parameters for a new patient, using a minimal number of pairwise comparisons between candidate stimuli. We demonstrate the viability of this approach on a novel, state-of-the-art visual prosthesis model. We show that our approach quickly learns a personalized stimulus encoder, leads to dramatic improvements in the quality of restored vision, and is robust to noisy patient feedback and misspecifications in the underlying forward model. Overall, our results suggest that combining the strengths of deep learning and Bayesian optimization could significantly improve the perceptual experience of patients fitted with visual prostheses and may prove a viable solution for a range of neuroprosthetic technologies.

Fabrication and Characterization of a Flexible Thin-Film-Based Array of Microelectrodes for Corneal Electrical Stimulation

The electric stimulation (ES) of the cornea is a novel therapeutic approach to the treatment of degenerative visual diseases. Currently, ES is delivered by placing a mono-element electrode on the surface of the cornea that uniformly stimulates the eye along the electrode site. It has been reported that a certain degree of correlation exists between the location of the stimulated retinal area and the position of the electrode. Therefore, in this study, we present the development of a sectioned surface electrode for selective electric stimulation of the human cornea. The proposed device consists of 16 independent microelectrodes, a reference electrode, and 18 contact pads. The microelectrodes have a size of 200 µm × 200 µm, are arranged in a 4 × 4 matrix, and cover a total stimulation area of 16 mm2. The proposed fabrication process, based on surface micromachining technology and flexible electronics, uses only three materials: polyimide, aluminum, and titanium, which allow us to obtain a simplified, ergonomic, and reproducible fabrication process. The fabricated prototype was validated to laboratory level by electrical and electrochemical tests, showing a relatively high electrical conductivity and average impedance from 712 kΩ to 1.4 MΩ at the clinically relevant frequency range (from 11 Hz to 30 Hz). Additionally, the biocompatibility of the electrode prototype was demonstrated by performing in vivo tests and by analyzing the polyimide films using Fourier transform infrared spectroscopy (FTIR). The resulting electrode prototype is robust, mechanically flexible, and biocompatible, with a high potential to be used for selective ES of the cornea.

Attitudes of potential recipients toward emerging visual prosthesis technologies

With the advent of multiple visual prosthesis devices to treat blindness, the question of how potential patients view such interventions becomes important in order to understand the levels of expectation and acceptance, and the perceived risk-reward balance across the different device approaches. Building on previous work on single device approaches done with blind individuals in Chicago and Detroit, USA, Melbourne, Australia, and Bejing, China, we investigated attitudes in blind individuals in Athens, Greece with coverage expanded to three of the contemporary approaches, Retinal, Thalamic, and Cortical. We presented an informational lecture on the approaches, had potential participants fill out a preliminary Questionnaire 1, then organized selected subjects into focus groups for guided discussion on visual prostheses, and finally had these subjects fill out a more detailed Questionnaire 2. We report here the first quantitative data that compares multiple prosthesis approaches. Our primary findings are that for these potential patients, perceived risk continues to outweigh perceived benefits, with the Retinal approach having the least negative overall impression and the Cortical approach the most negative. Concerns about the quality of restored vision were primary. Factors that drove the choice of hypothetical participation in a clinical trial were age and years of blindness. Secondary factors focused on positive clinical outcomes. The focus groups served to swing the impressions of each approach from neutrality toward the extremes of a Likert scale, and shifted the overall willingness to participate in a clinical trial from neutral to negative. These results, coupled with informal assessment of audience questions after the informational lecture, suggest that a substantial improvement in performance over currently available devices will be necessary before visual prostheses gain wide acceptance.

A systematic review of extended reality (XR) for understanding and augmenting vision loss

Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. A defining quality of these XR technologies is their ability to update the stimulus based on the user's eye, head, or body movements. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. Here we present a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology to further visual accessibility. In contrast to other reviews, we sample studies from multiple scientific disciplines, focus on technology that augments a person's residual vision, and require studies to feature a quantitative evaluation with appropriate end users. We summarize prominent findings from different XR research areas, show how the landscape has changed over the past decade, and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the usability of different XR-based accessibility aids.

Brain structural changes in blindness: a systematic review and an anatomical likelihood estimation (ALE) meta-analysis

In recent decades, numerous structural brain imaging studies investigated purported morphometric changes in early (EB) and late onset blindness (LB). The results of these studies have not yielded very consistent results, neither with respect to the type, nor to the anatomical locations of the brain morphometric alterations. To better characterize the effects of blindness on brain morphometry, we performed a systematic review and an Anatomical-Likelihood-Estimation (ALE) coordinate-based-meta-analysis of 65 eligible studies on brain structural changes in EB and LB, including 890 EB, 466 LB and 1257 sighted controls. Results revealed atrophic changes throughout the whole extent of the retino-geniculo-striate system in both EB and LB, whereas changes in areas beyond the occipital lobe occurred in EB only. We discuss the nature of some of the contradictory findings with respect to the used brain imaging methodologies and characteristics of the blind populations such as the onset, duration and cause of blindness. Future studies should aim for much larger sample sizes, eventually by merging data from different brain imaging centers using the same imaging sequences, opt for multimodal structural brain imaging, and go beyond a purely structural approach by combining functional with structural connectivity network analyses.

Object recognition and localization enhancement in visual prostheses: a real-time mixed reality simulation

Blindness is a main threat that affects the daily life activities of any human. Visual prostheses have been introduced to provide artificial vision to the blind with the aim of allowing them to restore confidence and independence. In this article, we propose an approach that involves four image enhancement techniques to facilitate object recognition and localization for visual prostheses users. These techniques are clip art representation of the objects, edge sharpening, corner enhancement and electrode dropout handling. The proposed techniques are tested in a real-time mixed reality simulation environment that mimics vision perceived by visual prostheses users. Twelve experiments were conducted to measure the performance of the participants in object recognition and localization. The experiments involved single objects, multiple objects and navigation. To evaluate the performance of the participants in objects recognition, we measure their recognition time, recognition accuracy and confidence level. For object localization, two metrics were used to measure the performance of the participants which are the grasping attempt time and the grasping accuracy. The results demonstrate that using all enhancement techniques simultaneously gives higher accuracy, higher confidence level and less time for recognizing and grasping objects in comparison to not applying the enhancement techniques or applying pair-wise combinations of them. Visual prostheses could benefit from the proposed approach to provide users with an enhanced perception.

Toward a personalized closed-loop stimulation of the visual cortex: Advances and challenges

Current cortical visual prosthesis approaches are primarily unidirectional and do not consider the feed-back circuits that exist in just about every part of the nervous system. Herein, we provide a brief overview of some recent developments for better controlling brain stimulation and present preliminary human data indicating that closed-loop strategies could considerably enhance the effectiveness, safety, and long-term stability of visual cortex stimulation. We propose that the development of improved closed-loop strategies may help to enhance our capacity to communicate with the brain.

Towards aSmart Bionic Eye: AI-powered artificial vision for the treatment of incurable blindness

Objective.How can we return a functional form of sight to people who are living with incurable blindness? Despite recent advances in the development of visual neuroprostheses, the quality of current prosthetic vision is still rudimentary and does not differ much across different device technologies.Approach.Rather than aiming to represent the visual scene as naturally as possible, aSmart Bionic Eyecould provide visual augmentations through the means of artificial intelligence-based scene understanding, tailored to specific real-world tasks that are known to affect the quality of life of people who are blind, such as face recognition, outdoor navigation, and self-care.Main results.Complementary to existing research aiming to restore natural vision, we propose a patient-centered approach to incorporate deep learning-based visual augmentations into the next generation of devices.Significance.The ability of a visual prosthesis to support everyday tasks might make the difference between abandoned technology and a widely adopted next-generation neuroprosthetic device.

Hybrid Neural Autoencoders for Stimulus Encoding in Visual and Other Sensory Neuroprostheses

Sensory neuroprostheses are emerging as a promising technology to restore lost sensory function or augment human capabilities. However, sensations elicited by current devices often appear artificial and distorted. Although current models can predict the neural or perceptual response to an electrical stimulus, an optimal stimulation strategy solves the inverse problem: what is the required stimulus to produce a desired response? Here, we frame this as an end-to-end optimization problem, where a deep neural network stimulus encoder is trained to invert a known and fixed forward model that approximates the underlying biological system. As a proof of concept, we demonstrate the effectiveness of this hybrid neural autoencoder (HNA) in visual neuroprostheses. We find that HNA produces high-fidelity patient-specific stimuli representing handwritten digits and segmented images of everyday objects, and significantly outperforms conventional encoding strategies across all simulated patients. Overall this is an important step towards the long-standing challenge of restoring high-quality vision to people living with incurable blindness and may prove a promising solution for a variety of neuroprosthetic technologies.

Rapid electrophoretic deposition of biocompatible graphene coatings for high-performance recording neural electrodes

The electrical and biological interfacial properties of invasive electrodes have a significant impact on the performance and longevity of neural recordings in the brain. In this study, we demonstrated rapid electrophoretic deposition and electrochemical reduction of graphene oxide (GO) on metal-based neural electrodes. Scanning electron microscopy (SEM), X-ray photoelectron spectroscopy (XPS) and other characterizations confirmed the existence of a uniform and effectively reduced graphene oxide coating. Electrochemically reduced graphene oxide (ErGO) coated Pt/Ir neural electrodes exhibited 15.2-fold increase in charge storage capacity (CSC) and 90% decrease in impedance with only 3.8% increase in electrode diameter. Patch clamp electrophysiology and calcium imaging of primary rat hippocampus neurons cultured on ErGO demonstrated that there was no adverse impact on the functional development of neurons. Immunostaining showed a balanced growth of excitatory and inhibitory neurons, and astrocytes. Acute recordings from the auditory cortex and chronic recordings (19 days) from the somatosensory cortex found ErGO coating improved the performance of neural electrodes in signal-to-noise ratio (SNR) and amplitude of signals. The proposed approach not only provides an in-depth evaluation of the effect of ErGO coating on neural electrodes but also widens the coating methods of commercial neural electrodes.

Bioelectronic medicine: Preclinical insights and clinical advances

The nervous system maintains homeostasis and health. Homeostatic disruptions underlying the pathobiology of many diseases can be controlled by bioelectronic devices targeting CNS and peripheral neural circuits. New insights into the regulatory functions of the nervous system and technological developments in bioelectronics drive progress in the emerging field of bioelectronic medicine. Here, we provide an overview of key aspects of preclinical research, translation, and clinical advances in bioelectronic medicine.

Clinical Progress and Optimization of Information Processing in Artificial Visual Prostheses

Visual prostheses, used to assist in restoring functional vision to the visually impaired, convert captured external images into corresponding electrical stimulation patterns that are stimulated by implanted microelectrodes to induce phosphenes and eventually visual perception. Detecting and providing useful visual information to the prosthesis wearer under limited artificial vision has been an important concern in the field of visual prosthesis. Along with the development of prosthetic device design and stimulus encoding methods, researchers have explored the possibility of the application of computer vision by simulating visual perception under prosthetic vision. Effective image processing in computer vision is performed to optimize artificial visual information and improve the ability to restore various important visual functions in implant recipients, allowing them to better achieve their daily demands. This paper first reviews the recent clinical implantation of different types of visual prostheses, summarizes the artificial visual perception of implant recipients, and especially focuses on its irregularities, such as dropout and distorted phosphenes. Then, the important aspects of computer vision in the optimization of visual information processing are reviewed, and the possibilities and shortcomings of these solutions are discussed. Ultimately, the development direction and emphasis issues for improving the performance of visual prosthesis devices are summarized.

Ultrasound stimulation for non-invasive visual prostheses

Globally, it is estimated there are more than 2.2 billion visually impaired people. Visual diseases such as retinitis pigmentosa, age-related macular degeneration, glaucoma, and optic neuritis can cause irreversible profound vision loss. Many groups have investigated different approaches such as microelectronic prostheses, optogenetics, stem cell therapy, and gene therapy to restore vision. However, these methods have some limitations such as invasive implantation surgery and unknown long-term risk of genetic manipulation. In addition to the safety of ultrasound as a medical imaging modality, ultrasound stimulation can be a viable non-invasive alternative approach for the sight restoration because of its ability to non-invasively control neuronal activities. Indeed, recent studies have demonstrated ultrasound stimulation can successfully modulate retinal/brain neuronal activities without causing any damage to the nerve cells. Superior penetration depth and high spatial resolution of focused ultrasound can open a new avenue in neuromodulation researches. This review summarizes the latest research results about neural responses to ultrasound stimulation. Also, this work provides an overview of technical viewpoints in the future design of a miniaturized ultrasound transducer for a non-invasive acoustic visual prosthesis for non-surgical and painless restoration of vision.

Beyond the Cane: Describing Urban Scenes to Blind People for Mobility Tasks

Blind people face difficulties with independent mobility, impacting employment prospects, social inclusion, and quality of life. Given the advancements in computer vision, with more efficient and effective automated information extraction from visual scenes, it is important to determine what information is worth conveying to blind travelers, especially since people have a limited capacity to receive and process sensory information. We aimed to investigate which objects in a street scene are useful to describe and how those objects should be described. Thirteen cane-using participants, five of whom were early blind, took part in two urban walking experiments. In the first experiment, participants were asked to voice their information needs in the form of questions to the experimenter. In the second experiment, participants were asked to score scene descriptions and navigation instructions, provided by the experimenter, in terms of their usefulness. The descriptions included a variety of objects with various annotations per object. Additionally, we asked participants to rank order the objects and the different descriptions per object in terms of priority and explain why the provided information is or is not useful to them. The results reveal differences between early and late blind participants. Late blind participants requested information more frequently and prioritized information about objects’ locations. Our results illustrate how different factors, such as the level of detail, relative position, and what type of information is provided when describing an object, affected the usefulness of scene descriptions. Participants explained how they (indirectly) used information, but they were frequently unable to explain their ratings. The results distinguish between various types of travel information, underscore the importance of featuring these types at multiple levels of abstraction, and highlight gaps in current understanding of travel information needs. Elucidating the information needs of blind travelers is critical for the development of more useful assistive technologies.

Immersive Virtual Reality Simulations of Bionic Vision

Bionic vision uses neuroprostheses to restore useful vision to people living with incurable blindness. However, a major outstanding challenge is predicting what people "see" when they use their devices. The limited field of view of current devices necessitates head movements to scan the scene, which is difficult to simulate on a computer screen. In addition, many computational models of bionic vision lack biological realism. To address these challenges, we present VR-SPV, an open-source virtual reality toolbox for simulated prosthetic vision that uses a psychophysically validated computational model to allow sighted participants to "see through the eyes" of a bionic eye user. To demonstrate its utility, we systematically evaluated how clinically reported visual distortions affect performance in a letter recognition and an immersive obstacle avoidance task. Our results highlight the importance of using an appropriate phosphene model when predicting visual outcomes for bionic vision.

End-to-end optimization of prosthetic vision

Neural prosthetics may provide a promising solution to restore visual perception in some forms of blindness. The restored prosthetic percept is rudimentary compared to normal vision and can be optimized with a variety of image preprocessing techniques to maximize relevant information transfer. Extracting the most useful features from a visual scene is a nontrivial task and optimal preprocessing choices strongly depend on the context. Despite rapid advancements in deep learning, research currently faces a difficult challenge in finding a general and automated preprocessing strategy that can be tailored to specific tasks or user requirements. In this paper, we present a novel deep learning approach that explicitly addresses this issue by optimizing the entire process of phosphene generation in an end-to-end fashion. The proposed model is based on a deep auto-encoder architecture and includes a highly adjustable simulation module of prosthetic vision. In computational validation experiments, we show that such an approach is able to automatically find a task-specific stimulation protocol. The results of these proof-of-principle experiments illustrate the potential of end-to-end optimization for prosthetic vision. The presented approach is highly modular and our approach could be extended to automated dynamic optimization of prosthetic vision for everyday tasks, given any specific constraints, accommodating individual requirements of the end-user.

Deciphering the Retinal Epigenome during Development, Disease and Reprogramming: Advancements, Challenges and Perspectives

Retinal neurogenesis is driven by concerted actions of transcription factors, some of which are expressed in a continuum and across several cell subtypes throughout development. While seemingly redundant, many factors diversify their regulatory outcome on gene expression, by coordinating variations in chromatin landscapes to drive divergent retinal specification programs. Recent studies have furthered the understanding of the epigenetic contribution to the progression of age-related macular degeneration, a leading cause of blindness in the elderly. The knowledge of the epigenomic mechanisms that control the acquisition and stabilization of retinal cell fates and are evoked upon damage, holds the potential for the treatment of retinal degeneration. Herein, this review presents the state-of-the-art approaches to investigate the retinal epigenome during development, disease, and reprogramming. A pipeline is then reviewed to functionally interrogate the epigenetic and transcriptional networks underlying cell fate specification, relying on a truly unbiased screening of open chromatin states. The related work proposes an inferential model to identify gene regulatory networks, features the first footprinting analysis and the first tentative, systematic query of candidate pioneer factors in the retina ever conducted in any model organism, leading to the identification of previously uncharacterized master regulators of retinal cell identity, such as the nuclear factor I, NFI. This pipeline is virtually applicable to the study of genetic programs and candidate pioneer factors in any developmental context. Finally, challenges and limitations intrinsic to the current next-generation sequencing techniques are discussed, as well as recent advances in super-resolution imaging, enabling spatio-temporal resolution of the genome.

Machine Learning Method for Functional Assessment of Retinal Models

Challenges in the field of retinal prostheses motivate the development of retinal models to accurately simulate Retinal Ganglion Cells (RGCs) responses. The goal of retinal prostheses is to enable blind individuals to solve complex, reallife visual tasks. In this paper, we introduce the functional assessment (FA) of retinal models, which describes the concept of evaluating the performance of retinal models on visual understanding tasks. We present a machine learning method for FA: we feed traditional machine learning classifiers with RGC responses generated by retinal models, to solve object and digit recognition tasks (CIFAR-10, MNIST, Fashion MNIST, Imagenette). We examined critical FA aspects, including how the performance of FA depends on the task, how to optimally feed RGC responses to the classifiers and how the number of output neurons correlates with the model's accuracy. To increase the number of output neurons, we manipulated input images - by splitting and then feeding them to the retinal model and we found that image splitting does not significantly improve the model's accuracy. We also show that differences in the structure of datasets result in largely divergent performance of the retinal model (MNIST and Fashion MNIST exceeded 80% accuracy, while CIFAR-10 and Imagenette achieved ∼40%). Furthermore, retinal models which perform better in standard evaluation, i.e. more accurately predict RGC response, perform better in FA as well. However, unlike standard evaluation, FA results can be straightforwardly interpreted in the context of comparing the quality of visual perception.

Using Biologically-inspired Image Features to Model Retinal Response: Evidence from Biological Datasets

Retinal models are needed to simulate the translation of visual percepts to Retinal Ganglion Cells (RGCs) neural spike trains, through which visual information is transmitted to the brain. Restoring vision through neural prostheses motivates the development of accurate retinal models. We integrate biologically-inspired image features to RGC models. We trained Linear-Nonlinear models using response data from biological retinae. We show that augmenting raw image input with retina-inspired image features leads to performance improvements: in a smaller (30sec. of retina recordings) set integration of features leads to improved models in approximately $\frac{2}{3}$ of the modeled RGCS; in a larger (4min. recording) we show that utilizing Spike Triggered Average analysis to localize RGCs in input images and extract features in a cell-based manner leads to improved models in all (except two) of the modeled RGCs.

A quantitative analysis of the retinofugal projections in congenital and late-onset blindness

•

Congenital (CB) and late blind (LB) affects the integrity brain visual structures.

•

We measured the integrity of the retino-fugal system using structural MRI images.

•

Optic nerve, optic tract, optic chiasm and LGN were reduced by 50 to 60% in CB and LB.

•

There were no differences between CB and LB.

•

In LB, optic nerve volume correlated negatively with blindness duration.

Vision loss early in life has dramatic consequences on the organization of the visual system and hence on structural plasticity of its remnant components. Most of the studies on the anatomical changes in the brain following visual deprivation have focused on the re-organization of the visual cortex and its afferent and efferent projections. In this study, we performed a quantitative analysis of the volume and size of the optic chiasm, optic nerve, optic tract and the lateral geniculate nucleus (LGN), the retino recipient thalamic nucleus. Analysis was carried out on structural T1-weighted MRIs from 22 congenitally blind (CB), 14 late blind (LB) and 29 age -and sex-matched sighted control (SC) subjects. We manually segmented the optic nerve, optic chiasm and optic tract, while LGN volumes were extracted using in-house software. We also measured voxel intensity of optic nerve, optic chiasm and optic tract. Mean volumes of the optic nerve, optic tract and optic chiasm were reduced by 50 to 60% in both CB and LB participants. No significant differences were found between the congenitally and late-onset blind participants for any of the measures. Our data further revealed reduced white matter voxel intensities in optic nerve, optic chiasm and optic tract in blind compared to sighted participants, suggesting decreased myelin content in the atrophied white matter. The LGN was reduced by 50% and 44% in CB and LB, respectively. In LB, optic nerve volume correlated negatively with the blindness duration index; no such correlation was found for optic chiasm, optic tract and LGN. The observation that despite the absence of visual input about half of the subcortical retinofugal projections are structurally preserved raises the question of their functional role. One possibility is that the surviving fibers play a role in the maintenance of circadian rhythms in the blind through the intrinsically photosensitive melanopsin-containing retinal ganglion cells.

Current Review of Optical Neural Interfaces for Clinical Applications

Neural interfaces, which enable the recording and stimulation of living neurons, have emerged as valuable tools in understanding the brain in health and disease, as well as serving as neural prostheses. While neural interfaces are typically based on electrical transduction, alternative energy modalities have been explored to create safe and effective approaches. Among these approaches, optical methods of linking neurons to the outside world have gained attention because light offers high spatial selectivity and decreased invasiveness. Here, we review the current state-of-art of optical neural interfaces and their clinical applications. Optical neural interfaces can be categorized into optical control and optical readout, each of which can be divided into intrinsic and extrinsic approaches. We discuss the advantages and disadvantages of each of these methods and offer a comparison of relative performance. Future directions, including their clinical opportunities, are discussed with regard to the optical properties of biological tissue.

Nanostructured PEDOT Coatings for Electrode-Neuron Integration

Neural electrodes have been developed for the diagnosis and treatment of stroke, sensory deficits, and neurological disorders based on the electrical stimulation of nerve tissue and recording of neural electrical activity. A low interface impedance and large active surface area for charge transfer and intimate contact between neurons and the electrode are critical to obtain high-quality neural signal and effective stimulation without causing damage to both tissue and electrode. In this study, a nanostructured poly(3,4-ethylenedioxythiophene) (PEDOT) coating with lots of long protrusions was created via a one-step electrochemical polymerization from a dichloromethane solution without any rigid or soft templates. The nanostructures on the PEDOT coating were basically formed by intertwined PEDOT nanofibers, which further enhanced the active surface area. The fuzzy PEDOT-modified microelectrodes exhibited an impedance as low as 1 kΩ at 1 kHz, which is much lower than those produced from aqueous 3,4-ethylenedioxythiophene (EDOT) solution, and it was comparable with PEDOT films or composites created from/with template materials. Also, more than 150 times larger charge storage capacity density was obtained compared to the unmodified microelectrode. An in vitro biocompatibility test performed on PC12 cells and primary cells suggested that the PEDOT coatings support cell adhesion, growth, and neurite extension. These results suggest the great potential of the nanostructured PEDOT coating as an electroactive and biosafe intimate contact between the implanted neural electrode and neurons.

Inducing lateralized phosphenes over the occipital lobe using transcranial magnetic stimulation to navigate a virtual environment

Electrical stimulation involving visual areas of the brain produces artificial light percepts called phosphenes. These visual percepts have been extensively investigated in previous studies involving intracortical microsimulation (ICMS) and serve as the basis for developing a visual prosthesis for the blind. Although advances have been achieved, many challenges still remain with implementing a functional ICMS for visual rehabilitation purposes. Transcranial magnetic stimulation (TMS) over the primary occipital lobe offers an alternative method to produce phosphenes non-invasively. A main challenge facing blind individuals involves navigation. Within the scientific community, methods to evaluate the ability of a visual prosthesis to facilitate in navigation has been neglected. In this study, we investigate the effectiveness of evoking lateralized phosphenes to navigate a computer simulated virtual environment. More importantly, we demonstrate how virtual environments along with the development of a visual prosthesis share a mutual relationship benefiting both patients and researchers. Using two TMS devices, a pair of 40mm figure-of-eight coils were placed over each occipital hemisphere resulting in lateralized phosphene perception. Participants were tasked with making a series of left and right turns using peripheral devices depending on the visual hemifield in which a phosphene is present. If a participant was able to accurately perceive all ten phosphenes, the simulated target is able to advance and fully exit the virtual environment. Our findings demonstrate that participants can interpret lateralized phosphenes while highlighting the integration of computer based virtual environments to evaluate the capability of a visual prosthesis during navigation.

Brain-Machine Interfaces to Assist the Blind

The loss or absence of vision is probably one of the most incapacitating events that can befall a human being. The importance of vision for humans is also reflected in brain anatomy as approximately one third of the human brain is devoted to vision. It is therefore unsurprising that throughout history many attempts have been undertaken to develop devices aiming at substituting for a missing visual capacity. In this review, we present two concepts that have been prevalent over the last two decades. The first concept is sensory substitution, which refers to the use of another sensory modality to perform a task that is normally primarily sub-served by the lost sense. The second concept is cross-modal plasticity, which occurs when loss of input in one sensory modality leads to reorganization in brain representation of other sensory modalities. Both phenomena are training-dependent. We also briefly describe the history of blindness from ancient times to modernity, and then proceed to address the means that have been used to help blind individuals, with an emphasis on modern technologies, invasive (various type of surgical implants) and non-invasive devices. With the advent of brain imaging, it has become possible to peer into the neural substrates of sensory substitution and highlight the magnitude of the plastic processes that lead to a rewired brain. Finally, we will address the important question of the value and practicality of the available technologies and future directions.

Toward Long-Term Communication With the Brain in the Blind by Intracortical Stimulation: Challenges and Future Prospects

The restoration of a useful visual sense in a profoundly blind person by direct electrical stimulation of the visual cortex has been a subject of study for many years. However, the field of cortically based sight restoration has made few advances in the last few decades, and many problems remain. In this context, the scientific and technological problems associated with safe and effective communication with the brain are very complex, and there are still many unresolved issues delaying its development. In this work, we review some of the biological and technical issues that still remain to be solved, including long-term biotolerability, the number of electrodes required to provide useful vision, and the delivery of information to the implants. Furthermore, we emphasize the possible role of the neuroplastic changes that follow vision loss in the success of this approach. We propose that increased collaborations among clinicians, basic researchers, and neural engineers will enhance our ability to send meaningful information to the brain and restore a limited but useful sense of vision to many blind individuals.

Neurolight: A Deep Learning Neural Interface for Cortical Visual Prostheses

Visual neuroprosthesis, that provide electrical stimulation along several sites of the human visual system, constitute a potential tool for vision restoration for the blind. Scientific and technological progress in the fields of neural engineering and artificial vision comes with new theories and tools that, along with the dawn of modern artificial intelligence, constitute a promising framework for the further development of neurotechnology. In the framework of the development of a Cortical Visual Neuroprosthesis for the blind (CORTIVIS), we are now facing the challenge of developing not only computationally powerful tools and flexible approaches that will allow us to provide some degree of functional vision to individuals who are profoundly blind. In this work, we propose a general neuroprosthesis framework composed of several task-oriented and visual encoding modules. We address the development and implementation of computational models of the firing rates of retinal ganglion cells and design a tool — Neurolight — that allows these models to be interfaced with intracortical microelectrodes in order to create electrical stimulation patterns that can evoke useful perceptions. In addition, the developed framework allows the deployment of a diverse array of state-of-the-art deep-learning techniques for task-oriented and general image pre-processing, such as semantic segmentation and object detection in our system’s pipeline. To the best of our knowledge, this constitutes the first deep-learning-based system designed to directly interface with the visual brain through an intracortical microelectrode array. We implement the complete pipeline, from obtaining a video stream to developing and deploying task-oriented deep-learning models and predictive models of retinal ganglion cells’ encoding of visual inputs under the control of a neurostimulation device able to send electrical train pulses to a microelectrode array implanted at the visual cortex.

Neurohybrid Memristive CMOS-Integrated Systems for Biosensors and Neuroprosthetics

Here we provide a perspective concept of neurohybrid memristive chip based on the combination of living neural networks cultivated in microfluidic/microelectrode system, metal-oxide memristive devices or arrays integrated with mixed-signal CMOS layer to control the analog memristive circuits, process the decoded information, and arrange a feedback stimulation of biological culture as parts of a bidirectional neurointerface. Our main focus is on the state-of-the-art approaches for cultivation and spatial ordering of the network of dissociated hippocampal neuron cells, fabrication of a large-scale cross-bar array of memristive devices tailored using device engineering, resistive state programming, or non-linear dynamics, as well as hardware implementation of spiking neural networks (SNNs) based on the arrays of memristive devices and integrated CMOS electronics. The concept represents an example of a brain-on-chip system belonging to a more general class of memristive neurohybrid systems for a new-generation robotics, artificial intelligence, and personalized medicine, discussed in the framework of the proposed roadmap for the next decade period.

Graphene Oxide-Based Nanomaterials: An Insight into Retinal Prosthesis

Retinal prosthesis has recently emerged as a treatment strategy for retinopathies, providing excellent assistance in the treatment of age-related macular degeneration (AMD) and retinitis pigmentosa. The potential application of graphene oxide (GO), a highly biocompatible nanomaterial with superior physicochemical properties, in the fabrication of electrodes for retinal prosthesis, is reviewed in this article. This review integrates insights from biological medicine and nanotechnology, with electronic and electrical engineering technological breakthroughs, and aims to highlight innovative objectives in developing biomedical applications of retinal prosthesis.

Bioelectronic Medicine: From Preclinical Studies on the Inflammatory Reflex to New Approaches in Disease Diagnosis and Treatment

Bioelectronic medicine is an evolving field in which new insights into the regulatory role of the nervous system and new developments in bioelectronic technology result in novel approaches in disease diagnosis and treatment. Studies on the immunoregulatory function of the vagus nerve and the inflammatory reflex have a specific place in bioelectronic medicine. These studies recently led to clinical trials with bioelectronic vagus nerve stimulation in inflammatory diseases and other conditions. Here, we outline key findings from this preclinical and clinical research. We also point to other aspects and pillars of interdisciplinary research and technological developments in bioelectronic medicine.

Metaheuristic Optimisation Algorithms for Tuning a Bioinspired Retinal Model

A significant challenge in neuroscience is understanding how visual information is encoded in the retina. Such knowledge is extremely important for the purpose of designing bioinspired sensors and artificial retinal systems that will, in so far as may be possible, be capable of mimicking vertebrate retinal behaviour. In this study, we report the tuning of a reliable computational bioinspired retinal model with various algorithms to improve the mimicry of the model. Its main contribution is two-fold. First, given the multi-objective nature of the problem, an automatic multi-objective optimisation strategy is proposed through the use of four biological-based metrics, which are used to adjust the retinal model for accurate prediction of retinal ganglion cell responses. Second, a subset of population-based search heuristics-genetic algorithms (SPEA2, NSGA-II and NSGA-III), particle swarm optimisation (PSO) and differential evolution (DE)-are explored to identify the best algorithm for fine-tuning the retinal model, by comparing performance across a hypervolume metric. Nonparametric statistical tests are used to perform a rigorous comparison between all the metaheuristics. The best results were achieved with the PSO algorithm on the basis of the largest hypervolume that was achieved, well-distributed elements and high numbers on the Pareto front.