Eye smarter than scientists believed: neural computations in circuits of the retina

Characterizing medaka visual features using a high-throughput optomotor response assay

Medaka fish (Oryzias latipes) is a powerful model to study genetics underlying the developmental and functional traits of the vertebrate visual system. We established a simple and high-throughput optomotor response (OMR) assay utilizing medaka larvae to study visual functions including visual acuity and contrast sensitivity. Our assay presents multiple adjustable stripes in motion to individual fish in a linear arena. For that the OMR assay employs a tablet display and the Fish Stripes software to adjust speed, width, color, and contrast of the stripes. Our results demonstrated that optomotor responses were robustly induced by black and white stripes presented from below in the linear-pool-arena. We detected robust strain specific differences in the OMR when comparing long established medaka inbred strains. We observed an interesting training effect upon the initial exposure of larvae to thick stripes, which allowed them to better respond to narrower stripes. The OMR setup and protocol presented here provide an efficient tool for quantitative phenotype mapping, addressing visual acuity, trainability of cortical neurons, color sensitivity, locomotor response, retinal regeneration and others. Our open-source setup presented here provides a crucial prerequisite for ultimately addressing the genetic basis of those processes.

The mechanism of human color vision and potential implanted devices for artificial color vision

Vision plays a major role in perceiving external stimuli and information in our daily lives. The neural mechanism of color vision is complicated, involving the co-ordinated functions of a variety of cells, such as retinal cells and lateral geniculate nucleus cells, as well as multiple levels of the visual cortex. In this work, we reviewed the history of experimental and theoretical studies on this issue, from the fundamental functions of the individual cells of the visual system to the coding in the transmission of neural signals and sophisticated brain processes at different levels. We discuss various hypotheses, models, and theories related to the color vision mechanism and present some suggestions for developing novel implanted devices that may help restore color vision in visually impaired people or introduce artificial color vision to those who need it.

Multidimensional vision sensors for information processing

The visual scene in the physical world integrates multidimensional information (spatial, temporal, polarization, spectrum and so on) and typically shows unstructured characteristics. Conventional image sensors cannot process this multidimensional vision data, creating a need for vision sensors that can efficiently extract features from substantial multidimensional vision data. Vision sensors are able to transform the unstructured visual scene into featured information without relying on sophisticated algorithms and complex hardware. The response characteristics of sensors can be abstracted into operators with specific functionalities, allowing for the efficient processing of perceptual information. In this Review, we delve into the hardware implementation of multidimensional vision sensors, exploring their working mechanisms and design principles. We exemplify multidimensional vision sensors built on emerging devices and silicon-based system integration. We further provide benchmarking metrics for multidimensional vision sensors and conclude with the principle of device-system co-design and co-optimization.

Deciphering the genetic code of neuronal type connectivity through bilinear modeling

Understanding how different neuronal types connect and communicate is critical to interpreting brain function and behavior. However, it has remained a formidable challenge to decipher the genetic underpinnings that dictate the specific connections formed between neuronal types. To address this, we propose a novel bilinear modeling approach that leverages the architecture similar to that of recommendation systems. Our model transforms the gene expressions of presynaptic and postsynaptic neuronal types, obtained from single-cell transcriptomics, into a covariance matrix. The objective is to construct this covariance matrix that closely mirrors a connectivity matrix, derived from connectomic data, reflecting the known anatomical connections between these neuronal types. When tested on a dataset of Caenorhabditis elegans, our model achieved a performance comparable to, if slightly better than, the previously proposed spatial connectome model (SCM) in reconstructing electrical synaptic connectivity based on gene expressions. Through a comparative analysis, our model not only captured all genetic interactions identified by the SCM but also inferred additional ones. Applied to a mouse retinal neuronal dataset, the bilinear model successfully recapitulated recognized connectivity motifs between bipolar cells and retinal ganglion cells, and provided interpretable insights into genetic interactions shaping the connectivity. Specifically, it identified unique genetic signatures associated with different connectivity motifs, including genes important to cell-cell adhesion and synapse formation, highlighting their role in orchestrating specific synaptic connections between these neurons. Our work establishes an innovative computational strategy for decoding the genetic programming of neuronal type connectivity. It not only sets a new benchmark for single-cell transcriptomic analysis of synaptic connections but also paves the way for mechanistic studies of neural circuit assembly and genetic manipulation of circuit wiring.

The vertebrate retina: a window into the evolution of computation in the brain

Animal brains are probably the most complex computational machines on our planet, and like everything in biology, they are the product of evolution. Advances in developmental and palaeobiology have been expanding our general understanding of how nervous systems can change at a molecular and structural level. However, how these changes translate into altered function - that is, into 'computation' - remains comparatively sparsely explored. What, concretely, does it mean for neuronal computation when neurons change their morphology and connectivity, when new neurons appear or old ones disappear, or when transmitter systems are slowly modified over many generations? And how does evolution use these many possible knobs and dials to constantly tune computation to give rise to the amazing diversity in animal behaviours we see today? Addressing these major gaps of understanding benefits from choosing a suitable model system. Here, I present the vertebrate retina as one perhaps unusually promising candidate. The retina is ancient and displays highly conserved core organisational principles across the entire vertebrate lineage, alongside a myriad of adjustments across extant species that were shaped by the history of their visual ecology. Moreover, the computational logic of the retina is readily interrogated experimentally, and our existing understanding of retinal circuits in a handful of species can serve as an anchor when exploring the visual circuit adaptations across the entire vertebrate tree of life, from fish deep in the aphotic zone of the oceans to eagles soaring high up in the sky.

Classification and analysis of retinal interneurons by computational structure under natural scenes

Inhibitory neurons are diverse across the brain, but for the visual system we lack the ability to functionally classify these neurons under complex natural stimuli. Here we take the approach of classifying retinal amacrine cell responses to natural scenes using optical recording and an interpretable neural network model. We fit mouse amacrine cell responses to a two-layer convolutional neural network model of a class shown previously to accurately capture salamander ganglion cell responses to natural scenes. Using an approach from interpretable machine learning, we determined for each stimulus the model interneurons that generated each amacrine response, analogous to the set of bipolar cells that target the amacrine population. From this analysis we clustered amacrine cells not by their natural scene responses, but by the model presynaptic neurons that constructed those responses, conservatively finding approximately seven groups by this approach. By analyzing the set of model presynaptic input neurons for each amacrine cluster, we find that distributed rather than dedicated inputs generate natural scene responses for different amacrine cell types. Additional analyses revealed distinct transient and sustained modes exhibited by the network during the response to simple flashes. These results give insight into the computational structure of how the diverse amacrine cell population responds to natural scenes, and generate multiple quantitative hypotheses for how synaptic inputs generate those responses.

How Does the Inner Retinal Network Shape the Ganglion Cells Receptive Field? A Computational Study

We consider a model of basic inner retinal connectivity where bipolar and amacrine cells interconnect and both cell types project onto ganglion cells, modulating their response output to the brain visual areas. We derive an analytical formula for the spatiotemporal response of retinal ganglion cells to stimuli, taking into account the effects of amacrine cells inhibition. This analysis reveals two important functional parameters of the network: (1) the intensity of the interactions between bipolar and amacrine cells and (2) the characteristic timescale of these responses. Both parameters have a profound combined impact on the spatiotemporal features of retinal ganglion cells' responses to light. The validity of the model is confirmed by faithfully reproducing pharmacogenetic experimental results obtained by stimulating excitatory DREADDs (Designer Receptors Exclusively Activated by Designer Drugs) expressed on ganglion cells and amacrine cells' subclasses, thereby modifying the inner retinal network activity to visual stimuli in a complex, entangled manner. Our mathematical model allows us to explore and decipher these complex effects in a manner that would not be feasible experimentally and provides novel insights in retinal dynamics.

Cone-Opponent Ganglion Cells in the Primate Fovea Tuned to Noncardinal Color Directions

A long-standing question in vision science is how the three cone photoreceptor types-long (L), medium (M), and short (S) wavelength sensitive-combine to generate our perception of color. Hue perception can be described along two opponent axes: red-green and blue-yellow. Psychophysical measurements of color appearance indicate that the cone inputs to the red-green and blue-yellow opponent axes are M vs. L + S and L vs. M + S, respectively. However, the "cardinal directions of color space" revealed by psychophysical measurements of color detection thresholds following adaptation are L vs. M and S vs. L + M. These cardinal directions match the most common cone-opponent retinal ganglion cells (RGCs) in the primate retina. Accordingly, the cone opponency necessary for color appearance is thought to be established in the cortex. While neurons with the appropriate M vs. L + S and L vs. M + S opponency have been reported in the retina and lateral geniculate nucleus, their existence continues to be debated. Resolving this long-standing debate is necessary because a complete account of the cone opponency in the retinal output is critical for understanding how downstream neural circuits process color. Here, we performed adaptive optics calcium imaging to noninvasively measure foveal RGC light responses in the living Macaca fascicularis eye. We confirm the presence of L vs. M + S and M vs. L + S neurons with noncardinal cone opponency and demonstrate that cone-opponent signals in the retinal output are more diverse than classically thought.

A recurrent neural circuit in Drosophila deblurs visual inputs

A critical goal of vision is to detect changes in light intensity, even when these changes are blurred by the spatial resolution of the eye and the motion of the animal. Here we describe a recurrent neural circuit in Drosophila that compensates for blur and thereby selectively enhances the perceived contrast of moving edges. Using in vivo , two-photon voltage imaging, we measured the temporal response properties of L1 and L2, two cell types that receive direct synaptic input from photoreceptors. These neurons have biphasic responses to brief flashes of light, a hallmark of cells that encode changes in stimulus intensity. However, the second phase was often much larger than the first, creating an unusual temporal filter. Genetic dissection revealed that recurrent neural circuitry strongly shapes the second phase of the response, informing the structure of a dynamical model. By applying this model to moving natural images, we demonstrate that rather than veridically representing stimulus changes, this temporal processing strategy systematically enhances them, amplifying and sharpening responses. Comparing the measured responses of L2 to model predictions across both artificial and natural stimuli revealed that L2 tunes its properties as the model predicts in order to deblur images. Since this strategy is tunable to behavioral context, generalizable to any time-varying sensory input, and implementable with a common circuit motif, we propose that it could be broadly used to selectively enhance sharp and salient changes.

Predictably manipulating photoreceptor light responses to reveal their role in downstream visual responses

Computation in neural circuits relies on judicious use of nonlinear circuit components. In many cases, multiple nonlinear components work collectively to control circuit outputs. Separating the contributions of these different components is difficult, and this hampers our understanding of the mechanistic basis of many important computations. Here, we introduce a tool that permits the design of light stimuli that predictably alter rod and cone phototransduction currents - including stimuli that compensate for nonlinear properties such as light adaptation. This tool, based on well-established models for the rod and cone phototransduction cascade, permits the separation of nonlinearities in phototransduction from those in downstream circuits. This will allow, for example, direct tests of how adaptation in rod and cone phototransduction affects downstream visual signals and perception.

High performance artificial visual perception and recognition with a plasmon-enhanced 2D material neural network

The development of neuromorphic visual systems has recently gained momentum due to their potential in areas such as autonomous vehicles and robotics. However, current machine visual systems based on silicon technology usually contain photosensor arrays, format conversion, memory and processing modules. As a result, the redundant data shuttling between each unit, resulting in large latency and high-power consumption, seriously limits the performance of neuromorphic vision chips. Here, we demonstrate an artificial neural network (ANN) architecture based on an integrated 2D MoS2/Ag nanograting phototransistor array, which can simultaneously sense, pre-process and recognize optical images without latency. The pre-processing function of the device under photoelectric synergy ensures considerable improvement of efficiency and accuracy of subsequent image recognition. The comprehensive performance of the proof-of-concept device demonstrates great potential for machine vision applications in terms of large dynamic range (180 dB), high speed (500 ns) and low energy consumption per spike (2.4 × 10-17 J).

Coupled Ferroelectric-Photonic Memory in a Retinomorphic Hardware for In-Sensor Computing

The development of all-in-one devices for artificial visual systems offers an attractive solution in terms of energy efficiency and real-time processing speed. In recent years, the proliferation of smart sensors in the growth of Internet-of-Things (IoT) has led to the increasing importance of in-sensor computing technology, which places computational power at the edge of the data-flow architecture. In this study, a prototype visual sensor inspired by the human retina is proposed, which integrates ferroelectricity and photosensitivity in two-dimensional (2D) α-In2Se3 material. This device mimics the functions of photoreceptors and amacrine cells in the retina, performing optical reception and memory computation functions through the use of electrical switching polarization in the channel. The gate-tunable linearity of excitatory and inhibitory functions in photon-induced short-term plasticity enables to encode and classify 12 000 images in the Mixed National Institute of Standards and Technology (MNIST) dataset with remarkable accuracy, achieving ≈94%. Additionally, in-sensor convolution image processing through a network of phototransistors, with five convolutional kernels electrically pre-programmed into the transistors is demonstrated. The convoluted photocurrent matrices undergo straightforward arithmetic calculations to produce edge and feature-enhanced scenarios. The findings demonstrate the potential of ferroelectric α-In2Se3 for highly compact and efficient retinomorphic hardware implementation, regardless of ambipolar transport in the channel.

Bionic visual-audio photodetectors with in-sensor perception and preprocessing

Serving as the "eyes" and "ears" of the Internet of Things, optical and acoustic sensors are the fundamental components in hardware systems. Nowadays, mainstream hardware systems, often comprising numerous discrete sensors, conversion modules, and processing units, tend to result in complex architectures that are less efficient compared to human sensory pathways. Here, a visual-audio photodetector inspired by the human perception system is proposed to enable all-in-one visual and acoustic signal detection with computing capability. This device not only captures light but also optically records sound waves, thus achieving "watching" and "listening" within a single unit. The gate-tunable positive, negative, and zero photoresponses lead to highly programmable responsivities. This programmability enables the execution of diverse functions, including visual feature extraction, object classification, and sound wave manipulation. These results showcase the potential of expanding perception approaches in neuromorphic devices, opening up new possibilities to craft intelligent and compact hardware systems.

PbS/CsPbBr3 Heterojunction for Broadband Neuromorphic Vision Sensing

Neuromorphic light sensors with analogue-domain image processing capability hold promise for overcoming the energy efficiency limitations and latency of von Neumann architecture-based vision chips. Recently, metal halide perovskites, with strong light-matter interaction, long carrier diffusion length, and exceptional photoelectric conversion efficiencies, exhibit reconfigurable photoresponsivity due to their intrinsic ion migration effect, which is expected to advance the development of visual sensors. However, suffering from a large bandgap, it is challenging to achieve highly tunable responsivity simultaneously with a wide-spectrum response in perovskites, which will significantly enhance the image recognition accuracy through the machine learning algorithm. Herein, we demonstrate a broadband neuromorphic visual sensor from visible (Vis) to near-infrared (NIR) by coupling all-inorganic metal halide perovskites (CsPbBr3) with narrow-bandgap lead sulfide (PbS). The PbS/CsPbBr3 heterostructure is composed of high-quality single crystals of PbS and CsPbBr3. Interestingly, the ion migration of CsPbBr3 with the implementation of an electric field induces the energy band dynamic bending at the interface of the PbS/CsPbBr3 heterojunction, leading to reversible, multilevel, and linearly tunable photoresponsivity. Furthermore, the reconfigurable and broadband photoresponse in the PbS/CsPbBr3 heterojunction allows convolutional neuronal network processing for pattern recognition and edge enhancements from the Vis to the NIR waveband, suggesting the great potential of the PbS/CsPbBr3 heterostructure in artificial intelligent vision sensing.

Self-Rectifying All-Optical Modulated Optoelectronic Multistates Memristor Crossbar Array for Neuromorphic Computing

Researching optoelectronic memristors capable of integrating sensory and processing functions is essential for advancing the development of efficient neuromorphic vision. Here, we experimentally demonstrated an all-optical controlled and self-rectifying optoelectronic memristor (OEM) crossbar array with the function of multilevel storage under light stimuli. The NiO/TiO2 device exhibits an ultrahigh (>104) rectifying ratio (RR) thus overcoming the presence of sneak current. The reversible conductance modulation without electric signal involvement provides a novel way to realize ultrafast information processing. The proposed OEM array realized synaptic functions observed in the human brain, including long-term potentiation (LTP), long-term depression (LTD), paired-pulse facilitation (PPF), the transition from short-term memory (STM) to long-term memory (LTM), and learning experience behaviors successfully. The authors present a novel OEM crossbar that possesses complete light-modulation capabilities, potentially advancing the future development of efficient neuromorphic vision.

Clarification on the understanding of contrast theory in relation to the article "ON and OFF receptive field processing in the presence of optical scattering": comment

We are writing to address errors of misrepresentation in the article "ON and OFF receptive field processing in the presence of optical scattering" [Biomed. Opt. Express14, 2618 (2023)10.1364/BOE.489117]. In their investigation of predictions of "contrast theory" to explain the efficacy of diffusion optics technology (DOT), a myopia control lens design [Br. J. Ophthalmol.107, 1709 (2023)10.1136/bjo-2021-321005], Breher et al. incorrectly indicated that our contrast theory proposed that the association between cone opsin gene splicing defects and myopia was due to differential involvement in ON- and OFF-visual pathways. In addition, the Authors write that we have "hypothesized enhanced ON contrast sensitivity in myopes," but we predict the opposite.

Spike desensitisation as a mechanism for high-contrast selectivity in retinal ganglion cells

In the vertebrate retina, several dozens of parallel channels relay information about the visual world to the brain. These channels are represented by the different types of retinal ganglion cells (RGCs), whose responses are rendered selective for distinct sets of visual features by various mechanisms. These mechanisms can be roughly grouped into synaptic interactions and cell-intrinsic mechanisms, with the latter including dendritic morphology as well as ion channel complement and distribution. Here, we investigate how strongly ion channel complement can shape RGC output by comparing two mouse RGC types, the well-described ON alpha cell and a little-studied ON cell that is EGFP-labelled in the Igfbp5 mouse line and displays an unusual selectivity for stimuli with high contrast. Using patch-clamp recordings and computational modelling, we show that a higher activation threshold and a pronounced slow inactivation of the voltage-gated Na+ channels contribute to the distinct contrast tuning and transient responses in ON Igfbp5 RGCs, respectively. In contrast, such a mechanism could not be observed in ON alpha cells. This study provides an example for the powerful role that the last stage of retinal processing can play in shaping RGC responses.

Melatonin suppression by light involves different retinal photoreceptors in young and older adults

Age-related sleep and circadian rhythm disturbances may be due to altered nonvisual photoreception. Here, we investigated the temporal dynamics of light-induced melatonin suppression in young and older individuals. In a within-subject design study, young and older participants were exposed for 60 min (0030-0130 at night) to nine narrow-band lights (range: 420-620 nm). Plasma melatonin suppression was calculated at 15, 30, 45, and 60 min time intervals. Individual spectral sensitivity of melatonin suppression and photoreceptor contribution were predicted for each interval and age group. In young participants, melanopsin solely drove melatonin suppression at all time intervals, with a peak sensitivity at 485.3 nm established only after 15 min of light exposure. Conversely, in older participants, spectral light-driven melatonin suppression was best explained by a more complex model combining melanopsin, S-cone, and M-cone functions, with a stable peak (~500 nm) at 30, 45, and 60 min of light exposure. Aging is associated with a distinct photoreceptor contribution to melatonin suppression by light. While in young adults melanopsin-only photoreception is a reliable predictor of melatonin suppression, in older individuals this process is jointly driven by melanopsin, S-cone, and M-cone functions. These findings offer new prospects for customizing light therapy for older individuals.

Synaptic Origins of the Complex Receptive Field Structure in Primate Smooth Monostratified Retinal Ganglion Cells

Considerable progress has been made in studying the receptive fields of the most common primate retinal ganglion cell (RGC) types, such as parasol RGCs. Much less is known about the rarer primate RGC types and the circuitry that gives rise to noncanonical receptive field structures. The goal of this study was to analyze synaptic inputs to smooth monostratified RGCs to determine the origins of their complex spatial receptive fields, which contain isolated regions of high sensitivity called "hotspots." Interestingly, smooth monostratified RGCs co-stratify with the well-studied parasol RGCs and are thus constrained to receiving input from bipolar and amacrine cells with processes sharing the same layer, raising the question of how their functional differences originate. Through 3D reconstructions of circuitry and synapses onto ON smooth monostratified and ON parasol RGCs from central macaque retina, we identified four distinct sampling strategies employed by smooth and parasol RGCs to extract diverse response properties from co-stratifying bipolar and amacrine cells. The two RGC types differed in the proportion of amacrine cell input, relative contributions of co-stratifying bipolar cell types, amount of synaptic input per bipolar cell, and spatial distribution of bipolar cell synapses. Our results indicate that the smooth RGC's complex receptive field structure arises through spatial asymmetries in excitatory bipolar cell input which formed several discrete clusters comparable with physiologically measured hotspots. Taken together, our results demonstrate how the striking differences between ON parasol and ON smooth monostratified RGCs arise from distinct strategies for sampling a common set of synaptic inputs.

Layers of inhibitory networks shape receptive field properties of AII amacrine cells

In the retina, rod and cone pathways mediate visual signals over a billion-fold range in luminance. AII ("A-two") amacrine cells (ACs) receive signals from both pathways via different bipolar cells, enabling AIIs to operate at night and during the day. Previous work has examined luminance-dependent changes in AII gap junction connectivity, but less is known about how surrounding circuitry shapes AII receptive fields across light levels. Here, we report that moderate contrast stimuli elicit surround inhibition in AIIs under all but the dimmest visual conditions, due to actions of horizontal cells and at least two ACs that inhibit presynaptic bipolar cells. Under photopic (daylight) conditions, surround inhibition transforms AII response kinetics, which are inherited by downstream ganglion cells. Ablating neuronal nitric oxide synthase type-1 (nNOS-1) ACs removes AII surround inhibition under mesopic (dusk/dawn), but not photopic, conditions. Our findings demonstrate how multiple layers of neural circuitry interact to encode signals across a wide physiological range.

Awake responses suggest inefficient dense coding in the mouse retina

The structure and function of the vertebrate retina have been extensively studied across species with an isolated, ex vivo preparation. Retinal function in vivo, however, remains elusive, especially in awake animals. Here, we performed single-unit extracellular recordings in the optic tract of head-fixed mice to compare the output of awake, anesthetized, and ex vivo retinas. While the visual response properties were overall similar across conditions, we found that awake retinal output had in general (1) faster kinetics with less variability in the response latencies; (2) a larger dynamic range; and (3) higher firing activity, by ~20 Hz on average, for both baseline and visually evoked responses. Our modeling analyses further showed that such awake response patterns convey comparable total information but less efficiently, and allow for a linear population decoder to perform significantly better than the anesthetized or ex vivo responses. These results highlight distinct retinal behavior in awake states, in particular suggesting that the retina employs dense coding in vivo, rather than sparse efficient coding as has been often assumed from ex vivo studies.

Reconfigurable, non-volatile neuromorphic photovoltaics

The neural network image sensor-which mimics neurobiological functions of the human retina-has recently been demonstrated to simultaneously sense and process optical images. However, highly tunable responsivity concurrent with non-volatile storage of image data in the neural network would allow a transformative leap in compactness and function of these artificial neural networks. Here, we demonstrate a reconfigurable and non-volatile neuromorphic device based on two-dimensional semiconducting metal sulfides that is concurrently a photovoltaic detector. The device is based on a metal-semiconductor-metal (MSM) two-terminal structure with pulse-tunable sulfur vacancies at the M-S junctions. By modulating sulfur vacancy concentrations, the polarities of short-circuit photocurrent can be changed with multiple stable magnitudes. The bias-induced motion of sulfur vacancies leads to highly reconfigurable responsivities by dynamically modulating the Schottky barriers. A convolutional neuromorphic network is finally designed for image processing and object detection using the same device. The results demonstrated that neuromorphic photodetectors can be the key components of visual perception hardware.

Emerging computational motifs: Lessons from the retina

The retinal neuronal circuit is the first stage of visual processing in the central nervous system. The efforts of scientists over the last few decades indicate that the retina is not merely an array of photosensitive cells, but also a processor that performs various computations. Within a thickness of only ∼200 µm, the retina consists of diverse forms of neuronal circuits, each of which encodes different visual features. Since the discovery of direction-selective cells by Horace Barlow and Richard Hill, the mechanisms that generate direction selectivity in the retina have remained a fascinating research topic. This review provides an overview of recent advances in our understanding of direction-selectivity circuits. Beyond the conventional wisdom of direction selectivity, emerging findings indicate that the retina utilizes complicated and sophisticated mechanisms in which excitatory and inhibitory pathways are involved in the efficient encoding of motion information. As will become evident, the discovery of computational motifs in the retina facilitates an understanding of how sensory systems establish feature selectivity.

An ON-type direction-selective ganglion cell in primate retina

To maintain a stable and clear image of the world, our eyes reflexively follow the direction in which a visual scene is moving. Such gaze-stabilization mechanisms reduce image blur as we move in the environment. In non-primate mammals, this behaviour is initiated by retinal output neurons called ON-type direction-selective ganglion cells (ON-DSGCs), which detect the direction of image motion and transmit signals to brainstem nuclei that drive compensatory eye movements1. However, ON-DSGCs have not yet been identified in the retina of primates, raising the possibility that this reflex is mediated by cortical visual areas. Here we mined single-cell RNA transcriptomic data from primate retina to identify a candidate ON-DSGC. We then combined two-photon calcium imaging, molecular identification and morphological analysis to reveal a population of ON-DSGCs in the macaque retina. The morphology, molecular signature and GABA (γ-aminobutyric acid)-dependent mechanisms that underlie direction selectivity in primate ON-DSGCs are highly conserved with those in other mammals. We further identify a candidate ON-DSGC in human retina. The presence of ON-DSGCs in primates highlights the need to examine the contribution of subcortical retinal mechanisms to normal and aberrant gaze stabilization in the developing and mature visual system.

Neuro-inspired optical sensor array for high-accuracy static image recognition and dynamic trace extraction

Neuro-inspired vision systems hold great promise to address the growing demands of mass data processing for edge computing, a distributed framework that brings computation and data storage closer to the sources of data. In addition to the capability of static image sensing and processing, the hardware implementation of a neuro-inspired vision system also requires the fulfilment of detecting and recognizing moving targets. Here, we demonstrated a neuro-inspired optical sensor based on two-dimensional NbS2/MoS2 hybrid films, which featured remarkable photo-induced conductance plasticity and low electrical energy consumption. A neuro-inspired optical sensor array with 10 × 10 NbS2/MoS2 phototransistors enabled highly integrated functions of sensing, memory, and contrast enhancement capabilities for static images, which benefits convolutional neural network (CNN) with a high image recognition accuracy. More importantly, in-sensor trajectory registration of moving light spots was experimentally implemented such that the post-processing could yield a high restoration accuracy. Our neuro-inspired optical sensor array could provide a fascinating platform for the implementation of high-performance artificial vision systems.

Internal feedback in the cortical perception-action loop enables fast and accurate behavior

Animals move smoothly and reliably in unpredictable environments. Models of sensorimotor control, drawing on control theory, have assumed that sensory information from the environment leads to actions, which then act back on the environment, creating a single, unidirectional perception-action loop. However, the sensorimotor loop contains internal delays in sensory and motor pathways, which can lead to unstable control. We show here that these delays can be compensated by internal feedback signals that flow backward, from motor toward sensory areas. This internal feedback is ubiquitous in neural sensorimotor systems, and we show how internal feedback compensates internal delays. This is accomplished by filtering out self-generated and other predictable changes so that unpredicted, actionable information can be rapidly transmitted toward action by the fastest components, effectively compressing the sensory input to more efficiently use feedforward pathways: Tracts of fast, giant neurons necessarily convey less accurate signals than tracts with many smaller neurons, but they are crucial for fast and accurate behavior. We use a mathematically tractable control model to show that internal feedback has an indispensable role in achieving state estimation, localization of function (how different parts of the cortex control different parts of the body), and attention, all of which are crucial for effective sensorimotor control. This control model can explain anatomical, physiological, and behavioral observations, including motor signals in the visual cortex, heterogeneous kinetics of sensory receptors, and the presence of giant cells in the cortex of humans as well as internal feedback patterns and unexplained heterogeneity in neural systems.

Light-enhanced molecular polarity enabling multispectral color-cognitive memristor for neuromorphic visual system

An optoelectronic synapse having a multispectral color-discriminating ability is an essential prerequisite to emulate the human retina for realizing a neuromorphic visual system. Several studies based on the three-terminal transistor architecture have shown its feasibility; however, its implementation with a two-terminal memristor architecture, advantageous to achieving high integration density as a simple crossbar array for an ultra-high-resolution vision chip, remains a challenge. Furthermore, regardless of the architecture, it requires specific material combinations to exhibit the photo-synaptic functionalities, and thus its integration into various systems is limited. Here, we suggest an approach that can universally introduce a color-discriminating synaptic functionality into a two-terminal memristor irrespective of the kinds of switching medium. This is possible by simply introducing the molecular interlayer with long-lasting photo-enhanced dipoles that can adjust the resistance of the memristor at the light-irradiation. We also propose the molecular design principle that can afford this feature. The optoelectronic synapse array having a color-discriminating functionality is confirmed to improve the inference accuracy of the convolutional neural network for the colorful image recognition tasks through a visual pre-processing. Additionally, the wavelength-dependent optoelectronic synapse can also be leveraged in the design of a light-programmable reservoir computing system.

Cone-Opponent Ganglion Cells in the Primate Fovea Tuned to Non-Cardinal Color Directions

A long-standing question in vision science is how the three cone photoreceptor types - long (L), medium (M) and short (S) wavelength sensitive - combine to generate our perception of color. Hue perception can be described along two opponent axes: red-green and blue-yellow. Psychophysical measurements of color appearance indicate that the cone inputs to the red-green and blue-yellow opponent axes are M vs. L+S and L vs. M+S, respectively. However, the "cardinal directions of color space" revealed by psychophysical measurements of color detection thresholds are L vs. M and S vs. L+M. The cardinal directions match the most common cone-opponent retinal ganglion cells (RGCs) in the primate retina. Accordingly, the cone opponency necessary for color appearance is thought to be established in cortex. However, small populations with the appropriate M vs. L+S and L vs. M+S cone-opponency have been reported in large surveys of cone inputs to primate RGCs and their projections to the lateral geniculate nucleus (LGN) yet their existence continues to be debated. Resolving this long-standing open question is needed as a complete account of the cone-opponency in the retinal output is critical for efforts to understand how downstream neural circuits process color. Here, we performed adaptive optics calcium imaging to longitudinally and noninvasively measurements of the foveal RGC light responses in the living macaque eye. We confirm the presence of L vs. M+S and M vs. L+S neurons with non-cardinal cone-opponency and demonstrate that cone-opponent signals in the retinal output are substantially more diverse than classically thought.

Two Sides of the Same Coin: Efficient and Predictive Neural Coding

Some visual properties are consistent across a wide range of environments, while other properties are more labile. The efficient coding hypothesis states that many of these regularities in the environment can be discarded from neural representations, thus allocating more of the brain's dynamic range to properties that are likely to vary. This paradigm is less clear about how the visual system prioritizes different pieces of information that vary across visual environments. One solution is to prioritize information that can be used to predict future events, particularly those that guide behavior. The relationship between the efficient coding and future prediction paradigms is an area of active investigation. In this review, we argue that these paradigms are complementary and often act on distinct components of the visual input. We also discuss how normative approaches to efficient coding and future prediction can be integrated.

Interpreting the retinal neural code for natural scenes: From computations to neurons

Understanding the circuit mechanisms of the visual code for natural scenes is a central goal of sensory neuroscience. We show that a three-layer network model predicts retinal natural scene responses with an accuracy nearing experimental limits. The model's internal structure is interpretable, as interneurons recorded separately and not modeled directly are highly correlated with model interneurons. Models fitted only to natural scenes reproduce a diverse set of phenomena related to motion encoding, adaptation, and predictive coding, establishing their ethological relevance to natural visual computation. A new approach decomposes the computations of model ganglion cells into the contributions of model interneurons, allowing automatic generation of new hypotheses for how interneurons with different spatiotemporal responses are combined to generate retinal computations, including predictive phenomena currently lacking an explanation. Our results demonstrate a unified and general approach to study the circuit mechanisms of ethological retinal computations under natural visual scenes.

Visual information processing through the interplay between fine and coarse signal pathways

Object recognition is often viewed as a feedforward, bottom-up process in machine learning, but in real neural systems, object recognition is a complicated process which involves the interplay between two signal pathways. One is the parvocellular pathway (P-pathway), which is slow and extracts fine features of objects; the other is the magnocellular pathway (M-pathway), which is fast and extracts coarse features of objects. It has been suggested that the interplay between the two pathways endows the neural system with the capacity of processing visual information rapidly, adaptively, and robustly. However, the underlying computational mechanism remains largely unknown. In this study, we build a two-pathway model to elucidate the computational properties associated with the interactions between two visual pathways. Specifically, we model two visual pathways using two convolution neural networks: one mimics the P-pathway, referred to as FineNet, which is deep, has small-size kernels, and receives detailed visual inputs; the other mimics the M-pathway, referred to as CoarseNet, which is shallow, has large-size kernels, and receives blurred visual inputs. We show that CoarseNet can learn from FineNet through imitation to improve its performance, FineNet can benefit from the feedback of CoarseNet to improve its robustness to noise; and the two pathways interact with each other to achieve rough-to-fine information processing. Using visual backward masking as an example, we further demonstrate that our model can explain visual cognitive behaviors that involve the interplay between two pathways. We hope that this study gives us insight into understanding the interaction principles between two visual pathways.

Retinal ganglion cells encode differently in the myopic mouse retina?

The etiology of myopia remains unclear. This study investigated whether retinal ganglion cells (RGCs) in the myopic retina encode visual information differently from the normal retina and to determine the role of Connexin (Cx) 36 in this process. Generalized linear models (GLMs), which can capture stimulus-dependent changes in real neurons with spike timing precision and reliability, were used to predict RGCs responses to focused and defocused images in the retinas of wild-type (normal) and Lens-Induced Myopia (LIM) mice. As the predominant subunit of gap junctions in the mouse retina and a plausible modulator in myopia development, Cx36 knockout (KO) mice were used as a control for an intact retinal circuit. The kinetics of excitatory postsynaptic currents (EPSCs) of a single αRGC could reflect projection of both focused and defocused images in the retinas of normal and LIM, but not in the Cx36 knockout mice. Poisson GLMs revealed that RGC encoding of visual stimuli in the LIM retina was similar to that of the normal retina. In the LIM retinas, the linear-Gaussian GLM model with offset was a better fit for predicting the spike count under a focused image than the defocused image. Akaike information criterion (AIC) indicated that nonparametric GLM (np-GLM) model predicted focused/defocused images better in both LIM and normal retinas. However, the spike counts in 33% of αRGCs in LIM retinas were better fitted by exponential GLM (exp-GLM) under defocus, compared to only 13% αRGCs in normal retinas. The differences in encoding performance between LIM and normal retinas indicated the possible amendment and plasticity of the retinal circuit in myopic retinas. The absence of a similar response between Cx36 KO mice and normal/LIM mice might suggest that Cx36, which is associated with myopia development, plays a role in encoding focused and defocused images.

IRIS: Integrated Retinal Functionality in Image Sensors

Neuromorphic image sensors draw inspiration from the biological retina to implement visual computations in electronic hardware. Gain control in phototransduction and temporal differentiation at the first retinal synapse inspired the first generation of neuromorphic sensors, but processing in downstream retinal circuits, much of which has been discovered in the past decade, has not been implemented in image sensor technology. We present a technology-circuit co-design solution that implements two motion computations-object motion sensitivity and looming detection-at the retina's output that could have wide applications for vision-based decision-making in dynamic environments. Our simulations on Globalfoundries 22 nm technology node show that the proposed retina-inspired circuits can be fabricated on image sensing platforms in existing semiconductor foundries by taking advantage of the recent advances in semiconductor chip stacking technology. Integrated Retinal Functionality in Image Sensors (IRIS) technology could drive advances in machine vision applications that demand energy-efficient and low-bandwidth real-time decision-making.

Diffusive Memristors with Uniform and Tunable Relaxation Time for Spike Generation in Event-Based Pattern Recognition

A diffusive memristor is a promising building block for brain-inspired computing hardware. However, the randomness in the device relaxation dynamics limits the wide-range adoption of diffusive memristors in large arrays. In this work, the device stack is engineered to achieve a much-improved uniformity in the relaxation time (standard deviation σ reduced from ≈12 to ≈0.32 ms). The memristor is further connected with a resistor or a capacitor and the relaxation time is tuned between 1.13 µs and 1.25 ms, ranging from three orders of magnitude. The hierarchy of time surfaces (HOTS) algorithm, to utilize the tunable and uniform relaxation behavior for spike generation, is implemented. An accuracy of 77.3% is achieved in recognizing moving objects in the neuromorphic MNIST (N-MNIST) dataset. The work paves the way for building emerging neuromorphic computing hardware systems with ultralow power consumption.

Stimulus invariant aspects of the retinal code drive discriminability of natural scenes

Everything that the brain sees must first be encoded by the retina, which maintains a reliable representation of the visual world in many different, complex natural scenes while also adapting to stimulus changes. Decomposing the population code into independent and cell-cell interactions reveals how broad scene structure is encoded in the adapted retinal output. By recording from the same retina while presenting many different natural movies, we see that the population structure, characterized by strong interactions, is consistent across both natural and synthetic stimuli. We show that these interactions contribute to encoding scene identity. We also demonstrate that this structure likely arises in part from shared bipolar cell input as well as from gap junctions between retinal ganglion cells and amacrine cells.

Cell anatomy and network input explain differences within but not between leech touch cells at two different locations

Mechanosensory cells in the leech share several common features with mechanoreceptors in the human glabrous skin. Previous studies showed that the six T (touch) cells in each body segment of the leech are highly variable in their responses to somatic current injection and change their excitability over time. Here, we investigate three potential reasons for this variability in excitability by comparing the responses of T cells at two soma locations (T2 and T3): (1) Differential effects of time-dependent changes in excitability, (2) divergent synaptic input from the network, and (3) different anatomical structures. These hypotheses were explored with a combination of electrophysiological double recordings, 3D reconstruction of neurobiotin-filled cells, and compartmental model simulations. Current injection triggered significantly more spikes with shorter latency and larger amplitudes in cells at soma location T2 than at T3. During longer recordings, cells at both locations increased their excitability over time in the same way. T2 and T3 cells received the same amount of synaptic input from the unstimulated network, and the polysynaptic connections between both T cells were mutually symmetric. However, we found a striking anatomical difference: While in our data set all T2 cells innervated two roots connecting the ganglion with the skin, 50% of the T3 cells had only one root process. The sub-sample of T3 cells with one root process was significantly less excitable than the T3 cells with two root processes and the T2 cells. To test if the additional root process causes higher excitability, we simulated the responses of 3D reconstructed cells of both anatomies with detailed multi-compartment models. The anatomical subtypes do not differ in excitability when identical biophysical parameters and a homogeneous channel distribution are assumed. Hence, all three hypotheses may contribute to the highly variable T cell responses, but none of them is the only factor accounting for the observed systematic difference in excitability between cells at T2 vs. T3 soma location. Therefore, future patch clamp and modeling studies are needed to analyze how biophysical properties and spatial distribution of ion channels on the cell surface contribute to the variability and systematic differences of electrophysiological phenotypes.

Optogenetic Stimulation of Single Ganglion Cells in the Living Primate Fovea

Though the responses of the rich variety of retinal ganglion cells (RGCs) reflect the totality of visual processing in the retina and provide the sole conduit for those processed responses to the brain, we have much to learn about how the brain uses these signals to guide behavior. An impediment to developing a comprehensive understanding of the role of retinal circuits in behavior is the paucity of causal studies in the intact primate visual system. Here we demonstrate the ability to optogenetically activate individual RGCs with flashes of light focused on single RGC somas in vivo , without activation of neighboring cells. The ability to selectively activate specific cells is the first step toward causal experiments that directly link retinal circuits to visual experience and behavior.

Signaling - transcription interactions in mouse retinal ganglion cells early axon pathfinding -a literature review

Sending an axon out of the eye and into the target brain nuclei is the defining feature of retinal ganglion cells (RGCs). The literature on RGC axon pathfinding is vast, but it focuses mostly on decision making events such as midline crossing at the optic chiasm or retinotopic mapping at the target nuclei. In comparison, the exit of RGC axons out of the eye is much less explored. The first checkpoint on the RGC axons' path is the optic cup - optic stalk junction (OC-OS). OC-OS development and the exit of the RGC pioneer axons out of the eye are coordinated spatially and temporally. By the time the optic nerve head domain is specified, the optic fissure margins are in contact and the fusion process is ongoing, the first RGCs are born in its proximity and send pioneer axons in the optic stalk. RGC differentiation continues in centrifugal waves. Later born RGC axons fasciculate with the more mature axons. Growth cones at the end of the axons respond to guidance cues to adopt a centripetal direction, maintain nerve fiber layer restriction and to leave the optic cup. Although there is extensive information on OC-OS development, we still have important unanswered questions regarding its contribution to the exit of the RGC axons out of the eye. We are still to distinguish the morphogens of the OC-OS from the axon guidance molecules which are expressed in the same place at the same time. The early RGC transcription programs responsible for axon emergence and pathfinding are also unknown. This review summarizes the molecular mechanisms for early RGC axon guidance by contextualizing mouse knock-out studies on OC-OS development with the recent transcriptomic studies on developing RGCs in an attempt to contribute to the understanding of human optic nerve developmental anomalies. The published data summarized here suggests that the developing optic nerve head provides a physical channel (the closing optic fissure) as well as molecular guidance cues for the pioneer RGC axons to exit the eye.

SpikeSEE: An energy-efficient dynamic scenes processing framework for retinal prostheses

Intelligent and low-power retinal prostheses are highly demanded in this era, where wearable and implantable devices are used for numerous healthcare applications. In this paper, we propose an energy-efficient dynamic scenes processing framework (SpikeSEE) that combines a spike representation encoding technique and a bio-inspired spiking recurrent neural network (SRNN) model to achieve intelligent processing and extreme low-power computation for retinal prostheses. The spike representation encoding technique could interpret dynamic scenes with sparse spike trains, decreasing the data volume. The SRNN model, inspired by the human retina's special structure and spike processing method, is adopted to predict the response of ganglion cells to dynamic scenes. Experimental results show that the Pearson correlation coefficient of the proposed SRNN model achieves 0.93, which outperforms the state-of-the-art processing framework for retinal prostheses. Thanks to the spike representation and SRNN processing, the model can extract visual features in a multiplication-free fashion. The framework achieves 8 times power reduction compared with the convolutional recurrent neural network (CRNN) processing-based framework. Our proposed SpikeSEE predicts the response of ganglion cells more accurately with lower energy consumption, which alleviates the precision and power issues of retinal prostheses and provides a potential solution for wearable or implantable prostheses.

Early-life exposure to environmentally relevant concentrations of triclocarban impairs ocular development in zebrafish larvae

Triclocarban (TCC), is an antimicrobial component in personal care products and it is one of the emerging contaminants since it has been detected in various environmental matrices. Its presence in human cord blood, breast milk, and maternal urine raised issues about its possible impact on development and increased concerns about the safety of daily exposure. This study aims to provide additional information about the effects of zebrafish early-life exposure to TCC on eye development and visual function. Zebrafish embryos were exposed to two concentrations of TCC (5 and 50 μg/L) for 4 days. TCC-mediated toxicity was assessed in larvae at the end of exposure and in the long term (20 days post fertilization; dpf), through different biological end-points. The experiments showed that TCC exposure influences the retinal architecture. In 4 dpf treated larvae, we found a less organized ciliary marginal zone, a decrease in the inner nuclear and inner plexiform layers, and a decrease in the retinal ganglion cell layer. Photoreceptor and inner plexiform layers showed an increase in 20 dpf larvae at lower and both concentrations, respectively. The expression levels of two genes involved in eye development (mitfb and pax6a) were both decreased at the concentration of 5 μg/L in 4 dpf larvae, and an increase in mitfb was observed in 5 μg/L-exposed 20 dpf larvae. Interestingly, 20 dpf larvae failed to discriminate between visual stimuli, demonstrating notable visual perception impairments due to compound. The results prompt us to hypothesize that early-life exposure to TCC may have severe and potentially long-term effect on zebrafish visual function.

Efficient coding of natural scenes improves neural system identification

Neural system identification aims at learning the response function of neurons to arbitrary stimuli using experimentally recorded data, but typically does not leverage normative principles such as efficient coding of natural environments. Visual systems, however, have evolved to efficiently process input from the natural environment. Here, we present a normative network regularization for system identification models by incorporating, as a regularizer, the efficient coding hypothesis, which states that neural response properties of sensory representations are strongly shaped by the need to preserve most of the stimulus information with limited resources. Using this approach, we explored if a system identification model can be improved by sharing its convolutional filters with those of an autoencoder which aims to efficiently encode natural stimuli. To this end, we built a hybrid model to predict the responses of retinal neurons to noise stimuli. This approach did not only yield a higher performance than the "stand-alone" system identification model, it also produced more biologically plausible filters, meaning that they more closely resembled neural representation in early visual systems. We found these results applied to retinal responses to different artificial stimuli and across model architectures. Moreover, our normatively regularized model performed particularly well in predicting responses of direction-of-motion sensitive retinal neurons. The benefit of natural scene statistics became marginal, however, for predicting the responses to natural movies. In summary, our results indicate that efficiently encoding environmental inputs can improve system identification models, at least for noise stimuli, and point to the benefit of probing the visual system with naturalistic stimuli.

Dynamic machine vision with retinomorphic photomemristor-reservoir computing

Dynamic machine vision requires recognizing the past and predicting the future of a moving object based on present vision. Current machine vision systems accomplish this by processing numerous image frames or using complex algorithms. Here, we report motion recognition and prediction in recurrent photomemristor networks. In our system, a retinomorphic photomemristor array, working as dynamic vision reservoir, embeds past motion frames as hidden states into the present frame through inherent dynamic memory. The informative present frame facilitates accurate recognition of past and prediction of future motions with machine learning algorithms. This in-sensor motion processing capability eliminates redundant data flows and promotes real-time perception of moving objects for dynamic machine vision.

Increasing cell culture density during a developmental window prevents fated rod precursors derailment toward hybrid rod-glia cells

In proliferating multipotent retinal progenitors, transcription factors dynamics set the fate of postmitotic daughter cells, but postmitotic cell fate plasticity driven by extrinsic factors remains controversial. Transcriptome analysis reveals the concurrent expression by postmitotic rod precursors of genes critical for the Müller glia cell fate, which are rarely generated from terminally-dividing progenitors as a pair with rod precursors. By combining gene expression and functional characterisation in single cultured rod precursors, we identified a time-restricted window where increasing cell culture density switches off the expression of genes critical for Müller glial cells. Intriguingly, rod precursors in low cell culture density maintain the expression of genes of rod and glial cell fate and develop a mixed rod/Muller glial cells electrophysiological fingerprint, revealing rods derailment toward a hybrid rod-glial phenotype. The notion of cell culture density as an extrinsic factor critical for preventing rod-fated cells diversion toward a hybrid cell state may explain the occurrence of hybrid rod/MG cells in the adult retina and provide a strategy to improve engraftment yield in regenerative approaches to retinal degenerative disease by stabilising the fate of grafted rod precursors.

Amacrine cells differentially balance zebrafish color circuits in the central and peripheral retina

The vertebrate inner retina is driven by photoreceptors whose outputs are already pre-processed; in zebrafish, outer retinal circuits split "color" from "grayscale" information across four cone-photoreceptor types. It remains unclear how the inner retina processes incoming spectral information while also combining cone signals to shape grayscale functions. We address this question by imaging the light-driven responses of amacrine cells (ACs) and bipolar cells (BCs) in larval zebrafish in the presence and pharmacological absence of inner retinal inhibition. We find that ACs enhance opponency in some bipolar cells while at the same time suppressing pre-existing opponency in others, so that, depending on the retinal region, the net change in the number of color-opponent units is essentially zero. To achieve this "dynamic balance," ACs counteract intrinsic color opponency of BCs via the On channel. Consistent with these observations, Off-stratifying ACs are exclusively achromatic, while all color-opponent ACs stratify in the On sublamina.

Hierarchical retinal computations rely on hybrid chemical-electrical signaling

Bipolar cells (BCs) are integral to the retinal circuits that extract diverse features from the visual environment. They bridge photoreceptors to ganglion cells, the source of retinal output. Understanding how such circuits encode visual features requires an accounting of the mechanisms that control glutamate release from bipolar cell axons. Here, we demonstrate orientation selectivity in a specific genetically identifiable type of mouse bipolar cell-type 5A (BC5A). Their synaptic terminals respond best when stimulated with vertical bars that are far larger than their dendritic fields. We provide evidence that this selectivity involves enhanced excitation for vertical stimuli that requires gap junctional coupling through connexin36. We also show that this orientation selectivity is detectable postsynaptically in direction-selective ganglion cells, which were not previously thought to be selective for orientation. Together, these results demonstrate how multiple features are extracted by a single hierarchical network, engaging distinct electrical and chemical synaptic pathways.

Artificial intelligence techniques for retinal prostheses: a comprehensive review and future direction

Objective. Retinal prostheses are promising devices to restore vision for patients with severe age-related macular degeneration or retinitis pigmentosa disease. The visual processing mechanism embodied in retinal prostheses play an important role in the restoration effect. Its performance depends on our understanding of the retina's working mechanism and the evolvement of computer vision models. Recently, remarkable progress has been made in the field of processing algorithm for retinal prostheses where the new discovery of the retina's working principle and state-of-the-arts computer vision models are combined together.Approach. We investigated the related research on artificial intelligence techniques for retinal prostheses. The processing algorithm in these studies could be attributed to three types: computer vision-related methods, biophysical models, and deep learning models.Main results. In this review, we first illustrate the structure and function of the normal and degenerated retina, then demonstrate the vision rehabilitation mechanism of three representative retinal prostheses. It is necessary to summarize the computational frameworks abstracted from the normal retina. In addition, the development and feature of three types of different processing algorithms are summarized. Finally, we analyze the bottleneck in existing algorithms and propose our prospect about the future directions to improve the restoration effect.Significance. This review systematically summarizes existing processing models for predicting the response of the retina to external stimuli. What's more, the suggestions for future direction may inspire researchers in this field to design better algorithms for retinal prostheses.

Graphene/MoS2-xOx/graphene photomemristor with tunable non-volatile responsivities for neuromorphic vision processing

Conventional artificial intelligence (AI) machine vision technology, based on the von Neumann architecture, uses separate sensing, computing, and storage units to process huge amounts of vision data generated in sensory terminals. The frequent movement of redundant data between sensors, processors and memory, however, results in high-power consumption and latency. A more efficient approach is to offload some of the memory and computational tasks to sensor elements that can perceive and process the optical signal simultaneously. Here, we proposed a non-volatile photomemristor, in which the reconfigurable responsivity can be modulated by the charge and/or photon flux through it and further stored in the device. The non-volatile photomemristor has a simple two-terminal architecture, in which photoexcited carriers and oxygen-related ions are coupled, leading to a displaced and pinched hysteresis in the current-voltage characteristics. For the first time, non-volatile photomemristors implement computationally complete logic with photoresponse-stateful operations, for which the same photomemristor serves as both a logic gate and memory, using photoresponse as a physical state variable instead of light, voltage and memresistance. The polarity reversal of photomemristors shows great potential for in-memory sensing and computing with feature extraction and image recognition for neuromorphic vision.

Role of Oxidative Stress in Ocular Diseases: A Balancing Act

Redox homeostasis is a delicate balancing act of maintaining appropriate levels of antioxidant defense mechanisms and reactive oxidizing oxygen and nitrogen species. Any disruption of this balance leads to oxidative stress, which is a key pathogenic factor in several ocular diseases. In this review, we present the current evidence for oxidative stress and mitochondrial dysfunction in conditions affecting both the anterior segment (e.g., dry eye disease, keratoconus, cataract) and posterior segment (age-related macular degeneration, proliferative vitreoretinopathy, diabetic retinopathy, glaucoma) of the human eye. We posit that further development of therapeutic interventions to promote pro-regenerative responses and maintenance of the redox balance may delay or prevent the progression of these major ocular pathologies. Continued efforts in this field will not only yield a better understanding of the molecular mechanisms underlying the pathogenesis of ocular diseases but also enable the identification of novel druggable redox targets and antioxidant therapies.

Nine insights from internet engineering that help us understand brain network communication

Philosophers have long recognized the value of metaphor as a tool that opens new avenues of investigation. By seeing brains as having the goal of representation, the computer metaphor in its various guises has helped systems neuroscience approach a wide array of neuronal behaviors at small and large scales. Here I advocate a complementary metaphor, the internet. Adopting this metaphor shifts our focus from computing to communication, and from seeing neuronal signals as localized representational elements to seeing neuronal signals as traveling messages. In doing so, we can take advantage of a comparison with the internet's robust and efficient routing strategies to understand how the brain might meet the challenges of network communication. I lay out nine engineering strategies that help the internet solve routing challenges similar to those faced by brain networks. The internet metaphor helps us by reframing neuronal activity across the brain as, in part, a manifestation of routing, which may, in different parts of the system, resemble the internet more, less, or not at all. I describe suggestive evidence consistent with the brain's use of internet-like routing strategies and conclude that, even if empirical data do not directly implicate internet-like routing, the metaphor is valuable as a reference point for those investigating the difficult problem of network communication in the brain and in particular the problem of routing.

Emerging memristive neurons for neuromorphic computing and sensing

Inspired by the principles of the biological nervous system, neuromorphic engineering has brought a promising alternative approach to intelligence computing with high energy efficiency and low consumption. As pivotal components of neuromorphic system, artificial spiking neurons are powerful information processing units and can achieve highly complex nonlinear computations. By leveraging the switching dynamic characteristics of memristive device, memristive neurons show rich spiking behaviors with simple circuit. This report reviews the memristive neurons and their applications in neuromorphic sensing and computing systems. The switching mechanisms that endow memristive devices with rich dynamics and nonlinearity are highlighted, and subsequently various nonlinear spiking neuron behaviors emulated in these memristive devices are reviewed. Then, recent development is introduced on neuromorphic system with memristive neurons for sensing and computing. Finally, we discuss challenges and outlooks of the memristive neurons toward high-performance neuromorphic hardware systems and provide an insightful perspective for the development of interactive neuromorphic electronic systems.