Seeing Things in Motion: Models, Circuits, and Mechanisms

Fiddler crabs (Afruca tangeri) detect second-order motion in both intensity and polarization

Motion vision is vital for a wide range of animal behaviors. Fiddler crabs, for example, rely heavily on motion to detect the movement of avian predators. They are known to detect first-order motion using both intensity (defined by spatiotemporal correlations in luminance) and polarization information (defined separately as spatiotemporal correlations in the degree and/or angle of polarization). However, little is known about their ability to detect second-order motion, another important form of motion information; defined separately by spatiotemporal correlations in higher-order image properties. In this work we used behavioral experiments to test how fiddler crabs (Afruca tangeri) responded to both second-order intensity and polarization stimuli. Fiddler crabs responded to a number of different intensity based second-order stimuli. Furthermore, the crabs also responded to second-order polarization stimuli, a behaviorally relevant stimulus applicable to an unpolarized flying bird when viewed against a polarized sky. The detection of second-order motion in polarization is, to the best of our knowledge, the first demonstration of this ability in any animal. This discovery therefore opens a new dimension in our understanding of how animals use polarization vision for target detection and the broader importance of second-order motion detection for animal behavior.

A Biomimetic Visual Detection Model: Event-Driven LGMDs Implemented With Fractional Spiking Neuron Circuits

OBJECTIVE

Lobula giant motion detectors (LGMDs) in locusts effectively predict collisions and trigger avoidance, with potential applications in autonomous driving and UAVs. Research on LGMD characteristics splits into two views: one focusing on the presynaptic visual pathway, the other on the postsynaptic LGMD neurons. Both perspectives have support, leading to two computational models, but they lack a biophysical description of the individual LGMD neuron behavior. This paper aims to mimic and explain LGMD behavior based on fractional spiking neurons (FSNs) and construct a biomimetic visual model for the LGMD compatible with these characteristics.

METHODS

We implement the visual model using an event camera to simulate photoreceptors and follow the ON/OFF visual pathway, incorporating lateral inhibition to mimic the LGMD system from the bottom up. Second, most computational models of motion perception use only the dendrites within the LGMD neurons as the ideal pathway for linear summation, ignoring dendritic effects inducing neuronal properties. Thus, we introduce FSN circuits by altering dendritic morphological parameters to simulate multi-scale spike frequency adaptation (SFA) observed in LGMDs. Additionally, we add one more circuit of dendritic trees into the FSNs to be compatible with the postsynaptic feed-forward inhibition (FFI) in LGMD neurons, providing a novel explanatory and predictive model.

RESULTS

We test that the event-driven biomimetic visual model can achieve collision detection and looming selection in different complex scenes, especially fast-moving objects.

A Learning Dendritic Neuron-Based Motion Direction Detective System and Its Application to Grayscale Images

In recent research, dendritic neuron-based models have shown promise in effectively learning and recognizing object motion direction within binary images. Leveraging the dendritic neuron structure and On-Off Response mechanism within the primary cortex, this approach has notably reduced learning time and costs compared to traditional neural networks. This paper advances the existing model by integrating bio-inspired components into a learnable dendritic neuron-based artificial visual system (AVS), specifically incorporating mechanisms from horizontal and bipolar cells. This enhancement enables the model to proficiently identify object motion directions in grayscale images, aligning its threshold with human-like perception. The enhanced model demonstrates superior efficiency in motion direction recognition, requiring less data (90% less than other deep models) and less time for training. Experimental findings highlight the model's remarkable robustness, indicating significant potential for real-world applications. The integration of bio-inspired features not only enhances performance but also opens avenues for further exploration in neural network research. Notably, the application of this model to realistic object recognition yields convincing accuracy at nearly 100%, underscoring its practical utility.

The environment to the rescue: can physics help predict predator-prey interactions?

Understanding the factors that determine the occurrence and strength of ecological interactions under specific abiotic and biotic conditions is fundamental since many aspects of ecological community stability and ecosystem functioning depend on patterns of interactions among species. Current approaches to mapping food webs are mostly based on traits, expert knowledge, experiments, and/or statistical inference. However, they do not offer clear mechanisms explaining how trophic interactions are affected by the interplay between organism characteristics and aspects of the physical environment, such as temperature, light intensity or viscosity. Hence, they cannot yet predict accurately how local food webs will respond to anthropogenic pressures, notably to climate change and species invasions. Herein, we propose a framework that synthesises recent developments in food-web theory, integrating body size and metabolism with the physical properties of ecosystems. We advocate for combination of the movement paradigm with a modular definition of the predation sequence, because movement is central to predator-prey interactions, and a generic, modular model is needed to describe all the possible variation in predator-prey interactions. Pending sufficient empirical and theoretical knowledge, our framework will help predict the food-web impacts of well-studied physical factors, such as temperature and oxygen availability, as well as less commonly considered variables such as wind, turbidity or electrical conductivity. An improved predictive capability will facilitate a better understanding of ecosystem responses to a changing world.

The vertebrate retina: a window into the evolution of computation in the brain

Animal brains are probably the most complex computational machines on our planet, and like everything in biology, they are the product of evolution. Advances in developmental and palaeobiology have been expanding our general understanding of how nervous systems can change at a molecular and structural level. However, how these changes translate into altered function - that is, into 'computation' - remains comparatively sparsely explored. What, concretely, does it mean for neuronal computation when neurons change their morphology and connectivity, when new neurons appear or old ones disappear, or when transmitter systems are slowly modified over many generations? And how does evolution use these many possible knobs and dials to constantly tune computation to give rise to the amazing diversity in animal behaviours we see today? Addressing these major gaps of understanding benefits from choosing a suitable model system. Here, I present the vertebrate retina as one perhaps unusually promising candidate. The retina is ancient and displays highly conserved core organisational principles across the entire vertebrate lineage, alongside a myriad of adjustments across extant species that were shaped by the history of their visual ecology. Moreover, the computational logic of the retina is readily interrogated experimentally, and our existing understanding of retinal circuits in a handful of species can serve as an anchor when exploring the visual circuit adaptations across the entire vertebrate tree of life, from fish deep in the aphotic zone of the oceans to eagles soaring high up in the sky.

Ancestral photoreceptor diversity as the basis of visual behaviour

Animal colour vision is based on comparing signals from different photoreceptors. It is generally assumed that processing different spectral types of photoreceptor mainly serves colour vision. Here I propose instead that photoreceptors are parallel feature channels that differentially support visual-motor programmes like motion vision behaviours, prey capture and predator evasion. Colour vision may have emerged as a secondary benefit of these circuits, which originally helped aquatic vertebrates to visually navigate and segment their underwater world. Specifically, I suggest that ancestral vertebrate vision was built around three main systems, including a high-resolution general purpose greyscale system based on ancestral red cones and rods to mediate visual body stabilization and navigation, a high-sensitivity specialized foreground system based on ancestral ultraviolet cones to mediate threat detection and prey capture, and a net-suppressive system based on ancestral green and blue cones for regulating red/rod and ultraviolet circuits. This ancestral strategy probably still underpins vision today, and different vertebrate lineages have since adapted their original photoreceptor circuits to suit their diverse visual ecologies.

A Bio-Inspired Probabilistic Neural Network Model for Noise-Resistant Collision Perception

Bio-inspired models based on the lobula giant movement detector (LGMD) in the locust's visual brain have received extensive attention and application for collision perception in various scenarios. These models offer advantages such as low power consumption and high computational efficiency in visual processing. However, current LGMD-based computational models, typically organized as four-layered neural networks, often encounter challenges related to noisy signals, particularly in complex dynamic environments. Biological studies have unveiled the intrinsic stochastic nature of synaptic transmission, which can aid neural computation in mitigating noise. In alignment with these biological findings, this paper introduces a probabilistic LGMD (Prob-LGMD) model that incorporates a probability into the synaptic connections between multiple layers, thereby capturing the uncertainty in signal transmission, interaction, and integration among neurons. Comparative testing of the proposed Prob-LGMD model and two conventional LGMD models was conducted using a range of visual stimuli, including indoor structured scenes and complex outdoor scenes, all subject to artificial noise. Additionally, the model's performance was compared to standard engineering noise-filtering methods. The results clearly demonstrate that the proposed model outperforms all comparative methods, exhibiting a significant improvement in noise tolerance. This study showcases a straightforward yet effective approach to enhance collision perception in noisy environments.

Spike desensitisation as a mechanism for high-contrast selectivity in retinal ganglion cells

In the vertebrate retina, several dozens of parallel channels relay information about the visual world to the brain. These channels are represented by the different types of retinal ganglion cells (RGCs), whose responses are rendered selective for distinct sets of visual features by various mechanisms. These mechanisms can be roughly grouped into synaptic interactions and cell-intrinsic mechanisms, with the latter including dendritic morphology as well as ion channel complement and distribution. Here, we investigate how strongly ion channel complement can shape RGC output by comparing two mouse RGC types, the well-described ON alpha cell and a little-studied ON cell that is EGFP-labelled in the Igfbp5 mouse line and displays an unusual selectivity for stimuli with high contrast. Using patch-clamp recordings and computational modelling, we show that a higher activation threshold and a pronounced slow inactivation of the voltage-gated Na+ channels contribute to the distinct contrast tuning and transient responses in ON Igfbp5 RGCs, respectively. In contrast, such a mechanism could not be observed in ON alpha cells. This study provides an example for the powerful role that the last stage of retinal processing can play in shaping RGC responses.

Emerging computational motifs: Lessons from the retina

The retinal neuronal circuit is the first stage of visual processing in the central nervous system. The efforts of scientists over the last few decades indicate that the retina is not merely an array of photosensitive cells, but also a processor that performs various computations. Within a thickness of only ∼200 µm, the retina consists of diverse forms of neuronal circuits, each of which encodes different visual features. Since the discovery of direction-selective cells by Horace Barlow and Richard Hill, the mechanisms that generate direction selectivity in the retina have remained a fascinating research topic. This review provides an overview of recent advances in our understanding of direction-selectivity circuits. Beyond the conventional wisdom of direction selectivity, emerging findings indicate that the retina utilizes complicated and sophisticated mechanisms in which excitatory and inhibitory pathways are involved in the efficient encoding of motion information. As will become evident, the discovery of computational motifs in the retina facilitates an understanding of how sensory systems establish feature selectivity.

Asymmetric connections with starburst amacrine cells underlie the upward motion selectivity of J-type retinal ganglion cells

Motion is an important aspect of visual information. The directions of visual motion are encoded in the retina by direction-selective ganglion cells (DSGCs). ON-OFF DSGCs and ON DSGCs co-stratify with starburst amacrine cells (SACs) in the inner plexiform layer and depend on SACs for their direction selectivity. J-type retinal ganglion cells (J-RGCs), a type of OFF DSGCs in the mouse retina, on the other hand, do not co-stratify with SACs, and how direction selectivity in J-RGCs emerges has not been understood. Here, we report that both the excitatory and inhibitory synaptic inputs to J-RGCs are direction-selective (DS), with the inhibitory inputs playing a more important role for direction selectivity. The DS inhibitory inputs come from SACs, and the functional connections between J-RGCs and SACs are spatially asymmetric. Thus, J-RGCs and SACs form functionally important synaptic contacts even though their dendritic arbors show little overlap. These findings underscore the need to look beyond the neurons' stratification patterns in retinal circuit studies. Our results also highlight the critical role of SACs for retinal direction selectivity.

A circuit suppressing retinal drive to the optokinetic system during fast image motion

Optokinetic nystagmus (OKN) assists stabilization of the retinal image during head rotation. OKN is driven by ON direction selective retinal ganglion cells (ON DSGCs), which encode both the direction and speed of global retinal slip. The synaptic circuits responsible for the direction selectivity of ON DSGCs are well understood, but those sculpting their slow-speed preference remain enigmatic. Here, we probe this mechanism in mouse retina through patch clamp recordings, functional imaging, genetic manipulation, and electron microscopic reconstructions. We confirm earlier evidence that feedforward glycinergic inhibition is the main suppressor of ON DSGC responses to fast motion, and reveal the source for this inhibition-the VGluT3 amacrine cell, a dual neurotransmitter, excitatory/inhibitory interneuron. Together, our results identify a role for VGluT3 cells in limiting the speed range of OKN. More broadly, they suggest VGluT3 cells shape the response of many retinal cell types to fast motion, suppressing it in some while enhancing it in others.

Circuits for self-motion estimation and walking control in Drosophila

The brain's evolution and operation are inextricably linked to animal movement, and critical functions, such as motor control, spatial perception, and navigation, rely on precise knowledge of body movement. Such internal estimates of self-motion emerge from the integration of mechanosensory and visual feedback with motor-related signals. Thus, this internal representation likely depends on the activity of circuits distributed across the central nervous system. However, the circuits responsible for self-motion estimation, and the exact mechanisms by which motor-sensory coordination occurs within these circuits remain poorly understood. Recent technological advances have positioned Drosophila melanogaster as an advantageous model for investigating the emergence, maintenance, and utilization of self-motion representations during naturalistic walking behaviors. In this review, I will illustrate how the adult fly is providing insights into the fundamental problems of self-motion computations and walking control, which have relevance for all animals.

Motion perception based on ON/OFF channels: A survey

Motion perception is an essential ability for animals and artificially intelligent systems interacting effectively, safely with surrounding objects and environments. Biological visual systems, that have naturally evolved over hundreds-million years, are quite efficient and robust for motion perception, whereas artificial vision systems are far from such capability. This paper argues that the gap can be significantly reduced by formulation of ON/OFF channels in motion perception models encoding luminance increment (ON) and decrement (OFF) responses within receptive field, separately. Such signal-bifurcating structure has been found in neural systems of many animal species articulating early motion is split and processed in segregated pathways. However, the corresponding biological substrates, and the necessity for artificial vision systems have never been elucidated together, leaving concerns on uniqueness and advantages of ON/OFF channels upon building dynamic vision systems to address real world challenges. This paper highlights the importance of ON/OFF channels in motion perception through surveying current progress covering both neuroscience and computationally modelling works with applications. Compared to related literature, this paper for the first time provides insights into implementation of different selectivity to directional motion of looming, translating, and small-sized target movement based on ON/OFF channels in keeping with soundness and robustness of biological principles. Existing challenges and future trends of such bio-plausible computational structure for visual perception in connection with hotspots of machine learning, advanced vision sensors like event-driven camera finally are discussed.

How Flies See Motion

How neurons detect the direction of motion is a prime example of neural computation: Motion vision is found in the visual systems of virtually all sighted animals, it is important for survival, and it requires interesting computations with well-defined linear and nonlinear processing steps-yet the whole process is of moderate complexity. The genetic methods available in the fruit fly Drosophila and the charting of a connectome of its visual system have led to rapid progress and unprecedented detail in our understanding of how neurons compute the direction of motion in this organism. The picture that emerged incorporates not only the identity, morphology, and synaptic connectivity of each neuron involved but also its neurotransmitters, its receptors, and their subcellular localization. Together with the neurons' membrane potential responses to visual stimulation, this information provides the basis for a biophysically realistic model of the circuit that computes the direction of visual motion.

Electrically-evoked responses for retinal prostheses are differentially altered depending on ganglion cell types in outer retinal neurodegeneration caused by Crb1 gene mutation

Background

Microelectronic prostheses for artificial vision stimulate neurons surviving outer retinal neurodegeneration such as retinitis pigmentosa (RP). Yet, the quality of prosthetic vision substantially varies across subjects, maybe due to different levels of retinal degeneration and/or distinct genotypes. Although the RP genotypes are remarkably diverse, prosthetic studies have primarily used retinal degeneration (rd) 1 and 10 mice, which both have Pde6b gene mutation. Here, we report the electric responses arising in retinal ganglion cells (RGCs) of the rd8 mouse model which has Crb1 mutation.

Methods

We first investigated age-dependent histological changes of wild-type (wt), rd8, and rd10 mice retinas by H&E staining. Then, we used cell-attached patch clamping to record spiking responses of ON, OFF and direction selective (DS) types of RGCs to a 4-ms-long electric pulse. The electric responses of rd8 RGCs were analyzed in comparison with those of wt RGCs in terms of individual RGC spiking patterns, populational characteristics, and spiking consistency across trials.

Results

In the histological examination, the rd8 mice showed partial retinal foldings, but the outer nuclear layer thicknesses remained comparable to those of the wt mice, indicating the early-stage of RP. Although spiking patterns of each RGC type seemed similar to those of the wt retinas, correlation levels between electric vs. light response features were different across the two mouse models. For example, in comparisons between light vs. electric response magnitudes, ON/OFF RGCs of the rd8 mice showed the same/opposite correlation polarity with those of wt mice, respectively. Also, the electric response spike counts of DS RGCs in the rd8 retinas showed a positive correlation with their direction selectivity indices (r = 0.40), while those of the wt retinas were negatively correlated (r = -0.90). Lastly, the spiking timing consistencies of late responses were largely decreased in both ON and OFF RGCs in the rd8 than the wt retinas, whereas no significant difference was found across DS RGCs of the two models.

Conclusion

Our results indicate the electric response features are altered depending on RGC types even from the early-stage RP caused by Crb1 mutation. Given the various degeneration patterns depending on mutation genes, our study suggests the importance of both genotype- and RGC type-dependent analyses for retinal prosthetic research.

A directionally selective collision-sensing visual neural network based on fractional-order differential operator

In this paper, we propose a directionally selective fractional-order lobular giant motion detector (LGMD) visual neural network. Unlike most collision-sensing network models based on LGMDs, our model can not only sense collision threats but also obtain the motion direction of the collision object. Firstly, this paper simulates the membrane potential response of neurons using the fractional-order differential operator to generate reliable collision response spikes. Then, a new correlation mechanism is proposed to obtain the motion direction of objects. Specifically, this paper performs correlation operation on the signals extracted from two pixels, utilizing the temporal delay of the signals to obtain their position relationship. In this way, the response characteristics of direction-selective neurons can be characterized. Finally, ON/OFF visual channels are introduced to encode increases and decreases in brightness, respectively, thereby modeling the bipolar response of special neurons. Extensive experimental results show that the proposed visual neural system conforms to the response characteristics of biological LGMD and direction-selective neurons, and that the performance of the system is stable and reliable.

A non-canonical retina-ipRGCs-SCN-PVT visual pathway for mediating contagious itch behavior

Contagious itch behavior informs conspecifics of adverse environment and is crucial for the survival of social animals. Gastrin-releasing peptide (GRP) and its receptor (GRPR) in the suprachiasmatic nucleus (SCN) of the hypothalamus mediates contagious itch behavior in mice. Here, we show that intrinsically photosensitive retina ganglion cells (ipRGCs) convey visual itch information, independently of melanopsin, from the retina to GRP neurons via PACAP-PAC1R signaling. Moreover, GRPR neurons relay itch information to the paraventricular nucleus of the thalamus (PVT). Surprisingly, neither the visual cortex nor superior colliculus is involved in contagious itch. In vivo calcium imaging and extracellular recordings reveal contagious itch-specific neural dynamics of GRPR neurons. Thus, we propose that the retina-ipRGC-SCN-PVT pathway constitutes a previously unknown visual pathway that probably evolved for motion vision that encodes salient environmental cues and enables animals to imitate behaviors of conspecifics as an anticipatory mechanism to cope with adverse conditions.

It has been shown that GRP-GRPR neuropeptide signaling in the SCN is important for contagious itch behavior in mice. Gao et al. find that SCN-projecting ipRGCs are sufficient to relay itch information from the retina to the SCN by releasing neuropeptide PACAP to activate the GRP-GRPR pathway.

Motion vision: Drosophila neural pathways that go with the visual flow

Processing visual motion cues to interpret self-motion, the movement of others, and the environment's structure is vital to all animals, whether prey or predator. A new study in Drosophila identifies multiple pathways likely contributing to visual motion-dependent computations and behaviors.

Functional convergence of on-off direction-selective ganglion cells in the visual thalamus

In the mouse visual system, multiple types of retinal ganglion cells (RGCs) each encode distinct features of the visual space. A clear understanding of how this information is parsed in their downstream target, the dorsal lateral geniculate nucleus (dLGN), remains elusive. Here, we characterized retinogeniculate connectivity in Cart-IRES2-Cre-D and BD-CreER2 mice, which labels subsets of on-off direction-selective ganglion cells (ooDSGCs) tuned to the vertical directions and to only ventral motion, respectively. Our immunohistochemical, electrophysiological, and optogenetic experiments reveal that only a small fraction (<15%) of thalamocortical (TC) neurons in the dLGN receives primary retinal drive from these subtypes of ooDSGCs. The majority of the functionally identifiable ooDSGC inputs in the dLGN are weak and converge together with inputs from other RGC types. Yet our modeling indicates that this mixing is not random: BD-CreER⁺ ooDSGC inputs converge less frequently with ooDSGCs tuned to the opposite direction than with non-CART-Cre⁺ RGC types. Taken together, these results indicate that convergence of distinct information lines in dLGN follows specific rules of organization.

Endocrine cybernetics: neuropeptides as molecular switches in behavioural decisions

Plasticity in animal behaviour relies on the ability to integrate external and internal cues from the changing environment and hence modulate activity in synaptic circuits of the brain. This context-dependent neuromodulation is largely based on non-synaptic signalling with neuropeptides. Here, we describe select peptidergic systems in the Drosophila brain that act at different levels of a hierarchy to modulate behaviour and associated physiology. These systems modulate circuits in brain regions, such as the central complex and the mushroom bodies, which supervise specific behaviours. At the top level of the hierarchy there are small numbers of large peptidergic neurons that arborize widely in multiple areas of the brain to orchestrate or modulate global activity in a state and context-dependent manner. At the bottom level local peptidergic neurons provide executive neuromodulation of sensory gain and intrinsically in restricted parts of specific neuronal circuits. The orchestrating neurons receive interoceptive signals that mediate energy and sleep homeostasis, metabolic state and circadian timing, as well as external cues that affect food search, aggression or mating. Some of these cues can be triggers of conflicting behaviours such as mating versus aggression, or sleep versus feeding, and peptidergic neurons participate in circuits, enabling behaviour choices and switches.

Distinct inhibitory pathways control velocity and directional tuning in the mouse retina

The sensory periphery is responsible for detecting ethologically relevant features of the external world, using compact, predominantly feedforward circuits. Visual motion is a particularly prevalent sensory feature, the presence of which can be a signal to enact diverse behaviors ranging from gaze stabilization reflexes to predator avoidance or prey capture. To understand how the retina constructs the distinct neural representations required for these behaviors, we investigated two circuits responsible for encoding different aspects of image motion: ON and ON-OFF direction-selective ganglion cells (DSGCs). Using a combination of two-photon targeted whole-cell electrophysiology, pharmacology, and conditional knockout mice, we show that distinct inhibitory pathways independently control tuning for motion velocity and motion direction in these two cell types. We further employ dynamic clamp and numerical modeling techniques to show that asymmetric inhibition provides a velocity-invariant mechanism of directional tuning, despite the strong velocity dependence of classical models of direction selectivity. We therefore demonstrate that invariant representations of motion features by inhibitory interneurons act as computational building blocks to construct distinct, behaviorally relevant signals at the earliest stages of the visual system.

Conserved circuits for direction selectivity in the primate retina

The detection of motion direction is a fundamental visual function and a classic model for neural computation. In the non-primate retina, direction selectivity arises in starburst amacrine cell (SAC) dendrites, which provide selective inhibition to direction-selective retinal ganglion cells (dsRGCs). Although SACs are present in primates, their connectivity and the existence of dsRGCs remain open questions. Here, we present a connectomic reconstruction of the primate ON SAC circuit from a serial electron microscopy volume of the macaque central retina. We show that the structural basis for the SACs' ability to confer directional selectivity on postsynaptic neurons is conserved. SACs selectively target a candidate homolog to the mammalian ON-sustained dsRGCs that project to the accessory optic system (AOS) and contribute to gaze-stabilizing reflexes. These results indicate that the capacity to compute motion direction is present in the retina, which is earlier in the primate visual system than classically thought.

Gene-independent therapeutic interventions to maintain and restore light sensitivity in degenerating photoreceptors

Neurodegenerative retinal diseases are a prime cause of blindness in industrialized countries. In many cases, there are no therapeutic treatments, although they are essential to improve patients' quality of life. A set of disease-causing genes, which primarily affect photoreceptors, has already been identified and is of major interest for developing gene therapies. Nevertheless, depending on the nature and the state of the disease, gene-independent strategies are needed. Various strategies to halt disease progression or maintain function of the retina are under research. These therapeutic interventions include neuroprotection, direct reprogramming of affected photoreceptors, the application of non-coding RNAs, the generation of artificial photoreceptors by optogenetics and cell replacement strategies. During recent years, major breakthroughs have been made such as the first optogenetic application to a blind patient whose visual function partially recovered by targeting retinal ganglion cells. Also, RPE cell transplantation therapies are under clinical investigation and show great promise to improve visual function in blind patients. These cells are generated from human stem cells. Similar therapies for replacing photoreceptors are extensively tested in pre-clinical models. This marks just the start of promising new cures taking advantage of developments in the areas of genetic engineering, optogenetics, and stem-cell research. In this review, we present the recent therapeutic advances of gene-independent approaches that are currently under clinical evaluation. Our main focus is on photoreceptors as these sensory cells are highly vulnerable to degenerative diseases, and are crucial for light detection.

Multisensory correlation computations in the human brain identified by a time-resolved encoding model

Neural mechanisms that arbitrate between integrating and segregating multisensory information are essential for complex scene analysis and for the resolution of the multisensory correspondence problem. However, these mechanisms and their dynamics remain largely unknown, partly because classical models of multisensory integration are static. Here, we used the Multisensory Correlation Detector, a model that provides a good explanatory power for human behavior while incorporating dynamic computations. Participants judged whether sequences of auditory and visual signals originated from the same source (causal inference) or whether one modality was leading the other (temporal order), while being recorded with magnetoencephalography. First, we confirm that the Multisensory Correlation Detector explains causal inference and temporal order behavioral judgments well. Second, we found strong fits of brain activity to the two outputs of the Multisensory Correlation Detector in temporo-parietal cortices. Finally, we report an asymmetry in the goodness of the fits, which were more reliable during the causal inference task than during the temporal order judgment task. Overall, our results suggest the existence of multisensory correlation detectors in the human brain, which explain why and how causal inference is strongly driven by the temporal correlation of multisensory signals.

An Artificial Visual System for Motion Direction Detection Based on the Hassenstein–Reichardt Correlator Model

The perception of motion direction is essential for the survival of visual animals. Despite various theoretical and biophysical investigations that have been conducted to elucidate directional selectivity at the neural level, the systemic mechanism of motion direction detection remains elusive. Here, we develop an artificial visual system (AVS) based on the core computation of the Hassenstein–Reichardt correlator (HRC) model for global motion direction detection. With reference to the biological investigations of Drosophila, we first describe a local motion-sensitive, directionally detective neuron that only responds to ON motion signals with high pattern contrast in a particular direction. Then, we use the full-neurons scheme motion direction detection mechanism to detect the global motion direction based on our previous research. The mechanism enables our AVS to detect multiple directions in a two-dimensional view, and the global motion direction is inferred from the outputs of all local motion-sensitive directionally detective neurons. To verify the reliability of our AVS, we conduct a series of experiments and compare its performance with the time-considered convolution neural network (CNN) and the EfficientNetB0 under the same conditions. The experimental results demonstrated that our system is reliable in detecting the direction of motion, and among the three models, our AVS has better motion direction detection capabilities.

A Looming Spatial Localization Neural Network Inspired by MLG1 Neurons in the Crab Neohelice

Similar to most visual animals, the crab Neohelice granulata relies predominantly on visual information to escape from predators, to track prey and for selecting mates. It, therefore, needs specialized neurons to process visual information and determine the spatial location of looming objects. In the crab Neohelice granulata, the Monostratified Lobula Giant type1 (MLG1) neurons have been found to manifest looming sensitivity with finely tuned capabilities of encoding spatial location information. MLG1s neuronal ensemble can not only perceive the location of a looming stimulus, but are also thought to be able to influence the direction of movement continuously, for example, escaping from a threatening, looming target in relation to its position. Such specific characteristics make the MLG1s unique compared to normal looming detection neurons in invertebrates which can not localize spatial looming. Modeling the MLG1s ensemble is not only critical for elucidating the mechanisms underlying the functionality of such neural circuits, but also important for developing new autonomous, efficient, directionally reactive collision avoidance systems for robots and vehicles. However, little computational modeling has been done for implementing looming spatial localization analogous to the specific functionality of MLG1s ensemble. To bridge this gap, we propose a model of MLG1s and their pre-synaptic visual neural network to detect the spatial location of looming objects. The model consists of 16 homogeneous sectors arranged in a circular field inspired by the natural arrangement of 16 MLG1s' receptive fields to encode and convey spatial information concerning looming objects with dynamic expanding edges in different locations of the visual field. Responses of the proposed model to systematic real-world visual stimuli match many of the biological characteristics of MLG1 neurons. The systematic experiments demonstrate that our proposed MLG1s model works effectively and robustly to perceive and localize looming information, which could be a promising candidate for intelligent machines interacting within dynamic environments free of collision. This study also sheds light upon a new type of neuromorphic visual sensor strategy that can extract looming objects with locational information in a quick and reliable manner.

Realistic retinal modeling unravels the differential role of excitation and inhibition to starburst amacrine cells in direction selectivity

Retinal direction-selectivity originates in starburst amacrine cells (SACs), which display a centrifugal preference, responding with greater depolarization to a stimulus expanding from soma to dendrites than to a collapsing stimulus. Various mechanisms were hypothesized to underlie SAC centrifugal preference, but dissociating them is experimentally challenging and the mechanisms remain debatable. To address this issue, we developed the Retinal Stimulation Modeling Environment (RSME), a multifaceted data-driven retinal model that encompasses detailed neuronal morphology and biophysical properties, retina-tailored connectivity scheme and visual input. Using a genetic algorithm, we demonstrated that spatiotemporally diverse excitatory inputs-sustained in the proximal and transient in the distal processes-are sufficient to generate experimentally validated centrifugal preference in a single SAC. Reversing these input kinetics did not produce any centrifugal-preferring SAC. We then explored the contribution of SAC-SAC inhibitory connections in establishing the centrifugal preference. SAC inhibitory network enhanced the centrifugal preference, but failed to generate it in its absence. Embedding a direction selective ganglion cell (DSGC) in a SAC network showed that the known SAC-DSGC asymmetric connectivity by itself produces direction selectivity. Still, this selectivity is sharpened in a centrifugal-preferring SAC network. Finally, we use RSME to demonstrate the contribution of SAC-SAC inhibitory connections in mediating direction selectivity and recapitulate recent experimental findings. Thus, using RSME, we obtained a mechanistic understanding of SACs' centrifugal preference and its contribution to direction selectivity.

Spiking Characteristics of Network-Mediated Responses Arising in Direction-Selective Ganglion Cells of Rabbit and Mouse Retinas to Electric Stimulation for Retinal Prostheses

To restore the sight of individuals blinded by outer retinal degeneration, numerous retinal prostheses have been developed. However, the performance of those implants is still hampered by some factors including the lack of comprehensive understanding of the electrically-evoked responses arising in various retinal ganglion cell (RGC) types. In this study, we characterized the electrically-evoked network-mediated responses (hereafter referred to as electric responses) of ON-OFF direction-selective (DS) RGCs in rabbit and mouse retinas for the first time. Interestingly, both species in common demonstrated strong negative correlations between spike counts of electric responses and direction selective indices (DSIs), suggesting electric stimulation activates inhibitory presynaptic neurons that suppress null direction responses for high direction tuning in their light responses. The DS cells of the two species showed several differences including different numbers of bursts. Also, spiking patterns were more heterogeneous across DS RGCs of rabbits than those of mice. The electric response magnitudes of rabbit DS cells showed positive and negative correlations with ON and OFF light response magnitudes to preferred direction motion, respectively. But the mouse DS cells showed positive correlations in both comparisons. Our Fano Factor (FF) and spike time tiling coefficient (STTC) analyses revealed that spiking consistencies across repeats were reduced in late electric responses in both species. Moreover, the response consistencies of DS RGCs were lower than those of non-DS RGCs. Our results indicate the species-dependent retinal circuits may result in different electric response features and therefore suggest a proper animal model may be crucial in prosthetic researches.

Visuomotor control in mice and primates

We conduct a comparative evaluation of the visual systems from the retina to the muscles of the mouse and the macaque monkey noting the differences and similarities between these two species. The topics covered include (1) visual-field overlap, (2) visual spatial resolution, (3) V1 cortical point-image [i.e., V1 tissue dedicated to analyzing a unit receptive field], (4) object versus motion encoding, (5) oculomotor range, (6) eye, head, and body movement coordination, and (7) neocortical and cerebellar function. We also discuss blindsight in rodents and primates which provides insights on how the neocortex mediates conscious vision in these species. This review is timely because the field of visuomotor neurophysiology is expanding beyond the macaque monkey to include the mouse; there is therefore a need for a comparative analysis between these two species on how the brain generates visuomotor responses.

Development of the vertebrate retinal direction-selective circuit

The vertebrate retina contains an array of neural circuits that detect distinct features in visual space. Direction-selective (DS) circuits are an evolutionarily conserved retinal circuit motif - from zebrafish to rodents to primates - specialized for motion detection. During retinal development, neuronal subtypes that wire DS circuits form exquisitely precise connections with each other to shape the output of retinal ganglion cells tuned for specific speeds and directions of motion. In this review, we follow the chronology of DS circuit development in the vertebrate retina, including the cellular, molecular, and activity-dependent mechanisms that regulate the formation of DS circuits, from cell birth and migration to synapse formation and refinement. We highlight recent findings that identify genetic programs critical for specifying neuronal subtypes within DS circuits and molecular interactions essential for responses along the cardinal axes of motion. Finally, we discuss the roles of DS circuits in visual behavior and in certain human visual disease conditions. As one of the best-characterized circuits in the vertebrate retina, DS circuits represent an ideal model system for studying the development of neural connectivity at the level of individual genes, cells, and behavior.

Binocular integration of retinal motion information underlies optic flow processing by the cortex

Locomotion creates various patterns of optic flow on the retina, which provide the observer with information about their movement relative to the environment. However, it is unclear how these optic flow patterns are encoded by the cortex. Here, we use two-photon calcium imaging in awake mice to systematically map monocular and binocular responses to horizontal motion in four areas of the visual cortex. We find that neurons selective to translational or rotational optic flow are abundant in higher visual areas, whereas neurons suppressed by binocular motion are more common in the primary visual cortex. Disruption of retinal direction selectivity in Frmd7 mutant mice reduces the number of translation-selective neurons in the primary visual cortex and translation- and rotation-selective neurons as well as binocular direction-selective neurons in the rostrolateral and anterior visual cortex, blurring the functional distinction between primary and higher visual areas. Thus, optic flow representations in specific areas of the visual cortex rely on binocular integration of motion information from the retina.

•

Translation- and rotation-selective neurons are abundant in higher visual areas

•

Optic-flow-selective neurons in V1 and RL/A rely on retinal direction selectivity

•

Retinal direction selectivity controls functional segregation between V1 and RL/A

•

Binocular integration of retinal motion information underlies optic flow selectivity

A bioinspired angular velocity decoding neural network model for visually guided flights

Efficient and robust motion perception systems are important pre-requisites for achieving visually guided flights in future micro air vehicles. As a source of inspiration, the visual neural networks of flying insects such as honeybee and Drosophila provide ideal examples on which to base artificial motion perception models. In this paper, we have used this approach to develop a novel method that solves the fundamental problem of estimating angular velocity for visually guided flights. Compared with previous models, our elementary motion detector (EMD) based model uses a separate texture estimation pathway to effectively decode angular velocity, and demonstrates considerable independence from the spatial frequency and contrast of the gratings. Using the Unity development platform the model is further tested for tunnel centering and terrain following paradigms in order to reproduce the visually guided flight behaviors of honeybees. In a series of controlled trials, the virtual bee utilizes the proposed angular velocity control schemes to accurately navigate through a patterned tunnel, maintaining a suitable distance from the undulating textured terrain. The results are consistent with both neuron spike recordings and behavioral path recordings of real honeybees, thereby demonstrating the model's potential for implementation in micro air vehicles which have only visual sensors.

On the potential role of lateral connectivity in retinal anticipation

We analyse the potential effects of lateral connectivity (amacrine cells and gap junctions) on motion anticipation in the retina. Our main result is that lateral connectivity can-under conditions analysed in the paper-trigger a wave of activity enhancing the anticipation mechanism provided by local gain control (Berry et al. in Nature 398(6725):334-338, 1999; Chen et al. in J. Neurosci. 33(1):120-132, 2013). We illustrate these predictions by two examples studied in the experimental literature: differential motion sensitive cells (Baccus and Meister in Neuron 36(5):909-919, 2002) and direction sensitive cells where direction sensitivity is inherited from asymmetry in gap junctions connectivity (Trenholm et al. in Nat. Neurosci. 16:154-156, 2013). We finally present reconstructions of retinal responses to 2D visual inputs to assess the ability of our model to anticipate motion in the case of three different 2D stimuli.

A Robust Collision Perception Visual Neural Network With Specific Selectivity to Darker Objects

Building an efficient and reliable collision perception visual system is a challenging problem for future robots and autonomous vehicles. The biological visual neural networks, which have evolved over millions of years in nature and are working perfectly in the real world, could be ideal models for designing artificial vision systems. In the locust's visual pathways, a lobula giant movement detector (LGMD), that is, the LGMD2, has been identified as a looming perception neuron that responds most strongly to darker approaching objects relative to their backgrounds; similar situations which many ground vehicles and robots are often faced with. However, little has been done on modeling the LGMD2 and investigating its potential in robotics and vehicles. In this article, we build an LGMD2 visual neural network which possesses the similar collision selectivity of an LGMD2 neuron in locust via the modeling of biased-ON and -OFF pathways splitting visual signals into parallel ON/OFF channels. With stronger inhibition (bias) in the ON pathway, this model responds selectively to darker looming objects. The proposed model has been tested systematically with a range of stimuli including real-world scenarios. It has also been implemented in a micro-mobile robot and tested with real-time experiments. The experimental results have verified the effectiveness and robustness of the proposed model for detecting darker looming objects against various dynamic and cluttered backgrounds.

Modelling Drosophila motion vision pathways for decoding the direction of translating objects against cluttered moving backgrounds

Decoding the direction of translating objects in front of cluttered moving backgrounds, accurately and efficiently, is still a challenging problem. In nature, lightweight and low-powered flying insects apply motion vision to detect a moving target in highly variable environments during flight, which are excellent paradigms to learn motion perception strategies. This paper investigates the fruit fly Drosophila motion vision pathways and presents computational modelling based on cutting-edge physiological researches. The proposed visual system model features bio-plausible ON and OFF pathways, wide-field horizontal-sensitive (HS) and vertical-sensitive (VS) systems. The main contributions of this research are on two aspects: (1) the proposed model articulates the forming of both direction-selective and direction-opponent responses, revealed as principal features of motion perception neural circuits, in a feed-forward manner; (2) it also shows robust direction selectivity to translating objects in front of cluttered moving backgrounds, via the modelling of spatiotemporal dynamics including combination of motion pre-filtering mechanisms and ensembles of local correlators inside both the ON and OFF pathways, which works effectively to suppress irrelevant background motion or distractors, and to improve the dynamic response. Accordingly, the direction of translating objects is decoded as global responses of both the HS and VS systems with positive or negative output indicating preferred-direction or null-direction translation. The experiments have verified the effectiveness of the proposed neural system model, and demonstrated its responsive preference to faster-moving, higher-contrast and larger-size targets embedded in cluttered moving backgrounds.

Arousal Modulates Retinal Output

At various stages of the visual system, visual responses are modulated by arousal. Here, we find that in mice this modulation operates as early as in the first synapse from the retina and even in retinal axons. To measure retinal activity in the awake, intact brain, we imaged the synaptic boutons of retinal axons in the superior colliculus. Their activity depended not only on vision but also on running speed and pupil size, regardless of retinal illumination. Arousal typically reduced their visual responses and selectivity for direction and orientation. Recordings from retinal axons in the optic tract revealed that arousal modulates the firing of some retinal ganglion cells. Arousal had similar effects postsynaptically in colliculus neurons, independent of activity in the other main source of visual inputs to the colliculus, the primary visual cortex. These results indicate that arousal modulates activity at every stage of the mouse visual system.

Ventral motion parallax enhances fruit fly steering to visual sideslip

Flies and other insects use incoherent motion (parallax) to the front and sides to measure distances and identify obstacles during translation. Although additional depth information could be drawn from below, there is no experimental proof that they use it. The finding that blowflies encode motion disparities in their ventral visual fields suggests this may be an important region for depth information. We used a virtual flight arena to measure fruit fly responses to optic flow. The stimuli appeared below (n = 51) or above the fly (n = 44), at different speeds, with or without parallax cues. Dorsal parallax does not affect responses, and similar motion disparities in rotation have no effect anywhere in the visual field. But responses to strong ventral sideslip (206° s⁻¹) change drastically depending on the presence or absence of parallax. Ventral parallax could help resolve ambiguities in cluttered motion fields, and enhance corrective responses to nearby objects.

Revisiting the role of Dcc in visual system development with a novel eye clearing method

The Deleted in Colorectal Carcinoma (Dcc) receptor plays a critical role in optic nerve development. Whilst Dcc is expressed postnatally in the eye, its function remains unknown as Dcc knockouts die at birth. To circumvent this drawback, we generated an eye-specific Dcc mutant. To study the organization of the retina and visual projections in these mice, we also established EyeDISCO, a novel tissue clearing protocol that removes melanin allowing 3D imaging of whole eyes and visual pathways. We show that in the absence of Dcc, some ganglion cell axons stalled at the optic disc, whereas others perforated the retina, separating photoreceptors from the retinal pigment epithelium. A subset of visual axons entered the CNS, but these projections are perturbed. Moreover, Dcc-deficient retinas displayed a massive postnatal loss of retinal ganglion cells and a large fraction of photoreceptors. Thus, Dcc is essential for the development and maintenance of the retina.

Understanding the retinal basis of vision across species

The vertebrate retina first evolved some 500 million years ago in ancestral marine chordates. Since then, the eyes of different species have been tuned to best support their unique visuoecological lifestyles. Visual specializations in eye designs, large-scale inhomogeneities across the retinal surface and local circuit motifs mean that all species' retinas are unique. Computational theories, such as the efficient coding hypothesis, have come a long way towards an explanation of the basic features of retinal organization and function; however, they cannot explain the full extent of retinal diversity within and across species. To build a truly general understanding of vertebrate vision and the retina's computational purpose, it is therefore important to more quantitatively relate different species' retinal functions to their specific natural environments and behavioural requirements. Ultimately, the goal of such efforts should be to build up to a more general theory of vision.

Nystagmus in patients with congenital stationary night blindness (CSNB) originates from synchronously firing retinal ganglion cells

Congenital nystagmus, involuntary oscillating small eye movements, is commonly thought to originate from aberrant interactions between brainstem nuclei and foveal cortical pathways. Here, we investigated whether nystagmus associated with congenital stationary night blindness (CSNB) results from primary deficits in the retina. We found that CSNB patients as well as an animal model (nob mice), both of which lacked functional nyctalopin protein (NYX, nyx) in ON bipolar cells (BCs) at their synapse with photoreceptors, showed oscillating eye movements at a frequency of 4-7 Hz. nob ON direction-selective ganglion cells (DSGCs), which detect global motion and project to the accessory optic system (AOS), oscillated with the same frequency as their eyes. In the dark, individual ganglion cells (GCs) oscillated asynchronously, but their oscillations became synchronized by light stimulation. Likewise, both patient and nob mice oscillating eye movements were only present in the light when contrast was present. Retinal pharmacological and genetic manipulations that blocked nob GC oscillations also eliminated their oscillating eye movements, and retinal pharmacological manipulations that reduced the oscillation frequency of nob GCs also reduced the oscillation frequency of their eye movements. We conclude that, in nob mice, synchronized oscillations of retinal GCs, most likely the ON-DCGCs, cause nystagmus with properties similar to those associated with CSNB in humans. These results show that the nob mouse is the first animal model for a form of congenital nystagmus, paving the way for development of therapeutic strategies.

Unusual Physiological Properties of Smooth Monostratified Ganglion Cell Types in Primate Retina

The functions of the diverse retinal ganglion cell types in primates and the parallel visual pathways they initiate remain poorly understood. Here, unusual physiological and computational properties of the ON and OFF smooth monostratified ganglion cells are explored. Large-scale multi-electrode recordings from 48 macaque retinas revealed that these cells exhibit irregular receptive field structure composed of spatially segregated hotspots, quite different from the classic center-surround model of retinal receptive fields. Surprisingly, visual stimulation of different hotspots in the same cell produced spikes with subtly different spatiotemporal voltage signatures, consistent with a dendritic contribution to hotspot structure. Targeted visual stimulation and computational inference demonstrated strong nonlinear subunit properties associated with each hotspot, supporting a model in which the hotspots apply nonlinearities at a larger spatial scale than bipolar cells. These findings reveal a previously unreported nonlinear mechanism in the output of the primate retina that contributes to signaling spatial information.

A Plastic Visual Pathway Regulates Cooperative Behavior in Drosophila Larvae

Cooperative behavior emerges in biological systems through coordinated actions among individuals [1, 2]. Although widely observed across animal species, the cellular and molecular mechanisms underlying the establishment and maintenance of cooperative behaviors remain largely unknown [3]. To characterize the circuit mechanisms serving the needs of independent individuals and social groups, we investigated cooperative digging behavior in Drosophila larvae [4-6]. Although chemical and mechanical sensations are important for larval aggregation at specific sites [7-9], an individual larva's ability to participate in a cooperative burrowing cluster relies on direct visual input as well as visual and social experience during development. In addition, vision modulates cluster dynamics by promoting coordinated movements between pairs of larvae [5]. To determine the specific pathways within the larval visual circuit underlying cooperative social clustering, we examined larval photoreceptors (PRs) and the downstream local interneurons (lOLPs) using anatomical and functional studies [10, 11]. Our results indicate that rhodopsin-6-expressing-PRs (Rh6-PRs) and lOLPs are required for both cooperative clustering and movement detection. Remarkably, visual deprivation and social isolation strongly impact the structural and functional connectivity between Rh6-PRs and lOLPs, while at the same time having no effect on the adjacent rhodopsin-5-expressing PRs (Rh5-PRs). Together, our findings demonstrate that a specific larval visual pathway involved in social interactions undergoes experience-dependent modifications during development, suggesting that plasticity in sensory circuits could act as the cellular substrate for social learning, a possible mechanism allowing an animal to integrate into a malleable social environment and engage in complex social behaviors.