A biophysical mechanism for preferred direction enhancement in fly motion vision

Multilevel visual motion opponency in Drosophila

Inhibitory interactions between opponent neuronal pathways constitute a common circuit motif across brain areas and species. However, in most cases, synaptic wiring and biophysical, cellular and network mechanisms generating opponency are unknown. Here, we combine optogenetics, voltage and calcium imaging, connectomics, electrophysiology and modeling to reveal multilevel opponent inhibition in the fly visual system. We uncover a circuit architecture in which a single cell type implements direction-selective, motion-opponent inhibition at all three network levels. This inhibition, mediated by GluClα receptors, is balanced with excitation in strength, despite tenfold fewer synapses. The different opponent network levels constitute a nested, hierarchical structure operating at increasing spatiotemporal scales. Electrophysiology and modeling suggest that distributing this computation over consecutive network levels counteracts a reduction in gain, which would result from integrating large opposing conductances at a single instance. We propose that this neural architecture provides resilience to noise while enabling high selectivity for relevant sensory information.

Motion perception based on ON/OFF channels: A survey

Motion perception is an essential ability for animals and artificially intelligent systems interacting effectively, safely with surrounding objects and environments. Biological visual systems, that have naturally evolved over hundreds-million years, are quite efficient and robust for motion perception, whereas artificial vision systems are far from such capability. This paper argues that the gap can be significantly reduced by formulation of ON/OFF channels in motion perception models encoding luminance increment (ON) and decrement (OFF) responses within receptive field, separately. Such signal-bifurcating structure has been found in neural systems of many animal species articulating early motion is split and processed in segregated pathways. However, the corresponding biological substrates, and the necessity for artificial vision systems have never been elucidated together, leaving concerns on uniqueness and advantages of ON/OFF channels upon building dynamic vision systems to address real world challenges. This paper highlights the importance of ON/OFF channels in motion perception through surveying current progress covering both neuroscience and computationally modelling works with applications. Compared to related literature, this paper for the first time provides insights into implementation of different selectivity to directional motion of looming, translating, and small-sized target movement based on ON/OFF channels in keeping with soundness and robustness of biological principles. Existing challenges and future trends of such bio-plausible computational structure for visual perception in connection with hotspots of machine learning, advanced vision sensors like event-driven camera finally are discussed.

How Flies See Motion

How neurons detect the direction of motion is a prime example of neural computation: Motion vision is found in the visual systems of virtually all sighted animals, it is important for survival, and it requires interesting computations with well-defined linear and nonlinear processing steps-yet the whole process is of moderate complexity. The genetic methods available in the fruit fly Drosophila and the charting of a connectome of its visual system have led to rapid progress and unprecedented detail in our understanding of how neurons compute the direction of motion in this organism. The picture that emerged incorporates not only the identity, morphology, and synaptic connectivity of each neuron involved but also its neurotransmitters, its receptors, and their subcellular localization. Together with the neurons' membrane potential responses to visual stimulation, this information provides the basis for a biophysically realistic model of the circuit that computes the direction of visual motion.

Disynaptic inhibition shapes tuning of OFF-motion detectors in Drosophila

The circuitry underlying the detection of visual motion in Drosophila melanogaster is one of the best studied networks in neuroscience. Lately, electron microscopy reconstructions, algorithmic models, and functional studies have proposed a common motif for the cellular circuitry of an elementary motion detector based on both supralinear enhancement for preferred direction and sublinear suppression for null-direction motion. In T5 cells, however, all columnar input neurons (Tm1, Tm2, Tm4, and Tm9) are excitatory. So, how is null-direction suppression realized there? Using two-photon calcium imaging in combination with thermogenetics, optogenetics, apoptotics, and pharmacology, we discovered that it is via CT1, the GABAergic large-field amacrine cell, where the different processes have previously been shown to act in an electrically isolated way. Within each column, CT1 receives excitatory input from Tm9 and Tm1 and provides the sign-inverted, now inhibitory input signal onto T5. Ablating CT1 or knocking down GABA-receptor subunit Rdl significantly broadened the directional tuning of T5 cells. It thus appears that the signal of Tm1 and Tm9 is used both as an excitatory input for preferred direction enhancement and, through a sign inversion within the Tm1/Tm9-CT1 microcircuit, as an inhibitory input for null-direction suppression.

Optic flow based spatial vision in insects

The optic flow, i.e., the displacement of retinal images of objects in the environment induced by self-motion, is an important source of spatial information, especially for fast-flying insects. Spatial information over a wide range of distances, from the animal's immediate surroundings over several hundred metres to kilometres, is necessary for mediating behaviours, such as landing manoeuvres, collision avoidance in spatially complex environments, learning environmental object constellations and path integration in spatial navigation. To facilitate the processing of spatial information, the complexity of the optic flow is often reduced by active vision strategies. These result in translations and rotations being largely separated by a saccadic flight and gaze mode. Only the translational components of the optic flow contain spatial information. In the first step of optic flow processing, an array of local motion detectors provides a retinotopic spatial proximity map of the environment. This local motion information is then processed in parallel neural pathways in a task-specific manner and used to control the different components of spatial behaviour. A particular challenge here is that the distance information extracted from the optic flow does not represent the distances unambiguously, but these are scaled by the animal's speed of locomotion. Possible ways of coping with this ambiguity are discussed.

Excitatory and inhibitory neural dynamics jointly tune motion detection

Neurons integrate excitatory and inhibitory signals to produce their outputs, but the role of input timing in this integration remains poorly understood. Motion detection is a paradigmatic example of this integration, since theories of motion detection rely on different delays in visual signals. These delays allow circuits to compare scenes at different times to calculate the direction and speed of motion. Different motion detection circuits have different velocity sensitivity, but it remains untested how the response dynamics of individual cell types drive this tuning. Here, we sped up or slowed down specific neuron types in Drosophila's motion detection circuit by manipulating ion channel expression. Altering the dynamics of individual neuron types upstream of motion detectors increased their sensitivity to fast or slow visual motion, exposing distinct roles for excitatory and inhibitory dynamics in tuning directional signals, including a role for the amacrine cell CT1. A circuit model constrained by functional data and anatomy qualitatively reproduced the observed tuning changes. Overall, these results reveal how excitatory and inhibitory dynamics together tune a canonical circuit computation.

The Understanding of ON-Edge Motion Detection Through the Simulation Based on the Connectome of ’s Optic Lobe

The optic lobe of the fly is one of the prominent model systems for the neural mechanism of the motion detection. How a fly who lives under various visual situations of the nature processes the information from at most a few thousands of ommatidia in their neural circuit for the detection of moving objects is not exactly clear though many computational models of the fly optic lobe as a moving objects detector were suggested. Here we attempted to elucidate the mechanisms of ON-edge motion detection by a simulation approach based on the TEM connectome of Drosophila. Our simulation model of the optic lobe with the NEURON simulator that covers the full scale of ommatidia, reproduced the characteristics of the receptor neurons, lamina monopolar neurons, and T4 cells in the lobula. The contribution of each neuron can be estimated by changing synaptic connection strengths in the simulation and measuring the response to the motion stimulus. Those show the paradelle pathway provide motion detection in the fly optic lobe has more robustness and is more sophisticated than a simple combination of HR and BL systems.

Identifying Inputs to Visual Projection Neurons in Drosophila Lobula by Analyzing Connectomic Data

Electron microscopy (EM)-based connectomes provide important insights into how visual circuitry of fruit fly Drosophila computes various visual features, guiding and complementing behavioral and physiological studies. However, connectomic analyses of the lobula, a neuropil putatively dedicated to detecting object-like features, remains underdeveloped, largely because of incomplete data on the inputs to the brain region. Here, we attempted to map the columnar inputs into the Drosophila lobula neuropil by performing connectivity-based and morphology-based clustering on a densely reconstructed connectome dataset. While the dataset mostly lacked visual neuropils other than lobula, which would normally help identify inputs to lobula, our clustering analysis successfully extracted clusters of cells with homogeneous connectivity and morphology, likely representing genuine cell types. We were able to draw a correspondence between the resulting clusters and previously identified cell types, revealing previously undocumented connectivity between lobula input and output neurons. While future, more complete connectomic reconstructions are necessary to verify the results presented here, they can serve as a useful basis for formulating hypotheses on mechanisms of visual feature detection in lobula.

A biophysical account of multiplication by a single neuron

Nonlinear, multiplication-like operations carried out by individual nerve cells greatly enhance the computational power of a neural system1-3, but our understanding of their biophysical implementation is scant. Here we pursue this problem in the Drosophila melanogaster ON motion vision circuit4,5, in which we record the membrane potentials of direction-selective T4 neurons and of their columnar input elements6,7 in response to visual and pharmacological stimuli in vivo. Our electrophysiological measurements and conductance-based simulations provide evidence for a passive supralinear interaction between two distinct types of synapse on T4 dendrites. We show that this multiplication-like nonlinearity arises from the coincidence of cholinergic excitation and release from glutamatergic inhibition. The latter depends on the expression of the glutamate-gated chloride channel GluClα8,9 in T4 neurons, which sharpens the directional tuning of the cells and shapes the optomotor behaviour of the animals. Interacting pairs of shunting inhibitory and excitatory synapses have long been postulated as an analogue approximation of a multiplication, which is integral to theories of motion detection10,11, sound localization12 and sensorimotor control13.

Quantitative Characterization of Output from the Directionally Selective Visual Interneuron H1 in the Grey Flesh Fly Sarcophaga bullata

H1, a very well-studied insect visual interneuron, has a panoramic receptive field and is directionally selective in responding to optic flow. The synaptic basis for the directional selectivity of the H1 neuron has been studied using both theoretical and cellular approaches. Extracellular single-unit recordings are readily obtained by beginning students using commercially available adults of the grey flesh fly Sarcophaga bullata. We describe an apparatus which allows students to present a series of moving visual stimuli to the eye of the restrained, minimally dissected adult Sarcophaga, while recording both the single unit responses of the H1 neuron and the position and velocity of the moving stimulus. Students obtain quantitative and reproducible responses of H1, probing the response properties of the neuron by modulating stimulus parameters such as: direction and speed of movement, visual contrast, spatial wavelength, or the extent of the visual field occupied. Students learn to perform quantitative analysis of their data and to generate graphical representations of their results characterizing the tuning and receptive field of this neuron. This exercise demonstrates the utility of single unit recording of an identified interneuron in an awake restrained insect and promotes interpretation of these results in terms of the visual stimuli normally encountered by freely flying flies in their natural environment.

Non-preferred contrast responses in the Drosophila motion pathways reveal a receptive field structure that explains a common visual illusion

Diverse sensory systems, from audition to thermosensation, feature a separation of inputs into ON (increments) and OFF (decrements) signals. In the Drosophila visual system, separate ON and OFF pathways compute the direction of motion, yet anatomical and functional studies have identified some crosstalk between these channels. We used this well-studied circuit to ask whether the motion computation depends on ON-OFF pathway crosstalk. Using whole-cell electrophysiology, we recorded visual responses of T4 (ON) and T5 (OFF) cells, mapped their composite ON-OFF receptive fields, and found that they share a similar spatiotemporal structure. We fit a biophysical model to these receptive fields that accurately predicts directionally selective T4 and T5 responses to both ON and OFF moving stimuli. This model also provides a detailed mechanistic explanation for the directional preference inversion in response to the prominent reverse-phi illusion. Finally, we used the steering responses of tethered flying flies to validate the model's predicted effects of varying stimulus parameters on the behavioral turning inversion.

Predicting individual neuron responses with anatomically constrained task optimization

Artificial neural networks trained to solve sensory tasks can develop statistical representations that match those in biological circuits. However, it remains unclear whether they can reproduce properties of individual neurons. Here, we investigated how artificial networks predict individual neuron properties in the visual motion circuits of the fruit fly Drosophila. We trained anatomically constrained networks to predict movement in natural scenes, solving the same inference problem as fly motion detectors. Units in the artificial networks adopted many properties of analogous individual neurons, even though they were not explicitly trained to match these properties. Among these properties was the split into ON and OFF motion detectors, which is not predicted by classical motion detection models. The match between model and neurons was closest when models were trained to be robust to noise. These results demonstrate how anatomical, task, and noise constraints can explain properties of individual neurons in a small neural network.

Mechanism of Motion Direction Detection Based on Barlow’s Retina Inhibitory Scheme in Direction-Selective Ganglion Cells

Previous studies have reported that directionally selective ganglion cells respond strongly in their preferred direction, but are only weakly excited by stimuli moving in the opposite null direction. Various studies have attempted to elucidate the mechanisms underlying direction selectivity with cellular basis. However, these studies have not elucidated the mechanism underlying motion direction detection. In this study, we propose the mechanism based on Barlow’s inhibitory scheme for motion direction detection. We described the local motion-sensing direction-selective neurons. Next, this model was used to construct the two-dimensional multi-directional detection neurons which detect the local motion directions. The information of local motion directions was finally used to infer the global motion direction. To verify the validity of the proposed mechanism, we conducted a series of experiments involving a dataset with a number of images. The proposed mechanism exhibited good performance in all experiments with high detection accuracy. Furthermore, we compare the performance of our proposed system and traditional Convolution Neural Network (CNN) on motion direction prediction. It is found that the performance of our system is much better than that of CNN in terms of accuracy, calculation speed and cost.

Autonomous Flying With Neuromorphic Sensing

Autonomous flight for large aircraft appears to be within our reach. However, launching autonomous systems for everyday missions still requires an immense interdisciplinary research effort supported by pointed policies and funding. We believe that concerted endeavors in the fields of neuroscience, mathematics, sensor physics, robotics, and computer science are needed to address remaining crucial scientific challenges. In this paper, we argue for a bio-inspired approach to solve autonomous flying challenges, outline the frontier of sensing, data processing, and flight control within a neuromorphic paradigm, and chart directions of research needed to achieve operational capabilities comparable to those we observe in nature. One central problem of neuromorphic computing is learning. In biological systems, learning is achieved by adaptive and relativistic information acquisition characterized by near-continuous information retrieval with variable rates and sparsity. This results in both energy and computational resource savings being an inspiration for autonomous systems. We consider pertinent features of insect, bat and bird flight behavior as examples to address various vital aspects of autonomous flight. Insects exhibit sophisticated flight dynamics with comparatively reduced complexity of the brain. They represent excellent objects for the study of navigation and flight control. Bats and birds enable more complex models of attention and point to the importance of active sensing for conducting more complex missions. The implementation of neuromorphic paradigms for autonomous flight will require fundamental changes in both traditional hardware and software. We provide recommendations for sensor hardware and processing algorithm development to enable energy efficient and computationally effective flight control.

Conditional protein tagging methods reveal highly specific subcellular distribution of ion channels in motion-sensing neurons

Neurotransmitter receptors and ion channels shape the biophysical properties of neurons, from the sign of the response mediated by neurotransmitter receptors to the dynamics shaped by voltage-gated ion channels. Therefore, knowing the localizations and types of receptors and channels present in neurons is fundamental to our understanding of neural computation. Here, we developed two approaches to visualize the subcellular localization of specific proteins in Drosophila: The flippase-dependent expression of GFP-tagged receptor subunits in single neurons and ‘FlpTag’, a versatile new tool for the conditional labelling of endogenous proteins. Using these methods, we investigated the subcellular distribution of the receptors GluClα, Rdl, and Dα7 and the ion channels para and Ih in motion-sensing T4/T5 neurons of the Drosophila visual system. We discovered a strictly segregated subcellular distribution of these proteins and a sequential spatial arrangement of glutamate, acetylcholine, and GABA receptors along the dendrite that matched the previously reported EM-reconstructed synapse distributions.

A minimal synaptic model for direction selective neurons in Drosophila

Visual motion estimation is a canonical neural computation. In Drosophila, recent advances have identified anatomic and functional circuitry underlying direction-selective computations. Models with varying levels of abstraction have been proposed to explain specific experimental results but have rarely been compared across experiments. Here we use the wealth of available anatomical and physiological data to construct a minimal, biophysically inspired synaptic model for Drosophila’s first-order direction-selective T4 cells. We show how this model relates mathematically to classical models of motion detection, including the Hassenstein-Reichardt correlator model. We used numerical simulation to test how well this synaptic model could reproduce measurements of T4 cells across many datasets and stimulus modalities. These comparisons include responses to sinusoid gratings, to apparent motion stimuli, to stochastic stimuli, and to natural scenes. Without fine-tuning this model, it sufficed to reproduce many, but not all, response properties of T4 cells. Since this model is flexible and based on straightforward biophysical properties, it provides an extensible framework for developing a mechanistic understanding of T4 neural response properties. Moreover, it can be used to assess the sufficiency of simple biophysical mechanisms to describe features of the direction-selective computation and identify where our understanding must be improved.

The computation of directional selectivity in the Drosophila OFF motion pathway

In flies, the direction of moving ON and OFF features is computed separately. T4 (ON) and T5 (OFF) are the first neurons in their respective pathways to extract a directionally selective response from their non-selective inputs. Our recent study of T4 found that the integration of offset depolarizing and hyperpolarizing inputs is critical for the generation of directional selectivity. However, T5s lack small-field inhibitory inputs, suggesting they may use a different mechanism. Here we used whole-cell recordings of T5 neurons and found a similar receptive field structure: fast depolarization and persistent, spatially offset hyperpolarization. By assaying pairwise interactions of local stimulation across the receptive field, we found no amplifying responses, only suppressive responses to the non-preferred motion direction. We then evaluated passive, biophysical models and found that a model using direct inhibition, but not the removal of excitation, can accurately predict T5 responses to a range of moving stimuli.

Mechanisms of vision in the fruit fly

Vision is essential to maximize the efficiency of daily tasks such as feeding, avoiding predators or finding mating partners. An advantageous model is Drosophila melanogaster, since it offers tools that allow genetic and neuronal manipulation with high spatial and temporal resolution, which can be combined with behavioral, anatomical and physiological assays. Recent advances have expanded our knowledge on the neural circuitry underlying such important behaviors as color vision (role of reciprocal inhibition to enhance color signal at the level of the ommatidia); motion vision (motion-detection neurones receive both excitatory and inhibitory input), and sensory processing (role of the central complex in spatial navigation, and in orchestrating the information from other senses and the inner state). Research on synergies between pathways is shaping the field.

How fly neurons compute the direction of visual motion

Detecting the direction of image motion is a fundamental component of visual computation, essential for survival of the animal. However, at the level of individual photoreceptors, the direction in which the image is shifting is not explicitly represented. Rather, directional motion information needs to be extracted from the photoreceptor array by comparing the signals of neighboring units over time. The exact nature of this process as implemented in the visual system of the fruit fly Drosophila melanogaster has been studied in great detail, and much progress has recently been made in determining the neural circuits giving rise to directional motion information. The results reveal the following: (1) motion information is computed in parallel ON and OFF pathways. (2) Within each pathway, T4 (ON) and T5 (OFF) cells are the first neurons to represent the direction of motion. Four subtypes of T4 and T5 cells exist, each sensitive to one of the four cardinal directions. (3) The core process of direction selectivity as implemented on the dendrites of T4 and T5 cells comprises both an enhancement of signals for motion along their preferred direction as well as a suppression of signals for motion along the opposite direction. This combined strategy ensures a high degree of direction selectivity right at the first stage where the direction of motion is computed. (4) At the subsequent processing stage, tangential cells spatially integrate direct excitation from ON and OFF-selective T4 and T5 cells and indirect inhibition from bi-stratified LPi cells activated by neighboring T4/T5 terminals, thus generating flow-field-selective responses.

Constraining computational models using electron microscopy wiring diagrams

Numerous efforts to generate "connectomes," or synaptic wiring diagrams, of large neural circuits or entire nervous systems are currently underway. These efforts promise an abundance of data to guide theoretical models of neural computation and test their predictions. However, there is not yet a standard set of tools for incorporating the connectivity constraints that these datasets provide into the models typically studied in theoretical neuroscience. This article surveys recent approaches to building models with constrained wiring diagrams and the insights they have provided. It also describes challenges and the need for new techniques to scale these approaches to ever more complex datasets.

Comparisons between the ON- and OFF-edge motion pathways in the Drosophila brain

Understanding the circuit mechanisms behind motion detection is a long-standing question in visual neuroscience. In Drosophila melanogaster, recently discovered synapse-level connectomes in the optic lobe, particularly in ON-pathway (T4) receptive-field circuits, in concert with physiological studies, suggest a motion model that is increasingly intricate when compared with the ubiquitous Hassenstein-Reichardt model. By contrast, our knowledge of OFF-pathway (T5) has been incomplete. Here, we present a conclusive and comprehensive connectome that, for the first time, integrates detailed connectivity information for inputs to both the T4 and T5 pathways in a single EM dataset covering the entire optic lobe. With novel reconstruction methods using automated synapse prediction suited to such a large connectome, we successfully corroborate previous findings in the T4 pathway and comprehensively identify inputs and receptive fields for T5. Although the two pathways are probably evolutionarily linked and exhibit many similarities, we uncover interesting differences and interactions that may underlie their distinct functional properties.

The Neuronal Basis of an Illusory Motion Percept Is Explained by Decorrelation of Parallel Motion Pathways

Both vertebrates and invertebrates perceive illusory motion, known as "reverse-phi," in visual stimuli that contain sequential luminance increments and decrements. However, increment (ON) and decrement (OFF) signals are initially processed by separate visual neurons, and parallel elementary motion detectors downstream respond selectively to the motion of light or dark edges, often termed ON- and OFF-edges. It remains unknown how and where ON and OFF signals combine to generate reverse-phi motion signals. Here, we show that each of Drosophila's elementary motion detectors encodes motion by combining both ON and OFF signals. Their pattern of responses reflects combinations of increments and decrements that co-occur in natural motion, serving to decorrelate their outputs. These results suggest that the general principle of signal decorrelation drives the functional specialization of parallel motion detection channels, including their selectivity for moving light or dark edges.

Glutamate Signaling in the Fly Visual System

For a proper understanding of neural circuit function, it is important to know which signals neurons relay to their downstream partners. Calcium imaging with genetically encoded calcium sensors like GCaMP has become the default approach for mapping these responses. How well such measurements represent the true neurotransmitter output of any given cell, however, remains unclear. Here, we demonstrate the viability of the glutamate sensor iGluSnFR for 2-photon in vivo imaging in Drosophila melanogaster and prove its usefulness for estimating spatiotemporal receptive fields in the visual system. We compare the results obtained with iGluSnFR with the ones obtained with GCaMP6f and find that the spatial aspects of the receptive fields are preserved between indicators. In the temporal domain, however, measurements obtained with iGluSnFR reveal the underlying response properties to be much faster than those acquired with GCaMP6f. Our approach thus offers a more accurate description of glutamatergic neurons in the fruit fly.