1
|
Wu N, Zhou B, Agrochao M, Clark DA. Broken time reversal symmetry in visual motion detection. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.06.08.598068. [PMID: 38915608 PMCID: PMC11195140 DOI: 10.1101/2024.06.08.598068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/26/2024]
Abstract
Our intuition suggests that when a movie is played in reverse, our perception of motion in the reversed movie will be perfectly inverted compared to the original. This intuition is also reflected in many classical theoretical and practical models of motion detection. However, here we demonstrate that this symmetry of motion perception upon time reversal is often broken in real visual systems. In this work, we designed a set of visual stimuli to investigate how stimulus symmetries affect time reversal symmetry breaking in the fruit fly Drosophila's well-studied optomotor rotation behavior. We discovered a suite of new stimuli with a wide variety of different properties that can lead to broken time reversal symmetries in fly behavioral responses. We then trained neural network models to predict the velocity of scenes with both natural and artificial contrast distributions. Training with naturalistic contrast distributions yielded models that break time reversal symmetry, even when the training data was time reversal symmetric. We show analytically and numerically that the breaking of time reversal symmetry in the model responses can arise from contrast asymmetry in the training data, but can also arise from other features of the contrast distribution. Furthermore, shallower neural network models can exhibit stronger symmetry breaking than deeper ones, suggesting that less flexible neural networks promote some forms of time reversal symmetry breaking. Overall, these results reveal a surprising feature of biological motion detectors and suggest that it could arise from constrained optimization in natural environments.
Collapse
Affiliation(s)
- Nathan Wu
- Yale College, New Haven, CT 06511, USA
| | - Baohua Zhou
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA
| | - Margarida Agrochao
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA
| | - Damon A. Clark
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA
- Department of Physics, Yale University, New Haven, CT 06511, USA
- Department of Neuroscience, Yale University, New Haven, CT 06511, USA
- Quantitative Biology Institute, Yale University, New Haven, CT 06511, USA
- Wu Tsai Institute, Yale University, New Haven, CT 06511, USA
| |
Collapse
|
2
|
Moreno-Sanchez A, Vasserman AN, Jang H, Hina BW, von Reyn CR, Ausborn J. Morphology and synapse topography optimize linear encoding of synapse numbers in Drosophila looming responsive descending neurons. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.24.591016. [PMID: 38712267 PMCID: PMC11071487 DOI: 10.1101/2024.04.24.591016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2024]
Abstract
Synapses are often precisely organized on dendritic arbors, yet the role of synaptic topography in dendritic integration remains poorly understood. Utilizing electron microscopy (EM) connectomics we investigate synaptic topography in Drosophila melanogaster looming circuits, focusing on retinotopically tuned visual projection neurons (VPNs) that synapse onto descending neurons (DNs). Synapses of a given VPN type project to non-overlapping regions on DN dendrites. Within these spatially constrained clusters, synapses are not retinotopically organized, but instead adopt near random distributions. To investigate how this organization strategy impacts DN integration, we developed multicompartment models of DNs fitted to experimental data and using precise EM morphologies and synapse locations. We find that DN dendrite morphologies normalize EPSP amplitudes of individual synaptic inputs and that near random distributions of synapses ensure linear encoding of synapse numbers from individual VPNs. These findings illuminate how synaptic topography influences dendritic integration and suggest that linear encoding of synapse numbers may be a default strategy established through connectivity and passive neuron properties, upon which active properties and plasticity can then tune as needed.
Collapse
Affiliation(s)
- Anthony Moreno-Sanchez
- Department of Neurobiology and Anatomy, Drexel University College of Medicine, Philadelphia, United States
| | - Alexander N. Vasserman
- Department of Neurobiology and Anatomy, Drexel University College of Medicine, Philadelphia, United States
| | - HyoJong Jang
- School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA, United States
| | - Bryce W. Hina
- School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA, United States
| | - Catherine R. von Reyn
- Department of Neurobiology and Anatomy, Drexel University College of Medicine, Philadelphia, United States
- School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA, United States
| | - Jessica Ausborn
- Department of Neurobiology and Anatomy, Drexel University College of Medicine, Philadelphia, United States
| |
Collapse
|
3
|
Dai Z, Fu Q, Peng J, Li H. SLoN: a spiking looming perception network exploiting neural encoding and processing in ON/OFF channels. Front Neurosci 2024; 18:1291053. [PMID: 38510466 PMCID: PMC10950957 DOI: 10.3389/fnins.2024.1291053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Accepted: 02/14/2024] [Indexed: 03/22/2024] Open
Abstract
Looming perception, the ability to sense approaching objects, is crucial for the survival of humans and animals. After hundreds of millions of years of evolutionary development, biological entities have evolved efficient and robust looming perception visual systems. However, current artificial vision systems fall short of such capabilities. In this study, we propose a novel spiking neural network for looming perception that mimics biological vision to communicate motion information through action potentials or spikes, providing a more realistic approach than previous artificial neural networks based on sum-then-activate operations. The proposed spiking looming perception network (SLoN) comprises three core components. Neural encoding, known as phase coding, transforms video signals into spike trains, introducing the concept of phase delay to depict the spatial-temporal competition between phasic excitatory and inhibitory signals shaping looming selectivity. To align with biological substrates where visual signals are bifurcated into parallel ON/OFF channels encoding brightness increments and decrements separately to achieve specific selectivity to ON/OFF-contrast stimuli, we implement eccentric down-sampling at the entrance of ON/OFF channels, mimicking the foveal region of the mammalian receptive field with higher acuity to motion, computationally modeled with a leaky integrate-and-fire (LIF) neuronal network. The SLoN model is deliberately tested under various visual collision scenarios, ranging from synthetic to real-world stimuli. A notable achievement is that the SLoN selectively spikes for looming features concealed in visual streams against other categories of movements, including translating, receding, grating, and near misses, demonstrating robust selectivity in line with biological principles. Additionally, the efficacy of the ON/OFF channels, the phase coding with delay, and the eccentric visual processing are further investigated to demonstrate their effectiveness in looming perception. The cornerstone of this study rests upon showcasing a new paradigm for looming perception that is more biologically plausible in light of biological motion perception.
Collapse
|
4
|
Hong J, Sun X, Peng J, Fu Q. A Bio-Inspired Probabilistic Neural Network Model for Noise-Resistant Collision Perception. Biomimetics (Basel) 2024; 9:136. [PMID: 38534821 DOI: 10.3390/biomimetics9030136] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2023] [Revised: 09/19/2023] [Accepted: 02/20/2024] [Indexed: 03/28/2024] Open
Abstract
Bio-inspired models based on the lobula giant movement detector (LGMD) in the locust's visual brain have received extensive attention and application for collision perception in various scenarios. These models offer advantages such as low power consumption and high computational efficiency in visual processing. However, current LGMD-based computational models, typically organized as four-layered neural networks, often encounter challenges related to noisy signals, particularly in complex dynamic environments. Biological studies have unveiled the intrinsic stochastic nature of synaptic transmission, which can aid neural computation in mitigating noise. In alignment with these biological findings, this paper introduces a probabilistic LGMD (Prob-LGMD) model that incorporates a probability into the synaptic connections between multiple layers, thereby capturing the uncertainty in signal transmission, interaction, and integration among neurons. Comparative testing of the proposed Prob-LGMD model and two conventional LGMD models was conducted using a range of visual stimuli, including indoor structured scenes and complex outdoor scenes, all subject to artificial noise. Additionally, the model's performance was compared to standard engineering noise-filtering methods. The results clearly demonstrate that the proposed model outperforms all comparative methods, exhibiting a significant improvement in noise tolerance. This study showcases a straightforward yet effective approach to enhance collision perception in noisy environments.
Collapse
Affiliation(s)
- Jialan Hong
- Machine Life and Intelligence Research Centre, School of Mathematics and Information Science, Guangzhou University, Guangzhou 510006, China
| | - Xuelong Sun
- Machine Life and Intelligence Research Centre, School of Mathematics and Information Science, Guangzhou University, Guangzhou 510006, China
| | - Jigen Peng
- Machine Life and Intelligence Research Centre, School of Mathematics and Information Science, Guangzhou University, Guangzhou 510006, China
| | - Qinbing Fu
- Machine Life and Intelligence Research Centre, School of Mathematics and Information Science, Guangzhou University, Guangzhou 510006, China
| |
Collapse
|
5
|
Wu H, Yue S, Hu C. Re-framing bio-plausible collision detection: identifying shared meta-properties through strategic prototyping. Front Neurorobot 2024; 18:1349498. [PMID: 38333372 PMCID: PMC10850265 DOI: 10.3389/fnbot.2024.1349498] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Accepted: 01/12/2024] [Indexed: 02/10/2024] Open
Abstract
Insects exhibit remarkable abilities in navigating complex natural environments, whether it be evading predators, capturing prey, or seeking out con-specifics, all of which rely on their compact yet reliable neural systems. We explore the field of bio-inspired robotic vision systems, focusing on the locust inspired Lobula Giant Movement Detector (LGMD) models. The existing LGMD models are thoroughly evaluated, identifying their common meta-properties that are essential for their functionality. This article reveals a common framework, characterized by layered structures and computational strategies, which is crucial for enhancing the capability of bio-inspired models for diverse applications. The result of this analysis is the Strategic Prototype, which embodies the identified meta-properties. It represents a modular and more flexible method for developing more responsive and adaptable robotic visual systems. The perspective highlights the potential of the Strategic Prototype: LGMD-Universally Prototype (LGMD-UP), the key to re-framing LGMD models and advancing our understanding and implementation of bio-inspired visual systems in robotics. It might open up more flexible and adaptable avenues for research and practical applications.
Collapse
Affiliation(s)
- Haotian Wu
- School of Mechanical and Electrical Engineering, Guangzhou University, Guangzhou, China
- Machine Life and Intelligence Research Center, Guangzhou University, Guangzhou, China
| | - Shigang Yue
- Machine Life and Intelligence Research Center, Guangzhou University, Guangzhou, China
- School of Computing and Mathematical Sciences, University of Leicester, Leicester, United Kingdom
| | - Cheng Hu
- School of Mechanical and Electrical Engineering, Guangzhou University, Guangzhou, China
- Machine Life and Intelligence Research Center, Guangzhou University, Guangzhou, China
| |
Collapse
|
6
|
Rubinov M. Circular and unified analysis in network neuroscience. eLife 2023; 12:e79559. [PMID: 38014843 PMCID: PMC10684154 DOI: 10.7554/elife.79559] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Accepted: 10/18/2023] [Indexed: 11/29/2023] Open
Abstract
Genuinely new discovery transcends existing knowledge. Despite this, many analyses in systems neuroscience neglect to test new speculative hypotheses against benchmark empirical facts. Some of these analyses inadvertently use circular reasoning to present existing knowledge as new discovery. Here, I discuss that this problem can confound key results and estimate that it has affected more than three thousand studies in network neuroscience over the last decade. I suggest that future studies can reduce this problem by limiting the use of speculative evidence, integrating existing knowledge into benchmark models, and rigorously testing proposed discoveries against these models. I conclude with a summary of practical challenges and recommendations.
Collapse
Affiliation(s)
- Mika Rubinov
- Departments of Biomedical Engineering, Computer Science, and Psychology, Vanderbilt UniversityNashvilleUnited States
- Janelia Research Campus, Howard Hughes Medical InstituteAshburnUnited States
| |
Collapse
|
7
|
Tanaka R, Zhou B, Agrochao M, Badwan BA, Au B, Matos NCB, Clark DA. Neural mechanisms to incorporate visual counterevidence in self-movement estimation. Curr Biol 2023; 33:4960-4979.e7. [PMID: 37918398 PMCID: PMC10848174 DOI: 10.1016/j.cub.2023.10.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2023] [Revised: 10/07/2023] [Accepted: 10/09/2023] [Indexed: 11/04/2023]
Abstract
In selecting appropriate behaviors, animals should weigh sensory evidence both for and against specific beliefs about the world. For instance, animals measure optic flow to estimate and control their own rotation. However, existing models of flow detection can be spuriously triggered by visual motion created by objects moving in the world. Here, we show that stationary patterns on the retina, which constitute evidence against observer rotation, suppress inappropriate stabilizing rotational behavior in the fruit fly Drosophila. In silico experiments show that artificial neural networks (ANNs) that are optimized to distinguish observer movement from external object motion similarly detect stationarity and incorporate negative evidence. Employing neural measurements and genetic manipulations, we identified components of the circuitry for stationary pattern detection, which runs parallel to the fly's local motion and optic-flow detectors. Our results show how the fly brain incorporates negative evidence to improve heading stability, exemplifying how a compact brain exploits geometrical constraints of the visual world.
Collapse
Affiliation(s)
- Ryosuke Tanaka
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA
| | - Baohua Zhou
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA; Department of Statistics and Data Science, Yale University, New Haven, CT 06511, USA
| | - Margarida Agrochao
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA
| | - Bara A Badwan
- School of Engineering and Applied Science, Yale University, New Haven, CT 06511, USA
| | - Braedyn Au
- Department of Physics, Yale University, New Haven, CT 06511, USA
| | - Natalia C B Matos
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA
| | - Damon A Clark
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA; Department of Physics, Yale University, New Haven, CT 06511, USA; Department of Neuroscience, Yale University, New Haven, CT 06511, USA; Wu Tsai Institute, Yale University, New Haven, CT 06511, USA; Quantitative Biology Institute, Yale University, New Haven, CT 06511, USA.
| |
Collapse
|
8
|
Barredo E, Theobald J. Insect neurobiology: What to do with conflicting evidence? Curr Biol 2023; 33:R1188-R1190. [PMID: 37989095 DOI: 10.1016/j.cub.2023.09.060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2023]
Abstract
Sensory systems gather information from the environment so the nervous system can formulate appropriate responses. But what happens when sensory information is inconsistent? A new study demonstrates how flies respond to incompatible visual evidence of their own motion.
Collapse
Affiliation(s)
- Elina Barredo
- Institute of Environment and Department of Biological Sciences, Florida International University, Miami, FL 33199, USA
| | - Jamie Theobald
- Institute of Environment and Department of Biological Sciences, Florida International University, Miami, FL 33199, USA.
| |
Collapse
|
9
|
Matsumoto A, Yonehara K. Emerging computational motifs: Lessons from the retina. Neurosci Res 2023; 196:11-22. [PMID: 37352934 DOI: 10.1016/j.neures.2023.06.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Revised: 06/03/2023] [Accepted: 06/08/2023] [Indexed: 06/25/2023]
Abstract
The retinal neuronal circuit is the first stage of visual processing in the central nervous system. The efforts of scientists over the last few decades indicate that the retina is not merely an array of photosensitive cells, but also a processor that performs various computations. Within a thickness of only ∼200 µm, the retina consists of diverse forms of neuronal circuits, each of which encodes different visual features. Since the discovery of direction-selective cells by Horace Barlow and Richard Hill, the mechanisms that generate direction selectivity in the retina have remained a fascinating research topic. This review provides an overview of recent advances in our understanding of direction-selectivity circuits. Beyond the conventional wisdom of direction selectivity, emerging findings indicate that the retina utilizes complicated and sophisticated mechanisms in which excitatory and inhibitory pathways are involved in the efficient encoding of motion information. As will become evident, the discovery of computational motifs in the retina facilitates an understanding of how sensory systems establish feature selectivity.
Collapse
Affiliation(s)
- Akihiro Matsumoto
- Danish Research Institute of Translational Neuroscience - DANDRITE, Nordic-EMBL Partnership for Molecular Medicine, Department of Biomedicine, Aarhus University, Aarhus, Denmark; Department of Gene Function and Phenomics, National Institute of Genetics, Mishima, Japan; Department of Genetics, The Graduate University for Advanced Studies (SOKENDAI), Mishima, Japan.
| | - Keisuke Yonehara
- Danish Research Institute of Translational Neuroscience - DANDRITE, Nordic-EMBL Partnership for Molecular Medicine, Department of Biomedicine, Aarhus University, Aarhus, Denmark; Department of Gene Function and Phenomics, National Institute of Genetics, Mishima, Japan; Department of Genetics, The Graduate University for Advanced Studies (SOKENDAI), Mishima, Japan
| |
Collapse
|
10
|
Chen J, Gish CM, Fransen JW, Salazar-Gatzimas E, Clark DA, Borghuis BG. Direct comparison reveals algorithmic similarities in fly and mouse visual motion detection. iScience 2023; 26:107928. [PMID: 37810236 PMCID: PMC10550730 DOI: 10.1016/j.isci.2023.107928] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Revised: 08/07/2023] [Accepted: 09/12/2023] [Indexed: 10/10/2023] Open
Abstract
Evolution has equipped vertebrates and invertebrates with neural circuits that selectively encode visual motion. While similarities in the computations performed by these circuits in mouse and fruit fly have been noted, direct experimental comparisons have been lacking. Because molecular mechanisms and neuronal morphology in the two species are distinct, we directly compared motion encoding in these two species at the algorithmic level, using matched stimuli and focusing on a pair of analogous neurons, the mouse ON starburst amacrine cell (ON SAC) and Drosophila T4 neurons. We find that the cells share similar spatiotemporal receptive field structures, sensitivity to spatiotemporal correlations, and tuning to sinusoidal drifting gratings, but differ in their responses to apparent motion stimuli. Both neuron types showed a response to summed sinusoids that deviates from models for motion processing in these cells, underscoring the similarities in their processing and identifying response features that remain to be explained.
Collapse
Affiliation(s)
- Juyue Chen
- Interdepartmental Neurosciences Program, Yale University, New Haven, CT 06511, USA
| | - Caitlin M Gish
- Department of Physics, Yale University, New Haven, CT 06511, USA
| | - James W Fransen
- Department of Anatomical Sciences and Neurobiology, University of Louisville, Louisville, KY 40202, USA
| | | | - Damon A Clark
- Interdepartmental Neurosciences Program, Yale University, New Haven, CT 06511, USA
- Department of Physics, Yale University, New Haven, CT 06511, USA
- Department of Molecular, Cellular, Developmental Biology, Yale University, New Haven, CT 06511, USA
- Department of Neuroscience, Yale University, New Haven, CT 06511, USA
| | - Bart G Borghuis
- Department of Anatomical Sciences and Neurobiology, University of Louisville, Louisville, KY 40202, USA
| |
Collapse
|
11
|
Mabuchi Y, Cui X, Xie L, Kim H, Jiang T, Yapici N. Visual feedback neurons fine-tune Drosophila male courtship via GABA-mediated inhibition. Curr Biol 2023; 33:3896-3910.e7. [PMID: 37673068 PMCID: PMC10529139 DOI: 10.1016/j.cub.2023.08.034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Revised: 06/27/2023] [Accepted: 08/11/2023] [Indexed: 09/08/2023]
Abstract
Many species of animals use vision to regulate their social behaviors. However, the molecular and circuit mechanisms underlying visually guided social interactions remain largely unknown. Here, we show that the Drosophila ortholog of the human GABAA-receptor-associated protein (GABARAP) is required in a class of visual feedback neurons, lamina tangential (Lat) cells, to fine-tune male courtship. GABARAP is a ubiquitin-like protein that maintains cell-surface levels of GABAA receptors. We demonstrate that knocking down GABARAP or GABAAreceptors in Lat neurons or hyperactivating them induces male courtship toward other males. Inhibiting Lat neurons, on the other hand, delays copulation by impairing the ability of males to follow females. Remarkably, the fly GABARAP protein and its human ortholog share a strong sequence identity, and the fly GABARAP function in Lat neurons can be rescued by its human ortholog. Using in vivo two-photon imaging and optogenetics, we reveal that Lat neurons are functionally connected to neural circuits that mediate visually guided courtship pursuits in males. Our work identifies a novel physiological function for GABARAP in regulating visually guided courtship pursuits in Drosophila males. Reduced GABAA signaling has been linked to social deficits observed in the autism spectrum and bipolar disorders. The functional similarity between the human and the fly GABARAP raises the possibility of a conserved role for this gene in regulating social behaviors across insects and mammals.
Collapse
Affiliation(s)
- Yuta Mabuchi
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY 14853, USA
| | - Xinyue Cui
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY 14853, USA
| | - Lily Xie
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY 14853, USA
| | - Haein Kim
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY 14853, USA
| | - Tianxing Jiang
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY 14853, USA
| | - Nilay Yapici
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY 14853, USA.
| |
Collapse
|
12
|
Fu Q. Motion perception based on ON/OFF channels: A survey. Neural Netw 2023; 165:1-18. [PMID: 37263088 DOI: 10.1016/j.neunet.2023.05.031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2022] [Revised: 04/02/2023] [Accepted: 05/17/2023] [Indexed: 06/03/2023]
Abstract
Motion perception is an essential ability for animals and artificially intelligent systems interacting effectively, safely with surrounding objects and environments. Biological visual systems, that have naturally evolved over hundreds-million years, are quite efficient and robust for motion perception, whereas artificial vision systems are far from such capability. This paper argues that the gap can be significantly reduced by formulation of ON/OFF channels in motion perception models encoding luminance increment (ON) and decrement (OFF) responses within receptive field, separately. Such signal-bifurcating structure has been found in neural systems of many animal species articulating early motion is split and processed in segregated pathways. However, the corresponding biological substrates, and the necessity for artificial vision systems have never been elucidated together, leaving concerns on uniqueness and advantages of ON/OFF channels upon building dynamic vision systems to address real world challenges. This paper highlights the importance of ON/OFF channels in motion perception through surveying current progress covering both neuroscience and computationally modelling works with applications. Compared to related literature, this paper for the first time provides insights into implementation of different selectivity to directional motion of looming, translating, and small-sized target movement based on ON/OFF channels in keeping with soundness and robustness of biological principles. Existing challenges and future trends of such bio-plausible computational structure for visual perception in connection with hotspots of machine learning, advanced vision sensors like event-driven camera finally are discussed.
Collapse
Affiliation(s)
- Qinbing Fu
- Machine Life and Intelligence Research Centre, School of Mathematics and Information Science, Guangzhou University, Guangzhou, 510006, China.
| |
Collapse
|
13
|
Dorkenwald S, Matsliah A, Sterling AR, Schlegel P, Yu SC, McKellar CE, Lin A, Costa M, Eichler K, Yin Y, Silversmith W, Schneider-Mizell C, Jordan CS, Brittain D, Halageri A, Kuehner K, Ogedengbe O, Morey R, Gager J, Kruk K, Perlman E, Yang R, Deutsch D, Bland D, Sorek M, Lu R, Macrina T, Lee K, Bae JA, Mu S, Nehoran B, Mitchell E, Popovych S, Wu J, Jia Z, Castro M, Kemnitz N, Ih D, Bates AS, Eckstein N, Funke J, Collman F, Bock DD, Jefferis GS, Seung HS, Murthy M. Neuronal wiring diagram of an adult brain. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.06.27.546656. [PMID: 37425937 PMCID: PMC10327113 DOI: 10.1101/2023.06.27.546656] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/11/2023]
Abstract
Connections between neurons can be mapped by acquiring and analyzing electron microscopic (EM) brain images. In recent years, this approach has been applied to chunks of brains to reconstruct local connectivity maps that are highly informative, yet inadequate for understanding brain function more globally. Here, we present the first neuronal wiring diagram of a whole adult brain, containing 5×107 chemical synapses between ~130,000 neurons reconstructed from a female Drosophila melanogaster. The resource also incorporates annotations of cell classes and types, nerves, hemilineages, and predictions of neurotransmitter identities. Data products are available by download, programmatic access, and interactive browsing and made interoperable with other fly data resources. We show how to derive a projectome, a map of projections between regions, from the connectome. We demonstrate the tracing of synaptic pathways and the analysis of information flow from inputs (sensory and ascending neurons) to outputs (motor, endocrine, and descending neurons), across both hemispheres, and between the central brain and the optic lobes. Tracing from a subset of photoreceptors all the way to descending motor pathways illustrates how structure can uncover putative circuit mechanisms underlying sensorimotor behaviors. The technologies and open ecosystem of the FlyWire Consortium set the stage for future large-scale connectome projects in other species.
Collapse
Affiliation(s)
- Sven Dorkenwald
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
- Computer Science Department, Princeton University, Princeton, USA
| | - Arie Matsliah
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | - Amy R Sterling
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
- Eyewire, Boston, USA
| | - Philipp Schlegel
- Neurobiology Division, MRC Laboratory of Molecular Biology, Cambridge, UK
- Drosophila Connectomics Group, Department of Zoology, University of Cambridge, Cambridge, UK
| | - Szi-chieh Yu
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | | | - Albert Lin
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
- Center for the Physics of Biological Function, Princeton University, Princeton, USA
| | - Marta Costa
- Drosophila Connectomics Group, Department of Zoology, University of Cambridge, Cambridge, UK
| | - Katharina Eichler
- Drosophila Connectomics Group, Department of Zoology, University of Cambridge, Cambridge, UK
| | - Yijie Yin
- Drosophila Connectomics Group, Department of Zoology, University of Cambridge, Cambridge, UK
| | - Will Silversmith
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | | | - Chris S. Jordan
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | | | - Akhilesh Halageri
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | - Kai Kuehner
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | | | - Ryan Morey
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | - Jay Gager
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | | | | | - Runzhe Yang
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
- Computer Science Department, Princeton University, Princeton, USA
| | - David Deutsch
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
- Department of Neurobiology, University of Haifa, Haifa, Israel
| | - Doug Bland
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | - Marissa Sorek
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
- Eyewire, Boston, USA
| | - Ran Lu
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | - Thomas Macrina
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
- Computer Science Department, Princeton University, Princeton, USA
| | - Kisuk Lee
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
- Brain & Cognitive Sciences Department, Massachusetts Institute of Technology, Cambridge, USA
| | - J. Alexander Bae
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
- Electrical and Computer Engineering Department, Princeton University, Princeton, USA
| | - Shang Mu
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | - Barak Nehoran
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
- Computer Science Department, Princeton University, Princeton, USA
| | - Eric Mitchell
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | - Sergiy Popovych
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
- Computer Science Department, Princeton University, Princeton, USA
| | - Jingpeng Wu
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | - Zhen Jia
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | - Manuel Castro
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | - Nico Kemnitz
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | - Dodam Ih
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | - Alexander Shakeel Bates
- Neurobiology Division, MRC Laboratory of Molecular Biology, Cambridge, UK
- Drosophila Connectomics Group, Department of Zoology, University of Cambridge, Cambridge, UK
- Harvard Medical School, Boston, USA
- Centre for Neural Circuits and Behaviour, The University of Oxford, Oxford, UK
| | - Nils Eckstein
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, USA
| | - Jan Funke
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, USA
| | | | - Davi D. Bock
- Department of Neurological Sciences, Larner College of Medicine, University of Vermont, Burlington, USA
| | - Gregory S.X.E Jefferis
- Neurobiology Division, MRC Laboratory of Molecular Biology, Cambridge, UK
- Drosophila Connectomics Group, Department of Zoology, University of Cambridge, Cambridge, UK
| | - H. Sebastian Seung
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
- Computer Science Department, Princeton University, Princeton, USA
| | - Mala Murthy
- Princeton Neuroscience Institute, Princeton University, Princeton, USA
| | | |
Collapse
|
14
|
Abstract
How neurons detect the direction of motion is a prime example of neural computation: Motion vision is found in the visual systems of virtually all sighted animals, it is important for survival, and it requires interesting computations with well-defined linear and nonlinear processing steps-yet the whole process is of moderate complexity. The genetic methods available in the fruit fly Drosophila and the charting of a connectome of its visual system have led to rapid progress and unprecedented detail in our understanding of how neurons compute the direction of motion in this organism. The picture that emerged incorporates not only the identity, morphology, and synaptic connectivity of each neuron involved but also its neurotransmitters, its receptors, and their subcellular localization. Together with the neurons' membrane potential responses to visual stimulation, this information provides the basis for a biophysically realistic model of the circuit that computes the direction of visual motion.
Collapse
Affiliation(s)
- Alexander Borst
- Max Planck Institute for Biological Intelligence, Martinsried, Germany; ,
| | - Lukas N Groschner
- Max Planck Institute for Biological Intelligence, Martinsried, Germany; ,
| |
Collapse
|
15
|
Zhao Y, Ke S, Cheng G, Lv X, Chang J, Zhou W. Direction Selectivity of TmY Neurites in Drosophila. Neurosci Bull 2023; 39:759-773. [PMID: 36399278 PMCID: PMC10169962 DOI: 10.1007/s12264-022-00966-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Accepted: 06/29/2022] [Indexed: 11/19/2022] Open
Abstract
The perception of motion is an important function of vision. Neural wiring diagrams for extracting directional information have been obtained by connectome reconstruction. Direction selectivity in Drosophila is thought to originate in T4/T5 neurons through integrating inputs with different temporal filtering properties. Through genetic screening based on synaptic distribution, we isolated a new type of TmY neuron, termed TmY-ds, that form reciprocal synaptic connections with T4/T5 neurons. Its neurites responded to grating motion along the four cardinal directions and showed a variety of direction selectivity. Intriguingly, its direction selectivity originated from temporal filtering neurons rather than T4/T5. Genetic silencing and activation experiments showed that TmY-ds neurons are functionally upstream of T4/T5. Our results suggest that direction selectivity is generated in a tripartite circuit formed among these three neurons-temporal filtering, TmY-ds, and T4/T5 neurons, in which TmY-ds plays a role in the enhancement of direction selectivity in T4/T5 neurons.
Collapse
Affiliation(s)
- Yinyin Zhao
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, 430074, China
| | - Shanshan Ke
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, 430074, China
| | - Guo Cheng
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, 430074, China
| | - Xiaohua Lv
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, 430074, China
| | - Jin Chang
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, 430074, China.
| | - Wei Zhou
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, 430074, China.
| |
Collapse
|
16
|
Mishra A, Serbe-Kamp E, Borst A, Haag J. Voltage to Calcium Transformation Enhances Direction Selectivity in Drosophila T4 Neurons. J Neurosci 2023; 43:2497-2514. [PMID: 36849417 PMCID: PMC10082464 DOI: 10.1523/jneurosci.2297-22.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Revised: 02/03/2023] [Accepted: 02/08/2023] [Indexed: 03/01/2023] Open
Abstract
An important step in neural information processing is the transformation of membrane voltage into calcium signals leading to transmitter release. However, the effect of voltage to calcium transformation on neural responses to different sensory stimuli is not well understood. Here, we use in vivo two-photon imaging of genetically encoded voltage and calcium indicators, ArcLight and GCaMP6f, respectively, to measure responses in direction-selective T4 neurons of female Drosophila Comparison between ArcLight and GCaMP6f signals reveals calcium signals to have a significantly higher direction selectivity compared with voltage signals. Using these recordings, we build a model which transforms T4 voltage responses into calcium responses. Using a cascade of thresholding, temporal filtering and a stationary nonlinearity, the model reproduces experimentally measured calcium responses across different visual stimuli. These findings provide a mechanistic underpinning of the voltage to calcium transformation and show how this processing step, in addition to synaptic mechanisms on the dendrites of T4 cells, enhances direction selectivity in the output signal of T4 neurons. Measuring the directional tuning of postsynaptic vertical system (VS)-cells with inputs from other cells blocked, we found that, indeed, it matches the one of the calcium signal in presynaptic T4 cells.SIGNIFICANCE STATEMENT The transformation of voltage to calcium influx is an important step in the signaling cascade within a nerve cell. While this process has been intensely studied in the context of transmitter release mechanism, its consequences for information transmission and neural computation are unclear. Here, we measured both membrane voltage and cytosolic calcium levels in direction-selective cells of Drosophila in response to a large set of visual stimuli. We found direction selectivity in the calcium signal to be significantly enhanced compared with membrane voltage through a nonlinear transformation of voltage to calcium. Our findings highlight the importance of an additional step in the signaling cascade for information processing within single nerve cells.
Collapse
Affiliation(s)
- Abhishek Mishra
- Max Planck Institute for Biological Intelligence, 82152 Martinsried, Germany
- Graduate School of Systemic Neurosciences, Ludwig Maximilian University of Munich, 82152 Martinsried, Germany
| | - Etienne Serbe-Kamp
- Max Planck Institute for Biological Intelligence, 82152 Martinsried, Germany
| | - Alexander Borst
- Max Planck Institute for Biological Intelligence, 82152 Martinsried, Germany
- Graduate School of Systemic Neurosciences, Ludwig Maximilian University of Munich, 82152 Martinsried, Germany
| | - Juergen Haag
- Max Planck Institute for Biological Intelligence, 82152 Martinsried, Germany
| |
Collapse
|
17
|
Fu Q, Li Z, Peng J. Harmonizing motion and contrast vision for robust looming detection. ARRAY 2023. [DOI: 10.1016/j.array.2022.100272] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022] Open
|
18
|
Bengochea M, Hassan B. Numerosity as a visual property: Evidence from two highly evolutionary distant species. Front Physiol 2023; 14:1086213. [PMID: 36846325 PMCID: PMC9949967 DOI: 10.3389/fphys.2023.1086213] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Accepted: 01/31/2023] [Indexed: 02/11/2023] Open
Abstract
Most animals, from humans to invertebrates, possess an ability to estimate numbers. This evolutionary advantage facilitates animals' choice of environments with more food sources, more conspecifics to increase mating success, and/or reduced predation risk among others. However, how the brain processes numerical information remains largely unknown. There are currently two lines of research interested in how numerosity of visual objects is perceived and analyzed in the brain. The first argues that numerosity is an advanced cognitive ability processed in high-order brain areas, while the second proposes that "numbers" are attributes of the visual scene and thus numerosity is processed in the visual sensory system. Recent evidence points to a sensory involvement in estimating magnitudes. In this Perspective, we highlight this evidence in two highly evolutionary distant species: humans and flies. We also discuss the advantages of studying numerical processing in fruit flies in order to dissect the neural circuits involved in and required for numerical processing. Based on experimental manipulation and the fly connectome, we propose a plausible neural network for number sense in invertebrates.
Collapse
Affiliation(s)
| | - Bassem Hassan
- *Correspondence: Mercedes Bengochea, ; Bassem Hassan,
| |
Collapse
|
19
|
Wang Y, Li H, Zheng Y, Peng J. A directionally selective collision-sensing visual neural network based on fractional-order differential operator. Front Neurorobot 2023; 17:1149675. [PMID: 37152416 PMCID: PMC10160397 DOI: 10.3389/fnbot.2023.1149675] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2023] [Accepted: 03/30/2023] [Indexed: 05/09/2023] Open
Abstract
In this paper, we propose a directionally selective fractional-order lobular giant motion detector (LGMD) visual neural network. Unlike most collision-sensing network models based on LGMDs, our model can not only sense collision threats but also obtain the motion direction of the collision object. Firstly, this paper simulates the membrane potential response of neurons using the fractional-order differential operator to generate reliable collision response spikes. Then, a new correlation mechanism is proposed to obtain the motion direction of objects. Specifically, this paper performs correlation operation on the signals extracted from two pixels, utilizing the temporal delay of the signals to obtain their position relationship. In this way, the response characteristics of direction-selective neurons can be characterized. Finally, ON/OFF visual channels are introduced to encode increases and decreases in brightness, respectively, thereby modeling the bipolar response of special neurons. Extensive experimental results show that the proposed visual neural system conforms to the response characteristics of biological LGMD and direction-selective neurons, and that the performance of the system is stable and reliable.
Collapse
|
20
|
de Jong DB, Paredes-Valles F, de Croon GCHE. How Do Neural Networks Estimate Optical Flow? A Neuropsychology-Inspired Study. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2022; 44:8290-8305. [PMID: 34033535 DOI: 10.1109/tpami.2021.3083538] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
End-to-end trained convolutional neural networks have led to a breakthrough in optical flow estimation. The most recent advances focus on improving the optical flow estimation by improving the architecture and setting a new benchmark on the publicly available MPI-Sintel dataset. Instead, in this article, we investigate how deep neural networks estimate optical flow. A better understanding of how these networks function is important for (i) assessing their generalization capabilities to unseen inputs, and (ii) suggesting changes to improve their performance. For our investigation, we focus on FlowNetS, as it is the prototype of an encoder-decoder neural network for optical flow estimation. Furthermore, we use a filter identification method that has played a major role in uncovering the motion filters present in animal brains in neuropsychological research. The method shows that the filters in the deepest layer of FlowNetS are sensitive to a variety of motion patterns. Not only do we find translation filters, as demonstrated in animal brains, but thanks to the easier measurements in artificial neural networks, we even unveil dilation, rotation, and occlusion filters. Furthermore, we find similarities in the refinement part of the network and the perceptual filling-in process which occurs in the mammal primary visual cortex.
Collapse
|
21
|
Ling J, Wang H, Xu M, Chen H, Li H, Peng J. Mathematical study of neural feedback roles in small target motion detection. Front Neurorobot 2022; 16:984430. [PMID: 36203523 PMCID: PMC9530796 DOI: 10.3389/fnbot.2022.984430] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2022] [Accepted: 08/26/2022] [Indexed: 11/28/2022] Open
Abstract
Building an efficient and reliable small target motion detection visual system is challenging for artificial intelligence robotics because a small target only occupies few pixels and hardly displays visual features in images. Biological visual systems that have evolved over millions of years could be ideal templates for designing artificial visual systems. Insects benefit from a class of specialized neurons, called small target motion detectors (STMDs), which endow them with an excellent ability to detect small moving targets against a cluttered dynamic environment. Some bio-inspired models featured in feed-forward information processing architectures have been proposed to imitate the functions of the STMD neurons. However, feedback, a crucial mechanism for visual system regulation, has not been investigated deeply in the STMD-based neural circuits and its roles in small target motion detection remain unclear. In this paper, we propose a time-delay feedback STMD model for small target motion detection in complex backgrounds. The main contributions of this study are as follows. First, a feedback pathway is designed by transmitting information from output-layer neurons to lower-layer interneurons in the STMD pathway and the role of the feedback is analyzed from the view of mathematical analysis. Second, to estimate the feedback constant, the existence and uniqueness of solutions for nonlinear dynamical systems formed by feedback loop are analyzed via Schauder's fixed point theorem and contraction mapping theorem. Finally, an iterative algorithm is designed to solve the nonlinear problem and the performance of the proposed model is tested by experiments. Experimental results demonstrate that the feedback is able to weaken background false positives while maintaining a minor effect on small targets. It outperforms existing STMD-based models regarding the accuracy of fast-moving small target detection in visual clutter. The proposed feedback approach could inspire the relevant modeling of robust motion perception robotics visual systems.
Collapse
Affiliation(s)
- Jun Ling
- School of Mathematics and Information Science, Guangzhou University, Guangzhou, China
| | - Hongxin Wang
- Machine Life and Intelligence Research Center, Guangzhou University, Guangzhou, China
- Computational Intelligence Lab (CIL), School of Computer Science, University of Lincoln, Lincoln, United Kingdom
| | - Mingshuo Xu
- School of Mathematics and Information Science, Guangzhou University, Guangzhou, China
| | - Hao Chen
- School of Mathematics and Information Science, Guangzhou University, Guangzhou, China
| | - Haiyang Li
- School of Mathematics and Information Science, Guangzhou University, Guangzhou, China
- *Correspondence: Haiyang Li
| | - Jigen Peng
- School of Mathematics and Information Science, Guangzhou University, Guangzhou, China
- Jigen Peng
| |
Collapse
|
22
|
Abstract
The nematode worm Caenorhabditis elegans has a relatively simple neural system for analysis of information transmission from sensory organ to muscle fiber. Consequently, this study includes an example of a neural circuit from the nematode worm, and a procedure is shown for measuring its information optimality by use of a logic gate model. This approach is useful where the assumptions are applicable for a neural circuit, and also for choosing between competing mathematical hypotheses that explain the function of a neural circuit. In this latter case, the logic gate model can estimate computational complexity and distinguish which of the mathematical models require fewer computations. In addition, the concept of information optimality is generalized to other biological systems, along with an extended discussion of its role in genetic-based pathways of organisms.
Collapse
|
23
|
Huang X, Qiao H, Li H, Jiang Z. Bioinspired approach-sensitive neural network for collision detection in cluttered and dynamic backgrounds. Appl Soft Comput 2022. [DOI: 10.1016/j.asoc.2022.108782] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
24
|
Patterson SS, Bembry BN, Mazzaferri MA, Neitz M, Rieke F, Soetedjo R, Neitz J. Conserved circuits for direction selectivity in the primate retina. Curr Biol 2022; 32:2529-2538.e4. [PMID: 35588744 DOI: 10.1016/j.cub.2022.04.056] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2021] [Revised: 02/25/2022] [Accepted: 04/20/2022] [Indexed: 02/06/2023]
Abstract
The detection of motion direction is a fundamental visual function and a classic model for neural computation. In the non-primate retina, direction selectivity arises in starburst amacrine cell (SAC) dendrites, which provide selective inhibition to direction-selective retinal ganglion cells (dsRGCs). Although SACs are present in primates, their connectivity and the existence of dsRGCs remain open questions. Here, we present a connectomic reconstruction of the primate ON SAC circuit from a serial electron microscopy volume of the macaque central retina. We show that the structural basis for the SACs' ability to confer directional selectivity on postsynaptic neurons is conserved. SACs selectively target a candidate homolog to the mammalian ON-sustained dsRGCs that project to the accessory optic system (AOS) and contribute to gaze-stabilizing reflexes. These results indicate that the capacity to compute motion direction is present in the retina, which is earlier in the primate visual system than classically thought.
Collapse
Affiliation(s)
- Sara S Patterson
- Center for Visual Science, University of Rochester, Rochester, NY 14620, USA; Department of Ophthalmology, University of Washington, Seattle, WA 98109, USA.
| | - Briyana N Bembry
- Department of Ophthalmology, University of Washington, Seattle, WA 98109, USA
| | - Marcus A Mazzaferri
- Department of Ophthalmology, University of Washington, Seattle, WA 98109, USA
| | - Maureen Neitz
- Department of Ophthalmology, University of Washington, Seattle, WA 98109, USA
| | - Fred Rieke
- Department of Physiology and Biophysics, University of Washington, Seattle, WA 98195, USA
| | - Robijanto Soetedjo
- Department of Physiology and Biophysics, University of Washington, Seattle, WA 98195, USA; Washington National Primate Research Center, University of Washington, Seattle, WA 98195, USA
| | - Jay Neitz
- Department of Ophthalmology, University of Washington, Seattle, WA 98109, USA.
| |
Collapse
|
25
|
An Artificial Visual System for Motion Direction Detection Based on the Hassenstein–Reichardt Correlator Model. ELECTRONICS 2022. [DOI: 10.3390/electronics11091423] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
The perception of motion direction is essential for the survival of visual animals. Despite various theoretical and biophysical investigations that have been conducted to elucidate directional selectivity at the neural level, the systemic mechanism of motion direction detection remains elusive. Here, we develop an artificial visual system (AVS) based on the core computation of the Hassenstein–Reichardt correlator (HRC) model for global motion direction detection. With reference to the biological investigations of Drosophila, we first describe a local motion-sensitive, directionally detective neuron that only responds to ON motion signals with high pattern contrast in a particular direction. Then, we use the full-neurons scheme motion direction detection mechanism to detect the global motion direction based on our previous research. The mechanism enables our AVS to detect multiple directions in a two-dimensional view, and the global motion direction is inferred from the outputs of all local motion-sensitive directionally detective neurons. To verify the reliability of our AVS, we conduct a series of experiments and compare its performance with the time-considered convolution neural network (CNN) and the EfficientNetB0 under the same conditions. The experimental results demonstrated that our system is reliable in detecting the direction of motion, and among the three models, our AVS has better motion direction detection capabilities.
Collapse
|
26
|
Neural mechanisms to exploit positional geometry for collision avoidance. Curr Biol 2022; 32:2357-2374.e6. [PMID: 35508172 DOI: 10.1016/j.cub.2022.04.023] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2022] [Revised: 03/21/2022] [Accepted: 04/08/2022] [Indexed: 11/21/2022]
Abstract
Visual motion provides rich geometrical cues about the three-dimensional configuration of the world. However, how brains decode the spatial information carried by motion signals remains poorly understood. Here, we study a collision-avoidance behavior in Drosophila as a simple model of motion-based spatial vision. With simulations and psychophysics, we demonstrate that walking Drosophila exhibit a pattern of slowing to avoid collisions by exploiting the geometry of positional changes of objects on near-collision courses. This behavior requires the visual neuron LPLC1, whose tuning mirrors the behavior and whose activity drives slowing. LPLC1 pools inputs from object and motion detectors, and spatially biased inhibition tunes it to the geometry of collisions. Connectomic analyses identified circuitry downstream of LPLC1 that faithfully inherits its response properties. Overall, our results reveal how a small neural circuit solves a specific spatial vision task by combining distinct visual features to exploit universal geometrical constraints of the visual world.
Collapse
|
27
|
A functionally ordered visual feature map in the Drosophila brain. Neuron 2022; 110:1700-1711.e6. [DOI: 10.1016/j.neuron.2022.02.013] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2021] [Revised: 11/30/2021] [Accepted: 02/16/2022] [Indexed: 12/19/2022]
|
28
|
Detecting Square Grid Structure in an Animal Neuronal Network. NEUROSCI 2022. [DOI: 10.3390/neurosci3010007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
An animal neural system ranges from a cluster of a few neurons to a brain of billions. At the lower range, it is possible to test each neuron for its role across a set of environmental conditions. However, the higher range requires another approach. One method is to disentangle the organization of the neuronal network. In the case of the entorhinal cortex in a rodent, a set of neuronal cells involved in spatial location activate in a regular grid-like arrangement. Therefore, it is of interest to develop methods to find these kinds of patterns in a neural network. For this study, a square grid arrangement of neurons is quantified by network metrics and then applied for identification of square grid structure in areas of the fruit fly brain. The results show several regions with contiguous clusters of square grid arrangements in the neural network, supportive of specialization in the information processing of the system.
Collapse
|
29
|
Luan H, Fu Q, Zhang Y, Hua M, Chen S, Yue S. A Looming Spatial Localization Neural Network Inspired by MLG1 Neurons in the Crab Neohelice. Front Neurosci 2022; 15:787256. [PMID: 35126038 PMCID: PMC8814358 DOI: 10.3389/fnins.2021.787256] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Accepted: 12/23/2021] [Indexed: 11/13/2022] Open
Abstract
Similar to most visual animals, the crab Neohelice granulata relies predominantly on visual information to escape from predators, to track prey and for selecting mates. It, therefore, needs specialized neurons to process visual information and determine the spatial location of looming objects. In the crab Neohelice granulata, the Monostratified Lobula Giant type1 (MLG1) neurons have been found to manifest looming sensitivity with finely tuned capabilities of encoding spatial location information. MLG1s neuronal ensemble can not only perceive the location of a looming stimulus, but are also thought to be able to influence the direction of movement continuously, for example, escaping from a threatening, looming target in relation to its position. Such specific characteristics make the MLG1s unique compared to normal looming detection neurons in invertebrates which can not localize spatial looming. Modeling the MLG1s ensemble is not only critical for elucidating the mechanisms underlying the functionality of such neural circuits, but also important for developing new autonomous, efficient, directionally reactive collision avoidance systems for robots and vehicles. However, little computational modeling has been done for implementing looming spatial localization analogous to the specific functionality of MLG1s ensemble. To bridge this gap, we propose a model of MLG1s and their pre-synaptic visual neural network to detect the spatial location of looming objects. The model consists of 16 homogeneous sectors arranged in a circular field inspired by the natural arrangement of 16 MLG1s' receptive fields to encode and convey spatial information concerning looming objects with dynamic expanding edges in different locations of the visual field. Responses of the proposed model to systematic real-world visual stimuli match many of the biological characteristics of MLG1 neurons. The systematic experiments demonstrate that our proposed MLG1s model works effectively and robustly to perceive and localize looming information, which could be a promising candidate for intelligent machines interacting within dynamic environments free of collision. This study also sheds light upon a new type of neuromorphic visual sensor strategy that can extract looming objects with locational information in a quick and reliable manner.
Collapse
Affiliation(s)
- Hao Luan
- School of Computer Science and Engineering, Tianjin University of Technology, Tianjin, China
| | - Qinbing Fu
- Machine Life and Intelligence Research Centre, School of Mathematics and Information Science, Guangzhou University, Guangzhou, China
- Computational Intelligence Laboratory (CIL), School of Computer Science, University of Lincoln, Lincoln, United Kingdom
| | - Yicheng Zhang
- Machine Life and Intelligence Research Centre, School of Mathematics and Information Science, Guangzhou University, Guangzhou, China
| | - Mu Hua
- Machine Life and Intelligence Research Centre, School of Mathematics and Information Science, Guangzhou University, Guangzhou, China
| | - Shengyong Chen
- School of Computer Science and Engineering, Tianjin University of Technology, Tianjin, China
| | - Shigang Yue
- Machine Life and Intelligence Research Centre, School of Mathematics and Information Science, Guangzhou University, Guangzhou, China
- Computational Intelligence Laboratory (CIL), School of Computer Science, University of Lincoln, Lincoln, United Kingdom
- *Correspondence: Shigang Yue
| |
Collapse
|
30
|
Barsotti E, Correia A, Cardona A. Neural architectures in the light of comparative connectomics. Curr Opin Neurobiol 2021; 71:139-149. [PMID: 34837731 PMCID: PMC8694100 DOI: 10.1016/j.conb.2021.10.006] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2021] [Revised: 10/20/2021] [Accepted: 10/26/2021] [Indexed: 01/06/2023]
Abstract
Since the Cambrian, animals diversified from a few body forms or bauplans, into many extinct and all extant species. A characteristic neural architecture serves each bauplan. How the connectome of each animal differs from that of closely related species or whether it converged into an optimal architecture shared with more distant ones is unknown. Recent technological innovations in molecular biology, microscopy, digital data storage and processing, and computational neuroscience have lowered the barriers for whole-brain connectomics. Comparative connectomics of suitable, relatively small, representative species across the phylogenetic tree can infer the archetypal neural architecture of each bauplan and identify any circuits that possibly converged onto a shared and potentially optimal, structure.
Collapse
Affiliation(s)
- Elizabeth Barsotti
- MRC Laboratory of Molecular Biology, Cambridge, UK; Department of Physiology, Development and Neuroscience, University of Cambridge, UK
| | - Ana Correia
- MRC Laboratory of Molecular Biology, Cambridge, UK; Department of Physiology, Development and Neuroscience, University of Cambridge, UK
| | - Albert Cardona
- MRC Laboratory of Molecular Biology, Cambridge, UK; Department of Physiology, Development and Neuroscience, University of Cambridge, UK.
| |
Collapse
|
31
|
Clemens J, Schöneich S, Kostarakos K, Hennig RM, Hedwig B. A small, computationally flexible network produces the phenotypic diversity of song recognition in crickets. eLife 2021; 10:e61475. [PMID: 34761750 PMCID: PMC8635984 DOI: 10.7554/elife.61475] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2020] [Accepted: 11/03/2021] [Indexed: 01/31/2023] Open
Abstract
How neural networks evolved to generate the diversity of species-specific communication signals is unknown. For receivers of the signals, one hypothesis is that novel recognition phenotypes arise from parameter variation in computationally flexible feature detection networks. We test this hypothesis in crickets, where males generate and females recognize the mating songs with a species-specific pulse pattern, by investigating whether the song recognition network in the cricket brain has the computational flexibility to recognize different temporal features. Using electrophysiological recordings from the network that recognizes crucial properties of the pulse pattern on the short timescale in the cricket Gryllus bimaculatus, we built a computational model that reproduces the neuronal and behavioral tuning of that species. An analysis of the model's parameter space reveals that the network can provide all recognition phenotypes for pulse duration and pause known in crickets and even other insects. Phenotypic diversity in the model is consistent with known preference types in crickets and other insects, and arises from computations that likely evolved to increase energy efficiency and robustness of pattern recognition. The model's parameter to phenotype mapping is degenerate - different network parameters can create similar changes in the phenotype - which likely supports evolutionary plasticity. Our study suggests that computationally flexible networks underlie the diverse pattern recognition phenotypes, and we reveal network properties that constrain and support behavioral diversity.
Collapse
Affiliation(s)
- Jan Clemens
- European Neuroscience Institute Göttingen – A Joint Initiative of the University Medical Center Göttingen and the Max-Planck SocietyGöttingenGermany
- BCCN GöttingenGöttingenGermany
| | - Stefan Schöneich
- University of Cambridge, Department of ZoologyCambridgeUnited Kingdom
- Friedrich-Schiller-University Jena, Institute for Zoology and Evolutionary ResearchJenaGermany
| | - Konstantinos Kostarakos
- University of Cambridge, Department of ZoologyCambridgeUnited Kingdom
- Institute of Biology, University of GrazUniversitätsplatzAustria
| | - R Matthias Hennig
- Humboldt-Universität zu Berlin, Department of BiologyPhilippstrasseGermany
| | - Berthold Hedwig
- University of Cambridge, Department of ZoologyCambridgeUnited Kingdom
| |
Collapse
|
32
|
Mano O, Creamer MS, Badwan BA, Clark DA. Predicting individual neuron responses with anatomically constrained task optimization. Curr Biol 2021; 31:4062-4075.e4. [PMID: 34324832 DOI: 10.1016/j.cub.2021.06.090] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2021] [Revised: 05/24/2021] [Accepted: 06/29/2021] [Indexed: 01/28/2023]
Abstract
Artificial neural networks trained to solve sensory tasks can develop statistical representations that match those in biological circuits. However, it remains unclear whether they can reproduce properties of individual neurons. Here, we investigated how artificial networks predict individual neuron properties in the visual motion circuits of the fruit fly Drosophila. We trained anatomically constrained networks to predict movement in natural scenes, solving the same inference problem as fly motion detectors. Units in the artificial networks adopted many properties of analogous individual neurons, even though they were not explicitly trained to match these properties. Among these properties was the split into ON and OFF motion detectors, which is not predicted by classical motion detection models. The match between model and neurons was closest when models were trained to be robust to noise. These results demonstrate how anatomical, task, and noise constraints can explain properties of individual neurons in a small neural network.
Collapse
Affiliation(s)
- Omer Mano
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA; Department of Neuroscience, Yale University, New Haven, CT 06511, USA
| | - Matthew S Creamer
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA
| | - Bara A Badwan
- School of Engineering and Applied Science, Yale University, New Haven, CT 06511, USA
| | - Damon A Clark
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA; Department of Neuroscience, Yale University, New Haven, CT 06511, USA; Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA; Department of Physics, Yale University, New Haven, CT 06511, USA.
| |
Collapse
|
33
|
Abstract
Cognition is often defined as a dual process of physical and non-physical mechanisms. This duality originated from past theory on the constituent parts of the natural world. Even though material causation is not an explanation for all natural processes, phenomena at the cellular level of life are modeled by physical causes. These phenomena include explanations for the function of organ systems, including the nervous system and information processing in the cerebrum. This review restricts the definition of cognition to a mechanistic process and enlists studies that support an abstract set of proximate mechanisms. Specifically, this process is approached from a large-scale perspective, the flow of information in a neural system. Study at this scale further constrains the possible explanations for cognition since the information flow is amenable to theory, unlike a lower-level approach where the problem becomes intractable. These possible hypotheses include stochastic processes for explaining the processes of cognition along with principles that support an abstract format for the encoded information.
Collapse
|
34
|
Dalgaty T, Miller JP, Vianello E, Casas J. Bio-Inspired Architectures Substantially Reduce the Memory Requirements of Neural Network Models. Front Neurosci 2021; 15:612359. [PMID: 33708069 PMCID: PMC7940538 DOI: 10.3389/fnins.2021.612359] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2020] [Accepted: 02/02/2021] [Indexed: 11/30/2022] Open
Abstract
We propose a neural network model for the jumping escape response behavior observed in the cricket cercal sensory system. This sensory system processes low-intensity air currents in the animal's immediate environment generated by predators, competitors, and mates. Our model is inspired by decades of physiological and anatomical studies. We compare the performance of our model with a model derived through a universal approximation, or a generic deep learning, approach, and demonstrate that, to achieve the same performance, these models required between one and two orders of magnitude more parameters. Furthermore, since the architecture of the bio-inspired model is defined by a set of logical relations between neurons, we find that the model is open to interpretation and can be understood. This work demonstrates the potential of incorporating bio-inspired architectural motifs, which have evolved in animal nervous systems, into memory efficient neural network models.
Collapse
Affiliation(s)
| | - John P Miller
- Department of Microbiology and Immunology, Montana State University, Bozeman, MT, United States
| | | | - Jérôme Casas
- Insect Biology Research Institute IRBI, UMR CNRS 7261, Université de Tours, Tours, France
| |
Collapse
|
35
|
Catalani E, Silvestri F, Bongiorni S, Taddei AR, Fanelli G, Rinalducci S, De Palma C, Perrotta C, Prantera G, Cervia D. Retinal damage in a new model of hyperglycemia induced by high-sucrose diets. Pharmacol Res 2021; 166:105488. [PMID: 33582248 DOI: 10.1016/j.phrs.2021.105488] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/01/2021] [Revised: 02/08/2021] [Accepted: 02/08/2021] [Indexed: 12/13/2022]
Abstract
Loss of retinal neurons may precede clinical signs of diabetic retinopathy (DR). We studied for the first time the effects of hyperglycemia on the visual system of the fruit fly Drosophila melanogaster to characterize a model for glucose-induced retinal neurodegeneration, thus complementing more traditional vertebrate systems. Adult flies were fed with increased high-sucrose regimens which did not modify the locomotion ability, muscle phenotype and mobility after 10 days. The increased availability of dietary sucrose induced hyperglycemia and phosphorylation of Akt in fat tissue, without significant effects on adult growth and viability, consistent with the early phase of insulin signaling and a low impact on the overall metabolic profile of flies at short term. Noteworthy, high-sucrose diets significantly decreased Drosophila responsiveness to the light as a consequence of vision defects. Hyperglycemia did not alter the gross anatomical architecture of the external eye phenotype although a progressive damage of photosensitive units was observed. Appreciable levels of cleaved caspase 3 and nitrotyrosine were detected in the internal retina network as well as punctate staining of Light-Chain 3 and p62, and accumulated autophagosomes, indicating apoptotic features, peroxynitrite formation and autophagy turnover defects. In summary, our results in Drosophila support the view that hyperglycemia induced by high-sucrose diets lead to eye defects, apoptosis/autophagy dysregulation, oxidative stress, and visual dysfunctions which are evolutionarily conserved, thus offering a meaningful opportunity of using a simple in vivo model to study the pathophysiology of neuroretinal alterations that develop in patients at the early stages of DR.
Collapse
Affiliation(s)
- Elisabetta Catalani
- Department for Innovation in Biological, Agro-food and Forest Systems (DIBAF), Università degli Studi della Tuscia, largo dell'Università snc, 01100 Viterbo, Italy
| | - Federica Silvestri
- Department for Innovation in Biological, Agro-food and Forest Systems (DIBAF), Università degli Studi della Tuscia, largo dell'Università snc, 01100 Viterbo, Italy
| | - Silvia Bongiorni
- Department of Ecological and Biological Sciences (DEB), Università degli Studi della Tuscia, largo dell'Università snc, 01100 Viterbo, Italy
| | - Anna Rita Taddei
- Section of Electron Microscopy, Great Equipment Center, Università degli Studi della Tuscia, largo dell'Università snc, 01100 Viterbo, Italy
| | - Giuseppina Fanelli
- Department of Ecological and Biological Sciences (DEB), Università degli Studi della Tuscia, largo dell'Università snc, 01100 Viterbo, Italy
| | - Sara Rinalducci
- Department of Ecological and Biological Sciences (DEB), Università degli Studi della Tuscia, largo dell'Università snc, 01100 Viterbo, Italy
| | - Clara De Palma
- Department of Medical Biotechnology and Translational Medicine (BioMeTra), Università degli Studi di Milano, via L. Vanvitelli 32, 20129 Milano, Italy
| | - Cristiana Perrotta
- Department of Biomedical and Clinical Sciences "Luigi Sacco" (DIBIC), Università degli Studi di Milano, via G.B. Grassi 74, 20157 Milano, Italy
| | - Giorgio Prantera
- Department of Ecological and Biological Sciences (DEB), Università degli Studi della Tuscia, largo dell'Università snc, 01100 Viterbo, Italy
| | - Davide Cervia
- Department for Innovation in Biological, Agro-food and Forest Systems (DIBAF), Università degli Studi della Tuscia, largo dell'Università snc, 01100 Viterbo, Italy.
| |
Collapse
|
36
|
Friedman R. Themes of advanced information processing in the primate brain. AIMS Neurosci 2020; 7:373-388. [PMID: 33263076 PMCID: PMC7701368 DOI: 10.3934/neuroscience.2020023] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2020] [Accepted: 10/09/2020] [Indexed: 11/30/2022] Open
Abstract
Here is a review of several empirical examples of information processing that occur in the primate cerebral cortex. These include visual processing, object identification and perception, information encoding, and memory. Also, there is a discussion of the higher scale neural organization, mainly theoretical, which suggests hypotheses on how the brain internally represents objects. Altogether they support the general attributes of the mechanisms of brain computation, such as efficiency, resiliency, data compression, and a modularization of neural function and their pathways. Moreover, the specific neural encoding schemes are expectedly stochastic, abstract and not easily decoded by theoretical or empirical approaches.
Collapse
Affiliation(s)
- Robert Friedman
- Department of Biological Sciences, University of South Carolina, Columbia 29208, USA
| |
Collapse
|
37
|
Fu Q, Hu C, Peng J, Rind FC, Yue S. A Robust Collision Perception Visual Neural Network With Specific Selectivity to Darker Objects. IEEE TRANSACTIONS ON CYBERNETICS 2020; 50:5074-5088. [PMID: 31804947 DOI: 10.1109/tcyb.2019.2946090] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Building an efficient and reliable collision perception visual system is a challenging problem for future robots and autonomous vehicles. The biological visual neural networks, which have evolved over millions of years in nature and are working perfectly in the real world, could be ideal models for designing artificial vision systems. In the locust's visual pathways, a lobula giant movement detector (LGMD), that is, the LGMD2, has been identified as a looming perception neuron that responds most strongly to darker approaching objects relative to their backgrounds; similar situations which many ground vehicles and robots are often faced with. However, little has been done on modeling the LGMD2 and investigating its potential in robotics and vehicles. In this article, we build an LGMD2 visual neural network which possesses the similar collision selectivity of an LGMD2 neuron in locust via the modeling of biased-ON and -OFF pathways splitting visual signals into parallel ON/OFF channels. With stronger inhibition (bias) in the ON pathway, this model responds selectively to darker looming objects. The proposed model has been tested systematically with a range of stimuli including real-world scenarios. It has also been implemented in a micro-mobile robot and tested with real-time experiments. The experimental results have verified the effectiveness and robustness of the proposed model for detecting darker looming objects against various dynamic and cluttered backgrounds.
Collapse
|
38
|
Zhao F, Zeng Y, Guo A, Su H, Xu B. A neural algorithm for Drosophila linear and nonlinear decision-making. Sci Rep 2020; 10:18660. [PMID: 33122701 PMCID: PMC7596070 DOI: 10.1038/s41598-020-75628-y] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2020] [Accepted: 10/16/2020] [Indexed: 11/15/2022] Open
Abstract
It has been evidenced that vision-based decision-making in Drosophila consists of both simple perceptual (linear) decision and value-based (non-linear) decision. This paper proposes a general computational spiking neural network (SNN) model to explore how different brain areas are connected contributing to Drosophila linear and nonlinear decision-making behavior. First, our SNN model could successfully describe all the experimental findings in fly visual reinforcement learning and action selection among multiple conflicting choices as well. Second, our computational modeling shows that dopaminergic neuron-GABAergic neuron-mushroom body (DA-GABA-MB) works in a recurrent loop providing a key circuit for gain and gating mechanism of nonlinear decision making. Compared with existing models, our model shows more biologically plausible on the network design and working mechanism, and could amplify the small differences between two conflicting cues more clearly. Finally, based on the proposed model, the UAV could quickly learn to make clear-cut decisions among multiple visual choices and flexible reversal learning resembling to real fly. Compared with linear and uniform decision-making methods, the DA-GABA-MB mechanism helps UAV complete the decision-making task with fewer steps.
Collapse
Affiliation(s)
- Feifei Zhao
- Research Center for Brain-inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China
| | - Yi Zeng
- Research Center for Brain-inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China. .,Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China. .,National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China. .,University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Aike Guo
- Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China. .,University of Chinese Academy of Sciences, Beijing, 100049, China. .,Institute of Neuroscience and State Key Laboratory of Neuroscience, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences, Shanghai, 200031, China. .,State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing, 100101, China. .,Shanghai Institute of Microsystem and Information Technology, Shanghai, 200050, China.
| | - Haifeng Su
- Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China.,Institute of Neuroscience and State Key Laboratory of Neuroscience, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences, Shanghai, 200031, China.,Laboratory of Molecular Neural Biology, School of Life Sciences, Shanghai University, Shanghai, 200444, China
| | - Bo Xu
- Research Center for Brain-inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China.,Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China.,University of Chinese Academy of Sciences, Beijing, 100049, China
| |
Collapse
|
39
|
Fendl S, Vieira RM, Borst A. Conditional protein tagging methods reveal highly specific subcellular distribution of ion channels in motion-sensing neurons. eLife 2020; 9:62953. [PMID: 33079061 PMCID: PMC7655108 DOI: 10.7554/elife.62953] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2020] [Accepted: 10/14/2020] [Indexed: 11/25/2022] Open
Abstract
Neurotransmitter receptors and ion channels shape the biophysical properties of neurons, from the sign of the response mediated by neurotransmitter receptors to the dynamics shaped by voltage-gated ion channels. Therefore, knowing the localizations and types of receptors and channels present in neurons is fundamental to our understanding of neural computation. Here, we developed two approaches to visualize the subcellular localization of specific proteins in Drosophila: The flippase-dependent expression of GFP-tagged receptor subunits in single neurons and ‘FlpTag’, a versatile new tool for the conditional labelling of endogenous proteins. Using these methods, we investigated the subcellular distribution of the receptors GluClα, Rdl, and Dα7 and the ion channels para and Ih in motion-sensing T4/T5 neurons of the Drosophila visual system. We discovered a strictly segregated subcellular distribution of these proteins and a sequential spatial arrangement of glutamate, acetylcholine, and GABA receptors along the dendrite that matched the previously reported EM-reconstructed synapse distributions.
Collapse
Affiliation(s)
- Sandra Fendl
- Max Planck Institute of Neurobiology, Martinsried, Germany.,Graduate School of Systemic Neurosciences, LMU Munich, Martinsried, Germany
| | | | - Alexander Borst
- Max Planck Institute of Neurobiology, Martinsried, Germany.,Graduate School of Systemic Neurosciences, LMU Munich, Martinsried, Germany
| |
Collapse
|
40
|
Fu Q, Yue S. Modelling Drosophila motion vision pathways for decoding the direction of translating objects against cluttered moving backgrounds. BIOLOGICAL CYBERNETICS 2020; 114:443-460. [PMID: 32623517 PMCID: PMC7554016 DOI: 10.1007/s00422-020-00841-x] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/31/2020] [Accepted: 06/19/2020] [Indexed: 06/03/2023]
Abstract
Decoding the direction of translating objects in front of cluttered moving backgrounds, accurately and efficiently, is still a challenging problem. In nature, lightweight and low-powered flying insects apply motion vision to detect a moving target in highly variable environments during flight, which are excellent paradigms to learn motion perception strategies. This paper investigates the fruit fly Drosophila motion vision pathways and presents computational modelling based on cutting-edge physiological researches. The proposed visual system model features bio-plausible ON and OFF pathways, wide-field horizontal-sensitive (HS) and vertical-sensitive (VS) systems. The main contributions of this research are on two aspects: (1) the proposed model articulates the forming of both direction-selective and direction-opponent responses, revealed as principal features of motion perception neural circuits, in a feed-forward manner; (2) it also shows robust direction selectivity to translating objects in front of cluttered moving backgrounds, via the modelling of spatiotemporal dynamics including combination of motion pre-filtering mechanisms and ensembles of local correlators inside both the ON and OFF pathways, which works effectively to suppress irrelevant background motion or distractors, and to improve the dynamic response. Accordingly, the direction of translating objects is decoded as global responses of both the HS and VS systems with positive or negative output indicating preferred-direction or null-direction translation. The experiments have verified the effectiveness of the proposed neural system model, and demonstrated its responsive preference to faster-moving, higher-contrast and larger-size targets embedded in cluttered moving backgrounds.
Collapse
Affiliation(s)
- Qinbing Fu
- Machine Life and Intelligence Research Centre, Guangzhou University, Guangzhou, China.
- Computational Intelligence Lab/Lincoln Centre for Autonomous Systems, University of Lincoln, Lincoln, UK.
| | - Shigang Yue
- Machine Life and Intelligence Research Centre, Guangzhou University, Guangzhou, China.
- Computational Intelligence Lab/Lincoln Centre for Autonomous Systems, University of Lincoln, Lincoln, UK.
| |
Collapse
|
41
|
Zavatone-Veth JA, Badwan BA, Clark DA. A minimal synaptic model for direction selective neurons in Drosophila. J Vis 2020; 20:2. [PMID: 32040161 PMCID: PMC7343402 DOI: 10.1167/jov.20.2.2] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/07/2023] Open
Abstract
Visual motion estimation is a canonical neural computation. In Drosophila, recent advances have identified anatomic and functional circuitry underlying direction-selective computations. Models with varying levels of abstraction have been proposed to explain specific experimental results but have rarely been compared across experiments. Here we use the wealth of available anatomical and physiological data to construct a minimal, biophysically inspired synaptic model for Drosophila’s first-order direction-selective T4 cells. We show how this model relates mathematically to classical models of motion detection, including the Hassenstein-Reichardt correlator model. We used numerical simulation to test how well this synaptic model could reproduce measurements of T4 cells across many datasets and stimulus modalities. These comparisons include responses to sinusoid gratings, to apparent motion stimuli, to stochastic stimuli, and to natural scenes. Without fine-tuning this model, it sufficed to reproduce many, but not all, response properties of T4 cells. Since this model is flexible and based on straightforward biophysical properties, it provides an extensible framework for developing a mechanistic understanding of T4 neural response properties. Moreover, it can be used to assess the sufficiency of simple biophysical mechanisms to describe features of the direction-selective computation and identify where our understanding must be improved.
Collapse
|
42
|
Serotonergic modulation of visual neurons in Drosophila melanogaster. PLoS Genet 2020; 16:e1009003. [PMID: 32866139 PMCID: PMC7485980 DOI: 10.1371/journal.pgen.1009003] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2020] [Revised: 09/11/2020] [Accepted: 07/22/2020] [Indexed: 02/06/2023] Open
Abstract
Sensory systems rely on neuromodulators, such as serotonin, to provide flexibility for information processing as stimuli vary, such as light intensity throughout the day. Serotonergic neurons broadly innervate the optic ganglia of Drosophila melanogaster, a widely used model for studying vision. It remains unclear whether serotonin modulates the physiology of interneurons in the optic ganglia. To address this question, we first mapped the expression patterns of serotonin receptors in the visual system, focusing on a subset of cells with processes in the first optic ganglion, the lamina. Serotonin receptor expression was found in several types of columnar cells in the lamina including 5-HT2B in lamina monopolar cell L2, required for spatiotemporal luminance contrast, and both 5-HT1A and 5-HT1B in T1 cells, whose function is unknown. Subcellular mapping with GFP-tagged 5-HT2B and 5-HT1A constructs indicated that these receptors localize to layer M2 of the medulla, proximal to serotonergic boutons, suggesting that the medulla neuropil is the primary site of serotonergic regulation for these neurons. Exogenous serotonin increased basal intracellular calcium in L2 terminals in layer M2 and modestly decreased the duration of visually induced calcium transients in L2 neurons following repeated dark flashes, but otherwise did not alter the calcium transients. Flies without functional 5-HT2B failed to show an increase in basal calcium in response to serotonin. 5-HT2B mutants also failed to show a change in amplitude in their response to repeated light flashes but other calcium transient parameters were relatively unaffected. While we did not detect serotonin receptor expression in L1 neurons, they, like L2, underwent serotonin-induced changes in basal calcium, presumably via interactions with other cells. These data demonstrate that serotonin modulates the physiology of interneurons involved in early visual processing in Drosophila. Serotonergic neurons innervate the Drosophila melanogaster eye, but it was not known whether serotonin signaling could induce acute physiological responses in visual interneurons. We found serotonin receptors expressed in all neuropils of the optic lobe and identified specific neurons involved in visual information processing that express serotonin receptors. Activation of these receptors increased intracellular calcium in first order interneurons L1 and L2 and may enhance visually induced calcium transients in L2 neurons. These data support a role for the serotonergic neuromodulation of interneurons in the Drosophila visual system.
Collapse
|
43
|
Paredes-Valles F, Scheper KYW, de Croon GCHE. Unsupervised Learning of a Hierarchical Spiking Neural Network for Optical Flow Estimation: From Events to Global Motion Perception. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2020; 42:2051-2064. [PMID: 30843817 DOI: 10.1109/tpami.2019.2903179] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
The combination of spiking neural networks and event-based vision sensors holds the potential of highly efficient and high-bandwidth optical flow estimation. This paper presents the first hierarchical spiking architecture in which motion (direction and speed) selectivity emerges in an unsupervised fashion from the raw stimuli generated with an event-based camera. A novel adaptive neuron model and stable spike-timing-dependent plasticity formulation are at the core of this neural network governing its spike-based processing and learning, respectively. After convergence, the neural architecture exhibits the main properties of biological visual motion systems, namely feature extraction and local and global motion perception. Convolutional layers with input synapses characterized by single and multiple transmission delays are employed for feature and local motion perception, respectively; while global motion selectivity emerges in a final fully-connected layer. The proposed solution is validated using synthetic and real event sequences. Along with this paper, we provide the cuSNN library, a framework that enables GPU-accelerated simulations of large-scale spiking neural networks. Source code and samples are available at https://github.com/tudelft/cuSNN.
Collapse
|
44
|
|
45
|
Neuroethology of acoustic communication in field crickets - from signal generation to song recognition in an insect brain. Prog Neurobiol 2020; 194:101882. [PMID: 32673695 DOI: 10.1016/j.pneurobio.2020.101882] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2020] [Revised: 06/25/2020] [Accepted: 07/05/2020] [Indexed: 11/22/2022]
Abstract
Field crickets are best known for the loud calling songs produced by males to attract conspecific females. This review aims to summarize the current knowledge of the neurobiological basis underlying the acoustic communication for mate finding in field crickets with emphasis on the recent research progress to understand the neuronal networks for motor pattern generation and auditory pattern recognition of the calling song in Gryllus bimaculatus. Strong scientific interest into the neural mechanisms underlying intraspecific communication has driven persistently advancing research efforts to study the male singing behaviour and female phonotaxis for mate finding in these insects. The growing neurobiological understanding also inspired many studies testing verifiable hypotheses in sensory ecology, bioacoustics and on the genetics and evolution of behaviour. Over last decades, acoustic communication in field crickets served as a very successful neuroethological model system. It has contributed significantly to the scientific process of establishing, reconsidering and refining fundamental concepts in behavioural neurosciences such as command neurons, central motor pattern generation, corollary discharge processing and pattern recognition by sensory feature detection, which are basic building blocks of our modern understanding on how nervous systems control and generate behaviour in all animals.
Collapse
|
46
|
Palavalli-Nettimi R, Theobald J. Insect Neurobiology: How a Small Spot Stops a Fly. Curr Biol 2020; 30:R761-R763. [PMID: 32634415 DOI: 10.1016/j.cub.2020.05.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
Animals often respond to small moving features very differently than they do to large moving fields. A new study finds that viewing small spots causes walking fruit flies to stop in their tracks, and identifies the cellular pathway that processes this signal.
Collapse
Affiliation(s)
| | - Jamie Theobald
- Florida International University, Department of Biological Sciences, Miami, FL 33199, USA.
| |
Collapse
|
47
|
Städele C, Keleş MF, Mongeau JM, Frye MA. Non-canonical Receptive Field Properties and Neuromodulation of Feature-Detecting Neurons in Flies. Curr Biol 2020; 30:2508-2519.e6. [PMID: 32442460 PMCID: PMC7343589 DOI: 10.1016/j.cub.2020.04.069] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2020] [Revised: 03/10/2020] [Accepted: 04/24/2020] [Indexed: 10/24/2022]
Abstract
Several fundamental aspects of motion vision circuitry are prevalent across flies and mice. Both taxa segregate ON and OFF signals. For any given spatial pattern, motion detectors in both taxa are tuned to speed, selective for one of four cardinal directions, and modulated by catecholamine neurotransmitters. These similarities represent conserved, canonical properties of the functional circuits and computational algorithms for motion vision. Less is known about feature detectors, including how receptive field properties differ from the motion pathway or whether they are under neuromodulatory control to impart functional plasticity for the detection of salient objects from a moving background. Here, we investigated 19 types of putative feature selective lobula columnar (LC) neurons in the optic lobe of the fruit fly Drosophila melanogaster to characterize divergent properties of feature selection. We identified LC12 and LC15 as feature detectors. LC15 encodes moving bars, whereas LC12 is selective for the motion of discrete objects, mostly independent of size. Neither is selective for contrast polarity, speed, or direction, highlighting key differences in the underlying algorithms for feature detection and motion vision. We show that the onset of background motion suppresses object responses by LC12 and LC15. Surprisingly, the application of octopamine, which is released during flight, reverses the suppressive influence of background motion, rendering both LCs able to track moving objects superimposed against background motion. Our results provide a comparative framework for the function and modulation of feature detectors and new insights into the underlying neuronal mechanisms involved in visual feature detection.
Collapse
Affiliation(s)
- Carola Städele
- Department of Integrative Biology and Physiology, University of California, Los Angeles, Los Angeles, CA 90095-7239, USA
| | - Mehmet F Keleş
- Department of Integrative Biology and Physiology, University of California, Los Angeles, Los Angeles, CA 90095-7239, USA
| | - Jean-Michel Mongeau
- Department of Integrative Biology and Physiology, University of California, Los Angeles, Los Angeles, CA 90095-7239, USA
| | - Mark A Frye
- Department of Integrative Biology and Physiology, University of California, Los Angeles, Los Angeles, CA 90095-7239, USA.
| |
Collapse
|
48
|
Schuetzenberger A, Borst A. Seeing Natural Images through the Eye of a Fly with Remote Focusing Two-Photon Microscopy. iScience 2020; 23:101170. [PMID: 32502966 PMCID: PMC7270611 DOI: 10.1016/j.isci.2020.101170] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2020] [Revised: 04/02/2020] [Accepted: 05/12/2020] [Indexed: 11/30/2022] Open
Abstract
Visual systems of many animals, including the fruit fly Drosophila, represent the surrounding space as 2D maps, formed by populations of neurons. Advanced genetic tools make the fly visual system especially well accessible. However, in typical in vivo preparations for two-photon calcium imaging, relatively few neurons can be recorded at the same time. Here, we present an extension to a conventional two-photon microscope, based on remote focusing, which enables real-time rotation of the imaging plane, and thus flexible alignment to cellular structures, without resolution or speed trade-off. We simultaneously record from over 100 neighboring cells spanning the 2D retinotopic map. We characterize its representation of moving natural images, which we find is comparable to noise predictions. Our method increases throughput 10-fold and allows us to visualize a significant fraction of the fly's visual field. Furthermore, our system can be applied in general for a more flexible investigation of neural circuits.
Collapse
Affiliation(s)
- Anna Schuetzenberger
- Department Circuits - Computation - Models, Max-Planck-Institute of Neurobiology, 82152 Planegg, Germany; Graduate School of Systemic Neurosciences, Ludwig-Maximilians-Universität, 82152 Planegg, Germany.
| | - Alexander Borst
- Department Circuits - Computation - Models, Max-Planck-Institute of Neurobiology, 82152 Planegg, Germany; Graduate School of Systemic Neurosciences, Ludwig-Maximilians-Universität, 82152 Planegg, Germany.
| |
Collapse
|
49
|
Yuan D, Ji X, Hao S, Gestrich JY, Duan W, Wang X, Xiang Y, Yang J, Hu P, Xu M, Liu L, Wei H. Lamina feedback neurons regulate the bandpass property of the flicker-induced orientation response in Drosophila. J Neurochem 2020; 156:59-75. [PMID: 32383496 DOI: 10.1111/jnc.15036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2019] [Revised: 04/28/2020] [Accepted: 04/30/2020] [Indexed: 11/28/2022]
Abstract
Natural scenes contain complex visual cues with specific features, including color, motion, flicker, and position. It is critical to understand how different visual features are processed at the early stages of visual perception to elicit appropriate cellular responses, and even behavioral output. Here, we studied the visual orientation response induced by flickering stripes in a novel behavioral paradigm in Drosophila melanogaster. We found that free walking flies exhibited bandpass orientation response to flickering stripes of different frequencies. The most sensitive frequency spectrum was confined to low frequencies of 2-4 Hz. Through genetic silencing, we showed that lamina L1 and L2 neurons, which receive visual inputs from R1 to R6 neurons, were the main components in mediating flicker-induced orientation behavior. Moreover, specific blocking of different types of lamina feedback neurons Lawf1, Lawf2, C2, C3, and T1 modulated orientation responses to flickering stripes of particular frequencies, suggesting that bandpass orientation response was generated through cooperative modulation of lamina feedback neurons. Furthermore, we found that lamina feedback neurons Lawf1 were glutamatergic. Thermal activation of Lawf1 neurons could suppress neural activities in L1 and L2 neurons, which could be blocked by the glutamate-gated chloride channel inhibitor picrotoxin (PTX). In summary, lamina monopolar neurons L1 and L2 are the primary components in mediating flicker-induced orientation response. Meanwhile, lamina feedback neurons cooperatively modulate the orientation response in a frequency-dependent way, which might be achieved through modulating neural activities of L1 and L2 neurons.
Collapse
Affiliation(s)
- Deliang Yuan
- State Key Laboratory of Brain and Cognitive Science, CAS Center for Excellence in Biomacromolecules, Institute of Biophysics, Chinese Academy of Sciences, Beijing, P. R. China.,College of Life Sciences, University of the Chinese Academy of Sciences, Beijing, P. R. China
| | - Xiaoxiao Ji
- State Key Laboratory of Brain and Cognitive Science, CAS Center for Excellence in Biomacromolecules, Institute of Biophysics, Chinese Academy of Sciences, Beijing, P. R. China.,College of Life Sciences, University of the Chinese Academy of Sciences, Beijing, P. R. China
| | - Shun Hao
- State Key Laboratory of Brain and Cognitive Science, CAS Center for Excellence in Biomacromolecules, Institute of Biophysics, Chinese Academy of Sciences, Beijing, P. R. China.,College of Life Sciences, University of the Chinese Academy of Sciences, Beijing, P. R. China
| | - Julia Yvonne Gestrich
- State Key Laboratory of Brain and Cognitive Science, CAS Center for Excellence in Biomacromolecules, Institute of Biophysics, Chinese Academy of Sciences, Beijing, P. R. China
| | - Wenlan Duan
- State Key Laboratory of Brain and Cognitive Science, CAS Center for Excellence in Biomacromolecules, Institute of Biophysics, Chinese Academy of Sciences, Beijing, P. R. China.,College of Life Sciences, University of the Chinese Academy of Sciences, Beijing, P. R. China
| | - Xinwei Wang
- State Key Laboratory of Brain and Cognitive Science, CAS Center for Excellence in Biomacromolecules, Institute of Biophysics, Chinese Academy of Sciences, Beijing, P. R. China.,College of Life Sciences, University of the Chinese Academy of Sciences, Beijing, P. R. China
| | - Yuanhang Xiang
- State Key Laboratory of Brain and Cognitive Science, CAS Center for Excellence in Biomacromolecules, Institute of Biophysics, Chinese Academy of Sciences, Beijing, P. R. China.,College of Life Sciences, University of the Chinese Academy of Sciences, Beijing, P. R. China
| | - Jihua Yang
- State Key Laboratory of Brain and Cognitive Science, CAS Center for Excellence in Biomacromolecules, Institute of Biophysics, Chinese Academy of Sciences, Beijing, P. R. China.,College of Life Sciences, University of the Chinese Academy of Sciences, Beijing, P. R. China
| | - Pengbo Hu
- State Key Laboratory of Brain and Cognitive Science, CAS Center for Excellence in Biomacromolecules, Institute of Biophysics, Chinese Academy of Sciences, Beijing, P. R. China.,College of Life Sciences, University of the Chinese Academy of Sciences, Beijing, P. R. China
| | - Mengbo Xu
- State Key Laboratory of Brain and Cognitive Science, CAS Center for Excellence in Biomacromolecules, Institute of Biophysics, Chinese Academy of Sciences, Beijing, P. R. China
| | - Li Liu
- State Key Laboratory of Brain and Cognitive Science, CAS Center for Excellence in Biomacromolecules, Institute of Biophysics, Chinese Academy of Sciences, Beijing, P. R. China.,College of Life Sciences, University of the Chinese Academy of Sciences, Beijing, P. R. China.,CAS Key Laboratory of Mental Health, Institute of Psychology, Chinese Academy of Sciences, Beijing, P. R. China
| | - Hongying Wei
- State Key Laboratory of Brain and Cognitive Science, CAS Center for Excellence in Biomacromolecules, Institute of Biophysics, Chinese Academy of Sciences, Beijing, P. R. China.,College of Life Sciences, University of the Chinese Academy of Sciences, Beijing, P. R. China
| |
Collapse
|
50
|
D'Angelo G, Janotte E, Schoepe T, O'Keeffe J, Milde MB, Chicca E, Bartolozzi C. Event-Based Eccentric Motion Detection Exploiting Time Difference Encoding. Front Neurosci 2020; 14:451. [PMID: 32457575 PMCID: PMC7227134 DOI: 10.3389/fnins.2020.00451] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2019] [Accepted: 04/14/2020] [Indexed: 11/13/2022] Open
Abstract
Attentional selectivity tends to follow events considered as interesting stimuli. Indeed, the motion of visual stimuli present in the environment attract our attention and allow us to react and interact with our surroundings. Extracting relevant motion information from the environment presents a challenge with regards to the high information content of the visual input. In this work we propose a novel integration between an eccentric down-sampling of the visual field, taking inspiration from the varying size of receptive fields (RFs) in the mammalian retina, and the Spiking Elementary Motion Detector (sEMD) model. We characterize the system functionality with simulated data and real world data collected with bio-inspired event driven cameras, successfully implementing motion detection along the four cardinal directions and diagonally.
Collapse
Affiliation(s)
- Giulia D'Angelo
- Event Driven Perception for Robotics, Italian Institute of Technology, iCub Facility, Genoa, Italy
| | - Ella Janotte
- Faculty of Technology and Center of Cognitive Interaction Technology (CITEC), Bielefeld University, Bielefeld, Germany
| | - Thorben Schoepe
- Faculty of Technology and Center of Cognitive Interaction Technology (CITEC), Bielefeld University, Bielefeld, Germany
| | - James O'Keeffe
- Biosciences Institute, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Moritz B Milde
- International Centre for Neuromorphic Systems, The MARCS Institute, Western Sydney University, Sydney, NSW, Australia
| | - Elisabetta Chicca
- Faculty of Technology and Center of Cognitive Interaction Technology (CITEC), Bielefeld University, Bielefeld, Germany
| | - Chiara Bartolozzi
- Event Driven Perception for Robotics, Italian Institute of Technology, iCub Facility, Genoa, Italy
| |
Collapse
|