1
|
Wu N, Zhou B, Agrochao M, Clark DA. Broken time reversal symmetry in visual motion detection. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.06.08.598068. [PMID: 38915608 PMCID: PMC11195140 DOI: 10.1101/2024.06.08.598068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/26/2024]
Abstract
Our intuition suggests that when a movie is played in reverse, our perception of motion in the reversed movie will be perfectly inverted compared to the original. This intuition is also reflected in many classical theoretical and practical models of motion detection. However, here we demonstrate that this symmetry of motion perception upon time reversal is often broken in real visual systems. In this work, we designed a set of visual stimuli to investigate how stimulus symmetries affect time reversal symmetry breaking in the fruit fly Drosophila's well-studied optomotor rotation behavior. We discovered a suite of new stimuli with a wide variety of different properties that can lead to broken time reversal symmetries in fly behavioral responses. We then trained neural network models to predict the velocity of scenes with both natural and artificial contrast distributions. Training with naturalistic contrast distributions yielded models that break time reversal symmetry, even when the training data was time reversal symmetric. We show analytically and numerically that the breaking of time reversal symmetry in the model responses can arise from contrast asymmetry in the training data, but can also arise from other features of the contrast distribution. Furthermore, shallower neural network models can exhibit stronger symmetry breaking than deeper ones, suggesting that less flexible neural networks promote some forms of time reversal symmetry breaking. Overall, these results reveal a surprising feature of biological motion detectors and suggest that it could arise from constrained optimization in natural environments.
Collapse
Affiliation(s)
- Nathan Wu
- Yale College, New Haven, CT 06511, USA
| | - Baohua Zhou
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA
| | - Margarida Agrochao
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA
| | - Damon A. Clark
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA
- Department of Physics, Yale University, New Haven, CT 06511, USA
- Department of Neuroscience, Yale University, New Haven, CT 06511, USA
- Quantitative Biology Institute, Yale University, New Haven, CT 06511, USA
- Wu Tsai Institute, Yale University, New Haven, CT 06511, USA
| |
Collapse
|
2
|
Tanaka R, Zhou B, Agrochao M, Badwan BA, Au B, Matos NCB, Clark DA. Neural mechanisms to incorporate visual counterevidence in self-movement estimation. Curr Biol 2023; 33:4960-4979.e7. [PMID: 37918398 PMCID: PMC10848174 DOI: 10.1016/j.cub.2023.10.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2023] [Revised: 10/07/2023] [Accepted: 10/09/2023] [Indexed: 11/04/2023]
Abstract
In selecting appropriate behaviors, animals should weigh sensory evidence both for and against specific beliefs about the world. For instance, animals measure optic flow to estimate and control their own rotation. However, existing models of flow detection can be spuriously triggered by visual motion created by objects moving in the world. Here, we show that stationary patterns on the retina, which constitute evidence against observer rotation, suppress inappropriate stabilizing rotational behavior in the fruit fly Drosophila. In silico experiments show that artificial neural networks (ANNs) that are optimized to distinguish observer movement from external object motion similarly detect stationarity and incorporate negative evidence. Employing neural measurements and genetic manipulations, we identified components of the circuitry for stationary pattern detection, which runs parallel to the fly's local motion and optic-flow detectors. Our results show how the fly brain incorporates negative evidence to improve heading stability, exemplifying how a compact brain exploits geometrical constraints of the visual world.
Collapse
Affiliation(s)
- Ryosuke Tanaka
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA
| | - Baohua Zhou
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA; Department of Statistics and Data Science, Yale University, New Haven, CT 06511, USA
| | - Margarida Agrochao
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA
| | - Bara A Badwan
- School of Engineering and Applied Science, Yale University, New Haven, CT 06511, USA
| | - Braedyn Au
- Department of Physics, Yale University, New Haven, CT 06511, USA
| | - Natalia C B Matos
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA
| | - Damon A Clark
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA; Department of Physics, Yale University, New Haven, CT 06511, USA; Department of Neuroscience, Yale University, New Haven, CT 06511, USA; Wu Tsai Institute, Yale University, New Haven, CT 06511, USA; Quantitative Biology Institute, Yale University, New Haven, CT 06511, USA.
| |
Collapse
|
3
|
Chen J, Gish CM, Fransen JW, Salazar-Gatzimas E, Clark DA, Borghuis BG. Direct comparison reveals algorithmic similarities in fly and mouse visual motion detection. iScience 2023; 26:107928. [PMID: 37810236 PMCID: PMC10550730 DOI: 10.1016/j.isci.2023.107928] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Revised: 08/07/2023] [Accepted: 09/12/2023] [Indexed: 10/10/2023] Open
Abstract
Evolution has equipped vertebrates and invertebrates with neural circuits that selectively encode visual motion. While similarities in the computations performed by these circuits in mouse and fruit fly have been noted, direct experimental comparisons have been lacking. Because molecular mechanisms and neuronal morphology in the two species are distinct, we directly compared motion encoding in these two species at the algorithmic level, using matched stimuli and focusing on a pair of analogous neurons, the mouse ON starburst amacrine cell (ON SAC) and Drosophila T4 neurons. We find that the cells share similar spatiotemporal receptive field structures, sensitivity to spatiotemporal correlations, and tuning to sinusoidal drifting gratings, but differ in their responses to apparent motion stimuli. Both neuron types showed a response to summed sinusoids that deviates from models for motion processing in these cells, underscoring the similarities in their processing and identifying response features that remain to be explained.
Collapse
Affiliation(s)
- Juyue Chen
- Interdepartmental Neurosciences Program, Yale University, New Haven, CT 06511, USA
| | - Caitlin M Gish
- Department of Physics, Yale University, New Haven, CT 06511, USA
| | - James W Fransen
- Department of Anatomical Sciences and Neurobiology, University of Louisville, Louisville, KY 40202, USA
| | | | - Damon A Clark
- Interdepartmental Neurosciences Program, Yale University, New Haven, CT 06511, USA
- Department of Physics, Yale University, New Haven, CT 06511, USA
- Department of Molecular, Cellular, Developmental Biology, Yale University, New Haven, CT 06511, USA
- Department of Neuroscience, Yale University, New Haven, CT 06511, USA
| | - Bart G Borghuis
- Department of Anatomical Sciences and Neurobiology, University of Louisville, Louisville, KY 40202, USA
| |
Collapse
|
4
|
Mano O, Choi M, Tanaka R, Creamer MS, Matos NCB, Shomar JW, Badwan BA, Clandinin TR, Clark DA. Long-timescale anti-directional rotation in Drosophila optomotor behavior. eLife 2023; 12:e86076. [PMID: 37751469 PMCID: PMC10522332 DOI: 10.7554/elife.86076] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2023] [Accepted: 09/12/2023] [Indexed: 09/28/2023] Open
Abstract
Locomotor movements cause visual images to be displaced across the eye, a retinal slip that is counteracted by stabilizing reflexes in many animals. In insects, optomotor turning causes the animal to turn in the direction of rotating visual stimuli, thereby reducing retinal slip and stabilizing trajectories through the world. This behavior has formed the basis for extensive dissections of motion vision. Here, we report that under certain stimulus conditions, two Drosophila species, including the widely studied Drosophila melanogaster, can suppress and even reverse the optomotor turning response over several seconds. Such 'anti-directional turning' is most strongly evoked by long-lasting, high-contrast, slow-moving visual stimuli that are distinct from those that promote syn-directional optomotor turning. Anti-directional turning, like the syn-directional optomotor response, requires the local motion detecting neurons T4 and T5. A subset of lobula plate tangential cells, CH cells, show involvement in these responses. Imaging from a variety of direction-selective cells in the lobula plate shows no evidence of dynamics that match the behavior, suggesting that the observed inversion in turning direction emerges downstream of the lobula plate. Further, anti-directional turning declines with age and exposure to light. These results show that Drosophila optomotor turning behaviors contain rich, stimulus-dependent dynamics that are inconsistent with simple reflexive stabilization responses.
Collapse
Affiliation(s)
- Omer Mano
- Department of Molecular, Cellular, and Developmental Biology, Yale UniversityNew HavenUnited States
| | - Minseung Choi
- Department of Neurobiology, Stanford UniversityStanfordUnited States
| | - Ryosuke Tanaka
- Interdepartmental Neuroscience Program, Yale UniversityNew HavenUnited States
| | - Matthew S Creamer
- Interdepartmental Neuroscience Program, Yale UniversityNew HavenUnited States
| | - Natalia CB Matos
- Interdepartmental Neuroscience Program, Yale UniversityNew HavenUnited States
| | - Joseph W Shomar
- Department of Physics, Yale UniversityNew HavenUnited States
| | - Bara A Badwan
- Department of Chemical Engineering, Yale UniversityNew HavenUnited States
| | | | - Damon A Clark
- Department of Molecular, Cellular, and Developmental Biology, Yale UniversityNew HavenUnited States
- Interdepartmental Neuroscience Program, Yale UniversityNew HavenUnited States
- Department of Physics, Yale UniversityNew HavenUnited States
- Department of Neuroscience, Yale UniversityNew HavenUnited States
| |
Collapse
|
5
|
Tanaka R, Zhou B, Agrochao M, Badwan BA, Au B, Matos NCB, Clark DA. Neural mechanisms to incorporate visual counterevidence in self motion estimation. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.01.04.522814. [PMID: 36711843 PMCID: PMC9881891 DOI: 10.1101/2023.01.04.522814] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Abstract
In selecting appropriate behaviors, animals should weigh sensory evidence both for and against specific beliefs about the world. For instance, animals measure optic flow to estimate and control their own rotation. However, existing models of flow detection can confuse the movement of external objects with genuine self motion. Here, we show that stationary patterns on the retina, which constitute negative evidence against self rotation, are used by the fruit fly Drosophila to suppress inappropriate stabilizing rotational behavior. In silico experiments show that artificial neural networks optimized to distinguish self and world motion similarly detect stationarity and incorporate negative evidence. Employing neural measurements and genetic manipulations, we identified components of the circuitry for stationary pattern detection, which runs parallel to the fly's motion- and optic flow-detectors. Our results exemplify how the compact brain of the fly incorporates negative evidence to improve heading stability, exploiting geometrical constraints of the visual world.
Collapse
Affiliation(s)
- Ryosuke Tanaka
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA
- Present Address: Institute of Neuroscience, Technical University of Munich, Munich 80802, Germany
| | - Baohua Zhou
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA
- Department of Statistics and Data Science, Yale University, New Haven, CT 06511, USA
| | - Margarida Agrochao
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA
| | - Bara A. Badwan
- School of Engineering and Applied Science, Yale University, New Haven, CT 06511, USA
| | - Braedyn Au
- Department of Physics, Yale University, New Haven, CT 06511, USA
| | - Natalia C. B. Matos
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA
| | - Damon A. Clark
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA
- Department of Physics, Yale University, New Haven, CT 06511, USA
- Department of Neuroscience, Yale University, New Haven, CT 06511, USA
- Wu Tsai Institute, Yale University, New Haven, CT 06511, USA
| |
Collapse
|
6
|
Currier TA, Pang MM, Clandinin TR. Visual processing in the fly, from photoreceptors to behavior. Genetics 2023; 224:iyad064. [PMID: 37128740 PMCID: PMC10213501 DOI: 10.1093/genetics/iyad064] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2022] [Accepted: 03/22/2023] [Indexed: 05/03/2023] Open
Abstract
Originally a genetic model organism, the experimental use of Drosophila melanogaster has grown to include quantitative behavioral analyses, sophisticated perturbations of neuronal function, and detailed sensory physiology. A highlight of these developments can be seen in the context of vision, where pioneering studies have uncovered fundamental and generalizable principles of sensory processing. Here we begin with an overview of vision-guided behaviors and common methods for probing visual circuits. We then outline the anatomy and physiology of brain regions involved in visual processing, beginning at the sensory periphery and ending with descending motor control. Areas of focus include contrast and motion detection in the optic lobe, circuits for visual feature selectivity, computations in support of spatial navigation, and contextual associative learning. Finally, we look to the future of fly visual neuroscience and discuss promising topics for further study.
Collapse
Affiliation(s)
- Timothy A Currier
- Department of Neurobiology, Stanford University School of Medicine, Stanford, CA 94305, USA
| | - Michelle M Pang
- Department of Neurobiology, Stanford University School of Medicine, Stanford, CA 94305, USA
| | - Thomas R Clandinin
- Department of Neurobiology, Stanford University School of Medicine, Stanford, CA 94305, USA
| |
Collapse
|
7
|
Mano O, Choi M, Tanaka R, Creamer MS, Matos NC, Shomar J, Badwan BA, Clandinin TR, Clark DA. Long timescale anti-directional rotation in Drosophila optomotor behavior. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.01.06.523055. [PMID: 36711627 PMCID: PMC9882005 DOI: 10.1101/2023.01.06.523055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Abstract
Locomotor movements cause visual images to be displaced across the eye, a retinal slip that is counteracted by stabilizing reflexes in many animals. In insects, optomotor turning causes the animal to turn in the direction of rotating visual stimuli, thereby reducing retinal slip and stabilizing trajectories through the world. This behavior has formed the basis for extensive dissections of motion vision. Here, we report that under certain stimulus conditions, two Drosophila species, including the widely studied D. melanogaster, can suppress and even reverse the optomotor turning response over several seconds. Such "anti-directional turning" is most strongly evoked by long-lasting, high-contrast, slow-moving visual stimuli that are distinct from those that promote syn-directional optomotor turning. Anti-directional turning, like the syn-directional optomotor response, requires the local motion detecting neurons T4 and T5. A subset of lobula plate tangential cells, CH cells, show involvement in these responses. Imaging from a variety of direction-selective cells in the lobula plate shows no evidence of dynamics that match the behavior, suggesting that the observed inversion in turning direction emerges downstream of the lobula plate. Further, anti-directional turning declines with age and exposure to light. These results show that Drosophila optomotor turning behaviors contain rich, stimulus-dependent dynamics that are inconsistent with simple reflexive stabilization responses.
Collapse
Affiliation(s)
- Omer Mano
- Department of Molecular, Cellular, and Developmental Biology, Yale University, New Haven, CT 06511, USA
| | - Minseung Choi
- Department of Neurobiology, Stanford University, Stanford, CA 94305, USA
| | - Ryosuke Tanaka
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA
| | - Matthew S. Creamer
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA
| | - Natalia C.B. Matos
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA
| | - Joseph Shomar
- Department of Physics, Yale University, New Haven, CT 06511, USA
| | - Bara A. Badwan
- Department of Chemical Engineering, Yale University, New Haven, CT 06511, USA
| | | | - Damon A. Clark
- Department of Molecular, Cellular, and Developmental Biology, Yale University, New Haven, CT 06511, USA
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA
- Department of Physics, Yale University, New Haven, CT 06511, USA
- Department of Neuroscience, Yale University, New Haven, CT 06511, USA
| |
Collapse
|
8
|
Kirubeswaran OR, Storrs KR. Inconsistent illusory motion in predictive coding deep neural networks. Vision Res 2023; 206:108195. [PMID: 36801664 DOI: 10.1016/j.visres.2023.108195] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2022] [Revised: 01/31/2023] [Accepted: 01/31/2023] [Indexed: 02/19/2023]
Abstract
Why do we perceive illusory motion in some static images? Several accounts point to eye movements, response latencies to different image elements, or interactions between image patterns and motion energy detectors. Recently PredNet, a recurrent deep neural network (DNN) based on predictive coding principles, was reported to reproduce the "Rotating Snakes" illusion, suggesting a role for predictive coding. We begin by replicating this finding, then use a series of "in silico" psychophysics and electrophysiology experiments to examine whether PredNet behaves consistently with human observers and non-human primate neural data. A pretrained PredNet predicted illusory motion for all subcomponents of the Rotating Snakes pattern, consistent with human observers. However, we found no simple response delays in internal units, unlike evidence from electrophysiological data. PredNet's detection of motion in gradients seemed dependent on contrast, but depends predominantly on luminance in humans. Finally, we examined the robustness of the illusion across ten PredNets of identical architecture, retrained on the same video data. There was large variation across network instances in whether they reproduced the Rotating Snakes illusion, and what motion, if any, they predicted for simplified variants. Unlike human observers, no network predicted motion for greyscale variants of the Rotating Snakes pattern. Our results sound a cautionary note: even when a DNN successfully reproduces some idiosyncrasy of human vision, more detailed investigation can reveal inconsistencies between humans and the network, and between different instances of the same network. These inconsistencies suggest that predictive coding does not reliably give rise to human-like illusory motion.
Collapse
Affiliation(s)
| | - Katherine R Storrs
- Department of Experimental Psychology, Justus Liebig University Giessen, Germany; Centre for Mind, Brain and Behaviour (CMBB), University of Marburg and Justus Liebig University Giessen, Germany; School of Psychology, University of Auckland, New Zealand
| |
Collapse
|
9
|
Seeing Things: A Community Science Investigation into Motion Illusion Susceptibility in Domestic Cats ( Felis silvestris catus) and Dogs ( Canis lupus familiaris). Animals (Basel) 2022; 12:ani12243562. [PMID: 36552482 PMCID: PMC9774501 DOI: 10.3390/ani12243562] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Revised: 11/25/2022] [Accepted: 12/14/2022] [Indexed: 12/23/2022] Open
Abstract
Illusions-visual fields that distort perception-can inform the understanding of visual perception and its evolution. An example of one such illusion, the Rotating Snakes illusion, causes the perception of motion in a series of static concentric circles. The current study investigated pet dogs' and cats' perception of the Rotating Snakes illusion in a community science paradigm. The results reveal that neither species spent significantly more time at the illusion than at either of the controls, failing to indicate susceptibility to the illusion. Specific behavioral data at each stimulus reveal that the most common behaviors of both species were Inactive and Stationary, while Locomotion and Pawing were the least common, supporting the finding that susceptibility may not be present. This study is the first to examine susceptibility to the Rotating Snakes illusion in dogs, as well as to directly compare the phenomenon between dogs and cats. We suggest future studies might consider exploring alternative methods in testing susceptibility to motion illusions in non-human animals.
Collapse
|
10
|
Gonzalez-Suarez AD, Zavatone-Veth JA, Chen J, Matulis CA, Badwan BA, Clark DA. Excitatory and inhibitory neural dynamics jointly tune motion detection. Curr Biol 2022; 32:3659-3675.e8. [PMID: 35868321 PMCID: PMC9474608 DOI: 10.1016/j.cub.2022.06.075] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2022] [Revised: 05/03/2022] [Accepted: 06/24/2022] [Indexed: 11/26/2022]
Abstract
Neurons integrate excitatory and inhibitory signals to produce their outputs, but the role of input timing in this integration remains poorly understood. Motion detection is a paradigmatic example of this integration, since theories of motion detection rely on different delays in visual signals. These delays allow circuits to compare scenes at different times to calculate the direction and speed of motion. Different motion detection circuits have different velocity sensitivity, but it remains untested how the response dynamics of individual cell types drive this tuning. Here, we sped up or slowed down specific neuron types in Drosophila's motion detection circuit by manipulating ion channel expression. Altering the dynamics of individual neuron types upstream of motion detectors increased their sensitivity to fast or slow visual motion, exposing distinct roles for excitatory and inhibitory dynamics in tuning directional signals, including a role for the amacrine cell CT1. A circuit model constrained by functional data and anatomy qualitatively reproduced the observed tuning changes. Overall, these results reveal how excitatory and inhibitory dynamics together tune a canonical circuit computation.
Collapse
Affiliation(s)
| | - Jacob A Zavatone-Veth
- Department of Physics, Harvard University, Cambridge, MA 02138, USA; Center for Brain Science, Harvard University, Cambridge, MA 02138, USA
| | - Juyue Chen
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA
| | | | - Bara A Badwan
- School of Engineering and Applied Science, Yale University, New Haven, CT 06511, USA
| | - Damon A Clark
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA; Department of Physics, Yale University, New Haven, CT 06511, USA; Department of Molecular, Cellular, and Developmental Biology, Yale University, New Haven, CT 06511, USA; Department of Neuroscience, Yale University, New Haven, CT 06511, USA.
| |
Collapse
|
11
|
Gatto E, Loukola OJ, Petrazzini MEM, Agrillo C, Cutini S. Illusional Perspective across Humans and Bees. Vision (Basel) 2022; 6:vision6020028. [PMID: 35737416 PMCID: PMC9231007 DOI: 10.3390/vision6020028] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Revised: 05/20/2022] [Accepted: 05/26/2022] [Indexed: 11/16/2022] Open
Abstract
For two centuries, visual illusions have attracted the attention of neurobiologists and comparative psychologists, given the possibility of investigating the complexity of perceptual mechanisms by using relatively simple patterns. Animal models, such as primates, birds, and fish, have played a crucial role in understanding the physiological circuits involved in the susceptibility of visual illusions. However, the comprehension of such mechanisms is still a matter of debate. Despite their different neural architectures, recent studies have shown that some arthropods, primarily Hymenoptera and Diptera, experience illusions similar to those humans do, suggesting that perceptual mechanisms are evolutionarily conserved among species. Here, we review the current state of illusory perception in bees. First, we introduce bees’ visual system and speculate which areas might make them susceptible to illusory scenes. Second, we review the current state of knowledge on misperception in bees (Apidae), focusing on the visual stimuli used in the literature. Finally, we discuss important aspects to be considered before claiming that a species shows higher cognitive ability while equally supporting alternative hypotheses. This growing evidence provides insights into the evolutionary origin of visual mechanisms across species.
Collapse
Affiliation(s)
- Elia Gatto
- Department of Chemical, Pharmaceutical and Agricultural Sciences, University of Ferrara, 44121 Ferrara, Italy
- Department of Life Sciences and Biotechnology, University of Ferrara, 44121 Ferrara, Italy
- Correspondence:
| | - Olli J. Loukola
- Ecology and Genetics Research Unit, University of Oulu, P.O. Box 3000, FI-90014 Oulu, Finland;
| | | | - Christian Agrillo
- Department of General Psychology, University of Padova, 35131 Padova, Italy; (M.E.M.P.); (C.A.)
- Department of Developmental and Social Psychology, University of Padova, 35131 Padova, Italy;
| | - Simone Cutini
- Department of Developmental and Social Psychology, University of Padova, 35131 Padova, Italy;
- Padua Neuroscience Center, University of Padova, 35129 Padova, Italy
| |
Collapse
|
12
|
Kobayashi T, Kitaoka A, Kosaka M, Tanaka K, Watanabe E. Motion illusion-like patterns extracted from photo and art images using predictive deep neural networks. Sci Rep 2022; 12:3893. [PMID: 35273206 PMCID: PMC8913633 DOI: 10.1038/s41598-022-07438-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2021] [Accepted: 02/18/2022] [Indexed: 11/09/2022] Open
Abstract
In our previous study, we successfully reproduced the illusory motion perceived in the rotating snakes illusion using deep neural networks incorporating predictive coding theory. In the present study, we further examined the properties of the network using a set of 1500 images, including ordinary static images of paintings and photographs and images of various types of motion illusions. Results showed that the networks clearly classified a group of illusory images and others and reproduced illusory motions against various types of illusions similar to human perception. Notably, the networks occasionally detected anomalous motion vectors, even in ordinally static images where humans were unable to perceive any illusory motion. Additionally, illusion-like designs with repeating patterns were generated using areas where anomalous vectors were detected, and psychophysical experiments were conducted, in which illusory motion perception in the generated designs was detected. The observed inaccuracy of the networks will provide useful information for further understanding information processing associated with human vision.
Collapse
Affiliation(s)
- Taisuke Kobayashi
- Laboratory of Neurophysiology, National Institute for Basic Biology, Higashiyama 5-1, Myodaiji-cho, Okazaki, Aichi, 444-8787, Japan.
| | - Akiyoshi Kitaoka
- College of Comprehensive Psychology, Ritsumeikan University, Iwakura-cho 2-150, Ibaraki, Osaka, 567-8570, Japan
| | - Manabu Kosaka
- Code_monsters group, Laboratory of Neurophysiology, National Institute for Basic Biology, Higashiyama 5-1, Myodaiji-cho, Okazaki, Aichi, 444-8787, Japan
| | - Kenta Tanaka
- Code_monsters group, Laboratory of Neurophysiology, National Institute for Basic Biology, Higashiyama 5-1, Myodaiji-cho, Okazaki, Aichi, 444-8787, Japan
| | - Eiji Watanabe
- Laboratory of Neurophysiology, National Institute for Basic Biology, Higashiyama 5-1, Myodaiji-cho, Okazaki, Aichi, 444-8787, Japan. .,Department of Basic Biology, The Graduate University for Advanced Studies (SOKENDAI), Miura, Kanagawa, 240-0193, Japan.
| |
Collapse
|
13
|
Identifying Inputs to Visual Projection Neurons in Drosophila Lobula by Analyzing Connectomic Data. eNeuro 2022; 9:ENEURO.0053-22.2022. [PMID: 35410869 PMCID: PMC9034759 DOI: 10.1523/eneuro.0053-22.2022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2022] [Revised: 03/26/2022] [Accepted: 03/30/2022] [Indexed: 11/21/2022] Open
Abstract
Electron microscopy (EM)-based connectomes provide important insights into how visual circuitry of fruit fly Drosophila computes various visual features, guiding and complementing behavioral and physiological studies. However, connectomic analyses of the lobula, a neuropil putatively dedicated to detecting object-like features, remains underdeveloped, largely because of incomplete data on the inputs to the brain region. Here, we attempted to map the columnar inputs into the Drosophila lobula neuropil by performing connectivity-based and morphology-based clustering on a densely reconstructed connectome dataset. While the dataset mostly lacked visual neuropils other than lobula, which would normally help identify inputs to lobula, our clustering analysis successfully extracted clusters of cells with homogeneous connectivity and morphology, likely representing genuine cell types. We were able to draw a correspondence between the resulting clusters and previously identified cell types, revealing previously undocumented connectivity between lobula input and output neurons. While future, more complete connectomic reconstructions are necessary to verify the results presented here, they can serve as a useful basis for formulating hypotheses on mechanisms of visual feature detection in lobula.
Collapse
|
14
|
Zhou B, Li Z, Kim S, Lafferty J, Clark DA. Shallow neural networks trained to detect collisions recover features of visual loom-selective neurons. eLife 2022; 11:72067. [PMID: 35023828 PMCID: PMC8849349 DOI: 10.7554/elife.72067] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2021] [Accepted: 01/11/2022] [Indexed: 11/13/2022] Open
Abstract
Animals have evolved sophisticated visual circuits to solve a vital inference problem: detecting whether or not a visual signal corresponds to an object on a collision course. Such events are detected by specific circuits sensitive to visual looming, or objects increasing in size. Various computational models have been developed for these circuits, but how the collision-detection inference problem itself shapes the computational structures of these circuits remains unknown. Here, inspired by the distinctive structures of LPLC2 neurons in the visual system of Drosophila, we build anatomically-constrained shallow neural network models and train them to identify visual signals that correspond to impending collisions. Surprisingly, the optimization arrives at two distinct, opposing solutions, only one of which matches the actual dendritic weighting of LPLC2 neurons. Both solutions can solve the inference problem with high accuracy when the population size is large enough. The LPLC2-like solutions reproduces experimentally observed LPLC2 neuron responses for many stimuli, and reproduces canonical tuning of loom sensitive neurons, even though the models are never trained on neural data. Thus, LPLC2 neuron properties and tuning are predicted by optimizing an anatomically-constrained neural network to detect impending collisions. More generally, these results illustrate how optimizing inference tasks that are important for an animal's perceptual goals can reveal and explain computational properties of specific sensory neurons.
Collapse
Affiliation(s)
- Baohua Zhou
- Department of Molecular, Cellular and Developmental Biology, Yale University, New Haven, United States
| | - Zifan Li
- Department of Statistics and Data Science, Yale University, New Haven, United States
| | - Sunnie Kim
- Department of Statistics and Data Science, Yale University, New Haven, United States
| | - John Lafferty
- Department of Statistics and Data Science, Yale University, New Haven, United States
| | - Damon A Clark
- Department of Molecular, Cellular and Developmental Biology, Yale University, New Haven, United States
| |
Collapse
|
15
|
Gruntman E, Reimers P, Romani S, Reiser MB. Non-preferred contrast responses in the Drosophila motion pathways reveal a receptive field structure that explains a common visual illusion. Curr Biol 2021; 31:5286-5298.e7. [PMID: 34672960 DOI: 10.1016/j.cub.2021.09.072] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2021] [Revised: 09/14/2021] [Accepted: 09/24/2021] [Indexed: 10/20/2022]
Abstract
Diverse sensory systems, from audition to thermosensation, feature a separation of inputs into ON (increments) and OFF (decrements) signals. In the Drosophila visual system, separate ON and OFF pathways compute the direction of motion, yet anatomical and functional studies have identified some crosstalk between these channels. We used this well-studied circuit to ask whether the motion computation depends on ON-OFF pathway crosstalk. Using whole-cell electrophysiology, we recorded visual responses of T4 (ON) and T5 (OFF) cells, mapped their composite ON-OFF receptive fields, and found that they share a similar spatiotemporal structure. We fit a biophysical model to these receptive fields that accurately predicts directionally selective T4 and T5 responses to both ON and OFF moving stimuli. This model also provides a detailed mechanistic explanation for the directional preference inversion in response to the prominent reverse-phi illusion. Finally, we used the steering responses of tethered flying flies to validate the model's predicted effects of varying stimulus parameters on the behavioral turning inversion.
Collapse
Affiliation(s)
- Eyal Gruntman
- Janelia Research Campus, Howard Hughes Medical Institute, 19700 Helix Drive, Ashburn, VA, USA.
| | - Pablo Reimers
- Janelia Research Campus, Howard Hughes Medical Institute, 19700 Helix Drive, Ashburn, VA, USA
| | - Sandro Romani
- Janelia Research Campus, Howard Hughes Medical Institute, 19700 Helix Drive, Ashburn, VA, USA
| | - Michael B Reiser
- Janelia Research Campus, Howard Hughes Medical Institute, 19700 Helix Drive, Ashburn, VA, USA.
| |
Collapse
|
16
|
Mano O, Creamer MS, Badwan BA, Clark DA. Predicting individual neuron responses with anatomically constrained task optimization. Curr Biol 2021; 31:4062-4075.e4. [PMID: 34324832 DOI: 10.1016/j.cub.2021.06.090] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2021] [Revised: 05/24/2021] [Accepted: 06/29/2021] [Indexed: 01/28/2023]
Abstract
Artificial neural networks trained to solve sensory tasks can develop statistical representations that match those in biological circuits. However, it remains unclear whether they can reproduce properties of individual neurons. Here, we investigated how artificial networks predict individual neuron properties in the visual motion circuits of the fruit fly Drosophila. We trained anatomically constrained networks to predict movement in natural scenes, solving the same inference problem as fly motion detectors. Units in the artificial networks adopted many properties of analogous individual neurons, even though they were not explicitly trained to match these properties. Among these properties was the split into ON and OFF motion detectors, which is not predicted by classical motion detection models. The match between model and neurons was closest when models were trained to be robust to noise. These results demonstrate how anatomical, task, and noise constraints can explain properties of individual neurons in a small neural network.
Collapse
Affiliation(s)
- Omer Mano
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA; Department of Neuroscience, Yale University, New Haven, CT 06511, USA
| | - Matthew S Creamer
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA
| | - Bara A Badwan
- School of Engineering and Applied Science, Yale University, New Haven, CT 06511, USA
| | - Damon A Clark
- Department of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511, USA; Department of Neuroscience, Yale University, New Haven, CT 06511, USA; Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06511, USA; Department of Physics, Yale University, New Haven, CT 06511, USA.
| |
Collapse
|
17
|
Liu X, Li H, Wang Y, Lei T, Wang J, Spillmann L, Andolina IM, Wang W. From Receptive to Perceptive Fields: Size-Dependent Asymmetries in Both Negative Afterimages and Subcortical On and Off Post-Stimulus Responses. J Neurosci 2021; 41:7813-7830. [PMID: 34326144 PMCID: PMC8445057 DOI: 10.1523/jneurosci.0300-21.2021] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2021] [Revised: 07/11/2021] [Accepted: 07/13/2021] [Indexed: 11/21/2022] Open
Abstract
Negative afterimages are perceptual phenomena that occur after physical stimuli disappear from sight. Their origin is linked to transient post-stimulus responses of visual neurons. The receptive fields (RFs) of these subcortical ON- and OFF-center neurons exhibit antagonistic interactions between central and surrounding visual space, resulting in selectivity for stimulus polarity and size. These two features are closely intertwined, yet their relationship to negative afterimage perception remains unknown. Here we tested whether size differentially affects the perception of bright and dark negative afterimages in humans of both sexes, and how this correlates with neural mechanisms in subcortical ON and OFF cells. Psychophysically, we found a size-dependent asymmetry whereby dark disks produce stronger and longer-lasting negative afterimages than bright disks of equal contrast at sizes >0.8°. Neurophysiological recordings from retinal and relay cells in female cat dorsal lateral geniculate nucleus showed that subcortical ON cells exhibited stronger sustained post-stimulus responses to dark disks, than OFF cells to bright disks, at sizes >1°. These sizes agree with the emergence of center-surround antagonism, revealing stronger suppression to opposite-polarity stimuli for OFF versus ON cells, particularly in dorsal lateral geniculate nucleus. Using a network-based retino-geniculate model, we confirmed stronger antagonism and temporal transience for OFF-cell post-stimulus rebound responses. A V1 population model demonstrated that both strength and duration asymmetries can be propagated to downstream cortical areas. Our results demonstrate how size-dependent antagonism impacts both the neuronal post-stimulus response and the resulting afterimage percepts, thereby supporting the idea of perceptual RFs reflecting the underlying neuronal RF organization of single cells.SIGNIFICANCE STATEMENT Visual illusions occur when sensory inputs and perceptual outcomes do not match, and provide a valuable tool to understand transformations from neural to perceptual responses. A classic example are negative afterimages that remain visible after a stimulus is removed from view. Such perceptions are linked to responses in early visual neurons, yet the details remain poorly understood. Combining human psychophysics, neurophysiological recordings in cats and retino-thalamo-cortical computational modeling, our study reveals how stimulus size and the receptive-field structure of subcortical ON and OFF cells contributes to the parallel asymmetries between neural and perceptual responses to bright versus dark afterimages. Thus, this work provides a deeper link from the underlying neural mechanisms to the resultant perceptual outcomes.
Collapse
Affiliation(s)
- Xu Liu
- Institute of Neuroscience, Center for Excellence in Brain Science and Intelligence Technology, State Key Laboratory of Neuroscience, Key Laboratory of Primate Neurobiology, Chinese Academy of Sciences, Shanghai, 200031, China
- University of Chinese Academy of Sciences, Beijing, 100049, China
| | - Hui Li
- Institute of Neuroscience, Center for Excellence in Brain Science and Intelligence Technology, State Key Laboratory of Neuroscience, Key Laboratory of Primate Neurobiology, Chinese Academy of Sciences, Shanghai, 200031, China
| | - Ye Wang
- State Key Laboratory of Media Convergence and Communication, Neuroscience and Intelligent Media Institute, Communication University of China, Beijing, 100024, China
| | - Tianhao Lei
- Institute of Neuroscience, Center for Excellence in Brain Science and Intelligence Technology, State Key Laboratory of Neuroscience, Key Laboratory of Primate Neurobiology, Chinese Academy of Sciences, Shanghai, 200031, China
| | - Jijun Wang
- Shanghai Mental Health Center, Shanghai Jiaotong University School of Medicine, Shanghai Key Laboratory of Psychotic Disorders, Shanghai, 200030, China
| | - Lothar Spillmann
- Department of Neurology, University of Freiburg, Freiburg, 79085, Germany
| | - Ian Max Andolina
- Institute of Neuroscience, Center for Excellence in Brain Science and Intelligence Technology, State Key Laboratory of Neuroscience, Key Laboratory of Primate Neurobiology, Chinese Academy of Sciences, Shanghai, 200031, China
- Shanghai Center for Brain and Brain-inspired Intelligence Technology, Shanghai, 200031, China
| | - Wei Wang
- Institute of Neuroscience, Center for Excellence in Brain Science and Intelligence Technology, State Key Laboratory of Neuroscience, Key Laboratory of Primate Neurobiology, Chinese Academy of Sciences, Shanghai, 200031, China
- Shanghai Center for Brain and Brain-inspired Intelligence Technology, Shanghai, 200031, China
- University of Chinese Academy of Sciences, Beijing, 100049, China
| |
Collapse
|
18
|
Ramos-Traslosheros G, Silies M. The physiological basis for contrast opponency in motion computation in Drosophila. Nat Commun 2021; 12:4987. [PMID: 34404776 PMCID: PMC8371135 DOI: 10.1038/s41467-021-24986-w] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2020] [Accepted: 07/07/2021] [Indexed: 12/02/2022] Open
Abstract
In Drosophila, direction-selective neurons implement a mechanism of motion computation similar to cortical neurons, using contrast-opponent receptive fields with ON and OFF subfields. It is not clear how the presynaptic circuitry of direction-selective neurons in the OFF pathway supports this computation if all major inputs are OFF-rectified neurons. Here, we reveal the biological substrate for motion computation in the OFF pathway. Three interneurons, Tm2, Tm9 and CT1, provide information about ON stimuli to the OFF direction-selective neuron T5 across its receptive field, supporting a contrast-opponent receptive field organization. Consistent with its prominent role in motion detection, variability in Tm9 receptive field properties transfers to T5, and calcium decrements in Tm9 in response to ON stimuli persist across behavioral states, while spatial tuning is sharpened by active behavior. Together, our work shows how a key neuronal computation is implemented by its constituent neuronal circuit elements to ensure direction selectivity.
Collapse
Affiliation(s)
- Giordano Ramos-Traslosheros
- Institute of Developmental Biology and Neurobiology, Johannes-Gutenberg University Mainz, Mainz, Germany
- International Max Planck Research School Neuroscienes and Göttingen Graduate School for Neurosciences, Biophysics, and Molecular Biosciences (GGNB) at the University of Göttingen, Göttingen, Germany
| | - Marion Silies
- Institute of Developmental Biology and Neurobiology, Johannes-Gutenberg University Mainz, Mainz, Germany.
| |
Collapse
|
19
|
Ding J, Chen A, Chung J, Acaron Ledesma H, Wu M, Berson DM, Palmer SE, Wei W. Spatially displaced excitation contributes to the encoding of interrupted motion by a retinal direction-selective circuit. eLife 2021; 10:e68181. [PMID: 34096504 PMCID: PMC8211448 DOI: 10.7554/elife.68181] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2021] [Accepted: 06/06/2021] [Indexed: 12/19/2022] Open
Abstract
Spatially distributed excitation and inhibition collectively shape a visual neuron's receptive field (RF) properties. In the direction-selective circuit of the mammalian retina, the role of strong null-direction inhibition of On-Off direction-selective ganglion cells (On-Off DSGCs) on their direction selectivity is well-studied. However, how excitatory inputs influence the On-Off DSGC's visual response is underexplored. Here, we report that On-Off DSGCs have a spatially displaced glutamatergic receptive field along their horizontal preferred-null motion axes. This displaced receptive field contributes to DSGC null-direction spiking during interrupted motion trajectories. Theoretical analyses indicate that population responses during interrupted motion may help populations of On-Off DSGCs signal the spatial location of moving objects in complex, naturalistic visual environments. Our study highlights that the direction-selective circuit exploits separate sets of mechanisms under different stimulus conditions, and these mechanisms may help encode multiple visual features.
Collapse
Affiliation(s)
- Jennifer Ding
- Committee on Neurobiology Graduate Program, The University of ChicagoChicagoUnited States
- Department of Neurobiology, The University of ChicagoChicagoUnited States
| | - Albert Chen
- Department of Organismal Biology, The University of ChicagoChicagoUnited States
| | - Janet Chung
- Department of Neurobiology, The University of ChicagoChicagoUnited States
| | - Hector Acaron Ledesma
- Graduate Program in Biophysical Sciences, The University of ChicagoChicagoUnited States
| | - Mofei Wu
- Department of Neurobiology, The University of ChicagoChicagoUnited States
| | - David M Berson
- Department of Neuroscience and Carney Institute for Brain Science, Brown UniversityProvidenceUnited States
| | - Stephanie E Palmer
- Committee on Neurobiology Graduate Program, The University of ChicagoChicagoUnited States
- Department of Organismal Biology, The University of ChicagoChicagoUnited States
- Grossman Institute for Neuroscience, Quantitative Biology and Human Behavior, The University of ChicagoChicagoUnited States
| | - Wei Wei
- Committee on Neurobiology Graduate Program, The University of ChicagoChicagoUnited States
- Department of Neurobiology, The University of ChicagoChicagoUnited States
- Grossman Institute for Neuroscience, Quantitative Biology and Human Behavior, The University of ChicagoChicagoUnited States
| |
Collapse
|