1
|
Ogawa Y, Nicholas S, Thyselius M, Leibbrandt R, Nowotny T, Knight JC, Nordström K. Descending neurons of the hoverfly respond to pursuits of artificial targets. Curr Biol 2023; 33:4392-4404.e5. [PMID: 37776861 DOI: 10.1016/j.cub.2023.08.091] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Revised: 07/11/2023] [Accepted: 08/31/2023] [Indexed: 10/02/2023]
Abstract
Many animals use motion vision information to control dynamic behaviors. Predatory animals, for example, show an exquisite ability to detect rapidly moving prey, followed by pursuit and capture. Such target detection is not only used by predators but is also important in conspecific interactions, such as for male hoverflies defending their territories against conspecific intruders. Visual target detection is believed to be subserved by specialized target-tuned neurons found in a range of species, including vertebrates and arthropods. However, how these target-tuned neurons respond to actual pursuit trajectories is currently not well understood. To redress this, we recorded extracellularly from target-selective descending neurons (TSDNs) in male Eristalis tenax hoverflies. We show that they have dorso-frontal receptive fields with a preferred direction up and away from the visual midline. We reconstructed visual flow fields as experienced during pursuits of artificial targets (black beads). We recorded TSDN responses to six reconstructed pursuits and found that each neuron responded consistently at remarkably specific time points but that these time points differed between neurons. We found that the observed spike probability was correlated with the spike probability predicted from each neuron's receptive field and size tuning. Interestingly, however, the overall response rate was low, with individual neurons responding to only a small part of each reconstructed pursuit. In contrast, the TSDN population responded to substantially larger proportions of the pursuits but with lower probability. This large variation between neurons could be useful if different neurons control different parts of the behavioral output.
Collapse
Affiliation(s)
- Yuri Ogawa
- Flinders Health and Medical Research Institute, Flinders University, GPO Box 2100, Adelaide, SA 5001, Australia
| | - Sarah Nicholas
- Flinders Health and Medical Research Institute, Flinders University, GPO Box 2100, Adelaide, SA 5001, Australia
| | - Malin Thyselius
- Department of Medical Cell Biology, Uppsala University, 75123 Uppsala, Sweden; School of Medical Sciences, Faculty of Medicine and Health, Örebro University, Örebro 701 82, Sweden
| | - Richard Leibbrandt
- College of Science and Engineering, Flinders University, GPO Box 2100, Adelaide, SA 5001, Australia
| | - Thomas Nowotny
- School of Engineering and Informatics, University of Sussex, Brighton BN1 9QJ, UK
| | - James C Knight
- School of Engineering and Informatics, University of Sussex, Brighton BN1 9QJ, UK
| | - Karin Nordström
- Flinders Health and Medical Research Institute, Flinders University, GPO Box 2100, Adelaide, SA 5001, Australia; Department of Medical Cell Biology, Uppsala University, 75123 Uppsala, Sweden.
| |
Collapse
|
2
|
Turner MH, Krieger A, Pang MM, Clandinin TR. Visual and motor signatures of locomotion dynamically shape a population code for feature detection in Drosophila. eLife 2022; 11:e82587. [PMID: 36300621 PMCID: PMC9651947 DOI: 10.7554/elife.82587] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2022] [Accepted: 10/25/2022] [Indexed: 01/07/2023] Open
Abstract
Natural vision is dynamic: as an animal moves, its visual input changes dramatically. How can the visual system reliably extract local features from an input dominated by self-generated signals? In Drosophila, diverse local visual features are represented by a group of projection neurons with distinct tuning properties. Here, we describe a connectome-based volumetric imaging strategy to measure visually evoked neural activity across this population. We show that local visual features are jointly represented across the population, and a shared gain factor improves trial-to-trial coding fidelity. A subset of these neurons, tuned to small objects, is modulated by two independent signals associated with self-movement, a motor-related signal, and a visual motion signal associated with rotation of the animal. These two inputs adjust the sensitivity of these feature detectors across the locomotor cycle, selectively reducing their gain during saccades and restoring it during intersaccadic intervals. This work reveals a strategy for reliable feature detection during locomotion.
Collapse
Affiliation(s)
- Maxwell H Turner
- Department of Neurobiology, Stanford UniversityStanfordUnited States
| | - Avery Krieger
- Department of Neurobiology, Stanford UniversityStanfordUnited States
| | - Michelle M Pang
- Department of Neurobiology, Stanford UniversityStanfordUnited States
| | | |
Collapse
|
3
|
Evans BJE, O’Carroll DC, Fabian JM, Wiederman SD. Dragonfly Neurons Selectively Attend to Targets Within Natural Scenes. Front Cell Neurosci 2022; 16:857071. [PMID: 35450210 PMCID: PMC9017788 DOI: 10.3389/fncel.2022.857071] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2022] [Accepted: 03/11/2022] [Indexed: 12/05/2022] Open
Abstract
Aerial predators, such as the dragonfly, determine the position and movement of their prey even when both are moving through complex, natural scenes. This task is likely supported by a group of neurons in the optic lobe which respond to moving targets that subtend less than a few degrees. These Small Target Motion Detector (STMD) neurons are tuned to both target size and velocity, whilst also exhibiting facilitated responses to targets traveling along continuous trajectories. When presented with a pair of targets, some STMDs generate spiking activity that represent a competitive selection of one target, as if the alternative does not exist (i.e., selective attention). Here, we describe intracellular responses of CSTMD1 (an identified STMD) to the visual presentation of targets embedded within cluttered, natural scenes. We examine CSTMD1 response changes to target contrast, as well as a range of target and background velocities. We find that background motion affects CSTMD1 responses via the competitive selection between features within the natural scene. Here, robust discrimination of our artificially embedded “target” is limited to scenarios when its velocity is matched to, or greater than, the background velocity. Additionally, the background’s direction of motion affects discriminability, though not in the manner observed in STMDs of other flying insects. Our results highlight that CSTMD1’s competitive responses are to those features best matched to the neuron’s underlying spatiotemporal tuning, whether from the embedded target or other features in the background clutter. In many scenarios, CSTMD1 responds robustly to targets moving through cluttered scenes. However, whether this neuronal system could underlie the task of competitively selecting slow moving prey against fast-moving backgrounds remains an open question.
Collapse
Affiliation(s)
- Bernard John Essex Evans
- School of Biomedicine, The University of Adelaide, Adelaide, SA, Australia
- *Correspondence: Bernard John Essex Evans
| | | | | | | |
Collapse
|
4
|
James JV, Cazzolato BS, Grainger S, Wiederman SD. Nonlinear, neuronal adaptation in insect vision models improves target discrimination within repetitively moving backgrounds. BIOINSPIRATION & BIOMIMETICS 2021; 16:066015. [PMID: 34555824 DOI: 10.1088/1748-3190/ac2988] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/23/2021] [Accepted: 09/23/2021] [Indexed: 06/13/2023]
Abstract
Neurons which respond selectively to small moving targets, even against a cluttered background, have been identified in several insect species. To investigate what underlies these robust and highly selective responses, researchers have probed the neuronal circuitry in target-detecting, visual pathways. Observations in flies reveal nonlinear adaptation over time, composed of a fast onset and gradual decay. This adaptive processing is seen in both of the independent, parallel pathways encoding either luminance increments (ON channel) or decrements (OFF channel). The functional significance of this adaptive phenomenon has not been determined from physiological studies, though the asymmetrical time course suggests a role in suppressing responses to repetitive stimuli. We tested this possibility by comparing an implementation of fast adaptation against alternatives, using a model of insect 'elementary small target motion detectors'. We conducted target-detecting simulations on various natural backgrounds, that were shifted via several movement profiles (and target velocities). Using performance metrics, we confirmed that the fast adaptation observed in neuronal systems enhances target detection against a repetitively moving background. Such background movement would be encountered via natural ego-motion as the insect travels through the world. These findings show that this form of nonlinear, fast-adaptation (suitably implementable via cellular biophysics) plays a role analogous to background subtraction techniques in conventional computer vision.
Collapse
Affiliation(s)
- John V James
- School of Mechanical Engineering, University of Adelaide, Adelaide SA, Australia
- Adelaide Medical School, University of Adelaide, Adelaide SA, Australia
| | - Benjamin S Cazzolato
- School of Mechanical Engineering, University of Adelaide, Adelaide SA, Australia
| | - Steven Grainger
- School of Mechanical Engineering, University of Adelaide, Adelaide SA, Australia
| | | |
Collapse
|
5
|
Facilitation of neural responses to targets moving against optic flow. Proc Natl Acad Sci U S A 2021; 118:2024966118. [PMID: 34531320 PMCID: PMC8463850 DOI: 10.1073/pnas.2024966118] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/27/2021] [Indexed: 01/08/2023] Open
Abstract
Target detection in visual clutter is a difficult computational task that insects, with their poor spatial resolution compound eyes and small brains, do successfully and with extremely short behavioral delays. We here show that the responses of target selective descending neurons are attenuated by background motion in the same direction as target motion but facilitated by background motion in the opposite direction. This finding is important for understanding how target pursuit can occur in tandem with gaze stabilization. Indeed, the neural facilitation would come into effect if the hoverfly is subjected to background motion in one direction but the target it is pursuing moves in the opposite direction and could therefore be used to override gaze stabilizing corrective turns. For the human observer, it can be difficult to follow the motion of small objects, especially when they move against background clutter. In contrast, insects efficiently do this, as evidenced by their ability to capture prey, pursue conspecifics, or defend territories, even in highly textured surrounds. We here recorded from target selective descending neurons (TSDNs), which likely subserve these impressive behaviors. To simulate the type of optic flow that would be generated by the pursuer’s own movements through the world, we used the motion of a perspective corrected sparse dot field. We show that hoverfly TSDN responses to target motion are suppressed when such optic flow moves syn-directional to the target. Indeed, neural responses are strongly suppressed when targets move over either translational sideslip or rotational yaw. More strikingly, we show that TSDNs are facilitated by optic flow moving counterdirectional to the target, if the target moves horizontally. Furthermore, we show that a small, frontal spatial window of optic flow is enough to fully facilitate or suppress TSDN responses to target motion. We argue that such TSDN response facilitation could be beneficial in modulating corrective turns during target pursuit.
Collapse
|
6
|
Strausfeld NJ. The lobula plate is exclusive to insects. ARTHROPOD STRUCTURE & DEVELOPMENT 2021; 61:101031. [PMID: 33711678 DOI: 10.1016/j.asd.2021.101031] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/20/2020] [Revised: 01/12/2021] [Accepted: 01/19/2021] [Indexed: 06/12/2023]
Abstract
Just one superorder of insects is known to possess a neuronal network that mediates extremely rapid reactions in flight in response to changes in optic flow. Research on the identity and functional organization of this network has over the course of almost half a century focused exclusively on the order Diptera, a member of the approximately 300-million-year-old clade Holometabola defined by its mode of development. However, it has been broadly claimed that the pivotal neuropil containing the network, the lobula plate, originated in the Cambrian before the divergence of Hexapoda and Crustacea from a mandibulate ancestor. This essay defines the traits that designate the lobula plate and argues against a homologue in Crustacea. It proposes that the origin of the lobula plate is relatively recent and may relate to the origin of flight.
Collapse
|
7
|
Lancer BH, Evans BJE, Wiederman SD. The visual neuroecology of anisoptera. CURRENT OPINION IN INSECT SCIENCE 2020; 42:14-22. [PMID: 32841784 DOI: 10.1016/j.cois.2020.07.002] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/23/2020] [Revised: 07/14/2020] [Accepted: 07/15/2020] [Indexed: 06/11/2023]
Abstract
Dragonflies belong to the oldest known lineage of flying animals, found across the globe around streams, ponds and forests. They are insect predators, specialising in ambush attack as aquatic larvae and rapid pursuit as adults. Dragonfly adults hunt amidst swarms in conditions that confuse many predatory species, and exhibit capture rates above 90%. Underlying the performance of such a remarkable predator is a finely tuned visual system capable of tracking targets amidst distractors and background clutter. The dragonfly performs a complex repertoire of flight behaviours, from near-motionless hovering to acute turns at high speeds. Here, we review the optical, neuronal, and behavioural adaptations that underlie the dragonflies' ability to achieve such remarkable predatory success.
Collapse
Affiliation(s)
- Benjamin Horatio Lancer
- Adelaide Medical School, The University of Adelaide, Adelaide, 5005 South Australia, Australia
| | | | - Steven D Wiederman
- Adelaide Medical School, The University of Adelaide, Adelaide, 5005 South Australia, Australia.
| |
Collapse
|
8
|
Städele C, Keleş MF, Mongeau JM, Frye MA. Non-canonical Receptive Field Properties and Neuromodulation of Feature-Detecting Neurons in Flies. Curr Biol 2020; 30:2508-2519.e6. [PMID: 32442460 PMCID: PMC7343589 DOI: 10.1016/j.cub.2020.04.069] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2020] [Revised: 03/10/2020] [Accepted: 04/24/2020] [Indexed: 10/24/2022]
Abstract
Several fundamental aspects of motion vision circuitry are prevalent across flies and mice. Both taxa segregate ON and OFF signals. For any given spatial pattern, motion detectors in both taxa are tuned to speed, selective for one of four cardinal directions, and modulated by catecholamine neurotransmitters. These similarities represent conserved, canonical properties of the functional circuits and computational algorithms for motion vision. Less is known about feature detectors, including how receptive field properties differ from the motion pathway or whether they are under neuromodulatory control to impart functional plasticity for the detection of salient objects from a moving background. Here, we investigated 19 types of putative feature selective lobula columnar (LC) neurons in the optic lobe of the fruit fly Drosophila melanogaster to characterize divergent properties of feature selection. We identified LC12 and LC15 as feature detectors. LC15 encodes moving bars, whereas LC12 is selective for the motion of discrete objects, mostly independent of size. Neither is selective for contrast polarity, speed, or direction, highlighting key differences in the underlying algorithms for feature detection and motion vision. We show that the onset of background motion suppresses object responses by LC12 and LC15. Surprisingly, the application of octopamine, which is released during flight, reverses the suppressive influence of background motion, rendering both LCs able to track moving objects superimposed against background motion. Our results provide a comparative framework for the function and modulation of feature detectors and new insights into the underlying neuronal mechanisms involved in visual feature detection.
Collapse
Affiliation(s)
- Carola Städele
- Department of Integrative Biology and Physiology, University of California, Los Angeles, Los Angeles, CA 90095-7239, USA
| | - Mehmet F Keleş
- Department of Integrative Biology and Physiology, University of California, Los Angeles, Los Angeles, CA 90095-7239, USA
| | - Jean-Michel Mongeau
- Department of Integrative Biology and Physiology, University of California, Los Angeles, Los Angeles, CA 90095-7239, USA
| | - Mark A Frye
- Department of Integrative Biology and Physiology, University of California, Los Angeles, Los Angeles, CA 90095-7239, USA.
| |
Collapse
|
9
|
Differential Tuning to Visual Motion Allows Robust Encoding of Optic Flow in the Dragonfly. J Neurosci 2019; 39:8051-8063. [PMID: 31481434 DOI: 10.1523/jneurosci.0143-19.2019] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2019] [Revised: 07/22/2019] [Accepted: 08/07/2019] [Indexed: 11/21/2022] Open
Abstract
Visual cues provide an important means for aerial creatures to ascertain their self-motion through the environment. In many insects, including flies, moths, and bees, wide-field motion-sensitive neurons in the third optic ganglion are thought to underlie such motion encoding; however, these neurons can only respond robustly over limited speed ranges. The task is more complicated for some species of dragonflies that switch between extended periods of hovering flight and fast-moving pursuit of prey and conspecifics, requiring motion detection over a broad range of velocities. Since little is known about motion processing in these insects, we performed intracellular recordings from hawking, emerald dragonflies (Hemicordulia spp.) and identified a diverse group of motion-sensitive neurons that we named lobula tangential cells (LTCs). Following prolonged visual stimulation with drifting gratings, we observed significant differences in both temporal and spatial tuning of LTCs. Cluster analysis of these changes confirmed several groups of LTCs with distinctive spatiotemporal tuning. These differences were associated with variation in velocity tuning in response to translated, natural scenes. LTCs with differences in velocity tuning ranges and optima may underlie how a broad range of motion velocities are encoded. In the hawking dragonfly, changes in LTC tuning over time are therefore likely to support their extensive range of behaviors, from hovering to fast-speed pursuits.SIGNIFICANCE STATEMENT Understanding how animals navigate the world is an inherently difficult and interesting problem. Insects are useful models for understanding neuronal mechanisms underlying these activities, with neurons that encode wide-field motion previously identified in insects, such as flies, hawkmoths, and butterflies. Like some Dipteran flies, dragonflies exhibit complex aerobatic behaviors, such as hovering, patrolling, and aerial combat. However, dragonflies lack halteres that support such diverse behavior in flies. To understand how dragonflies might address this problem using only visual cues, we recorded from their wide-field motion-sensitive neurons. We found these differ strongly in the ways they respond to sustained motion, allowing them collectively to encode the very broad range of velocities experienced during diverse behavior.
Collapse
|
10
|
Integration of Small- and Wide-Field Visual Features in Target-Selective Descending Neurons of both Predatory and Nonpredatory Dipterans. J Neurosci 2018; 38:10725-10733. [PMID: 30373766 PMCID: PMC6290295 DOI: 10.1523/jneurosci.1695-18.2018] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2018] [Revised: 10/18/2018] [Accepted: 10/21/2018] [Indexed: 11/21/2022] Open
Abstract
For many animals, target motion carries high ecological significance as this may be generated by a predator, prey, or potential mate. Indeed, animals whose survival depends on early target detection are often equipped with a sharply tuned visual system, yielding robust performance in challenging conditions. For example, many fast-flying insects use visual cues for identifying targets, such as prey (e.g., predatory dragonflies and robberflies) or conspecifics (e.g., nonpredatory hoverflies), and can often do so against self-generated background optic flow. Supporting these behaviors, the optic lobes of insects that pursue targets harbor neurons that respond robustly to the motion of small moving objects, even when displayed against syn-directional background clutter. However, in diptera, the encoding of target information by the descending neurons, which are more directly involved in generating the behavioral output, has received less attention. We characterized target-selective neurons by recording in the ventral nerve cord of male and female predatory Holcocephala fusca robberflies and of male nonpredatory Eristalis tenax hoverflies. We show that both species have dipteran target-selective descending neurons that only respond to target motion if the background is stationary or moving slowly, moves in the opposite direction, or has un-naturalistic spatial characteristics. The response to the target is suppressed when background and target move at similar velocities, which is strikingly different to the response of target neurons in the optic lobes. As the neurons we recorded from are premotor, our findings affect our interpretation of the neurophysiology underlying target-tracking behaviors. SIGNIFICANCE STATEMENT Many animals use sensory cues to detect moving targets that may represent predators, prey, or conspecifics. For example, birds of prey show superb sensitivity to the motion of small prey, and intercept these at high speeds. In a similar manner, predatory insects visually track moving prey, often against cluttered backgrounds. Accompanying this behavior, the brains of insects that pursue targets contain neurons that respond exclusively to target motion. We here show that dipteran insects also have target-selective descending neurons in the part of their nervous system that corresponds to the vertebrate spinal cord. Surprisingly, and in contrast to the neurons in the brain, these premotor neurons are inhibited by background patterns moving in the same direction as the target.
Collapse
|
11
|
Strydom R, Srinivasan MV. UAS stealth: target pursuit at constant distance using a bio-inspired motion camouflage guidance law. BIOINSPIRATION & BIOMIMETICS 2017; 12:055002. [PMID: 28675149 DOI: 10.1088/1748-3190/aa7d65] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
The aim of this study is to derive a guidance law by which an unmanned aerial system(s) (UAS) can pursue a moving target at a constant distance, while concealing its own motion. We derive a closed-form solution for the trajectory of the UAS by imposing two key constraints: (1) the shadower moves in such a way as to be perceived as a stationary object by the shadowee, and (2) the distance between the shadower and shadowee is kept constant. Additionally, the theory presented in this paper considers constraints on the maximum achievable speed and acceleration of the shadower. Our theory is tested through Matlab simulations, which validate the camouflage strategy for both 2D and 3D conditions. Furthermore, experiments using a realistic vision-based implementation are conducted in a virtual environment, where the results demonstrate that even with noisy state information it is possible to remain well camouflaged using the constant distance motion camouflage technique.
Collapse
Affiliation(s)
- Reuben Strydom
- The Queensland Brain Institute and the School of Information Technology and Electrical Engineering at The University of Queensland, Brisbane, Australia
| | | |
Collapse
|
12
|
Wiederman SD, Fabian JM, Dunbier JR, O'Carroll DC. A predictive focus of gain modulation encodes target trajectories in insect vision. eLife 2017; 6. [PMID: 28738970 PMCID: PMC5526664 DOI: 10.7554/elife.26478] [Citation(s) in RCA: 37] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2017] [Accepted: 06/28/2017] [Indexed: 12/13/2022] Open
Abstract
When a human catches a ball, they estimate future target location based on the current trajectory. How animals, small and large, encode such predictive processes at the single neuron level is unknown. Here we describe small target-selective neurons in predatory dragonflies that exhibit localized enhanced sensitivity for targets displaced to new locations just ahead of the prior path, with suppression elsewhere in the surround. This focused region of gain modulation is driven by predictive mechanisms, with the direction tuning shifting selectively to match the target’s prior path. It involves a large local increase in contrast gain which spreads forward after a delay (e.g. an occlusion) and can even transfer between brain hemispheres, predicting trajectories moved towards the visual midline from the other eye. The tractable nature of dragonflies for physiological experiments makes this a useful model for studying the neuronal mechanisms underlying the brain’s remarkable ability to anticipate moving stimuli. DOI:http://dx.doi.org/10.7554/eLife.26478.001 Catching a ball requires a person to track the speed and direction of a small moving target often against a cluttered and varying background. Predatory insects, like dragonflies, face a similar challenge when they pursue their prey through the air. The task is made a little easier, however, by the fact that most moving targets tend to follow predictable trajectories. Indeed, animals are also better at tracking targets that follow smooth continuous trajectories, suggesting that brains have evolved to exploit the normal behavior of visual stimuli to reduce their workload To find out how this process works, Wiederman, Fabian et al. studied the brains of dragonflies as they watched a black square intended to mimic prey. Brain cells called Small Target Motion Detectors (or STMD neurons for short) became more active in response to the target. But rather than simply following the target, the STMD neurons instead predicted its future location. In fact, individual neurons were more sensitive to movements occurring just ahead of the target’s current position, and less sensitive to movements occurring elsewhere. If the target abruptly disappeared, the point in space where the neurons were most sensitive to movement continued to gradually move forward over time. Given that real-life targets typically disappear when they move behind other objects, this suggests that the brain is predicting where the target is most likely to reappear. The STMD neurons became more sensitive to movement by increasing their ability to detect differences in brightness between the target and its background. In some cases, the neurons increased their sensitivity more than five-fold. Insects and mammals last shared a common ancestor more than 500 million years ago, and, in many respects, mammalian brains are substantially more complex than insect brains. Nevertheless, the results of Wiederman, Fabian et al. show that the insect brain can perform visual tasks that were previously associated only with mammals. Neuroscientists and engineers have used the insect brain for decades to study the circuits that support biological processes. In the coming years, insects such as the dragonfly may enable us to explore how visual regions of the brain predict future events. This knowledge could ultimately be applied to artificial vision systems, such as those in self-driving cars. DOI:http://dx.doi.org/10.7554/eLife.26478.002
Collapse
Affiliation(s)
- Steven D Wiederman
- Adelaide Medical School, The University of Adelaide, Adelaide, Australia
| | - Joseph M Fabian
- Adelaide Medical School, The University of Adelaide, Adelaide, Australia
| | - James R Dunbier
- Adelaide Medical School, The University of Adelaide, Adelaide, Australia
| | | |
Collapse
|
13
|
Bagheri ZM, Cazzolato BS, Grainger S, O’Carroll DC, Wiederman SD. An autonomous robot inspired by insect neurophysiology pursues moving features in natural environments. J Neural Eng 2017; 14:046030. [DOI: 10.1088/1741-2552/aa776c] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
14
|
Bagheri ZM, Wiederman SD, Cazzolato BS, Grainger S, O'Carroll DC. Performance of an insect-inspired target tracker in natural conditions. BIOINSPIRATION & BIOMIMETICS 2017; 12:025006. [PMID: 28112099 DOI: 10.1088/1748-3190/aa5b48] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
Robust and efficient target-tracking algorithms embedded on moving platforms, are a requirement for many computer vision and robotic applications. However, deployment of a real-time system is challenging, even with the computational power of modern hardware. As inspiration, we look to biological lightweight solutions-lightweight and low-powered flying insects. For example, dragonflies pursue prey and mates within cluttered, natural environments, deftly selecting their target amidst swarms. In our laboratory, we study the physiology and morphology of dragonfly 'small target motion detector' neurons likely to underlie this pursuit behaviour. Here we describe our insect-inspired tracking model derived from these data and compare its efficacy and efficiency with state-of-the-art engineering models. For model inputs, we use both publicly available video sequences, as well as our own task-specific dataset (small targets embedded within natural scenes). In the context of the tracking problem, we describe differences in object statistics within the video sequences. For the general dataset, our model often locks on to small components of larger objects, tracking these moving features. When input imagery includes small moving targets, for which our highly nonlinear filtering is matched, the robustness outperforms state-of-the-art trackers. In all scenarios, our insect-inspired tracker runs at least twice the speed of the comparison algorithms.
Collapse
Affiliation(s)
- Zahra M Bagheri
- Adelaide Medical School, The University of Adelaide, Adelaide, SA, 5005, Australia. School of Mechanical Engineering, The University of Adelaide, Adelaide, SA, 5005, Australia
| | | | | | | | | |
Collapse
|
15
|
Compass Cells in the Brain of an Insect Are Sensitive to Novel Events in the Visual World. PLoS One 2015; 10:e0144501. [PMID: 26636334 PMCID: PMC4670205 DOI: 10.1371/journal.pone.0144501] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2015] [Accepted: 11/19/2015] [Indexed: 11/29/2022] Open
Abstract
The central complex of the insect brain comprises a group of neuropils involved in spatial orientation and memory. In fruit flies it mediates place learning based on visual landmarks and houses neurons that encode the orientation for goal-directed locomotion, based on landmarks and self-motion cues for angular path-integration. In desert locusts, the central complex holds a compass-like representation of head directions, based on the polarization pattern of skylight. Through intracellular recordings from immobilized locusts, we investigated whether sky compass neurons of the central complex also represent the position or any salient feature of possible landmarks, in analogy to the observations in flies. Neurons showed strongest responses to the novel appearance of a small moving square, but we found no evidence for a topographic representation of object positions. Responses to an individual square were independent of direction of motion and trajectory, but showed rapid adaptation to successive stimulation, unaffected by changing the direction of motion. Responses reappeared, however, if the moving object changed its trajectory or if it suddenly reversed moving direction against the movement of similar objects that make up a coherent background-flow as induced by ego-motion. Response amplitudes co-varied with the precedent state of dynamic background activity, a phenomenon that has been related to attention-dependent saliency coding in neurons of the mammalian primary visual cortex. The data show that neurons of the central complex of the locust brain are visually bimodal, signaling sky compass direction and the novelty character of moving objects. These response properties might serve to attune compass-aided locomotor control to unexpected events in the environment. The difference to data obtained in fruit flies might relate to differences in the lifestyle of landmark learners (fly) and compass navigators (locust), point to the existence of parallel networks for the two orientation strategies, or reflect differences in experimental conditions.
Collapse
|
16
|
Bagheri ZM, Wiederman SD, Cazzolato BS, Grainger S, O'Carroll DC. Properties of neuronal facilitation that improve target tracking in natural pursuit simulations. J R Soc Interface 2015; 12:20150083. [PMID: 26063815 DOI: 10.1098/rsif.2015.0083] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
Although flying insects have limited visual acuity (approx. 1°) and relatively small brains, many species pursue tiny targets against cluttered backgrounds with high success. Our previous computational model, inspired by electrophysiological recordings from insect 'small target motion detector' (STMD) neurons, did not account for several key properties described from the biological system. These include the recent observations of response 'facilitation' (a slow build-up of response to targets that move on long, continuous trajectories) and 'selective attention', a competitive mechanism that selects one target from alternatives. Here, we present an elaborated STMD-inspired model, implemented in a closed loop target-tracking system that uses an active saccadic gaze fixation strategy inspired by insect pursuit. We test this system against heavily cluttered natural scenes. Inclusion of facilitation not only substantially improves success for even short-duration pursuits, but it also enhances the ability to 'attend' to one target in the presence of distracters. Our model predicts optimal facilitation parameters that are static in space and dynamic in time, changing with respect to the amount of background clutter and the intended purpose of the pursuit. Our results provide insights into insect neurophysiology and show the potential of this algorithm for implementation in artificial visual systems and robotic applications.
Collapse
Affiliation(s)
- Zahra M Bagheri
- Adelaide Centre for Neuroscience Research, The University of Adelaide, Adelaide, South Australia 5005, Australia School of Mechanical Engineering, The University of Adelaide, Adelaide, South Australia 5005, Australia
| | - Steven D Wiederman
- Adelaide Centre for Neuroscience Research, The University of Adelaide, Adelaide, South Australia 5005, Australia
| | - Benjamin S Cazzolato
- School of Mechanical Engineering, The University of Adelaide, Adelaide, South Australia 5005, Australia
| | - Steven Grainger
- School of Mechanical Engineering, The University of Adelaide, Adelaide, South Australia 5005, Australia
| | - David C O'Carroll
- Adelaide Centre for Neuroscience Research, The University of Adelaide, Adelaide, South Australia 5005, Australia Department of Biology, Lund University, Sölvegatan 35, S-22362 Lund, Sweden
| |
Collapse
|
17
|
Ullrich TW, Kern R, Egelhaaf M. Texture-defined objects influence responses of blowfly motion-sensitive neurons under natural dynamical conditions. Front Integr Neurosci 2014; 8:34. [PMID: 24808836 PMCID: PMC4010782 DOI: 10.3389/fnint.2014.00034] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2013] [Accepted: 04/10/2014] [Indexed: 11/13/2022] Open
Abstract
The responses of visual interneurons of flies involved in the processing of motion information do not only depend on the velocity, but also on other stimulus parameters, such as the contrast and the spatial frequency content of the stimulus pattern. These dependencies have been known for long, but it is still an open question how they affect the neurons' performance in extracting information about the structure of the environment under the specific dynamical conditions of natural flight. Free-flight of blowflies is characterized by sequences of phases of translational movements lasting for just 30-100 ms interspersed with even shorter and extremely rapid saccade-like rotational shifts in flight and gaze direction. Previous studies already analyzed how nearby objects, leading to relative motion on the retina with respect to a more distant background, influenced the response of a class of fly motion sensitive visual interneurons, the horizontal system (HS) cells. In the present study, we focused on objects that differed from their background by discontinuities either in their brightness contrast or in their spatial frequency content. We found strong object-induced effects on the membrane potential even during the short intersaccadic intervals, if the background contrast was small and the object contrast sufficiently high. The object evoked similar response increments provided that it contained higher spatial frequencies than the background, but not under reversed conditions. This asymmetry in the response behavior is partly a consequence of the depolarization level induced by the background. Thus, our results suggest that, under the specific dynamical conditions of natural flight, i.e., on a very short timescale, the responses of HS cells represent object information depending on the polarity of the difference between object and background contrast and spatial frequency content.
Collapse
Affiliation(s)
- Thomas W Ullrich
- Department of Neurobiology and Center of Excellence Cognitive Interaction Technology, Bielefeld University Bielefeld, Germany
| | - Roland Kern
- Department of Neurobiology and Center of Excellence Cognitive Interaction Technology, Bielefeld University Bielefeld, Germany
| | - Martin Egelhaaf
- Department of Neurobiology and Center of Excellence Cognitive Interaction Technology, Bielefeld University Bielefeld, Germany
| |
Collapse
|
18
|
Abstract
Visually-guided animals rely on their ability to stabilize the panorama and simultaneously track salient objects, or figures, that are distinct from the background in order to avoid predators, pursue food resources and mates, and navigate spatially. Visual figures are distinguished by luminance signals that produce coherent motion cues as well as more enigmatic 'higher-order' statistical features. Figure discrimination is thus a complex form of motion vision requiring specialized neural processing. In this minireview, we will highlight recent advances in understanding the perceptual, behavioral, and neurophysiological basis of higher-order figure detection in flies, much of which is grounded in the historical perspective and mechanistic underpinnings of human psychophysics.
Collapse
Affiliation(s)
- Jacob W Aptekar
- Howard Hughes Medical Institute, Department of Integrative Biology and Physiology, University of California Los Angeles, Los Angeles, CA 90095, USA
| | | |
Collapse
|
19
|
O'Carroll DC, Wiederman SD. Contrast sensitivity and the detection of moving patterns and features. Philos Trans R Soc Lond B Biol Sci 2014; 369:20130043. [PMID: 24395970 DOI: 10.1098/rstb.2013.0043] [Citation(s) in RCA: 49] [Impact Index Per Article: 4.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
Theories based on optimal sampling by the retina have been widely applied to visual ecology at the level of the optics of the eye, supported by visual behaviour. This leads to speculation about the additional processing that must lie in between-in the brain itself. But fewer studies have adopted a quantitative approach to evaluating the detectability of specific features in these neural pathways. We briefly review this approach with a focus on contrast sensitivity of two parallel pathways for motion processing in insects, one used for analysis of wide-field optic flow, the other for detection of small features. We further use a combination of optical modelling of image blur and physiological recording from both photoreceptors and higher-order small target motion detector neurons sensitive to small targets to show that such neurons operate right at the limits imposed by the optics of the eye and the noise level of single photoreceptors. Despite this, and the limitation of only being able to use information from adjacent receptors to detect target motion, they achieve a contrast sensitivity that rivals that of wide-field motion sensitive pathways in either insects or vertebrates-among the highest in absolute terms seen in any animal.
Collapse
Affiliation(s)
- David C O'Carroll
- Adelaide Centre for Neuroscience Research, School of Medical Sciences, The University of Adelaide, , Adelaide, South Australia 5000, Australia
| | | |
Collapse
|
20
|
Correlation between OFF and ON channels underlies dark target selectivity in an insect visual system. J Neurosci 2013; 33:13225-32. [PMID: 23926274 DOI: 10.1523/jneurosci.1277-13.2013] [Citation(s) in RCA: 37] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
In both vertebrates and invertebrates, evidence supports separation of luminance increments and decrements (ON and OFF channels) in early stages of visual processing (Hartline, 1938; Joesch et al., 2010); however, less is known about how these parallel pathways are recombined to encode form and motion. In Drosophila, genetic knockdown of inputs to putative ON and OFF pathways and direct recording from downstream neurons in the wide-field motion pathway reveal that local elementary motion detectors exist in pairs that separately correlate contrast polarity channels, ON with ON and OFF with OFF (Joesch et al., 2013). However, behavioral responses to reverse-phi motion of discrete features reveal additional correlations of the opposite signs (Clark et al., 2011). We here present intracellular recordings from feature detecting neurons in the dragonfly that provide direct physiological evidence for the correlation of OFF and ON pathways. These neurons show clear polarity selectivity for feature contrast, responding strongly to targets that are darker than the background and only weakly to dark contrasting edges. These dark target responses are much stronger than the linear combination of responses to ON and OFF edges. We compare these data with output from elementary motion detector-based models (Eichner et al., 2011; Clark et al., 2011), with and without stages of strong center-surround antagonism. Our data support an alternative elementary small target motion detector model, which derives dark target selectivity from the correlation of a delayed OFF with an un-delayed ON signal at each individual visual processing unit (Wiederman et al., 2008, 2009).
Collapse
|
21
|
Wiederman S, O’Carroll D. Selective Attention in an Insect Visual Neuron. Curr Biol 2013; 23:156-61. [DOI: 10.1016/j.cub.2012.11.048] [Citation(s) in RCA: 70] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2012] [Revised: 11/08/2012] [Accepted: 11/26/2012] [Indexed: 11/24/2022]
|
22
|
Dunbier JR, Wiederman SD, Shoemaker PA, O'Carroll DC. Facilitation of dragonfly target-detecting neurons by slow moving features on continuous paths. Front Neural Circuits 2012; 6:79. [PMID: 23112764 PMCID: PMC3483020 DOI: 10.3389/fncir.2012.00079] [Citation(s) in RCA: 30] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2012] [Accepted: 10/11/2012] [Indexed: 11/14/2022] Open
Abstract
Dragonflies detect and pursue targets such as other insects for feeding and conspecific interaction. They have a class of neurons highly specialized for this task in their lobula, the “small target motion detecting” (STMD) neurons. One such neuron, CSTMD1, reaches maximum response slowly over hundreds of milliseconds of target motion. Recording the intracellular response from CSTMD1 and a second neuron in this system, BSTMD1, we determined that for the neurons to reach maximum response levels, target motion must produce sequential local activation of elementary motion detecting elements. This facilitation effect is most pronounced when targets move at velocities slower than what was previously thought to be optimal. It is completely disrupted if targets are instantaneously displaced a few degrees from their current location. Additionally, we utilize a simple computational model to discount the parsimonious hypothesis that CSTMD1's slow build-up to maximum response is due to it incorporating a sluggish neural delay filter. Whilst the observed facilitation may be too slow to play a role in prey pursuit flights, which are typically rapidly resolved, we hypothesize that it helps maintain elevated sensitivity during prolonged, aerobatically intricate conspecific pursuits. Since the effect seems to be localized, it most likely enhances the relative salience of the most recently “seen” locations during such pursuit flights.
Collapse
Affiliation(s)
- James R Dunbier
- Adelaide Centre for Neuroscience Research, School of Medical Sciences, The University of Adelaide Adelaide, SA, Australia
| | | | | | | |
Collapse
|
23
|
Neural specializations for small target detection in insects. Curr Opin Neurobiol 2012; 22:272-8. [PMID: 22244741 DOI: 10.1016/j.conb.2011.12.013] [Citation(s) in RCA: 32] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2011] [Revised: 12/27/2011] [Accepted: 12/28/2011] [Indexed: 11/23/2022]
Abstract
Despite being equipped with low-resolution eyes and tiny brains, many insects show exquisite abilities to detect and pursue targets even in highly textured surrounds. Target tracking behavior is subserved by neurons that are sharply tuned to the motion of small high-contrast targets. These neurons respond robustly to target motion, even against self-generated optic flow. A recent model, supported by neurophysiology, generates target selectivity by being sharply tuned to the unique spatiotemporal profile associated with target motion. Target neurons are likely connected in a complex network where some provide more direct output to behavior, whereas others serve an inter-regulatory role. These interactions may regulate attention and aid in the robust detection of targets in clutter observed in behavior.
Collapse
|