A Target-Detecting Visual Neuron in the Dragonfly Locks on to Selectively Attended Targets

Visually guided swarm motion coordination via insect-inspired small target motion reactions

Despite progress developing experimentally-consistent models of insect in-flight sensing and feedback for individual agents, a lack of systematic understanding of the multi-agent and group performance of the resulting bio-inspired sensing and feedback approaches remains a barrier to robotic swarm implementations. This study introduces the small-target motion reactive (STMR) swarming approach by designing a concise engineering model of the small target motion detector (STMD) neurons found in insect lobula complexes. The STMD neuron model identifies the bearing angle at which peak optic flow magnitude occurs, and this angle is used to design an output feedback switched control system. A theoretical stability analysis provides bi-agent stability and state boundedness in group contexts. The approach is simulated and implemented on ground vehicles for validation and behavioral studies. The results indicate despite having the lowest connectivity of contemporary approaches (each agent instantaneously regards only a single neighbor), STMR achieves collective group motion. STMR group level metric analysis also highlights continuously varying polarization and decreasing heading variance.

Motion perception based on ON/OFF channels: A survey

Motion perception is an essential ability for animals and artificially intelligent systems interacting effectively, safely with surrounding objects and environments. Biological visual systems, that have naturally evolved over hundreds-million years, are quite efficient and robust for motion perception, whereas artificial vision systems are far from such capability. This paper argues that the gap can be significantly reduced by formulation of ON/OFF channels in motion perception models encoding luminance increment (ON) and decrement (OFF) responses within receptive field, separately. Such signal-bifurcating structure has been found in neural systems of many animal species articulating early motion is split and processed in segregated pathways. However, the corresponding biological substrates, and the necessity for artificial vision systems have never been elucidated together, leaving concerns on uniqueness and advantages of ON/OFF channels upon building dynamic vision systems to address real world challenges. This paper highlights the importance of ON/OFF channels in motion perception through surveying current progress covering both neuroscience and computationally modelling works with applications. Compared to related literature, this paper for the first time provides insights into implementation of different selectivity to directional motion of looming, translating, and small-sized target movement based on ON/OFF channels in keeping with soundness and robustness of biological principles. Existing challenges and future trends of such bio-plausible computational structure for visual perception in connection with hotspots of machine learning, advanced vision sensors like event-driven camera finally are discussed.

Predictive saccades and decision making in the beetle-predating saffron robber fly

Internal predictions about the sensory consequences of self-motion, encoded by corollary discharge, are ubiquitous in the animal kingdom, including for fruit flies, dragonflies, and humans. In contrast, predicting the future location of an independently moving external target requires an internal model. With the use of internal models for predictive gaze control, vertebrate predatory species compensate for their sluggish visual systems and long sensorimotor latencies. This ability is crucial for the timely and accurate decisions that underpin a successful attack. Here, we directly demonstrate that the robber fly Laphria saffrana, a specialized beetle predator, also uses predictive gaze control when head tracking potential prey. Laphria uses this predictive ability to perform the difficult categorization and perceptual decision task of differentiating a beetle from other flying insects with a low spatial resolution retina. Specifically, we show that (1) this predictive behavior is part of a saccade-and-fixate strategy, (2) the relative target angular position and velocity, acquired during fixation, inform the subsequent predictive saccade, and (3) the predictive saccade provides Laphria with additional fixation time to sample the frequency of the prey's specular wing reflections. We also demonstrate that Laphria uses such wing reflections as a proxy for the wingbeat frequency of the potential prey and that consecutively flashing LEDs to produce apparent motion elicits attacks when the LED flicker frequency matches that of the beetle's wingbeat cycle.

Preattentive facilitation of target trajectories in a dragonfly visual neuron

The ability to pursue targets in visually cluttered and distraction-rich environments is critical for predators such as dragonflies. Previously, we identified Centrifugal Small-Target Motion Detector 1 (CSTMD1), a dragonfly visual neuron likely involved in such target-tracking behaviour. CSTMD1 exhibits facilitated responses to targets moving along a continuous trajectory. Moreover, CSTMD1 competitively selects a single target out of a pair. Here, we conducted in vivo, intracellular recordings from CSTMD1 to examine the interplay between facilitation and selection, in response to the presentation of paired targets. We find that neuronal responses to both individual trajectories of simultaneous, paired targets are facilitated, rather than being constrained to the single, selected target. Additionally, switches in selection elicit suppression which is likely an important attribute underlying target pursuit. However, binocular experiments reveal these results are constrained to paired targets within the same visual hemifield, while selection of a target in one visual hemifield establishes ocular dominance that prevents facilitation or response to contralaterally presented targets. These results reveal that the dragonfly brain preattentively represents more than one target trajectory, to balance between attentional flexibility and resistance against distraction.

A dragonfly visual neuron independently facilitates responses to rival targets within the same visual field, mediating selective attention.

Behavioural and neural responses of crabs show evidence for selective attention in predator avoidance

Selective attention, the ability to focus on a specific stimulus and suppress distractions, plays a fundamental role for animals in many contexts, such as mating, feeding, and predation. Within natural environments, animals are often confronted with multiple stimuli of potential importance. Such a situation significantly complicates the decision-making process and imposes conflicting information on neural systems. In the context of predation, selectively attending to one of multiple threats is one possible solution. However, how animals make such escape decisions is rarely studied. A previous field study on the fiddler crab, Gelasimus dampieri, provided evidence of selective attention in the context of escape decisions. To identify the underlying mechanisms that guide their escape decisions, we measured the crabs' behavioural and neural responses to either a single, or two simultaneously approaching looming stimuli. The two stimuli were either identical or differed in contrast to represent different levels of threat certainty. Although our behavioural data provides some evidence that crabs perceive signals from both stimuli, we show that both the crabs and their looming-sensitive neurons almost exclusively respond to only one of two simultaneous threats. The crabs' body orientation played an important role in their decision about which stimulus to run away from. When faced with two stimuli of differing contrasts, both neurons and crabs were much more likely to respond to the stimulus with the higher contrast. Our data provides evidence that the crabs' looming-sensitive neurons play an important part in the mechanism that drives their selective attention in the context of predation. Our results support previous suggestions that the crabs' escape direction is calculated downstream of their looming-sensitive neurons by means of a population vector of the looming sensitive neuronal ensemble.

The Role of Central Complex Neurons in Prey Detection and Tracking in the Freely Moving Praying Mantis (Tenodera sinensis)

Complex tasks like hunting moving prey in an unpredictable environment require high levels of motor and sensory integration. An animal needs to detect and track suitable prey objects, measure their distance and orientation relative to its own position, and finally produce the correct motor output to approach and capture the prey. In the insect brain, the central complex (CX) is one target area where integration is likely to take place. In this study, we performed extracellular multi-unit recordings on the CX of freely hunting praying mantises (Tenodera sinensis). Initially, we recorded the neural activity of freely moving mantises as they hunted live prey. The recordings showed activity in cells that either reflected the mantis's own movements or the actions of a prey individual, which the mantises focused on. In the latter case, the activity increased as the prey moved and decreased when it stopped. Interestingly, cells ignored the movement of the other prey than the one to which the mantis attended. To obtain quantitative data, we generated simulated prey targets presented on an LCD screen positioned below the clear floor of the arena. The simulated target oscillated back and forth at various angles and distances. We identified populations of cells whose activity patterns were strongly linked to the appearance, movement, and relative position of the virtual prey. We refer to these as sensory responses. We also found cells whose activity preceded orientation movement toward the prey. We call these motor responses. Some cells showed both sensory and motor properties. Stimulation through tetrodes in some of the preparations could also generate similar movements. These results suggest the crucial importance of the CX to prey-capture behavior in predatory insects like the praying mantis and, hence, further emphasize its role in behaviorally and ecologically relevant contexts.

Dragonfly Neurons Selectively Attend to Targets Within Natural Scenes

Aerial predators, such as the dragonfly, determine the position and movement of their prey even when both are moving through complex, natural scenes. This task is likely supported by a group of neurons in the optic lobe which respond to moving targets that subtend less than a few degrees. These Small Target Motion Detector (STMD) neurons are tuned to both target size and velocity, whilst also exhibiting facilitated responses to targets traveling along continuous trajectories. When presented with a pair of targets, some STMDs generate spiking activity that represent a competitive selection of one target, as if the alternative does not exist (i.e., selective attention). Here, we describe intracellular responses of CSTMD1 (an identified STMD) to the visual presentation of targets embedded within cluttered, natural scenes. We examine CSTMD1 response changes to target contrast, as well as a range of target and background velocities. We find that background motion affects CSTMD1 responses via the competitive selection between features within the natural scene. Here, robust discrimination of our artificially embedded “target” is limited to scenarios when its velocity is matched to, or greater than, the background velocity. Additionally, the background’s direction of motion affects discriminability, though not in the manner observed in STMDs of other flying insects. Our results highlight that CSTMD1’s competitive responses are to those features best matched to the neuron’s underlying spatiotemporal tuning, whether from the embedded target or other features in the background clutter. In many scenarios, CSTMD1 responds robustly to targets moving through cluttered scenes. However, whether this neuronal system could underlie the task of competitively selecting slow moving prey against fast-moving backgrounds remains an open question.

Neural organization of the third optic neuropil, the lobula, in the highly visual semiterrestrial crab Neohelice granulata

The visual neuropils (lamina, medulla and lobula complex), of malacostracan crustaceans and hexapods have many organizational principles, cell types and functional properties in common. Information about the cellular elements that compose the crustacean lobula is scarce especially when focusing on small columnar cells. Semiterrestrial crabs possess a highly developed visual system and display conspicuous visually guided behaviors. In particular, Neohelice granulata has been previously used to describe the cellular components of the first two optic neuropils using Golgi impregnation technique. Here, we present a comprehensive description of individual elements composing the third optic neuropil, the lobula, of that same species. We characterized a wide variety of elements (140 types) including input terminals and lobula columnar, centrifugal and input columnar elements. Results reveal a very dense and complex neuropil. We found a frequently impregnated input element (suggesting a supernumerary cartridge representation) that arborizes in the third layer of the lobula and that presents four variants each with ramifications organized following one of the four cardinal axes suggesting a role in directional processing. We also describe input elements with two neurites branching in the third layer, probably connecting with the medulla and lobula plate. These facts suggest that this layer is involved in the directional motion detection pathway in crabs. We analyze and discuss our findings considering the similarities and differences found between the layered organization and components of this crustacean lobula and the lobula of insects. This article is protected by copyright. All rights reserved.

Nonlinear, neuronal adaptation in insect vision models improves target discrimination within repetitively moving backgrounds

Neurons which respond selectively to small moving targets, even against a cluttered background, have been identified in several insect species. To investigate what underlies these robust and highly selective responses, researchers have probed the neuronal circuitry in target-detecting, visual pathways. Observations in flies reveal nonlinear adaptation over time, composed of a fast onset and gradual decay. This adaptive processing is seen in both of the independent, parallel pathways encoding either luminance increments (ON channel) or decrements (OFF channel). The functional significance of this adaptive phenomenon has not been determined from physiological studies, though the asymmetrical time course suggests a role in suppressing responses to repetitive stimuli. We tested this possibility by comparing an implementation of fast adaptation against alternatives, using a model of insect 'elementary small target motion detectors'. We conducted target-detecting simulations on various natural backgrounds, that were shifted via several movement profiles (and target velocities). Using performance metrics, we confirmed that the fast adaptation observed in neuronal systems enhances target detection against a repetitively moving background. Such background movement would be encountered via natural ego-motion as the insect travels through the world. These findings show that this form of nonlinear, fast-adaptation (suitably implementable via cellular biophysics) plays a role analogous to background subtraction techniques in conventional computer vision.

Modeling Nonlinear Dendritic Processing of Facilitation in a Dragonfly Target-Tracking Neuron

Dragonflies are highly skilled and successful aerial predators that are even capable of selectively attending to one target within a swarm. Detection and tracking of prey is likely to be driven by small target motion detector (STMD) neurons identified from several insect groups. Prior work has shown that dragonfly STMD responses are facilitated by targets moving on a continuous path, enhancing the response gain at the present and predicted future location of targets. In this study, we combined detailed morphological data with computational modeling to test whether a combination of dendritic morphology and nonlinear properties of NMDA receptors could explain these observations. We developed a hybrid computational model of neurons within the dragonfly optic lobe, which integrates numerical and morphological components. The model was able to generate potent facilitation for targets moving on continuous trajectories, including a localized spotlight of maximal sensitivity close to the last seen target location, as also measured during in vivo recordings. The model did not, however, include a mechanism capable of producing a traveling or spreading wave of facilitation. Our data support a strong role for the high dendritic density seen in the dragonfly neuron in enhancing non-linear facilitation. An alternative model based on the morphology of an unrelated type of motion processing neuron from a dipteran fly required more than three times higher synaptic gain in order to elicit similar levels of facilitation, despite having only 20% fewer synapses. Our data support a potential role for NMDA receptors in target tracking and also demonstrate the feasibility of combining biologically plausible dendritic computations with more abstract computational models for basic processing as used in earlier studies.

The Radically Embodied Conscious Cybernetic Bayesian Brain: From Free Energy to Free Will and Back Again

Drawing from both enactivist and cognitivist perspectives on mind, I propose that explaining teleological phenomena may require reappraising both "Cartesian theaters" and mental homunculi in terms of embodied self-models (ESMs), understood as body maps with agentic properties, functioning as predictive-memory systems and cybernetic controllers. Quasi-homuncular ESMs are suggested to constitute a major organizing principle for neural architectures due to their initial and ongoing significance for solutions to inference problems in cognitive (and affective) development. Embodied experiences provide foundational lessons in learning curriculums in which agents explore increasingly challenging problem spaces, so answering an unresolved question in Bayesian cognitive science: what are biologically plausible mechanisms for equipping learners with sufficiently powerful inductive biases to adequately constrain inference spaces? Drawing on models from neurophysiology, psychology, and developmental robotics, I describe how embodiment provides fundamental sources of empirical priors (as reliably learnable posterior expectations). If ESMs play this kind of foundational role in cognitive development, then bidirectional linkages will be found between all sensory modalities and frontal-parietal control hierarchies, so infusing all senses with somatic-motoric properties, thereby structuring all perception by relevant affordances, so solving frame problems for embodied agents. Drawing upon the Free Energy Principle and Active Inference framework, I describe a particular mechanism for intentional action selection via consciously imagined (and explicitly represented) goal realization, where contrasts between desired and present states influence ongoing policy selection via predictive coding mechanisms and backward-chained imaginings (as self-realizing predictions). This embodied developmental legacy suggests a mechanism by which imaginings can be intentionally shaped by (internalized) partially-expressed motor acts, so providing means of agentic control for attention, working memory, imagination, and behavior. I further describe the nature(s) of mental causation and self-control, and also provide an account of readiness potentials in Libet paradigms wherein conscious intentions shape causal streams leading to enaction. Finally, I provide neurophenomenological handlings of prototypical qualia including pleasure, pain, and desire in terms of self-annihilating free energy gradients via quasi-synesthetic interoceptive active inference. In brief, this manuscript is intended to illustrate how radically embodied minds may create foundations for intelligence (as capacity for learning and inference), consciousness (as somatically-grounded self-world modeling), and will (as deployment of predictive models for enacting valued goals).

The lobula plate is exclusive to insects

Just one superorder of insects is known to possess a neuronal network that mediates extremely rapid reactions in flight in response to changes in optic flow. Research on the identity and functional organization of this network has over the course of almost half a century focused exclusively on the order Diptera, a member of the approximately 300-million-year-old clade Holometabola defined by its mode of development. However, it has been broadly claimed that the pivotal neuropil containing the network, the lobula plate, originated in the Cambrian before the divergence of Hexapoda and Crustacea from a mandibulate ancestor. This essay defines the traits that designate the lobula plate and argues against a homologue in Crustacea. It proposes that the origin of the lobula plate is relatively recent and may relate to the origin of flight.

Spike bursting in a dragonfly target-detecting neuron

Dragonflies visually detect prey and conspecifics, rapidly pursuing these targets via acrobatic flights. Over many decades, studies have investigated the elaborate neuronal circuits proposed to underlie this rapid behaviour. A subset of dragonfly visual neurons exhibit exquisite tuning to small, moving targets even when presented in cluttered backgrounds. In prior work, these neuronal responses were quantified by computing the rate of spikes fired during an analysis window of interest. However, neuronal systems can utilize a variety of neuronal coding principles to signal information, so a spike train’s information content is not necessarily encapsulated by spike rate alone. One example of this is burst coding, where neurons fire rapid bursts of spikes, followed by a period of inactivity. Here we show that the most studied target-detecting neuron in dragonflies, CSTMD1, responds to moving targets with a series of spike bursts. This spiking activity differs from those in other identified visual neurons in the dragonfly, indicative of different physiological mechanisms underlying CSTMD1’s spike generation. Burst codes present several advantages and disadvantages compared to other coding approaches. We propose functional implications of CSTMD1’s burst coding activity and show that spike bursts enhance the robustness of target-evoked responses.

The visual neuroecology of anisoptera

Dragonflies belong to the oldest known lineage of flying animals, found across the globe around streams, ponds and forests. They are insect predators, specialising in ambush attack as aquatic larvae and rapid pursuit as adults. Dragonfly adults hunt amidst swarms in conditions that confuse many predatory species, and exhibit capture rates above 90%. Underlying the performance of such a remarkable predator is a finely tuned visual system capable of tracking targets amidst distractors and background clutter. The dragonfly performs a complex repertoire of flight behaviours, from near-motionless hovering to acute turns at high speeds. Here, we review the optical, neuronal, and behavioural adaptations that underlie the dragonflies' ability to achieve such remarkable predatory success.

Oscillations in the central brain of Drosophila are phase locked to attended visual features

Object-based attention describes the brain's capacity to prioritize one set of stimuli while ignoring others. Human research suggests that the binding of diverse stimuli into one attended percept requires phase-locked oscillatory activity in the brain. Even insects display oscillatory brain activity during visual attention tasks, but it is unclear if neural oscillations in insects are selectively correlated to different features of attended objects. We addressed this question by recording local field potentials in the Drosophila central complex, a brain structure involved in visual navigation and decision making. We found that attention selectively increased the neural gain of visual features associated with attended objects and that attention could be redirected to unattended objects by activation of a reward circuit. Attention was associated with increased beta (20- to 30-Hz) oscillations that selectively locked onto temporal features of the attended visual objects. Our results suggest a conserved function for the beta frequency range in regulating selective attention to salient visual features.

Evidence of predictive selective attention in fiddler crabs during escape in the natural environment

Selective attention is of fundamental relevance to animals for performing a diversity of tasks such as mating, feeding, predation and avoiding predators. Within natural environments, prey animals are often exposed to multiple, simultaneous threats, which significantly complicates the decision-making process. However, selective attention is rarely studied in complex, natural environments or in the context of escape responses. We therefore asked how relatively simple animals integrate the information from multiple, concurrent threatening events. Do they identify and respond only to what they perceive as the most dangerous threat, or do they respond to multiple stimuli at the same time? Do simultaneous threats evoke an earlier or stronger response than single threats? We investigated these questions by conducting field experiments and compared escape responses of the fiddler crab Gelasimus dampieri when faced with either a single or two simultaneously approaching dummy predators. We used the dummies' approach trajectories to manipulate the threat level; a directly approaching dummy indicated higher risk while a tangentially approaching dummy that passed the crabs at a distance represented a lower risk. The crabs responded later, but on average more often, when approached more directly. However, when confronted with the two dummies simultaneously, the crabs responded as if approached only by the directly approaching dummy. This suggests that the crabs are able to predict how close the dummy's trajectory is to a collision course and selectively suppress their normally earlier response to the less dangerous dummy. We thus provide evidence of predictive selective attention within a natural environment.

Insect Neurobiology: Divergent Neural Computations in Predatory Insects

A comparative approach to neuroscience can greatly increase our understanding of how mechanisms map onto behaviour. A new study comparing two predatory insects demonstrates how neurons that are homologous can nonetheless mediate different computations and behaviour.