Insect detection of small targets moving in visual clutter

Visually guided swarm motion coordination via insect-inspired small target motion reactions

Despite progress developing experimentally-consistent models of insect in-flight sensing and feedback for individual agents, a lack of systematic understanding of the multi-agent and group performance of the resulting bio-inspired sensing and feedback approaches remains a barrier to robotic swarm implementations. This study introduces the small-target motion reactive (STMR) swarming approach by designing a concise engineering model of the small target motion detector (STMD) neurons found in insect lobula complexes. The STMD neuron model identifies the bearing angle at which peak optic flow magnitude occurs, and this angle is used to design an output feedback switched control system. A theoretical stability analysis provides bi-agent stability and state boundedness in group contexts. The approach is simulated and implemented on ground vehicles for validation and behavioral studies. The results indicate despite having the lowest connectivity of contemporary approaches (each agent instantaneously regards only a single neighbor), STMR achieves collective group motion. STMR group level metric analysis also highlights continuously varying polarization and decreasing heading variance.

Descending neurons of the hoverfly respond to pursuits of artificial targets

Many animals use motion vision information to control dynamic behaviors. Predatory animals, for example, show an exquisite ability to detect rapidly moving prey, followed by pursuit and capture. Such target detection is not only used by predators but is also important in conspecific interactions, such as for male hoverflies defending their territories against conspecific intruders. Visual target detection is believed to be subserved by specialized target-tuned neurons found in a range of species, including vertebrates and arthropods. However, how these target-tuned neurons respond to actual pursuit trajectories is currently not well understood. To redress this, we recorded extracellularly from target-selective descending neurons (TSDNs) in male Eristalis tenax hoverflies. We show that they have dorso-frontal receptive fields with a preferred direction up and away from the visual midline. We reconstructed visual flow fields as experienced during pursuits of artificial targets (black beads). We recorded TSDN responses to six reconstructed pursuits and found that each neuron responded consistently at remarkably specific time points but that these time points differed between neurons. We found that the observed spike probability was correlated with the spike probability predicted from each neuron's receptive field and size tuning. Interestingly, however, the overall response rate was low, with individual neurons responding to only a small part of each reconstructed pursuit. In contrast, the TSDN population responded to substantially larger proportions of the pursuits but with lower probability. This large variation between neurons could be useful if different neurons control different parts of the behavioral output.

Attention and Prediction-Guided Motion Detection for Low-Contrast Small Moving Targets

Small target motion detection within complex natural environments is an extremely challenging task for autonomous robots. Surprisingly, the visual systems of insects have evolved to be highly efficient in detecting mates and tracking prey, even though targets occupy as small as a few degrees of their visual fields. The excellent sensitivity to small target motion relies on a class of specialized neurons, called small target motion detectors (STMDs). However, existing STMD-based models are heavily dependent on visual contrast and perform poorly in complex natural environments, where small targets generally exhibit extremely low contrast against neighboring backgrounds. In this article, we develop an attention-and-prediction-guided visual system to overcome this limitation. The developed visual system comprises three main subsystems, namely: 1) an attention module; 2) an STMD-based neural network; and 3) a prediction module. The attention module searches for potential small targets in the predicted areas of the input image and enhances their contrast against a complex background. The STMD-based neural network receives the contrast-enhanced image and discriminates small moving targets from background false positives. The prediction module foresees future positions of the detected targets and generates a prediction map for the attention module. The three subsystems are connected in a recurrent architecture, allowing information to be processed sequentially to activate specific areas for small target detection. Extensive experiments on synthetic and real-world datasets demonstrate the effectiveness and superiority of the proposed visual system for detecting small, low-contrast moving targets against complex natural environments.

Motion perception based on ON/OFF channels: A survey

Motion perception is an essential ability for animals and artificially intelligent systems interacting effectively, safely with surrounding objects and environments. Biological visual systems, that have naturally evolved over hundreds-million years, are quite efficient and robust for motion perception, whereas artificial vision systems are far from such capability. This paper argues that the gap can be significantly reduced by formulation of ON/OFF channels in motion perception models encoding luminance increment (ON) and decrement (OFF) responses within receptive field, separately. Such signal-bifurcating structure has been found in neural systems of many animal species articulating early motion is split and processed in segregated pathways. However, the corresponding biological substrates, and the necessity for artificial vision systems have never been elucidated together, leaving concerns on uniqueness and advantages of ON/OFF channels upon building dynamic vision systems to address real world challenges. This paper highlights the importance of ON/OFF channels in motion perception through surveying current progress covering both neuroscience and computationally modelling works with applications. Compared to related literature, this paper for the first time provides insights into implementation of different selectivity to directional motion of looming, translating, and small-sized target movement based on ON/OFF channels in keeping with soundness and robustness of biological principles. Existing challenges and future trends of such bio-plausible computational structure for visual perception in connection with hotspots of machine learning, advanced vision sensors like event-driven camera finally are discussed.

Dual Receptive Fields Underlying Target and Wide-Field Motion Sensitivity in Looming-Sensitive Descending Neurons

Responding rapidly to visual stimuli is fundamental for many animals. For example, predatory birds and insects alike have amazing target detection abilities, with incredibly short neural and behavioral delays, enabling efficient prey capture. Similarly, looming objects need to be rapidly avoided to ensure immediate survival, as these could represent approaching predators. Male Eristalis tenax hoverflies are nonpredatory, highly territorial insects that perform high-speed pursuits of conspecifics and other territorial intruders. During the initial stages of the pursuit, the retinal projection of the target is very small, but this grows to a larger object before physical interaction. Supporting such behaviors, E. tenax and other insects have both target-tuned and loom-sensitive neurons in the optic lobes and the descending pathways. We here show that these visual stimuli are not necessarily encoded in parallel. Indeed, we describe a class of descending neurons that respond to small targets, to looming and to wide-field stimuli. We show that these descending neurons have two distinct receptive fields where the dorsal receptive field is sensitive to the motion of small targets and the ventral receptive field responds to larger objects or wide-field stimuli. Our data suggest that the two receptive fields have different presynaptic input, where the inputs are not linearly summed. This novel and unique arrangement could support different behaviors, including obstacle avoidance, flower landing, and target pursuit or capture.

Enhancing LGMD's Looming Selectivity for UAV With Spatial-Temporal Distributed Presynaptic Connections

Collision detection is one of the most challenging tasks for unmanned aerial vehicles (UAVs). This is especially true for small or micro-UAVs due to their limited computational power. In nature, flying insects with compact and simple visual systems demonstrate their remarkable ability to navigate and avoid collision in complex environments. A good example of this is provided by locusts. They can avoid collisions in a dense swarm through the activity of a motion-based visual neuron called the Lobula giant movement detector (LGMD). The defining feature of the LGMD neuron is its preference for looming. As a flying insect's visual neuron, LGMD is considered to be an ideal basis for building UAV's collision detecting system. However, existing LGMD models cannot distinguish looming clearly from other visual cues, such as complex background movements caused by UAV agile flights. To address this issue, we proposed a new model implementing distributed spatial-temporal synaptic interactions, which is inspired by recent findings in locusts' synaptic morphology. We first introduced the locally distributed excitation to enhance the excitation caused by visual motion with preferred velocities. Then, radially extending temporal latency for inhibition is incorporated to compete with the distributed excitation and selectively suppress the nonpreferred visual motions. This spatial-temporal competition between excitation and inhibition in our model is, therefore, tuned to preferred image angular velocity representing looming rather than background movements with these distributed synaptic interactions. Systematic experiments have been conducted to verify the performance of the proposed model for UAV agile flights. The results have demonstrated that this new model enhances the looming selectivity in complex flying scenes considerably and has the potential to be implemented on embedded collision detection systems for small or micro-UAVs.

Bioinspired figure-ground discrimination via visual motion smoothing

Flies detect and track moving targets among visual clutter, and this process mainly relies on visual motion. Visual motion is analyzed or computed with the pathway from the retina to T4/T5 cells. The computation of local directional motion was formulated as an elementary movement detector (EMD) model more than half a century ago. Solving target detection or figure-ground discrimination problems can be equivalent to extracting boundaries between a target and the background based on the motion discontinuities in the output of a retinotopic array of EMDs. Individual EMDs cannot measure true velocities, however, due to their sensitivity to pattern properties such as luminance contrast and spatial frequency content. It remains unclear how local directional motion signals are further integrated to enable figure-ground discrimination. Here, we present a computational model inspired by fly motion vision. Simulations suggest that the heavily fluctuating output of an EMD array is naturally surmounted by a lobula network, which is hypothesized to be downstream of the local motion detectors and have parallel pathways with distinct directional selectivity. The lobula network carries out a spatiotemporal smoothing operation for visual motion, especially across time, enabling the segmentation of moving figures from the background. The model qualitatively reproduces experimental observations in the visually evoked response characteristics of one type of lobula columnar (LC) cell. The model is further shown to be robust to natural scene variability. Our results suggest that the lobula is involved in local motion-based target detection.

Optic flow enrichment via Drosophila head and retina motions to support inflight position regulation

Developing a functional description of the neural control circuits and visual feedback paths underlying insect flight behaviors is an active research area. Feedback controllers incorporating engineering models of the insect visual system outputs have described some flight behaviors, yet they do not explain how insects are able to stabilize their body position relative to nearby targets such as neighbors or forage sources, especially in challenging environments in which optic flow is poor. The insect experimental community is simultaneously recording a growing library of in-flight head and eye motions that may be linked to increased perception. This study develops a quantitative model of the optic flow experienced by a flying insect or robot during head yawing rotations (distinct from lateral peering motions in previous work) with a single other target in view. This study then applies a model of insect visuomotor feedback to show via analysis and simulation of five species that these head motions sufficiently enrich the optic flow and that the output feedback can provide relative position regulation relative to the single target (asymptotic stability). In the simplifying case of pure rotation relative to the body, theoretical analysis provides a stronger stability guarantee. The results are shown to be robust to anatomical neck angle limits and body vibrations, persist with more detailed Drosophila lateral-directional flight dynamics simulations, and generalize to recent retinal motion studies. Together, these results suggest that the optic flow enrichment provided by head or pseudopupil rotation could be used in an insect's neural processing circuit to enable position regulation.

A Time-Delay Feedback Neural Network for Discriminating Small, Fast-Moving Targets in Complex Dynamic Environments

Discriminating small moving objects within complex visual environments is a significant challenge for autonomous micro-robots that are generally limited in computational power. By exploiting their highly evolved visual systems, flying insects can effectively detect mates and track prey during rapid pursuits, even though the small targets equate to only a few pixels in their visual field. The high degree of sensitivity to small target movement is supported by a class of specialized neurons called small target motion detectors (STMDs). Existing STMD-based computational models normally comprise four sequentially arranged neural layers interconnected via feedforward loops to extract information on small target motion from raw visual inputs. However, feedback, another important regulatory circuit for motion perception, has not been investigated in the STMD pathway and its functional roles for small target motion detection are not clear. In this article, we propose an STMD-based neural network with feedback connection (feedback STMD), where the network output is temporally delayed, then fed back to the lower layers to mediate neural responses. We compare the properties of the model with and without the time-delay feedback loop and find that it shows a preference for high-velocity objects. Extensive experiments suggest that the feedback STMD achieves superior detection performance for fast-moving small targets, while significantly suppressing background false positive movements which display lower velocities. The proposed feedback model provides an effective solution in robotic visual systems for detecting fast-moving small targets that are always salient and potentially threatening.

Exploration of motion inhibition for the suppression of false positives in biologically inspired small target detection algorithms from a moving platform

Detecting small moving targets against a cluttered background in visual data is a challenging task. The main problems include spatio-temporal target contrast enhancement, background suppression and accurate target segmentation. When targets are at great distances from a non-stationary camera, the difficulty of these challenges increases. In such cases the moving camera can introduce large spatial changes between frames which may cause issues in temporal algorithms; furthermore targets can approach a single pixel, thereby affecting spatial methods. Previous literature has shown that biologically inspired methods, based on the vision systems of insects, are robust to such conditions. It has also been shown that the use of divisive optic-flow inhibition with these methods enhances the detectability of small targets. However, the location within the visual pathway the inhibition should be applied was ambiguous. In this paper, we investigated the tunings of some of the optic-flow filters and use of a nonlinear transform on the optic-flow signal to modify motion responses for the purpose of suppressing false positives and enhancing small target detection. Additionally, we looked at multiple locations within the biologically inspired vision (BIV) algorithm where inhibition could further enhance detection performance, and look at driving the nonlinear transform with a global motion estimate. To get a better understanding of how the BIV algorithm performs, we compared to other state-of-the-art target detection algorithms, and look at how their performance can be enhanced with the optic-flow inhibition. Our explicit use of the nonlinear inhibition allows for the incorporation of a wider dynamic range of inhibiting signals, along with spatio-temporal filter refinement, which further increases target-background discrimination in the presence of camera motion. Extensive experiments shows that our proposed approach achieves an improvement of 25% over linearly conditioned inhibition schemes and 2.33 times the detection performance of the BIV model without inhibition. Moreover, our approach achieves between 10 and 104 times better detection performance compared to any conventional state-of-the-art moving object detection algorithm applied to the same, highly cluttered and moving scenes. Applying the nonlinear inhibition to other algorithms showed that their performance can be increased by up to 22 times. These findings show that the application of optic-flow- based signal suppression should be applied to enhance target detection from moving platforms. Furthermore, they indicate where best to look for evidence of such signals within the insect brain.

Visual and motor signatures of locomotion dynamically shape a population code for feature detection in Drosophila

Natural vision is dynamic: as an animal moves, its visual input changes dramatically. How can the visual system reliably extract local features from an input dominated by self-generated signals? In Drosophila, diverse local visual features are represented by a group of projection neurons with distinct tuning properties. Here, we describe a connectome-based volumetric imaging strategy to measure visually evoked neural activity across this population. We show that local visual features are jointly represented across the population, and a shared gain factor improves trial-to-trial coding fidelity. A subset of these neurons, tuned to small objects, is modulated by two independent signals associated with self-movement, a motor-related signal, and a visual motion signal associated with rotation of the animal. These two inputs adjust the sensitivity of these feature detectors across the locomotor cycle, selectively reducing their gain during saccades and restoring it during intersaccadic intervals. This work reveals a strategy for reliable feature detection during locomotion.

Mathematical study of neural feedback roles in small target motion detection

Building an efficient and reliable small target motion detection visual system is challenging for artificial intelligence robotics because a small target only occupies few pixels and hardly displays visual features in images. Biological visual systems that have evolved over millions of years could be ideal templates for designing artificial visual systems. Insects benefit from a class of specialized neurons, called small target motion detectors (STMDs), which endow them with an excellent ability to detect small moving targets against a cluttered dynamic environment. Some bio-inspired models featured in feed-forward information processing architectures have been proposed to imitate the functions of the STMD neurons. However, feedback, a crucial mechanism for visual system regulation, has not been investigated deeply in the STMD-based neural circuits and its roles in small target motion detection remain unclear. In this paper, we propose a time-delay feedback STMD model for small target motion detection in complex backgrounds. The main contributions of this study are as follows. First, a feedback pathway is designed by transmitting information from output-layer neurons to lower-layer interneurons in the STMD pathway and the role of the feedback is analyzed from the view of mathematical analysis. Second, to estimate the feedback constant, the existence and uniqueness of solutions for nonlinear dynamical systems formed by feedback loop are analyzed via Schauder's fixed point theorem and contraction mapping theorem. Finally, an iterative algorithm is designed to solve the nonlinear problem and the performance of the proposed model is tested by experiments. Experimental results demonstrate that the feedback is able to weaken background false positives while maintaining a minor effect on small targets. It outperforms existing STMD-based models regarding the accuracy of fast-moving small target detection in visual clutter. The proposed feedback approach could inspire the relevant modeling of robust motion perception robotics visual systems.

Dragonfly Neurons Selectively Attend to Targets Within Natural Scenes

Aerial predators, such as the dragonfly, determine the position and movement of their prey even when both are moving through complex, natural scenes. This task is likely supported by a group of neurons in the optic lobe which respond to moving targets that subtend less than a few degrees. These Small Target Motion Detector (STMD) neurons are tuned to both target size and velocity, whilst also exhibiting facilitated responses to targets traveling along continuous trajectories. When presented with a pair of targets, some STMDs generate spiking activity that represent a competitive selection of one target, as if the alternative does not exist (i.e., selective attention). Here, we describe intracellular responses of CSTMD1 (an identified STMD) to the visual presentation of targets embedded within cluttered, natural scenes. We examine CSTMD1 response changes to target contrast, as well as a range of target and background velocities. We find that background motion affects CSTMD1 responses via the competitive selection between features within the natural scene. Here, robust discrimination of our artificially embedded "target" is limited to scenarios when its velocity is matched to, or greater than, the background velocity. Additionally, the background's direction of motion affects discriminability, though not in the manner observed in STMDs of other flying insects. Our results highlight that CSTMD1's competitive responses are to those features best matched to the neuron's underlying spatiotemporal tuning, whether from the embedded target or other features in the background clutter. In many scenarios, CSTMD1 responds robustly to targets moving through cluttered scenes. However, whether this neuronal system could underlie the task of competitively selecting slow moving prey against fast-moving backgrounds remains an open question.

A Looming Spatial Localization Neural Network Inspired by MLG1 Neurons in the Crab Neohelice

Similar to most visual animals, the crab Neohelice granulata relies predominantly on visual information to escape from predators, to track prey and for selecting mates. It, therefore, needs specialized neurons to process visual information and determine the spatial location of looming objects. In the crab Neohelice granulata, the Monostratified Lobula Giant type1 (MLG1) neurons have been found to manifest looming sensitivity with finely tuned capabilities of encoding spatial location information. MLG1s neuronal ensemble can not only perceive the location of a looming stimulus, but are also thought to be able to influence the direction of movement continuously, for example, escaping from a threatening, looming target in relation to its position. Such specific characteristics make the MLG1s unique compared to normal looming detection neurons in invertebrates which can not localize spatial looming. Modeling the MLG1s ensemble is not only critical for elucidating the mechanisms underlying the functionality of such neural circuits, but also important for developing new autonomous, efficient, directionally reactive collision avoidance systems for robots and vehicles. However, little computational modeling has been done for implementing looming spatial localization analogous to the specific functionality of MLG1s ensemble. To bridge this gap, we propose a model of MLG1s and their pre-synaptic visual neural network to detect the spatial location of looming objects. The model consists of 16 homogeneous sectors arranged in a circular field inspired by the natural arrangement of 16 MLG1s' receptive fields to encode and convey spatial information concerning looming objects with dynamic expanding edges in different locations of the visual field. Responses of the proposed model to systematic real-world visual stimuli match many of the biological characteristics of MLG1 neurons. The systematic experiments demonstrate that our proposed MLG1s model works effectively and robustly to perceive and localize looming information, which could be a promising candidate for intelligent machines interacting within dynamic environments free of collision. This study also sheds light upon a new type of neuromorphic visual sensor strategy that can extract looming objects with locational information in a quick and reliable manner.

Neural organization of the third optic neuropil, the lobula, in the highly visual semiterrestrial crab Neohelice granulata

The visual neuropils (lamina, medulla and lobula complex), of malacostracan crustaceans and hexapods have many organizational principles, cell types and functional properties in common. Information about the cellular elements that compose the crustacean lobula is scarce especially when focusing on small columnar cells. Semiterrestrial crabs possess a highly developed visual system and display conspicuous visually guided behaviors. In particular, Neohelice granulata has been previously used to describe the cellular components of the first two optic neuropils using Golgi impregnation technique. Here, we present a comprehensive description of individual elements composing the third optic neuropil, the lobula, of that same species. We characterized a wide variety of elements (140 types) including input terminals and lobula columnar, centrifugal and input columnar elements. Results reveal a very dense and complex neuropil. We found a frequently impregnated input element (suggesting a supernumerary cartridge representation) that arborizes in the third layer of the lobula and that presents four variants each with ramifications organized following one of the four cardinal axes suggesting a role in directional processing. We also describe input elements with two neurites branching in the third layer, probably connecting with the medulla and lobula plate. These facts suggest that this layer is involved in the directional motion detection pathway in crabs. We analyze and discuss our findings considering the similarities and differences found between the layered organization and components of this crustacean lobula and the lobula of insects. This article is protected by copyright. All rights reserved.

Nonlinear, neuronal adaptation in insect vision models improves target discrimination within repetitively moving backgrounds

Neurons which respond selectively to small moving targets, even against a cluttered background, have been identified in several insect species. To investigate what underlies these robust and highly selective responses, researchers have probed the neuronal circuitry in target-detecting, visual pathways. Observations in flies reveal nonlinear adaptation over time, composed of a fast onset and gradual decay. This adaptive processing is seen in both of the independent, parallel pathways encoding either luminance increments (ON channel) or decrements (OFF channel). The functional significance of this adaptive phenomenon has not been determined from physiological studies, though the asymmetrical time course suggests a role in suppressing responses to repetitive stimuli. We tested this possibility by comparing an implementation of fast adaptation against alternatives, using a model of insect 'elementary small target motion detectors'. We conducted target-detecting simulations on various natural backgrounds, that were shifted via several movement profiles (and target velocities). Using performance metrics, we confirmed that the fast adaptation observed in neuronal systems enhances target detection against a repetitively moving background. Such background movement would be encountered via natural ego-motion as the insect travels through the world. These findings show that this form of nonlinear, fast-adaptation (suitably implementable via cellular biophysics) plays a role analogous to background subtraction techniques in conventional computer vision.

Facilitation of neural responses to targets moving against optic flow

Target detection in visual clutter is a difficult computational task that insects, with their poor spatial resolution compound eyes and small brains, do successfully and with extremely short behavioral delays. We here show that the responses of target selective descending neurons are attenuated by background motion in the same direction as target motion but facilitated by background motion in the opposite direction. This finding is important for understanding how target pursuit can occur in tandem with gaze stabilization. Indeed, the neural facilitation would come into effect if the hoverfly is subjected to background motion in one direction but the target it is pursuing moves in the opposite direction and could therefore be used to override gaze stabilizing corrective turns.

For the human observer, it can be difficult to follow the motion of small objects, especially when they move against background clutter. In contrast, insects efficiently do this, as evidenced by their ability to capture prey, pursue conspecifics, or defend territories, even in highly textured surrounds. We here recorded from target selective descending neurons (TSDNs), which likely subserve these impressive behaviors. To simulate the type of optic flow that would be generated by the pursuer’s own movements through the world, we used the motion of a perspective corrected sparse dot field. We show that hoverfly TSDN responses to target motion are suppressed when such optic flow moves syn-directional to the target. Indeed, neural responses are strongly suppressed when targets move over either translational sideslip or rotational yaw. More strikingly, we show that TSDNs are facilitated by optic flow moving counterdirectional to the target, if the target moves horizontally. Furthermore, we show that a small, frontal spatial window of optic flow is enough to fully facilitate or suppress TSDN responses to target motion. We argue that such TSDN response facilitation could be beneficial in modulating corrective turns during target pursuit.

Modeling Nonlinear Dendritic Processing of Facilitation in a Dragonfly Target-Tracking Neuron

Dragonflies are highly skilled and successful aerial predators that are even capable of selectively attending to one target within a swarm. Detection and tracking of prey is likely to be driven by small target motion detector (STMD) neurons identified from several insect groups. Prior work has shown that dragonfly STMD responses are facilitated by targets moving on a continuous path, enhancing the response gain at the present and predicted future location of targets. In this study, we combined detailed morphological data with computational modeling to test whether a combination of dendritic morphology and nonlinear properties of NMDA receptors could explain these observations. We developed a hybrid computational model of neurons within the dragonfly optic lobe, which integrates numerical and morphological components. The model was able to generate potent facilitation for targets moving on continuous trajectories, including a localized spotlight of maximal sensitivity close to the last seen target location, as also measured during in vivo recordings. The model did not, however, include a mechanism capable of producing a traveling or spreading wave of facilitation. Our data support a strong role for the high dendritic density seen in the dragonfly neuron in enhancing non-linear facilitation. An alternative model based on the morphology of an unrelated type of motion processing neuron from a dipteran fly required more than three times higher synaptic gain in order to elicit similar levels of facilitation, despite having only 20% fewer synapses. Our data support a potential role for NMDA receptors in target tracking and also demonstrate the feasibility of combining biologically plausible dendritic computations with more abstract computational models for basic processing as used in earlier studies.

An Algorithmic Approach to Natural Behavior

Uncovering the mechanisms and implications of natural behavior is a goal that unites many fields of biology. Yet, the diversity, flexibility, and multi-scale nature of these behaviors often make understanding elusive. Here, we review studies of animal pursuit and evasion - two special classes of behavior where theory-driven experiments and new modeling techniques are beginning to uncover the general control principles underlying natural behavior. A key finding of these studies is that intricate sequences of pursuit and evasion behavior can often be constructed through simple, repeatable rules that link sensory input to motor output: we refer to these rules as behavioral algorithms. Identifying and mathematically characterizing these algorithms has led to important insights, including the discovery of guidance rules that attacking predators use to intercept mobile prey, and coordinated neural and biomechanical mechanisms that animals use to avoid impending collisions. Here, we argue that algorithms provide a good starting point for studies of natural behavior more generally. Rather than beginning at the neural or ecological levels of organization, we advocate starting in the middle, where the algorithms that link sensory input to behavioral output can provide a solid foundation from which to explore both the implementation and the ecological outcomes of behavior. We review insights that have been gained through such an algorithmic approach to pursuit and evasion behaviors. From these, we synthesize theoretical principles and lay out key modeling tools needed to apply an algorithmic approach to the study of other complex natural behaviors.

Detecting Small Size and Minimal Thermal Signature Targets in Infrared Imagery Using Biologically Inspired Vision

Thermal infrared imaging provides an effective sensing modality for detecting small moving objects at long range. Typical challenges that limit the efficiency and robustness of the detection performance include sensor noise, minimal target contrast and cluttered backgrounds. These issues become more challenging when the targets are of small physical size and present minimal thermal signatures. In this paper, we experimentally show that a four-stage biologically inspired vision (BIV) model of the flying insect visual system have an excellent ability to overcome these challenges simultaneously. The early two stages of the model suppress spatio-temporal clutter and enhance spatial target contrast while compressing the signal in a computationally manageable bandwidth. The later two stages provide target motion enhancement and sub-pixel motion detection capabilities. To show the superiority of the BIV target detector over existing traditional detection methods, we perform extensive experiments and performance comparisons using high bit-depth, real-world infrared image sequences of small size and minimal thermal signature targets at long ranges. Our results show that the BIV target detector significantly outperformed 10 conventional spatial-only and spatiotemporal methods for infrared small target detection. The BIV target detector resulted in over 25 dB improvement in the median signal-to-clutter-ratio over the raw input and achieved 43% better detection rate than the best performing existing method.

What gaze direction can tell us about cognitive processes in invertebrates

Most visually guided animals shift their gaze using body movements, eye movements, or both to gather information selectively from their environments. Psychological studies of eye movements have advanced our understanding of perceptual and cognitive processes that mediate visual attention in humans and other vertebrates. However, much less is known about how these processes operate in other organisms, particularly invertebrates. We here make the case that studies of invertebrate cognition can benefit by adding precise measures of gaze direction. To accomplish this, we briefly review the human visual attention literature and outline four research themes and several experimental paradigms that could be extended to invertebrates. We briefly review selected studies where the measurement of gaze direction in invertebrates has provided new insights, and we suggest future areas of exploration.

Insect Neurobiology: How a Small Spot Stops a Fly

Animals often respond to small moving features very differently than they do to large moving fields. A new study finds that viewing small spots causes walking fruit flies to stop in their tracks, and identifies the cellular pathway that processes this signal.

Non-canonical Receptive Field Properties and Neuromodulation of Feature-Detecting Neurons in Flies

Several fundamental aspects of motion vision circuitry are prevalent across flies and mice. Both taxa segregate ON and OFF signals. For any given spatial pattern, motion detectors in both taxa are tuned to speed, selective for one of four cardinal directions, and modulated by catecholamine neurotransmitters. These similarities represent conserved, canonical properties of the functional circuits and computational algorithms for motion vision. Less is known about feature detectors, including how receptive field properties differ from the motion pathway or whether they are under neuromodulatory control to impart functional plasticity for the detection of salient objects from a moving background. Here, we investigated 19 types of putative feature selective lobula columnar (LC) neurons in the optic lobe of the fruit fly Drosophila melanogaster to characterize divergent properties of feature selection. We identified LC12 and LC15 as feature detectors. LC15 encodes moving bars, whereas LC12 is selective for the motion of discrete objects, mostly independent of size. Neither is selective for contrast polarity, speed, or direction, highlighting key differences in the underlying algorithms for feature detection and motion vision. We show that the onset of background motion suppresses object responses by LC12 and LC15. Surprisingly, the application of octopamine, which is released during flight, reverses the suppressive influence of background motion, rendering both LCs able to track moving objects superimposed against background motion. Our results provide a comparative framework for the function and modulation of feature detectors and new insights into the underlying neuronal mechanisms involved in visual feature detection.

Object-Displacement-Sensitive Visual Neurons Drive Freezing in Drosophila

Visual systems are often equipped with neurons that detect small moving objects, which may represent prey, predators, or conspecifics. Although the processing properties of those neurons have been studied in diverse organisms, links between the proposed algorithms and animal behaviors or circuit mechanisms remain elusive. Here, we have investigated behavioral function, computational algorithm, and neurochemical mechanisms of an object-selective neuron, LC11, in Drosophila. With genetic silencing and optogenetic activation, we show that LC11 is necessary for a visual object-induced stopping behavior in walking flies, a form of short-term freezing, and its activity can promote stopping. We propose a new quantitative model for small object selectivity based on the physiology and anatomy of LC11 and its inputs. The model accurately reproduces LC11 responses by pooling fast-adapting, tightly size-tuned inputs. Direct visualization of neurotransmitter inputs to LC11 confirmed the model conjectures about upstream processing. Our results demonstrate how adaptation can enhance selectivity for behaviorally relevant, dynamic visual features.

Direction Selective Neurons Responsive to Horizontal Motion in a Crab Reflect an Adaptation to Prevailing Movements in Flat Environments

All animals need information about the direction of motion to be able to track the trajectory of a target (prey, predator, cospecific) or to control the course of navigation. This information is provided by direction selective (DS) neurons, which respond to images moving in a unique direction. DS neurons have been described in numerous species including many arthropods. In these animals, the majority of the studies have focused on DS neurons dedicated to processing the optic flow generated during navigation. In contrast, only a few studies were performed on DS neurons related to object motion processing. The crab Neohelice is an established experimental model for the study of neurons involved in visually-guided behaviors. Here, we describe in male crabs of this species a new group of DS neurons that are highly directionally selective to moving objects. The neurons were physiologically and morphologically characterized by intracellular recording and staining in the optic lobe of intact animals. Because of their arborization in the lobula complex, we called these cells lobula complex directional cells (LCDCs). LCDCs also arborize in a previously undescribed small neuropil of the lateral protocerebrum. LCDCs are responsive only to horizontal motion. This nicely fits in the behavioral adaptations of a crab inhabiting a flat, densely crowded environment, where most object motions are generated by neighboring crabs moving along the horizontal plane.SIGNIFICANCE STATEMENT Direction selective (DS) neurons are key to a variety of visual behaviors including, target tracking (preys, predators, cospecifics) and course control. Here, we describe the physiology and morphology of a new group of remarkably directional neurons exclusively responsive to horizontal motion in crabs. These neurons arborize in the lobula complex and in a previously undescribed small neuropil of the lateral protocerebrum. The strong sensitivity of these cells for horizontal motion represents a clear example of functional neuronal adaptation to the lifestyle of an animal inhabiting a flat environment.

A Directionally Selective Small Target Motion Detecting Visual Neural Network in Cluttered Backgrounds

Discriminating targets moving against a cluttered background is a huge challenge, let alone detecting a target as small as one or a few pixels and tracking it in flight. In the insect's visual system, a class of specific neurons, called small target motion detectors (STMDs), have been identified as showing exquisite selectivity for small target motion. Some of the STMDs have also demonstrated direction selectivity which means these STMDs respond strongly only to their preferred motion direction. Direction selectivity is an important property of these STMD neurons which could contribute to tracking small targets such as mates in flight. However, little has been done on systematically modeling these directionally selective STMD neurons. In this paper, we propose a directionally selective STMD-based neural network for small target detection in a cluttered background. In the proposed neural network, a new correlation mechanism is introduced for direction selectivity via correlating signals relayed from two pixels. Then, a lateral inhibition mechanism is implemented on the spatial field for size selectivity of the STMD neurons. Finally, a population vector algorithm is used to encode motion direction of small targets. Extensive experiments showed that the proposed neural network not only is in accord with current biological findings, i.e., showing directional preferences but also worked reliably in detecting the small targets against cluttered backgrounds.

A Robust Visual System for Small Target Motion Detection Against Cluttered Moving Backgrounds

Monitoring small objects against cluttered moving backgrounds is a huge challenge to future robotic vision systems. As a source of inspiration, insects are quite apt at searching for mates and tracking prey, which always appear as small dim speckles in the visual field. The exquisite sensitivity of insects for small target motion, as revealed recently, is coming from a class of specific neurons called small target motion detectors (STMDs). Although a few STMD-based models have been proposed, these existing models only use motion information for small target detection and cannot discriminate small targets from small-target-like background features (named fake features). To address this problem, this paper proposes a novel visual system model (STMD+) for small target motion detection, which is composed of four subsystems-ommatidia, motion pathway, contrast pathway, and mushroom body. Compared with the existing STMD-based models, the additional contrast pathway extracts directional contrast from luminance signals to eliminate false positive background motion. The directional contrast and the extracted motion information by the motion pathway are integrated into the mushroom body for small target discrimination. Extensive experiments showed the significant and consistent improvements of the proposed visual system model over the existing STMD-based models against fake features.

Visual motion sensitivity in descending neurons in the hoverfly

Many animals use motion vision information to control dynamic behaviors. For example, flying insects must decide whether to pursue a prey or not, to avoid a predator, to maintain their current flight trajectory, or to land. The neural mechanisms underlying the computation of visual motion have been particularly well investigated in the fly optic lobes. However, the descending neurons, which connect the optic lobes with the motor command centers of the ventral nerve cord, remain less studied. To address this deficiency, we describe motion vision sensitive descending neurons in the hoverfly Eristalis tenax. We describe how the neurons can be identified based on their receptive field properties, and how they respond to moving targets, looming stimuli and to widefield optic flow. We discuss their similarities with previously published visual neurons, in the optic lobes and ventral nerve cord, and suggest that they can be classified as target-selective, looming sensitive and optic flow sensitive, based on these similarities. Our results highlight the importance of using several visual stimuli as the neurons can rarely be identified based on only one response characteristic. In addition, they provide an understanding of the neurophysiology of visual neurons that are likely to affect behavior.

Towards Computational Models and Applications of Insect Visual Systems for Motion Perception: A Review

Motion perception is a critical capability determining a variety of aspects of insects' life, including avoiding predators, foraging, and so forth. A good number of motion detectors have been identified in the insects' visual pathways. Computational modeling of these motion detectors has not only been providing effective solutions to artificial intelligence, but also benefiting the understanding of complicated biological visual systems. These biological mechanisms through millions of years of evolutionary development will have formed solid modules for constructing dynamic vision systems for future intelligent machines. This article reviews the computational motion perception models originating from biological research on insects' visual systems in the literature. These motion perception models or neural networks consist of the looming-sensitive neuronal models of lobula giant movement detectors (LGMDs) in locusts, the translation-sensitive neural systems of direction-selective neurons (DSNs) in fruit flies, bees, and locusts, and the small-target motion detectors (STMDs) in dragonflies and hoverflies. We also review the applications of these models to robots and vehicles. Through these modeling studies, we summarize the methodologies that generate different direction and size selectivity in motion perception. Finally, we discuss multiple systems integration and hardware realization of these bio-inspired motion perception models.

Integration of Small- and Wide-Field Visual Features in Target-Selective Descending Neurons of both Predatory and Nonpredatory Dipterans

For many animals, target motion carries high ecological significance as this may be generated by a predator, prey, or potential mate. Indeed, animals whose survival depends on early target detection are often equipped with a sharply tuned visual system, yielding robust performance in challenging conditions. For example, many fast-flying insects use visual cues for identifying targets, such as prey (e.g., predatory dragonflies and robberflies) or conspecifics (e.g., nonpredatory hoverflies), and can often do so against self-generated background optic flow. Supporting these behaviors, the optic lobes of insects that pursue targets harbor neurons that respond robustly to the motion of small moving objects, even when displayed against syn-directional background clutter. However, in diptera, the encoding of target information by the descending neurons, which are more directly involved in generating the behavioral output, has received less attention. We characterized target-selective neurons by recording in the ventral nerve cord of male and female predatory Holcocephala fusca robberflies and of male nonpredatory Eristalis tenax hoverflies. We show that both species have dipteran target-selective descending neurons that only respond to target motion if the background is stationary or moving slowly, moves in the opposite direction, or has un-naturalistic spatial characteristics. The response to the target is suppressed when background and target move at similar velocities, which is strikingly different to the response of target neurons in the optic lobes. As the neurons we recorded from are premotor, our findings affect our interpretation of the neurophysiology underlying target-tracking behaviors.

SIGNIFICANCE STATEMENT Many animals use sensory cues to detect moving targets that may represent predators, prey, or conspecifics. For example, birds of prey show superb sensitivity to the motion of small prey, and intercept these at high speeds. In a similar manner, predatory insects visually track moving prey, often against cluttered backgrounds. Accompanying this behavior, the brains of insects that pursue targets contain neurons that respond exclusively to target motion. We here show that dipteran insects also have target-selective descending neurons in the part of their nervous system that corresponds to the vertebrate spinal cord. Surprisingly, and in contrast to the neurons in the brain, these premotor neurons are inhibited by background patterns moving in the same direction as the target.

The Neuronal Basis of an Illusory Motion Percept Is Explained by Decorrelation of Parallel Motion Pathways

Both vertebrates and invertebrates perceive illusory motion, known as "reverse-phi," in visual stimuli that contain sequential luminance increments and decrements. However, increment (ON) and decrement (OFF) signals are initially processed by separate visual neurons, and parallel elementary motion detectors downstream respond selectively to the motion of light or dark edges, often termed ON- and OFF-edges. It remains unknown how and where ON and OFF signals combine to generate reverse-phi motion signals. Here, we show that each of Drosophila's elementary motion detectors encodes motion by combining both ON and OFF signals. Their pattern of responses reflects combinations of increments and decrements that co-occur in natural motion, serving to decorrelate their outputs. These results suggest that the general principle of signal decorrelation drives the functional specialization of parallel motion detection channels, including their selectivity for moving light or dark edges.

Visual Projection Neurons Mediating Directed Courtship in Drosophila

Many animals rely on vision to detect, locate, and track moving objects. In Drosophila courtship, males primarily use visual cues to orient toward and follow females and to select the ipsilateral wing for courtship song. Here, we show that the LC10 visual projection neurons convey essential visual information during courtship. Males with LC10 neurons silenced are unable to orient toward or maintain proximity to the female and do not predominantly use the ipsilateral wing when singing. LC10 neurons preferentially respond to small moving objects using an antagonistic motion-based center-surround mechanism. Unilateral activation of LC10 neurons recapitulates the orienting and ipsilateral wing extension normally elicited by females, and the potency with which LC10 induces wing extension is enhanced in a state of courtship arousal controlled by male-specific P1 neurons. These data suggest that LC10 is a major pathway relaying visual input to the courtship circuits in the male brain.

Representation of the stomatopod's retinal midband in the optic lobes: Putative neural substrates for integrating chromatic, achromatic and polarization information

Stomatopods have an elaborate visual system served by a retina that is unique to this class of pancrustaceans. Its upper and lower eye hemispheres encode luminance and linear polarization while an equatorial band of photoreceptors termed the midband detects color, circularly polarized light and linear polarization in the ultraviolet. In common with many malacostracan crustaceans, stomatopods have stalked eyes, but they can move these independently within three degrees of rotational freedom. Both eyes separately use saccadic and scanning movements but they can also move in a coordinated fashion to track selected targets or maintain a forward eyestalk posture during swimming. Visual information is initially processed in the first two optic neuropils, the lamina and the medulla, where the eye's midband is represented by enlarged regions within each neuropil that contain populations of neurons, the axons of which are segregated from the neuropil regions subtending the hemispheres. Neuronal channels representing the midband extend from the medulla to the lobula where populations of putative inhibitory glutamic acid decarboxylase-positive neurons and tyrosine hydroxylase-positive neurons intrinsic to the lobula have specific associations with the midband. Here we investigate the organization of the midband representation in the medulla and the lobula in the context of their overall architecture. We discuss the implications of observed arrangements, in which midband inputs to the lobula send out collaterals that extend across the retinotopic mosaic pertaining to the hemispheres. This organization suggests an integrative design that diverges from the eumalacostracan ground pattern and, for the stomatopod, enables color and polarization information to be integrated with luminance information that presumably encodes shape and motion.

Photoreceptor signalling is sufficient to explain the detectability threshold of insect aerial pursuers

An essential biological task for many flying insects is the detection of small, moving targets, such as when pursuing prey or conspecifics. Neural pathways underlying such 'target-detecting' behaviours have been investigated for their sensitivity and tuning properties (size, velocity). However, which stage of neuronal processing limits target detection is not yet known. Here, we investigated several skilled, aerial pursuers (males of four insect species), measuring the target-detection limit (signal-to-noise ratio) of light-adapted photoreceptors. We recorded intracellular responses to moving targets of varying size, extended well below the nominal resolution of single ommatidia. We found that the signal detection limit (2× photoreceptor noise) matches physiological or behavioural target-detection thresholds observed in each species. Thus, across a diverse range of flying insects, individual photoreceptor responses to changes in light intensity establish the sensitivity of the feature detection pathway, indicating later stages of processing are dedicated to feature tuning, tracking and selection.

Insect Vision: A Neuron that Anticipates an Object's Path

Dragonflies are superb aerial predators, plucking tiny insect prey from the sky. This ability depends on a visual system that has fascinated scientists for decades, and now one of its visual-target-detecting neurons has been shown to anticipate the image path of prey.

Performance of an insect-inspired target tracker in natural conditions

Robust and efficient target-tracking algorithms embedded on moving platforms, are a requirement for many computer vision and robotic applications. However, deployment of a real-time system is challenging, even with the computational power of modern hardware. As inspiration, we look to biological lightweight solutions-lightweight and low-powered flying insects. For example, dragonflies pursue prey and mates within cluttered, natural environments, deftly selecting their target amidst swarms. In our laboratory, we study the physiology and morphology of dragonfly 'small target motion detector' neurons likely to underlie this pursuit behaviour. Here we describe our insect-inspired tracking model derived from these data and compare its efficacy and efficiency with state-of-the-art engineering models. For model inputs, we use both publicly available video sequences, as well as our own task-specific dataset (small targets embedded within natural scenes). In the context of the tracking problem, we describe differences in object statistics within the video sequences. For the general dataset, our model often locks on to small components of larger objects, tracking these moving features. When input imagery includes small moving targets, for which our highly nonlinear filtering is matched, the robustness outperforms state-of-the-art trackers. In all scenarios, our insect-inspired tracker runs at least twice the speed of the comparison algorithms.

Nonlinear circuits for naturalistic visual motion estimation

Many animals use visual signals to estimate motion. Canonical models suppose that animals estimate motion by cross-correlating pairs of spatiotemporally separated visual signals, but recent experiments indicate that humans and flies perceive motion from higher-order correlations that signify motion in natural environments. Here we show how biologically plausible processing motifs in neural circuits could be tuned to extract this information. We emphasize how known aspects of Drosophila's visual circuitry could embody this tuning and predict fly behavior. We find that segregating motion signals into ON/OFF channels can enhance estimation accuracy by accounting for natural light/dark asymmetries. Furthermore, a diversity of inputs to motion detecting neurons can provide access to more complex higher-order correlations. Collectively, these results illustrate how non-canonical computations improve motion estimation with naturalistic inputs. This argues that the complexity of the fly's motion computations, implemented in its elaborate circuits, represents a valuable feature of its visual motion estimator.

DOI:http://dx.doi.org/10.7554/eLife.09123.001

Many animals have evolved the ability to estimate the speed and direction of visual motion. They use these estimates to judge their own motion, so that they can navigate through an environment, and to judge how other animals are moving, which allows them to avoid predators or detect prey.

In the 1950s, a physicist and a biologist used measurements of beetle behavior in response to visual stimuli to develop a model for how the brain estimates motion. The model became known as the Hassenstein-Reichardt correlator (HRC). The HRC and related models accurately predict the behavioral and neural responses of insects and mammals to many types of motion stimuli.

However, there are visual stimuli that generate motion percepts in fruit flies (and humans) that cannot be accounted for by the HRC. Are these differences between real brains and the HRC simply imperfections in visual circuits, whose neurons cannot perform idealized mathematical operations, or are these deviations intentional, somehow improving motion estimates? In other words: are the observed deviations a bug or a feature of visual circuits?

To address this question, Fitzgerald and Clark evaluated how different models of motion detection performed when presented with natural scenes. Natural scenes are fundamentally different from most stimuli used in lab, since they contain a rich set of regularities that are not present in simple stimuli. Fitzgerald and Clark compared the ability of the HRC, along with new, more general models, to estimate the speed and direction at which images moved across a screen. This revealed that many models could out-perform the HRC by taking advantage of regularities in natural scenes. Those models that were tuned to perform well with natural scenes could also predict the paradoxical motion percepts that were not predicted by the HRC. This suggests that visual circuits may have evolved to perform well with natural inputs, and the paradoxical motion percepts represent a feature of the real circuit, rather than a bug.

Models that performed well with natural inputs treated light and dark visual information differently. This different treatment of light and dark is a property of most visual systems, but not of the HRC or related models. In the future, these models of motion processing may help us understand how biological details of the fruit fly's visual circuits help it to estimate motion.

DOI:http://dx.doi.org/10.7554/eLife.09123.002

The Killer Fly Hunger Games: Target Size and Speed Predict Decision to Pursuit

Predatory animals have evolved to optimally detect their prey using exquisite sensory systems such as vision, olfaction and hearing. It may not be so surprising that vertebrates, with large central nervous systems, excel at predatory behaviors. More striking is the fact that many tiny insects, with their miniscule brains and scaled down nerve cords, are also ferocious, highly successful predators. For predation, it is important to determine whether a prey is suitable before initiating pursuit. This is paramount since pursuing a prey that is too large to capture, subdue or dispatch will generate a substantial metabolic cost (in the form of muscle output) without any chance of metabolic gain (in the form of food). In addition, during all pursuits, the predator breaks its potential camouflage and thus runs the risk of becoming prey itself. Many insects use their eyes to initially detect and subsequently pursue prey. Dragonflies, which are extremely efficient predators, therefore have huge eyes with relatively high spatial resolution that allow efficient prey size estimation before initiating pursuit. However, much smaller insects, such as killer flies, also visualize and successfully pursue prey. This is an impressive behavior since the small size of the killer fly naturally limits the neural capacity and also the spatial resolution provided by the compound eye. Despite this, we here show that killer flies efficiently pursue natural (Drosophila melanogaster) and artificial (beads) prey. The natural pursuits are initiated at a distance of 7.9 ± 2.9 cm, which we show is too far away to allow for distance estimation using binocular disparities. Moreover, we show that rather than estimating absolute prey size prior to launching the attack, as dragonflies do, killer flies attack with high probability when the ratio of the prey's subtended retinal velocity and retinal size is 0.37. We also show that killer flies will respond to a stimulus of an angular size that is smaller than that of the photoreceptor acceptance angle, and that the predatory response is strongly modulated by the metabolic state. Our data thus provide an exciting example of a loosely designed matched filter to Drosophila, but one which will still generate successful pursuits of other suitable prey.

Cellular evidence for efference copy in Drosophila visuomotor processing

Each time a locomoting fly turns, the visual image sweeps over the retina and generates a motion stimulus. Classic behavioral experiments suggested that flies use active neural-circuit mechanisms to suppress the perception of self-generated visual motion during intended turns. Direct electrophysiological evidence, however, has been lacking. We found that visual neurons in Drosophila receive motor-related inputs during rapid flight turns. These inputs arrived with a sign and latency appropriate for suppressing each targeted cell's visual response to the turn. Precise measurements of behavioral and neuronal response latencies supported the idea that motor-related inputs to optic flow-processing cells represent internal predictions of the expected visual drive induced by voluntary turns. Motor-related inputs to small object-selective visual neurons could reflect either proprioceptive feedback from the turn or internally generated signals. Our results in Drosophila echo the suppression of visual perception during rapid eye movements in primates, demonstrating common functional principles of sensorimotor processing across phyla.

Neurons forming optic glomeruli compute figure-ground discriminations in Drosophila

Many animals rely on visual figure-ground discrimination to aid in navigation, and to draw attention to salient features like conspecifics or predators. Even figures that are similar in pattern and luminance to the visual surroundings can be distinguished by the optical disparity generated by their relative motion against the ground, and yet the neural mechanisms underlying these visual discriminations are not well understood. We show in flies that a diverse array of figure-ground stimuli containing a motion-defined edge elicit statistically similar behavioral responses to one another, and statistically distinct behavioral responses from ground motion alone. From studies in larger flies and other insect species, we hypothesized that the circuitry of the lobula--one of the four, primary neuropiles of the fly optic lobe--performs this visual discrimination. Using calcium imaging of input dendrites, we then show that information encoded in cells projecting from the lobula to discrete optic glomeruli in the central brain group these sets of figure-ground stimuli in a homologous manner to the behavior; "figure-like" stimuli are coded similar to one another and "ground-like" stimuli are encoded differently. One cell class responds to the leading edge of a figure and is suppressed by ground motion. Two other classes cluster any figure-like stimuli, including a figure moving opposite the ground, distinctly from ground alone. This evidence demonstrates that lobula outputs provide a diverse basis set encoding visual features necessary for figure detection.

A network of visual motion-sensitive neurons for computing object position in an arthropod

Highly active insects and crabs depend on visual motion information for detecting and tracking mates, prey, or predators, for which they require directional control systems containing internal maps of visual space. A neural map formed by large, motion-sensitive neurons implicated in processing panoramic flow is known to exist in an optic ganglion of the fly. However, an equivalent map for processing spatial positions of single objects has not been hitherto identified in any arthropod. Crabs can escape directly away from a visual threat wherever the stimulus is located in the 360° field of view. When tested in a walking simulator, the crab Neohelice granulata immediately adjusts its running direction after changes in the position of the visual danger stimulus smaller than 1°. Combining mass and single-cell staining with in vivo intracellular recording, we show that a particular class of motion-sensitive neurons of the crab's lobula that project to the midbrain, the monostratified lobula giants type 1 (MLG1), form a system of 16 retinotopically organized elements that map the 360° azimuthal space. The preference of these neurons for horizontally moving objects conforms the visual ecology of the crab's mudflat world. With a mean receptive field of 118°, MLG1s have a large superposition among neighboring elements. Our results suggest that the MLG1 system conveys information on object position as a population vector. Such computational code can enable the accurate directional control observed in the visually guided behaviors of crabs.

Properties of neuronal facilitation that improve target tracking in natural pursuit simulations

Although flying insects have limited visual acuity (approx. 1°) and relatively small brains, many species pursue tiny targets against cluttered backgrounds with high success. Our previous computational model, inspired by electrophysiological recordings from insect 'small target motion detector' (STMD) neurons, did not account for several key properties described from the biological system. These include the recent observations of response 'facilitation' (a slow build-up of response to targets that move on long, continuous trajectories) and 'selective attention', a competitive mechanism that selects one target from alternatives. Here, we present an elaborated STMD-inspired model, implemented in a closed loop target-tracking system that uses an active saccadic gaze fixation strategy inspired by insect pursuit. We test this system against heavily cluttered natural scenes. Inclusion of facilitation not only substantially improves success for even short-duration pursuits, but it also enhances the ability to 'attend' to one target in the presence of distracters. Our model predicts optimal facilitation parameters that are static in space and dynamic in time, changing with respect to the amount of background clutter and the intended purpose of the pursuit. Our results provide insights into insect neurophysiology and show the potential of this algorithm for implementation in artificial visual systems and robotic applications.

Method and software for using m-sequences to characterize parallel components of higher-order visual tracking behavior in Drosophila

A moving visual figure may contain first-order signals defined by variation in mean luminance, as well as second-order signals defined by constant mean luminance and variation in luminance envelope, or higher-order signals that cannot be estimated by taking higher moments of the luminance distribution. Separating these properties of a moving figure to experimentally probe the visual subsystems that encode them is technically challenging and has resulted in debated mechanisms of visual object detection by flies. Our prior work took a white noise systems identification approach using a commercially available electronic display system to characterize the spatial variation in the temporal dynamics of two distinct subsystems for first- and higher-order components of visual figure tracking. The method relied on the use of single pixel displacements of two visual stimuli according to two binary maximum length shift register sequences (m-sequences) and cross-correlation of each m-sequence with time-varying flight steering measurements. The resultant spatio-temporal action fields represent temporal impulse responses parameterized by the azimuthal location of the visual figure, one STAF for first-order and another for higher-order components of compound stimuli. Here we review m-sequence and reverse correlation procedures, then describe our application in detail, provide Matlab code, validate the STAFs, and demonstrate the utility and robustness of STAFs by predicting the results of other published experimental procedures. This method has demonstrated how two relatively modest innovations on classical white noise analysis—the inclusion of space as a way to organize response kernels and the use of linear decoupling to measure the response to two channels of visual information simultaneously—could substantially improve our basic understanding of visual processing in the fly.

Spatiotemporal stimulus properties modulate responses to trajectory changes in a locust looming-sensitive pathway

The lobula giant movement detector (LGMD) and descending contralateral movement detector (DCMD) constitute one motion-sensitive pathway in the locust visual system that is implicated in collision-avoidance behaviors. While this pathway is thought to respond preferentially to objects approaching on a direct collision course, emerging studies suggest the firing rate is able to monitor more complicated movements that would occur under natural conditions. While previous studies have compared the response of the DCMD to objects on collision courses that travel at different speeds, velocity has not been manipulated for other simple or compound trajectories. Here we test the possibility that the LGMD/DCMD pathway is capable of responding uniquely to complex aspects of object motion, including translation and trajectory changes at different velocities. We found that the response of the DCMD to translational motion initiated in the caudal visual field was a low-amplitude peak in firing rate that occurred before the object crossed 90° azimuth that was invariant to different object velocities. Direct looms at different velocities resulted in peak firing rates that occurred later in time and with greater amplitude for higher velocities. In response to transitions from translational motion to a collision course, the firing rate change depended on both the location within the visual field and the velocity. These results suggest that this pathway is capable of conveying information about multiple properties of a moving object's trajectory.

Moving Object Detection Based on Bionic Compound Eye

In view of the traditional image difference method for camera movement under the condition of low accuracy on moving target detection, the image difference method based on bionic compound eye moving target detection algorithm was proposed and optimized. It used the image difference method based on bionic compound eye to detect the moving targets, and optimize the result by pixel interpolation and the ratio. The experimental conclusions show that the accuracy about new motion detection method is 25% higher than the traditional method.