Retinotopic organization of small-field-target-detecting neurons in the insect visual system

Descending neurons of the hoverfly respond to pursuits of artificial targets

Many animals use motion vision information to control dynamic behaviors. Predatory animals, for example, show an exquisite ability to detect rapidly moving prey, followed by pursuit and capture. Such target detection is not only used by predators but is also important in conspecific interactions, such as for male hoverflies defending their territories against conspecific intruders. Visual target detection is believed to be subserved by specialized target-tuned neurons found in a range of species, including vertebrates and arthropods. However, how these target-tuned neurons respond to actual pursuit trajectories is currently not well understood. To redress this, we recorded extracellularly from target-selective descending neurons (TSDNs) in male Eristalis tenax hoverflies. We show that they have dorso-frontal receptive fields with a preferred direction up and away from the visual midline. We reconstructed visual flow fields as experienced during pursuits of artificial targets (black beads). We recorded TSDN responses to six reconstructed pursuits and found that each neuron responded consistently at remarkably specific time points but that these time points differed between neurons. We found that the observed spike probability was correlated with the spike probability predicted from each neuron's receptive field and size tuning. Interestingly, however, the overall response rate was low, with individual neurons responding to only a small part of each reconstructed pursuit. In contrast, the TSDN population responded to substantially larger proportions of the pursuits but with lower probability. This large variation between neurons could be useful if different neurons control different parts of the behavioral output.

Attention and Prediction-Guided Motion Detection for Low-Contrast Small Moving Targets

Small target motion detection within complex natural environments is an extremely challenging task for autonomous robots. Surprisingly, the visual systems of insects have evolved to be highly efficient in detecting mates and tracking prey, even though targets occupy as small as a few degrees of their visual fields. The excellent sensitivity to small target motion relies on a class of specialized neurons, called small target motion detectors (STMDs). However, existing STMD-based models are heavily dependent on visual contrast and perform poorly in complex natural environments, where small targets generally exhibit extremely low contrast against neighboring backgrounds. In this article, we develop an attention-and-prediction-guided visual system to overcome this limitation. The developed visual system comprises three main subsystems, namely: 1) an attention module; 2) an STMD-based neural network; and 3) a prediction module. The attention module searches for potential small targets in the predicted areas of the input image and enhances their contrast against a complex background. The STMD-based neural network receives the contrast-enhanced image and discriminates small moving targets from background false positives. The prediction module foresees future positions of the detected targets and generates a prediction map for the attention module. The three subsystems are connected in a recurrent architecture, allowing information to be processed sequentially to activate specific areas for small target detection. Extensive experiments on synthetic and real-world datasets demonstrate the effectiveness and superiority of the proposed visual system for detecting small, low-contrast moving targets against complex natural environments.

Bioinspired figure-ground discrimination via visual motion smoothing

Flies detect and track moving targets among visual clutter, and this process mainly relies on visual motion. Visual motion is analyzed or computed with the pathway from the retina to T4/T5 cells. The computation of local directional motion was formulated as an elementary movement detector (EMD) model more than half a century ago. Solving target detection or figure-ground discrimination problems can be equivalent to extracting boundaries between a target and the background based on the motion discontinuities in the output of a retinotopic array of EMDs. Individual EMDs cannot measure true velocities, however, due to their sensitivity to pattern properties such as luminance contrast and spatial frequency content. It remains unclear how local directional motion signals are further integrated to enable figure-ground discrimination. Here, we present a computational model inspired by fly motion vision. Simulations suggest that the heavily fluctuating output of an EMD array is naturally surmounted by a lobula network, which is hypothesized to be downstream of the local motion detectors and have parallel pathways with distinct directional selectivity. The lobula network carries out a spatiotemporal smoothing operation for visual motion, especially across time, enabling the segmentation of moving figures from the background. The model qualitatively reproduces experimental observations in the visually evoked response characteristics of one type of lobula columnar (LC) cell. The model is further shown to be robust to natural scene variability. Our results suggest that the lobula is involved in local motion-based target detection.

A Time-Delay Feedback Neural Network for Discriminating Small, Fast-Moving Targets in Complex Dynamic Environments

Discriminating small moving objects within complex visual environments is a significant challenge for autonomous micro-robots that are generally limited in computational power. By exploiting their highly evolved visual systems, flying insects can effectively detect mates and track prey during rapid pursuits, even though the small targets equate to only a few pixels in their visual field. The high degree of sensitivity to small target movement is supported by a class of specialized neurons called small target motion detectors (STMDs). Existing STMD-based computational models normally comprise four sequentially arranged neural layers interconnected via feedforward loops to extract information on small target motion from raw visual inputs. However, feedback, another important regulatory circuit for motion perception, has not been investigated in the STMD pathway and its functional roles for small target motion detection are not clear. In this article, we propose an STMD-based neural network with feedback connection (feedback STMD), where the network output is temporally delayed, then fed back to the lower layers to mediate neural responses. We compare the properties of the model with and without the time-delay feedback loop and find that it shows a preference for high-velocity objects. Extensive experiments suggest that the feedback STMD achieves superior detection performance for fast-moving small targets, while significantly suppressing background false positive movements which display lower velocities. The proposed feedback model provides an effective solution in robotic visual systems for detecting fast-moving small targets that are always salient and potentially threatening.

Mathematical study of neural feedback roles in small target motion detection

Building an efficient and reliable small target motion detection visual system is challenging for artificial intelligence robotics because a small target only occupies few pixels and hardly displays visual features in images. Biological visual systems that have evolved over millions of years could be ideal templates for designing artificial visual systems. Insects benefit from a class of specialized neurons, called small target motion detectors (STMDs), which endow them with an excellent ability to detect small moving targets against a cluttered dynamic environment. Some bio-inspired models featured in feed-forward information processing architectures have been proposed to imitate the functions of the STMD neurons. However, feedback, a crucial mechanism for visual system regulation, has not been investigated deeply in the STMD-based neural circuits and its roles in small target motion detection remain unclear. In this paper, we propose a time-delay feedback STMD model for small target motion detection in complex backgrounds. The main contributions of this study are as follows. First, a feedback pathway is designed by transmitting information from output-layer neurons to lower-layer interneurons in the STMD pathway and the role of the feedback is analyzed from the view of mathematical analysis. Second, to estimate the feedback constant, the existence and uniqueness of solutions for nonlinear dynamical systems formed by feedback loop are analyzed via Schauder's fixed point theorem and contraction mapping theorem. Finally, an iterative algorithm is designed to solve the nonlinear problem and the performance of the proposed model is tested by experiments. Experimental results demonstrate that the feedback is able to weaken background false positives while maintaining a minor effect on small targets. It outperforms existing STMD-based models regarding the accuracy of fast-moving small target detection in visual clutter. The proposed feedback approach could inspire the relevant modeling of robust motion perception robotics visual systems.

Neural organization of the third optic neuropil, the lobula, in the highly visual semiterrestrial crab Neohelice granulata

The visual neuropils (lamina, medulla and lobula complex), of malacostracan crustaceans and hexapods have many organizational principles, cell types and functional properties in common. Information about the cellular elements that compose the crustacean lobula is scarce especially when focusing on small columnar cells. Semiterrestrial crabs possess a highly developed visual system and display conspicuous visually guided behaviors. In particular, Neohelice granulata has been previously used to describe the cellular components of the first two optic neuropils using Golgi impregnation technique. Here, we present a comprehensive description of individual elements composing the third optic neuropil, the lobula, of that same species. We characterized a wide variety of elements (140 types) including input terminals and lobula columnar, centrifugal and input columnar elements. Results reveal a very dense and complex neuropil. We found a frequently impregnated input element (suggesting a supernumerary cartridge representation) that arborizes in the third layer of the lobula and that presents four variants each with ramifications organized following one of the four cardinal axes suggesting a role in directional processing. We also describe input elements with two neurites branching in the third layer, probably connecting with the medulla and lobula plate. These facts suggest that this layer is involved in the directional motion detection pathway in crabs. We analyze and discuss our findings considering the similarities and differences found between the layered organization and components of this crustacean lobula and the lobula of insects. This article is protected by copyright. All rights reserved.

Facilitation of neural responses to targets moving against optic flow

Target detection in visual clutter is a difficult computational task that insects, with their poor spatial resolution compound eyes and small brains, do successfully and with extremely short behavioral delays. We here show that the responses of target selective descending neurons are attenuated by background motion in the same direction as target motion but facilitated by background motion in the opposite direction. This finding is important for understanding how target pursuit can occur in tandem with gaze stabilization. Indeed, the neural facilitation would come into effect if the hoverfly is subjected to background motion in one direction but the target it is pursuing moves in the opposite direction and could therefore be used to override gaze stabilizing corrective turns.

For the human observer, it can be difficult to follow the motion of small objects, especially when they move against background clutter. In contrast, insects efficiently do this, as evidenced by their ability to capture prey, pursue conspecifics, or defend territories, even in highly textured surrounds. We here recorded from target selective descending neurons (TSDNs), which likely subserve these impressive behaviors. To simulate the type of optic flow that would be generated by the pursuer’s own movements through the world, we used the motion of a perspective corrected sparse dot field. We show that hoverfly TSDN responses to target motion are suppressed when such optic flow moves syn-directional to the target. Indeed, neural responses are strongly suppressed when targets move over either translational sideslip or rotational yaw. More strikingly, we show that TSDNs are facilitated by optic flow moving counterdirectional to the target, if the target moves horizontally. Furthermore, we show that a small, frontal spatial window of optic flow is enough to fully facilitate or suppress TSDN responses to target motion. We argue that such TSDN response facilitation could be beneficial in modulating corrective turns during target pursuit.

Modeling Nonlinear Dendritic Processing of Facilitation in a Dragonfly Target-Tracking Neuron

Dragonflies are highly skilled and successful aerial predators that are even capable of selectively attending to one target within a swarm. Detection and tracking of prey is likely to be driven by small target motion detector (STMD) neurons identified from several insect groups. Prior work has shown that dragonfly STMD responses are facilitated by targets moving on a continuous path, enhancing the response gain at the present and predicted future location of targets. In this study, we combined detailed morphological data with computational modeling to test whether a combination of dendritic morphology and nonlinear properties of NMDA receptors could explain these observations. We developed a hybrid computational model of neurons within the dragonfly optic lobe, which integrates numerical and morphological components. The model was able to generate potent facilitation for targets moving on continuous trajectories, including a localized spotlight of maximal sensitivity close to the last seen target location, as also measured during in vivo recordings. The model did not, however, include a mechanism capable of producing a traveling or spreading wave of facilitation. Our data support a strong role for the high dendritic density seen in the dragonfly neuron in enhancing non-linear facilitation. An alternative model based on the morphology of an unrelated type of motion processing neuron from a dipteran fly required more than three times higher synaptic gain in order to elicit similar levels of facilitation, despite having only 20% fewer synapses. Our data support a potential role for NMDA receptors in target tracking and also demonstrate the feasibility of combining biologically plausible dendritic computations with more abstract computational models for basic processing as used in earlier studies.

Comparison of Transparency and Shrinkage During Clearing of Insect Brains Using Media With Tunable Refractive Index

Improvement of imaging quality has the potential to visualize previously unseen building blocks of the brain and is therefore one of the great challenges in neuroscience. Rapid development of new tissue clearing techniques in recent years have attempted to solve imaging compromises in thick brain samples, particularly for high resolution optical microscopy, where the clearing medium needs to match the high refractive index of the objective immersion medium. These problems are exacerbated in insect tissue, where numerous (initially air-filled) tracheal tubes branching throughout the brain increase the scattering of light. To date, surprisingly few studies have systematically quantified the benefits of such clearing methods using objective transparency and tissue shrinkage measurements. In this study we compare a traditional and widely used insect clearing medium, methyl salicylate combined with permanent mounting in Permount (“MS/P”) with several more recently applied clearing media that offer tunable refractive index (n): 2,2′-thiodiethanol (TDE), “SeeDB2” (in variants SeeDB2S and SeeDB2G matched to oil and glycerol immersion, n = 1.52 and 1.47, respectively) and Rapiclear (also with n = 1.52 and 1.47). We measured transparency and tissue shrinkage by comparing freshly dissected brains with cleared brains from dipteran flies, with or without addition of vacuum or ethanol pre-treatments (dehydration and rehydration) to evacuate air from the tracheal system. The results show that ethanol pre-treatment is very effective for improving transparency, regardless of the subsequent clearing medium, while vacuum treatment offers little measurable benefit. Ethanol pre-treated SeeDB2G and Rapiclear brains show much less shrinkage than using the traditional MS/P method. Furthermore, at lower refractive index, closer to that of glycerol immersion, these recently developed media offer outstanding transparency compared to TDE and MS/P. Rapiclear protocols were less laborious compared to SeeDB2, but both offer sufficient transparency and refractive index tunability to permit super-resolution imaging of local volumes in whole mount brains from large insects, and even light-sheet microscopy. Although long-term permanency of Rapiclear stored samples remains to be established, our samples still showed good preservation of fluorescence after storage for more than a year at room temperature.

Experience, circuit dynamics, and forebrain recruitment in larval zebrafish prey capture

Experience influences behavior, but little is known about how experience is encoded in the brain, and how changes in neural activity are implemented at a network level to improve performance. Here we investigate how differences in experience impact brain circuitry and behavior in larval zebrafish prey capture. We find that experience of live prey compared to inert food increases capture success by boosting capture initiation. In response to live prey, animals with and without prior experience of live prey show activity in visual areas (pretectum and optic tectum) and motor areas (cerebellum and hindbrain), with similar visual area retinotopic maps of prey position. However, prey-experienced animals more readily initiate capture in response to visual area activity and have greater visually-evoked activity in two forebrain areas: the telencephalon and habenula. Consequently, disruption of habenular neurons reduces capture performance in prey-experienced fish. Together, our results suggest that experience of prey strengthens prey-associated visual drive to the forebrain, and that this lowers the threshold for prey-associated visual activity to trigger activity in motor areas, thereby improving capture performance.

Non-canonical Receptive Field Properties and Neuromodulation of Feature-Detecting Neurons in Flies

Several fundamental aspects of motion vision circuitry are prevalent across flies and mice. Both taxa segregate ON and OFF signals. For any given spatial pattern, motion detectors in both taxa are tuned to speed, selective for one of four cardinal directions, and modulated by catecholamine neurotransmitters. These similarities represent conserved, canonical properties of the functional circuits and computational algorithms for motion vision. Less is known about feature detectors, including how receptive field properties differ from the motion pathway or whether they are under neuromodulatory control to impart functional plasticity for the detection of salient objects from a moving background. Here, we investigated 19 types of putative feature selective lobula columnar (LC) neurons in the optic lobe of the fruit fly Drosophila melanogaster to characterize divergent properties of feature selection. We identified LC12 and LC15 as feature detectors. LC15 encodes moving bars, whereas LC12 is selective for the motion of discrete objects, mostly independent of size. Neither is selective for contrast polarity, speed, or direction, highlighting key differences in the underlying algorithms for feature detection and motion vision. We show that the onset of background motion suppresses object responses by LC12 and LC15. Surprisingly, the application of octopamine, which is released during flight, reverses the suppressive influence of background motion, rendering both LCs able to track moving objects superimposed against background motion. Our results provide a comparative framework for the function and modulation of feature detectors and new insights into the underlying neuronal mechanisms involved in visual feature detection.

Direction Selective Neurons Responsive to Horizontal Motion in a Crab Reflect an Adaptation to Prevailing Movements in Flat Environments

All animals need information about the direction of motion to be able to track the trajectory of a target (prey, predator, cospecific) or to control the course of navigation. This information is provided by direction selective (DS) neurons, which respond to images moving in a unique direction. DS neurons have been described in numerous species including many arthropods. In these animals, the majority of the studies have focused on DS neurons dedicated to processing the optic flow generated during navigation. In contrast, only a few studies were performed on DS neurons related to object motion processing. The crab Neohelice is an established experimental model for the study of neurons involved in visually-guided behaviors. Here, we describe in male crabs of this species a new group of DS neurons that are highly directionally selective to moving objects. The neurons were physiologically and morphologically characterized by intracellular recording and staining in the optic lobe of intact animals. Because of their arborization in the lobula complex, we called these cells lobula complex directional cells (LCDCs). LCDCs also arborize in a previously undescribed small neuropil of the lateral protocerebrum. LCDCs are responsive only to horizontal motion. This nicely fits in the behavioral adaptations of a crab inhabiting a flat, densely crowded environment, where most object motions are generated by neighboring crabs moving along the horizontal plane.SIGNIFICANCE STATEMENT Direction selective (DS) neurons are key to a variety of visual behaviors including, target tracking (preys, predators, cospecifics) and course control. Here, we describe the physiology and morphology of a new group of remarkably directional neurons exclusively responsive to horizontal motion in crabs. These neurons arborize in the lobula complex and in a previously undescribed small neuropil of the lateral protocerebrum. The strong sensitivity of these cells for horizontal motion represents a clear example of functional neuronal adaptation to the lifestyle of an animal inhabiting a flat environment.

A Directionally Selective Small Target Motion Detecting Visual Neural Network in Cluttered Backgrounds

Discriminating targets moving against a cluttered background is a huge challenge, let alone detecting a target as small as one or a few pixels and tracking it in flight. In the insect's visual system, a class of specific neurons, called small target motion detectors (STMDs), have been identified as showing exquisite selectivity for small target motion. Some of the STMDs have also demonstrated direction selectivity which means these STMDs respond strongly only to their preferred motion direction. Direction selectivity is an important property of these STMD neurons which could contribute to tracking small targets such as mates in flight. However, little has been done on systematically modeling these directionally selective STMD neurons. In this paper, we propose a directionally selective STMD-based neural network for small target detection in a cluttered background. In the proposed neural network, a new correlation mechanism is introduced for direction selectivity via correlating signals relayed from two pixels. Then, a lateral inhibition mechanism is implemented on the spatial field for size selectivity of the STMD neurons. Finally, a population vector algorithm is used to encode motion direction of small targets. Extensive experiments showed that the proposed neural network not only is in accord with current biological findings, i.e., showing directional preferences but also worked reliably in detecting the small targets against cluttered backgrounds.

A Robust Visual System for Small Target Motion Detection Against Cluttered Moving Backgrounds

Monitoring small objects against cluttered moving backgrounds is a huge challenge to future robotic vision systems. As a source of inspiration, insects are quite apt at searching for mates and tracking prey, which always appear as small dim speckles in the visual field. The exquisite sensitivity of insects for small target motion, as revealed recently, is coming from a class of specific neurons called small target motion detectors (STMDs). Although a few STMD-based models have been proposed, these existing models only use motion information for small target detection and cannot discriminate small targets from small-target-like background features (named fake features). To address this problem, this paper proposes a novel visual system model (STMD+) for small target motion detection, which is composed of four subsystems-ommatidia, motion pathway, contrast pathway, and mushroom body. Compared with the existing STMD-based models, the additional contrast pathway extracts directional contrast from luminance signals to eliminate false positive background motion. The directional contrast and the extracted motion information by the motion pathway are integrated into the mushroom body for small target discrimination. Extensive experiments showed the significant and consistent improvements of the proposed visual system model over the existing STMD-based models against fake features.

Visual motion sensitivity in descending neurons in the hoverfly

Many animals use motion vision information to control dynamic behaviors. For example, flying insects must decide whether to pursue a prey or not, to avoid a predator, to maintain their current flight trajectory, or to land. The neural mechanisms underlying the computation of visual motion have been particularly well investigated in the fly optic lobes. However, the descending neurons, which connect the optic lobes with the motor command centers of the ventral nerve cord, remain less studied. To address this deficiency, we describe motion vision sensitive descending neurons in the hoverfly Eristalis tenax. We describe how the neurons can be identified based on their receptive field properties, and how they respond to moving targets, looming stimuli and to widefield optic flow. We discuss their similarities with previously published visual neurons, in the optic lobes and ventral nerve cord, and suggest that they can be classified as target-selective, looming sensitive and optic flow sensitive, based on these similarities. Our results highlight the importance of using several visual stimuli as the neurons can rarely be identified based on only one response characteristic. In addition, they provide an understanding of the neurophysiology of visual neurons that are likely to affect behavior.

Strategies of locomotion composition

This review aims to highlight the importance of saccades during locomotion as a strategy to reduce sensory information loss while the subject is moving. Acquiring sensory data from the environment during movement results in a temporal flow of information, as the sensory precept changes with the position of the observer. Accordingly, the movement pattern shapes the sensory flow. Therefore, the requirements of locomotion and sensation have to be balanced in the behaviour of the organism. Insect vision provides deep insight into the interplay between action and perception. Insects can shape their optic flow by reducing their rotational movements to fast and short saccades. This generates prolonged phases of translations which provide depth information. Extensive behavioural and physiological studies on insects show how shaping the optic flow facilitates the coding of motion vision. Indeed the saccadic strategy provides an elegant solution to optimise sensory flow. Complementary studies in other taxa reported similar locomotion strategies emphasising the crucial influence of sensory flow on locomotion.

Towards Computational Models and Applications of Insect Visual Systems for Motion Perception: A Review

Motion perception is a critical capability determining a variety of aspects of insects' life, including avoiding predators, foraging, and so forth. A good number of motion detectors have been identified in the insects' visual pathways. Computational modeling of these motion detectors has not only been providing effective solutions to artificial intelligence, but also benefiting the understanding of complicated biological visual systems. These biological mechanisms through millions of years of evolutionary development will have formed solid modules for constructing dynamic vision systems for future intelligent machines. This article reviews the computational motion perception models originating from biological research on insects' visual systems in the literature. These motion perception models or neural networks consist of the looming-sensitive neuronal models of lobula giant movement detectors (LGMDs) in locusts, the translation-sensitive neural systems of direction-selective neurons (DSNs) in fruit flies, bees, and locusts, and the small-target motion detectors (STMDs) in dragonflies and hoverflies. We also review the applications of these models to robots and vehicles. Through these modeling studies, we summarize the methodologies that generate different direction and size selectivity in motion perception. Finally, we discuss multiple systems integration and hardware realization of these bio-inspired motion perception models.

Integration of Small- and Wide-Field Visual Features in Target-Selective Descending Neurons of both Predatory and Nonpredatory Dipterans

For many animals, target motion carries high ecological significance as this may be generated by a predator, prey, or potential mate. Indeed, animals whose survival depends on early target detection are often equipped with a sharply tuned visual system, yielding robust performance in challenging conditions. For example, many fast-flying insects use visual cues for identifying targets, such as prey (e.g., predatory dragonflies and robberflies) or conspecifics (e.g., nonpredatory hoverflies), and can often do so against self-generated background optic flow. Supporting these behaviors, the optic lobes of insects that pursue targets harbor neurons that respond robustly to the motion of small moving objects, even when displayed against syn-directional background clutter. However, in diptera, the encoding of target information by the descending neurons, which are more directly involved in generating the behavioral output, has received less attention. We characterized target-selective neurons by recording in the ventral nerve cord of male and female predatory Holcocephala fusca robberflies and of male nonpredatory Eristalis tenax hoverflies. We show that both species have dipteran target-selective descending neurons that only respond to target motion if the background is stationary or moving slowly, moves in the opposite direction, or has un-naturalistic spatial characteristics. The response to the target is suppressed when background and target move at similar velocities, which is strikingly different to the response of target neurons in the optic lobes. As the neurons we recorded from are premotor, our findings affect our interpretation of the neurophysiology underlying target-tracking behaviors.

SIGNIFICANCE STATEMENT Many animals use sensory cues to detect moving targets that may represent predators, prey, or conspecifics. For example, birds of prey show superb sensitivity to the motion of small prey, and intercept these at high speeds. In a similar manner, predatory insects visually track moving prey, often against cluttered backgrounds. Accompanying this behavior, the brains of insects that pursue targets contain neurons that respond exclusively to target motion. We here show that dipteran insects also have target-selective descending neurons in the part of their nervous system that corresponds to the vertebrate spinal cord. Surprisingly, and in contrast to the neurons in the brain, these premotor neurons are inhibited by background patterns moving in the same direction as the target.

A predictive focus of gain modulation encodes target trajectories in insect vision

When a human catches a ball, they estimate future target location based on the current trajectory. How animals, small and large, encode such predictive processes at the single neuron level is unknown. Here we describe small target-selective neurons in predatory dragonflies that exhibit localized enhanced sensitivity for targets displaced to new locations just ahead of the prior path, with suppression elsewhere in the surround. This focused region of gain modulation is driven by predictive mechanisms, with the direction tuning shifting selectively to match the target’s prior path. It involves a large local increase in contrast gain which spreads forward after a delay (e.g. an occlusion) and can even transfer between brain hemispheres, predicting trajectories moved towards the visual midline from the other eye. The tractable nature of dragonflies for physiological experiments makes this a useful model for studying the neuronal mechanisms underlying the brain’s remarkable ability to anticipate moving stimuli.

DOI:http://dx.doi.org/10.7554/eLife.26478.001

Catching a ball requires a person to track the speed and direction of a small moving target often against a cluttered and varying background. Predatory insects, like dragonflies, face a similar challenge when they pursue their prey through the air. The task is made a little easier, however, by the fact that most moving targets tend to follow predictable trajectories. Indeed, animals are also better at tracking targets that follow smooth continuous trajectories, suggesting that brains have evolved to exploit the normal behavior of visual stimuli to reduce their workload

To find out how this process works, Wiederman, Fabian et al. studied the brains of dragonflies as they watched a black square intended to mimic prey. Brain cells called Small Target Motion Detectors (or STMD neurons for short) became more active in response to the target. But rather than simply following the target, the STMD neurons instead predicted its future location. In fact, individual neurons were more sensitive to movements occurring just ahead of the target’s current position, and less sensitive to movements occurring elsewhere.

If the target abruptly disappeared, the point in space where the neurons were most sensitive to movement continued to gradually move forward over time. Given that real-life targets typically disappear when they move behind other objects, this suggests that the brain is predicting where the target is most likely to reappear. The STMD neurons became more sensitive to movement by increasing their ability to detect differences in brightness between the target and its background. In some cases, the neurons increased their sensitivity more than five-fold.

Insects and mammals last shared a common ancestor more than 500 million years ago, and, in many respects, mammalian brains are substantially more complex than insect brains. Nevertheless, the results of Wiederman, Fabian et al. show that the insect brain can perform visual tasks that were previously associated only with mammals. Neuroscientists and engineers have used the insect brain for decades to study the circuits that support biological processes. In the coming years, insects such as the dragonfly may enable us to explore how visual regions of the brain predict future events. This knowledge could ultimately be applied to artificial vision systems, such as those in self-driving cars.

DOI:http://dx.doi.org/10.7554/eLife.26478.002

Object-Detecting Neurons in Drosophila

Many animals rely on vision to detect objects such as conspecifics, predators, and prey. Hypercomplex cells found in feline cortex and small target motion detectors found in dragonfly and hoverfly optic lobes demonstrate robust tuning for small objects, with weak or no response to larger objects or movement of the visual panorama [1-3]. However, the relationship among anatomical, molecular, and functional properties of object detection circuitry is not understood. Here we characterize a specialized object detector in Drosophila, the lobula columnar neuron LC11 [4]. By imaging calcium dynamics with two-photon excitation microscopy, we show that LC11 responds to the omni-directional movement of a small object darker than the background, with little or no responses to static flicker, vertically elongated bars, or panoramic gratings. LC11 dendrites innervate multiple layers of the lobula, and each dendrite spans enough columns to sample 75° of visual space, yet the area that evokes calcium responses is only 20° wide and shows robust responses to a 2.2° object spanning less than half of one facet of the compound eye. The dendrites of neighboring LC11s encode object motion retinotopically, but the axon terminals fuse into a glomerular structure in the central brain where retinotopy is lost. Blocking inhibitory ionic currents abolishes small object sensitivity and facilitates responses to elongated bars and gratings. Our results reveal high-acuity object motion detection in the Drosophila optic lobe.

Arthropod eyes: The early Cambrian fossil record and divergent evolution of visual systems

Four types of eyes serve the visual neuropils of extant arthropods: compound retinas composed of adjacent facets; a visual surface populated by spaced eyelets; a smooth transparent cuticle providing inwardly directed lens cylinders; and single-lens eyes. The first type is a characteristic of pancrustaceans, the eyes of which comprise lenses arranged as hexagonal or rectilinear arrays, each lens crowning 8-9 photoreceptor neurons. Except for Scutigeromorpha, the second type typifies Myriapoda whose relatively large eyelets surmount numerous photoreceptive rhabdoms stacked together as tiers. Scutigeromorph eyes are facetted, each lens crowning some dozen photoreceptor neurons of a modified apposition-type eye. Extant chelicerate eyes are single-lensed except in xiphosurans, whose lateral eyes comprise a cuticle with a smooth outer surface and an inner one providing regular arrays of lens cylinders. This account discusses whether these disparate eye types speak for or against divergence from one ancestral eye type. Previous considerations of eye evolution, focusing on the eyes of trilobites and on facet proliferation in xiphosurans and myriapods, have proposed that the mode of development of eyes in those taxa is distinct from that of pancrustaceans and is the plesiomorphic condition from which facetted eyes have evolved. But the recent discovery of enormous regularly facetted compound eyes belonging to early Cambrian radiodontans suggests that high-resolution facetted eyes with superior optics may be the ground pattern organization for arthropods, predating the evolution of arthrodization and jointed post-protocerebral appendages. Here we provide evidence that compound eye organization in stem-group euarthropods of the Cambrian can be understood in terms of eye morphologies diverging from this ancestral radiodontan-type ground pattern. We show that in certain Cambrian groups apposition eyes relate to fixed or mobile eyestalks, whereas other groups reveal concomitant evolution of sessile eyes equipped with optics typical of extant xiphosurans. Observations of fossil material, including that of trilobites and eurypterids, support the proposition that the ancestral compound eye was the apposition type. Cambrian arthropods include possible precursors of mandibulate eyes. The latter are the modified compound eyes, now sessile, and their underlying optic lobes exemplified by scutigeromorph chilopods, and the mobile stalked compound eyes and more elaborate optic lobes typifying Pancrustacea. Radical divergence from an ancestral apposition type is demonstrated by the evolution of chelicerate eyes, from doublet sessile-eyed stem-group taxa to special apposition eyes of xiphosurans, the compound eyes of eurypterids, and single-lens eyes of arachnids. Different eye types are discussed with respect to possible modes of life of the extinct species that possessed them, comparing these to extant counterparts and the types of visual centers the eyes might have served.

Properties of neuronal facilitation that improve target tracking in natural pursuit simulations

Although flying insects have limited visual acuity (approx. 1°) and relatively small brains, many species pursue tiny targets against cluttered backgrounds with high success. Our previous computational model, inspired by electrophysiological recordings from insect 'small target motion detector' (STMD) neurons, did not account for several key properties described from the biological system. These include the recent observations of response 'facilitation' (a slow build-up of response to targets that move on long, continuous trajectories) and 'selective attention', a competitive mechanism that selects one target from alternatives. Here, we present an elaborated STMD-inspired model, implemented in a closed loop target-tracking system that uses an active saccadic gaze fixation strategy inspired by insect pursuit. We test this system against heavily cluttered natural scenes. Inclusion of facilitation not only substantially improves success for even short-duration pursuits, but it also enhances the ability to 'attend' to one target in the presence of distracters. Our model predicts optimal facilitation parameters that are static in space and dynamic in time, changing with respect to the amount of background clutter and the intended purpose of the pursuit. Our results provide insights into insect neurophysiology and show the potential of this algorithm for implementation in artificial visual systems and robotic applications.

Distinct representation and distribution of visual information by specific cell types in mouse superficial superior colliculus

The superficial superior colliculus (sSC) occupies a critical node in the mammalian visual system; it is one of two major retinorecipient areas, receives visual cortical input, and innervates visual thalamocortical circuits. Nonetheless, the contribution of sSC neurons to downstream neural activity and visually guided behavior is unknown and frequently neglected. Here we identified the visual stimuli to which specific classes of sSC neurons respond, the downstream regions they target, and transgenic mice enabling class-specific manipulations. One class responds to small, slowly moving stimuli and projects exclusively to lateral posterior thalamus; another, comprising GABAergic neurons, responds to the sudden appearance or rapid movement of large stimuli and projects to multiple areas, including the lateral geniculate nucleus. A third class exhibits direction-selective responses and targets deeper SC layers. Together, our results show how specific sSC neurons represent and distribute diverse information and enable direct tests of their functional role.

Correlation between OFF and ON channels underlies dark target selectivity in an insect visual system

In both vertebrates and invertebrates, evidence supports separation of luminance increments and decrements (ON and OFF channels) in early stages of visual processing (Hartline, 1938; Joesch et al., 2010); however, less is known about how these parallel pathways are recombined to encode form and motion. In Drosophila, genetic knockdown of inputs to putative ON and OFF pathways and direct recording from downstream neurons in the wide-field motion pathway reveal that local elementary motion detectors exist in pairs that separately correlate contrast polarity channels, ON with ON and OFF with OFF (Joesch et al., 2013). However, behavioral responses to reverse-phi motion of discrete features reveal additional correlations of the opposite signs (Clark et al., 2011). We here present intracellular recordings from feature detecting neurons in the dragonfly that provide direct physiological evidence for the correlation of OFF and ON pathways. These neurons show clear polarity selectivity for feature contrast, responding strongly to targets that are darker than the background and only weakly to dark contrasting edges. These dark target responses are much stronger than the linear combination of responses to ON and OFF edges. We compare these data with output from elementary motion detector-based models (Eichner et al., 2011; Clark et al., 2011), with and without stages of strong center-surround antagonism. Our data support an alternative elementary small target motion detector model, which derives dark target selectivity from the correlation of a delayed OFF with an un-delayed ON signal at each individual visual processing unit (Wiederman et al., 2008, 2009).

Spatial vision in insects is facilitated by shaping the dynamics of visual input through behavioral action

Insects such as flies or bees, with their miniature brains, are able to control highly aerobatic flight maneuvres and to solve spatial vision tasks, such as avoiding collisions with obstacles, landing on objects, or even localizing a previously learnt inconspicuous goal on the basis of environmental cues. With regard to solving such spatial tasks, these insects still outperform man-made autonomous flying systems. To accomplish their extraordinary performance, flies and bees have been shown by their characteristic behavioral actions to actively shape the dynamics of the image flow on their eyes ("optic flow"). The neural processing of information about the spatial layout of the environment is greatly facilitated by segregating the rotational from the translational optic flow component through a saccadic flight and gaze strategy. This active vision strategy thus enables the nervous system to solve apparently complex spatial vision tasks in a particularly efficient and parsimonious way. The key idea of this review is that biological agents, such as flies or bees, acquire at least part of their strength as autonomous systems through active interactions with their environment and not by simply processing passively gained information about the world. These agent-environment interactions lead to adaptive behavior in surroundings of a wide range of complexity. Animals with even tiny brains, such as insects, are capable of performing extraordinarily well in their behavioral contexts by making optimal use of the closed action-perception loop. Model simulations and robotic implementations show that the smart biological mechanisms of motion computation and visually-guided flight control might be helpful to find technical solutions, for example, when designing micro air vehicles carrying a miniaturized, low-weight on-board processor.

Facilitation of dragonfly target-detecting neurons by slow moving features on continuous paths

Dragonflies detect and pursue targets such as other insects for feeding and conspecific interaction. They have a class of neurons highly specialized for this task in their lobula, the “small target motion detecting” (STMD) neurons. One such neuron, CSTMD1, reaches maximum response slowly over hundreds of milliseconds of target motion. Recording the intracellular response from CSTMD1 and a second neuron in this system, BSTMD1, we determined that for the neurons to reach maximum response levels, target motion must produce sequential local activation of elementary motion detecting elements. This facilitation effect is most pronounced when targets move at velocities slower than what was previously thought to be optimal. It is completely disrupted if targets are instantaneously displaced a few degrees from their current location. Additionally, we utilize a simple computational model to discount the parsimonious hypothesis that CSTMD1's slow build-up to maximum response is due to it incorporating a sluggish neural delay filter. Whilst the observed facilitation may be too slow to play a role in prey pursuit flights, which are typically rapidly resolved, we hypothesize that it helps maintain elevated sensitivity during prolonged, aerobatically intricate conspecific pursuits. Since the effect seems to be localized, it most likely enhances the relative salience of the most recently “seen” locations during such pursuit flights.

Neuronal encoding of object and distance information: a model simulation study on naturalistic optic flow processing

We developed a model of the input circuitry of the FD1 cell, an identified motion-sensitive interneuron in the blowfly's visual system. The model circuit successfully reproduces the FD1 cell's most conspicuous property: its larger responses to objects than to spatially extended patterns. The model circuit also mimics the time-dependent responses of FD1 to dynamically complex naturalistic stimuli, shaped by the blowfly's saccadic flight and gaze strategy: the FD1 responses are enhanced when, as a consequence of self-motion, a nearby object crosses the receptive field during intersaccadic intervals. Moreover, the model predicts that these object-induced responses are superimposed by pronounced pattern-dependent fluctuations during movements on virtual test flights in a three-dimensional environment with systematic modifications of the environmental patterns. Hence, the FD1 cell is predicted to detect not unambiguously objects defined by the spatial layout of the environment, but to be also sensitive to objects distinguished by textural features. These ambiguous detection abilities suggest an encoding of information about objects-irrespective of the features by which the objects are defined-by a population of cells, with the FD1 cell presumably playing a prominent role in such an ensemble.

Neural specializations for small target detection in insects

Despite being equipped with low-resolution eyes and tiny brains, many insects show exquisite abilities to detect and pursue targets even in highly textured surrounds. Target tracking behavior is subserved by neurons that are sharply tuned to the motion of small high-contrast targets. These neurons respond robustly to target motion, even against self-generated optic flow. A recent model, supported by neurophysiology, generates target selectivity by being sharply tuned to the unique spatiotemporal profile associated with target motion. Target neurons are likely connected in a complex network where some provide more direct output to behavior, whereas others serve an inter-regulatory role. These interactions may regulate attention and aid in the robust detection of targets in clutter observed in behavior.

Discrimination of features in natural scenes by a dragonfly neuron

Flying insects engage in spectacular high-speed pursuit of targets, requiring visual discrimination of moving objects against cluttered backgrounds. As a first step toward understanding the neural basis for this complex task, we used computational modeling of insect small target motion detector (STMD) neurons to predict responses to features within natural scenes and then compared this with responses recorded from an identified STMD neuron in the dragonfly brain (Hemicordulia tau). A surprising model prediction confirmed by our electrophysiological recordings is that even heavily cluttered scenes contain very few features that excite these neurons, due largely to their exquisite tuning for small features. We also show that very subtle manipulations of the image cause dramatic changes in the response of this neuron, because of the complex inhibitory and facilitatory interactions within the receptive field.

Spatial facilitation by a high-performance dragonfly target-detecting neuron

Many animals visualize and track small moving targets at long distances—be they prey, approaching predators or conspecifics. Insects are an excellent model system for investigating the neural mechanisms that have evolved for this challenging task. Specialized small target motion detector (STMD) neurons in the optic lobes of the insect brain respond strongly even when the target size is below the resolution limit of the eye. Many STMDs also respond robustly to small targets against complex stationary or moving backgrounds. We hypothesized that this requires a complex mechanism to avoid breakthrough responses by background features, and yet to adequately amplify the weak signal of tiny targets. We compared responses of dragonfly STMD neurons to small targets that begin moving within the receptive field with responses to targets that approach the same location along longer trajectories. We find that responses along longer trajectories are strongly facilitated by a mechanism that builds up slowly over several hundred milliseconds. This allows the neurons to give sustained responses to continuous target motion, thus providing a possible explanation for their extraordinary sensitivity.

A syntax of hoverfly flight prototypes

Hoverflies such as Eristalis tenax Linnaeus are known for their distinctive flight style. They can hover on the same spot for several seconds and then burst into movement in apparently any possible direction. In order to determine a quantitative and structured description of complex flight manoeuvres, we searched for a set of repeatedly occurring prototypical movements (PMs) and a set of rules for their ordering. PMs were identified by applying clustering algorithms to the translational and rotational velocities of the body of Eristalis during free-flight sequences. This approach led to nine stable and reliable PMs, and thus provided a tremendous reduction in the complexity of behavioural description. This set of PMs together with the probabilities of transition between them constitute a syntactical description of flight behaviour. The PMs themselves can be roughly segregated into fast rotational turns (saccades) and a variety of distinct translational movements (intersaccadic intervals). We interpret this segregation as reflecting an active sensing strategy which facilitates the extraction of spatial information from retinal image displacements. Detailed analysis of saccades shows that they are performed around all rotational axes individually and in all possible combinations. We found the probability of occurrence of a given saccade type to depend on parameters such as the angle between the long body axis and the direction of flight.

Are bigger brains better?

Attempts to relate brain size to behaviour and cognition have rarely integrated information from insects with that from vertebrates. Many insects, however, demonstrate that highly differentiated motor repertoires, extensive social structures and cognition are possible with very small brains, emphasising that we need to understand the neural circuits, not just the size of brain regions, which underlie these feats. Neural network analyses show that cognitive features found in insects, such as numerosity, attention and categorisation-like processes, may require only very limited neuron numbers. Thus, brain size may have less of a relationship with behavioural repertoire and cognitive capacity than generally assumed, prompting the question of what large brains are for. Larger brains are, at least partly, a consequence of larger neurons that are necessary in large animals due to basic biophysical constraints. They also contain greater replication of neuronal circuits, adding precision to sensory processes, detail to perception, more parallel processing and enlarged storage capacity. Yet, these advantages are unlikely to produce the qualitative shifts in behaviour that are often assumed to accompany increased brain size. Instead, modularity and interconnectivity may be more important.

Local and large-range inhibition in feature detection

Lateral inhibition is perhaps the most ubiquitous of neuronal mechanisms, having been demonstrated in early stages of processing in many different sensory pathways of both mammals and invertebrates. Recent work challenges the long-standing view that assumes that similar mechanisms operate to tune neuronal responses to higher order properties. Scant evidence for lateral inhibition exists beyond the level of the most peripheral stages of visual processing, leading to suggestions that many features of the tuning of higher order visual neurons can be accounted for by the receptive field and other intrinsic coding properties of visual neurons. Using insect target neurons as a model, we present unequivocal evidence that feature tuning is shaped not by intrinsic properties but by potent spatial lateral inhibition operating well beyond the first stages of visual processing. In addition, we present evidence for a second form of higher-order spatial inhibition--a long-range interocular transfer of information that we argue serves a role in establishing interocular rivalry and thus potentially a neural substrate for directing attention to single targets in the presence of distracters. In so doing, we demonstrate not just one, but two levels of spatial inhibition acting beyond the level of peripheral processing.

Feature detection and the hypercomplex property in insects

Discerning a target amongst visual 'clutter' is a complicated task that has been elegantly solved by flying insects, as evidenced by their mid-air interactions with conspecifics and prey. The neurophysiology of small-target motion detectors (STMDs) underlying these complex behaviors has recently been described and suggests that insects use mechanisms similar to those of hypercomplex cells of the mammalian visual cortex to achieve target-specific tuning. Cortical hypercomplex cells are end-stopped, which means that they respond optimally to small moving targets, with responses to extended bars attenuated. We review not only the underlying mechanisms involved in this tuning but also how recently proposed models provide a possible explanation for another remarkable property of these neurons - their ability to respond robustly to the motion of targets even against moving backgrounds.

Distributed dendritic processing facilitates object detection: a computational analysis on the visual system of the fly

Background

Detecting objects is an important task when moving through a natural environment. Flies, for example, may land on salient objects or may avoid collisions with them. The neuronal ensemble of Figure Detection cells (FD-cells) in the visual system of the fly is likely to be involved in controlling these behaviours, as these cells are more sensitive to objects than to extended background structures. Until now the computations in the presynaptic neuronal network of FD-cells and, in particular, the functional significance of the experimentally established distributed dendritic processing of excitatory and inhibitory inputs is not understood.

Methodology/Principal Findings

We use model simulations to analyse the neuronal computations responsible for the preference of FD-cells for small objects. We employed a new modelling approach which allowed us to account for the spatial spread of electrical signals in the dendrites while avoiding detailed compartmental modelling. The models are based on available physiological and anatomical data. Three models were tested each implementing an inhibitory neural circuit, but differing by the spatial arrangement of the inhibitory interaction. Parameter optimisation with an evolutionary algorithm revealed that only distributed dendritic processing satisfies the constraints arising from electrophysiological experiments. In contrast to a direct dendro-dendritic inhibition of the FD-cell (Direct Distributed Inhibition model), an inhibition of its presynaptic retinotopic elements (Indirect Distributed Inhibition model) requires smaller changes in input resistance in the inhibited neurons during visual stimulation.

Conclusions/Significance

Distributed dendritic inhibition of retinotopic elements as implemented in our Indirect Distributed Inhibition model is the most plausible wiring scheme for the neuronal circuit of FD-cells. This microcircuit is computationally similar to lateral inhibition between the retinotopic elements. Hence, distributed inhibition might be an alternative explanation of perceptual phenomena currently explained by lateral inhibition networks.

A model for the detection of moving targets in visual clutter inspired by insect physiology

We present a computational model for target discrimination based on intracellular recordings from neurons in the fly visual system. Determining how insects detect and track small moving features, often against cluttered moving backgrounds, is an intriguing challenge, both from a physiological and a computational perspective. Previous research has characterized higher-order neurons within the fly brain, known as ‘small target motion detectors’ (STMD), that respond robustly to moving features, even when the velocity of the target is matched to the background (i.e. with no relative motion cues). We recorded from intermediate-order neurons in the fly visual system that are well suited as a component along the target detection pathway. This full-wave rectifying, transient cell (RTC) reveals independent adaptation to luminance changes of opposite signs (suggesting separate ON and OFF channels) and fast adaptive temporal mechanisms, similar to other cell types previously described. From this physiological data we have created a numerical model for target discrimination. This model includes nonlinear filtering based on the fly optics, the photoreceptors, the 1^st order interneurons (Large Monopolar Cells), and the newly derived parameters for the RTC. We show that our RTC-based target detection model is well matched to properties described for the STMDs, such as contrast sensitivity, height tuning and velocity tuning. The model output shows that the spatiotemporal profile of small targets is sufficiently rare within natural scene imagery to allow our highly nonlinear ‘matched filter’ to successfully detect most targets from the background. Importantly, this model can explain this type of feature discrimination without the need for relative motion cues.

The processing of color, motion, and stimulus timing are anatomically segregated in the bumblebee brain

Animals use vision to perform such diverse behaviors as finding food, interacting socially with other animals, choosing a mate, and avoiding predators. These behaviors are complex and the visual system must process color, motion, and pattern cues efficiently so that animals can respond to relevant stimuli. The visual system achieves this by dividing visual information into separate pathways, but to what extent are these parallel streams separated in the brain? To answer this question, we recorded intracellularly in vivo from 105 morphologically identified neurons in the lobula, a major visual processing structure of bumblebees (Bombus impatiens). We found that these cells have anatomically segregated dendritic inputs confined to one or two of six lobula layers. Lobula neurons exhibit physiological characteristics common to their respective input layer. Cells with arborizations in layers 1-4 are generally indifferent to color but sensitive to motion, whereas layer 5-6 neurons often respond to both color and motion cues. Furthermore, the temporal characteristics of these responses differ systematically with dendritic branching pattern. Some layers are more temporally precise, whereas others are less precise but more reliable across trials. Because different layers send projections to different regions of the central brain, we hypothesize that the anatomical layers of the lobula are the structural basis for the segregation of visual information into color, motion, and stimulus timing.

Neural mechanisms underlying target detection in a dragonfly centrifugal neuron

Visual identification of targets is an important task for many animals searching for prey or conspecifics. Dragonflies utilize specialized optics in the dorsal acute zone, accompanied by higher-order visual neurons in the lobula complex, and descending neural pathways tuned to the motion of small targets. While recent studies describe the physiology of insect small target motion detector (STMD) neurons, little is known about the mechanisms that underlie their exquisite sensitivity to target motion. Lobula plate tangential cells (LPTCs), a group of neurons in dipteran flies selective for wide-field motion, have been shown to take input from local motion detectors consistent with the classic correlation model developed by Hassenstein and Reichardt in the 1950s. We have tested the hypothesis that similar mechanisms underlie the response of dragonfly STMDs. We show that an anatomically characterized centrifugal STMD neuron (CSTMD1) gives responses that depend strongly on target contrast, a clear prediction of the correlation model. Target stimuli are more complex in spatiotemporal terms than the sinusoidal grating patterns used to study LPTCs, so we used a correlation-based computer model to predict response tuning to velocity and width of moving targets. We show that increasing target width in the direction of travel causes a shift in response tuning to higher velocities, consistent with our model. Finally, we show how the morphology of CSTMD1 allows for impressive spatial interactions when more than one target is present in the visual field.

Visual neuroscience: hypercomplex cells in the arthropod visual system

Newly described visual interneurons in flies have sophisticated receptive field properties reminiscent of neurons in the mammalian visual cortex. The cells are well-suited to compute motion of conspecific females that male flies aerially intercept.