Feature detection and the hypercomplex property in insects

Anatomical organization of the cerebrum of the praying mantis Hierodula membranacea

Many predatory animals, such as the praying mantis, use vision for prey detection and capture. Mantises are known in particular for their capability to estimate distances to prey by stereoscopic vision. While the initial visual processing centers have been extensively documented, we lack knowledge on the architecture of central brain regions, pivotal for sensory motor transformation and higher brain functions. To close this gap, we provide a three-dimensional (3D) reconstruction of the central brain of the Asian mantis, Hierodula membranacea. The atlas facilitates in-depth analysis of neuron ramification regions and aides in elucidating potential neuronal pathways. We integrated seven 3D-reconstructed visual interneurons into the atlas. In total, 42 distinct neuropils of the cerebrum were reconstructed based on synapsin-immunolabeled whole-mount brains. Backfills from the antenna and maxillary palps, as well as immunolabeling of γ-aminobutyric acid (GABA) and tyrosine hydroxylase (TH), further substantiate the identification and boundaries of brain areas. The composition and internal organization of the neuropils were compared to the anatomical organization of the brain of the fruit fly (Drosophila melanogaster) and the two available brain atlases of Polyneoptera-the desert locust (Schistocerca gregaria) and the Madeira cockroach (Rhyparobia maderae). This study paves the way for detailed analyses of neuronal circuitry and promotes cross-species brain comparisons. We discuss differences in brain organization between holometabolous and polyneopteran insects. Identification of ramification sites of the visual neurons integrated into the atlas supports previous claims about homologous structures in the optic lobes of flies and mantises.

Descending neurons of the hoverfly respond to pursuits of artificial targets

Many animals use motion vision information to control dynamic behaviors. Predatory animals, for example, show an exquisite ability to detect rapidly moving prey, followed by pursuit and capture. Such target detection is not only used by predators but is also important in conspecific interactions, such as for male hoverflies defending their territories against conspecific intruders. Visual target detection is believed to be subserved by specialized target-tuned neurons found in a range of species, including vertebrates and arthropods. However, how these target-tuned neurons respond to actual pursuit trajectories is currently not well understood. To redress this, we recorded extracellularly from target-selective descending neurons (TSDNs) in male Eristalis tenax hoverflies. We show that they have dorso-frontal receptive fields with a preferred direction up and away from the visual midline. We reconstructed visual flow fields as experienced during pursuits of artificial targets (black beads). We recorded TSDN responses to six reconstructed pursuits and found that each neuron responded consistently at remarkably specific time points but that these time points differed between neurons. We found that the observed spike probability was correlated with the spike probability predicted from each neuron's receptive field and size tuning. Interestingly, however, the overall response rate was low, with individual neurons responding to only a small part of each reconstructed pursuit. In contrast, the TSDN population responded to substantially larger proportions of the pursuits but with lower probability. This large variation between neurons could be useful if different neurons control different parts of the behavioral output.

Optic flow enrichment via Drosophila head and retina motions to support inflight position regulation

Developing a functional description of the neural control circuits and visual feedback paths underlying insect flight behaviors is an active research area. Feedback controllers incorporating engineering models of the insect visual system outputs have described some flight behaviors, yet they do not explain how insects are able to stabilize their body position relative to nearby targets such as neighbors or forage sources, especially in challenging environments in which optic flow is poor. The insect experimental community is simultaneously recording a growing library of in-flight head and eye motions that may be linked to increased perception. This study develops a quantitative model of the optic flow experienced by a flying insect or robot during head yawing rotations (distinct from lateral peering motions in previous work) with a single other target in view. This study then applies a model of insect visuomotor feedback to show via analysis and simulation of five species that these head motions sufficiently enrich the optic flow and that the output feedback can provide relative position regulation relative to the single target (asymptotic stability). In the simplifying case of pure rotation relative to the body, theoretical analysis provides a stronger stability guarantee. The results are shown to be robust to anatomical neck angle limits and body vibrations, persist with more detailed Drosophila lateral-directional flight dynamics simulations, and generalize to recent retinal motion studies. Together, these results suggest that the optic flow enrichment provided by head or pseudopupil rotation could be used in an insect's neural processing circuit to enable position regulation.

Challenging deep learning models with image distortion based on the abutting grating illusion

Even state-of-the-art deep learning models lack fundamental abilities compared with humans. While many image distortions have been proposed to compare deep learning with humans, they depend on mathematical transformations instead of human cognitive functions. Here, we propose an image distortion based on the abutting grating illusion, which is a phenomenon discovered in humans and animals. The distortion generates illusory contour perception using line gratings abutting each other. We applied the method to MNIST, high-resolution MNIST, and "16-class-ImageNet" silhouettes. Many models, including models trained from scratch and 109 models pretrained with ImageNet or various data augmentation techniques, were tested. Our results show that abutting grating distortion is challenging even for state-of-the-art deep learning models. We discovered that DeepAugment models outperformed other pretrained models. Visualization of early layers indicates that better-performing models exhibit the endstopping property, which is consistent with neuroscience discoveries. Twenty-four human subjects classified distorted samples to validate the distortion.

Visual and motor signatures of locomotion dynamically shape a population code for feature detection in Drosophila

Natural vision is dynamic: as an animal moves, its visual input changes dramatically. How can the visual system reliably extract local features from an input dominated by self-generated signals? In Drosophila, diverse local visual features are represented by a group of projection neurons with distinct tuning properties. Here, we describe a connectome-based volumetric imaging strategy to measure visually evoked neural activity across this population. We show that local visual features are jointly represented across the population, and a shared gain factor improves trial-to-trial coding fidelity. A subset of these neurons, tuned to small objects, is modulated by two independent signals associated with self-movement, a motor-related signal, and a visual motion signal associated with rotation of the animal. These two inputs adjust the sensitivity of these feature detectors across the locomotor cycle, selectively reducing their gain during saccades and restoring it during intersaccadic intervals. This work reveals a strategy for reliable feature detection during locomotion.

Mathematical study of neural feedback roles in small target motion detection

Building an efficient and reliable small target motion detection visual system is challenging for artificial intelligence robotics because a small target only occupies few pixels and hardly displays visual features in images. Biological visual systems that have evolved over millions of years could be ideal templates for designing artificial visual systems. Insects benefit from a class of specialized neurons, called small target motion detectors (STMDs), which endow them with an excellent ability to detect small moving targets against a cluttered dynamic environment. Some bio-inspired models featured in feed-forward information processing architectures have been proposed to imitate the functions of the STMD neurons. However, feedback, a crucial mechanism for visual system regulation, has not been investigated deeply in the STMD-based neural circuits and its roles in small target motion detection remain unclear. In this paper, we propose a time-delay feedback STMD model for small target motion detection in complex backgrounds. The main contributions of this study are as follows. First, a feedback pathway is designed by transmitting information from output-layer neurons to lower-layer interneurons in the STMD pathway and the role of the feedback is analyzed from the view of mathematical analysis. Second, to estimate the feedback constant, the existence and uniqueness of solutions for nonlinear dynamical systems formed by feedback loop are analyzed via Schauder's fixed point theorem and contraction mapping theorem. Finally, an iterative algorithm is designed to solve the nonlinear problem and the performance of the proposed model is tested by experiments. Experimental results demonstrate that the feedback is able to weaken background false positives while maintaining a minor effect on small targets. It outperforms existing STMD-based models regarding the accuracy of fast-moving small target detection in visual clutter. The proposed feedback approach could inspire the relevant modeling of robust motion perception robotics visual systems.

Facilitation of neural responses to targets moving against optic flow

Target detection in visual clutter is a difficult computational task that insects, with their poor spatial resolution compound eyes and small brains, do successfully and with extremely short behavioral delays. We here show that the responses of target selective descending neurons are attenuated by background motion in the same direction as target motion but facilitated by background motion in the opposite direction. This finding is important for understanding how target pursuit can occur in tandem with gaze stabilization. Indeed, the neural facilitation would come into effect if the hoverfly is subjected to background motion in one direction but the target it is pursuing moves in the opposite direction and could therefore be used to override gaze stabilizing corrective turns.

For the human observer, it can be difficult to follow the motion of small objects, especially when they move against background clutter. In contrast, insects efficiently do this, as evidenced by their ability to capture prey, pursue conspecifics, or defend territories, even in highly textured surrounds. We here recorded from target selective descending neurons (TSDNs), which likely subserve these impressive behaviors. To simulate the type of optic flow that would be generated by the pursuer’s own movements through the world, we used the motion of a perspective corrected sparse dot field. We show that hoverfly TSDN responses to target motion are suppressed when such optic flow moves syn-directional to the target. Indeed, neural responses are strongly suppressed when targets move over either translational sideslip or rotational yaw. More strikingly, we show that TSDNs are facilitated by optic flow moving counterdirectional to the target, if the target moves horizontally. Furthermore, we show that a small, frontal spatial window of optic flow is enough to fully facilitate or suppress TSDN responses to target motion. We argue that such TSDN response facilitation could be beneficial in modulating corrective turns during target pursuit.

Modeling Nonlinear Dendritic Processing of Facilitation in a Dragonfly Target-Tracking Neuron

Dragonflies are highly skilled and successful aerial predators that are even capable of selectively attending to one target within a swarm. Detection and tracking of prey is likely to be driven by small target motion detector (STMD) neurons identified from several insect groups. Prior work has shown that dragonfly STMD responses are facilitated by targets moving on a continuous path, enhancing the response gain at the present and predicted future location of targets. In this study, we combined detailed morphological data with computational modeling to test whether a combination of dendritic morphology and nonlinear properties of NMDA receptors could explain these observations. We developed a hybrid computational model of neurons within the dragonfly optic lobe, which integrates numerical and morphological components. The model was able to generate potent facilitation for targets moving on continuous trajectories, including a localized spotlight of maximal sensitivity close to the last seen target location, as also measured during in vivo recordings. The model did not, however, include a mechanism capable of producing a traveling or spreading wave of facilitation. Our data support a strong role for the high dendritic density seen in the dragonfly neuron in enhancing non-linear facilitation. An alternative model based on the morphology of an unrelated type of motion processing neuron from a dipteran fly required more than three times higher synaptic gain in order to elicit similar levels of facilitation, despite having only 20% fewer synapses. Our data support a potential role for NMDA receptors in target tracking and also demonstrate the feasibility of combining biologically plausible dendritic computations with more abstract computational models for basic processing as used in earlier studies.

What gaze direction can tell us about cognitive processes in invertebrates

Most visually guided animals shift their gaze using body movements, eye movements, or both to gather information selectively from their environments. Psychological studies of eye movements have advanced our understanding of perceptual and cognitive processes that mediate visual attention in humans and other vertebrates. However, much less is known about how these processes operate in other organisms, particularly invertebrates. We here make the case that studies of invertebrate cognition can benefit by adding precise measures of gaze direction. To accomplish this, we briefly review the human visual attention literature and outline four research themes and several experimental paradigms that could be extended to invertebrates. We briefly review selected studies where the measurement of gaze direction in invertebrates has provided new insights, and we suggest future areas of exploration.

Visual motion sensitivity in descending neurons in the hoverfly

Many animals use motion vision information to control dynamic behaviors. For example, flying insects must decide whether to pursue a prey or not, to avoid a predator, to maintain their current flight trajectory, or to land. The neural mechanisms underlying the computation of visual motion have been particularly well investigated in the fly optic lobes. However, the descending neurons, which connect the optic lobes with the motor command centers of the ventral nerve cord, remain less studied. To address this deficiency, we describe motion vision sensitive descending neurons in the hoverfly Eristalis tenax. We describe how the neurons can be identified based on their receptive field properties, and how they respond to moving targets, looming stimuli and to widefield optic flow. We discuss their similarities with previously published visual neurons, in the optic lobes and ventral nerve cord, and suggest that they can be classified as target-selective, looming sensitive and optic flow sensitive, based on these similarities. Our results highlight the importance of using several visual stimuli as the neurons can rarely be identified based on only one response characteristic. In addition, they provide an understanding of the neurophysiology of visual neurons that are likely to affect behavior.

Image statistics of the environment surrounding freely behaving hoverflies

Natural scenes are not as random as they might appear, but are constrained in both space and time. The 2-dimensional spatial constraints can be described by quantifying the image statistics of photographs. Human observers perceive images with naturalistic image statistics as more pleasant to view, and both fly and vertebrate peripheral and higher order visual neurons are tuned to naturalistic image statistics. However, for a given animal, what is natural differs depending on the behavior, and even if we have a broad understanding of image statistics, we know less about the scenes relevant for particular behaviors. To mitigate this, we here investigate the image statistics surrounding Episyrphus balteatus hoverflies, where the males hover in sun shafts created by surrounding trees, producing a rich and dense background texture and also intricate shadow patterns on the ground. We quantified the image statistics of photographs of the ground and the surrounding panorama, as the ventral and lateral visual field is particularly important for visual flight control, and found differences in spatial statistics in photos where the hoverflies were hovering compared to where they were flying. Our results can, in the future, be used to create more naturalistic stimuli for experimenter-controlled experiments in the laboratory.

A bio-inspired sighted robot chases like a hoverfly

Here we present a novel bio-inspired visual processing system, which enables a robot to locate and follow a target, using an artificial compound eye called CurvACE. This visual sensor actively scanned the environment at an imposed frequency (50 Hz) with an angular scanning amplitude of [Formula: see text] and succeeded in locating a textured cylindrical target with hyperacuity, i.e. much finer resolution than the coarse inter-receptor angle of the compound eye. Equipped with this small, lightweight visual scanning sensor, a Mecanum-wheeled mobile robot named ACEbot was able to follow a target at a constant distance by localizing the right and left edges of the target. The localization of the target's contrasted edges is based on a bio-inspired summation of Gaussian receptive fields in the visual system. By means of its auto-adaptive pixels, ACEbot consistently achieved similar pursuit performances under various lighting conditions with a high level of repeatability. The robotic pursuit pattern mimics finely the behavior of the male fly Syritta Pipens L. while pursuing the female. The high similarity in the trajectories as well as the biomimicry of the visual system provides strong support for the hypothesis that flies do maintain center the target and constant its subtended angle during smooth pursuit. Moreover, we discuss the fact that such simple strategy can also provide a trajectory compatible with motion camouflage.

Integration of Small- and Wide-Field Visual Features in Target-Selective Descending Neurons of both Predatory and Nonpredatory Dipterans

For many animals, target motion carries high ecological significance as this may be generated by a predator, prey, or potential mate. Indeed, animals whose survival depends on early target detection are often equipped with a sharply tuned visual system, yielding robust performance in challenging conditions. For example, many fast-flying insects use visual cues for identifying targets, such as prey (e.g., predatory dragonflies and robberflies) or conspecifics (e.g., nonpredatory hoverflies), and can often do so against self-generated background optic flow. Supporting these behaviors, the optic lobes of insects that pursue targets harbor neurons that respond robustly to the motion of small moving objects, even when displayed against syn-directional background clutter. However, in diptera, the encoding of target information by the descending neurons, which are more directly involved in generating the behavioral output, has received less attention. We characterized target-selective neurons by recording in the ventral nerve cord of male and female predatory Holcocephala fusca robberflies and of male nonpredatory Eristalis tenax hoverflies. We show that both species have dipteran target-selective descending neurons that only respond to target motion if the background is stationary or moving slowly, moves in the opposite direction, or has un-naturalistic spatial characteristics. The response to the target is suppressed when background and target move at similar velocities, which is strikingly different to the response of target neurons in the optic lobes. As the neurons we recorded from are premotor, our findings affect our interpretation of the neurophysiology underlying target-tracking behaviors.

SIGNIFICANCE STATEMENT Many animals use sensory cues to detect moving targets that may represent predators, prey, or conspecifics. For example, birds of prey show superb sensitivity to the motion of small prey, and intercept these at high speeds. In a similar manner, predatory insects visually track moving prey, often against cluttered backgrounds. Accompanying this behavior, the brains of insects that pursue targets contain neurons that respond exclusively to target motion. We here show that dipteran insects also have target-selective descending neurons in the part of their nervous system that corresponds to the vertebrate spinal cord. Surprisingly, and in contrast to the neurons in the brain, these premotor neurons are inhibited by background patterns moving in the same direction as the target.

Sensory matched filters

As animals move through their environments they are subjected to an endless barrage of sensory signals. Of these, some will be of utmost importance, such as the tell-tale aroma of a potential mate, the distinctive appearance of a vital food source or the unmistakable sound of an approaching predator. Others will be less important. Indeed some will not be important at all. There are, for instance, wide realms of the sensory world that remain entirely undetected, simply because an animal lacks the physiological capacity to detect and analyse the signals that characterise this realm. Take ourselves for example: we are completely insensitive to the Earth's magnetic field, a sensory cue of vital importance as a compass for steering the long distance migration of animals as varied as birds, lobsters and sea turtles. We are also totally oblivious to the rich palette of ultraviolet colours that exist all around us, colours seen by insects, crustaceans, birds, fish and lizards (in fact perhaps by most animals). Nor can we hear the ultrasonic sonar pulses emitted by bats in hot pursuit of flying insect prey. The simple reason for these apparent deficiencies is that we either lack the sensory capacity entirely (as in the case of magnetoreception) or that our existing senses are incapable of detecting specific ranges of the stimulus (such as the ultraviolet wavelength range of light).

Photoreceptor signalling is sufficient to explain the detectability threshold of insect aerial pursuers

An essential biological task for many flying insects is the detection of small, moving targets, such as when pursuing prey or conspecifics. Neural pathways underlying such 'target-detecting' behaviours have been investigated for their sensitivity and tuning properties (size, velocity). However, which stage of neuronal processing limits target detection is not yet known. Here, we investigated several skilled, aerial pursuers (males of four insect species), measuring the target-detection limit (signal-to-noise ratio) of light-adapted photoreceptors. We recorded intracellular responses to moving targets of varying size, extended well below the nominal resolution of single ommatidia. We found that the signal detection limit (2× photoreceptor noise) matches physiological or behavioural target-detection thresholds observed in each species. Thus, across a diverse range of flying insects, individual photoreceptor responses to changes in light intensity establish the sensitivity of the feature detection pathway, indicating later stages of processing are dedicated to feature tuning, tracking and selection.

A predictive focus of gain modulation encodes target trajectories in insect vision

When a human catches a ball, they estimate future target location based on the current trajectory. How animals, small and large, encode such predictive processes at the single neuron level is unknown. Here we describe small target-selective neurons in predatory dragonflies that exhibit localized enhanced sensitivity for targets displaced to new locations just ahead of the prior path, with suppression elsewhere in the surround. This focused region of gain modulation is driven by predictive mechanisms, with the direction tuning shifting selectively to match the target’s prior path. It involves a large local increase in contrast gain which spreads forward after a delay (e.g. an occlusion) and can even transfer between brain hemispheres, predicting trajectories moved towards the visual midline from the other eye. The tractable nature of dragonflies for physiological experiments makes this a useful model for studying the neuronal mechanisms underlying the brain’s remarkable ability to anticipate moving stimuli.

DOI:http://dx.doi.org/10.7554/eLife.26478.001

Catching a ball requires a person to track the speed and direction of a small moving target often against a cluttered and varying background. Predatory insects, like dragonflies, face a similar challenge when they pursue their prey through the air. The task is made a little easier, however, by the fact that most moving targets tend to follow predictable trajectories. Indeed, animals are also better at tracking targets that follow smooth continuous trajectories, suggesting that brains have evolved to exploit the normal behavior of visual stimuli to reduce their workload

To find out how this process works, Wiederman, Fabian et al. studied the brains of dragonflies as they watched a black square intended to mimic prey. Brain cells called Small Target Motion Detectors (or STMD neurons for short) became more active in response to the target. But rather than simply following the target, the STMD neurons instead predicted its future location. In fact, individual neurons were more sensitive to movements occurring just ahead of the target’s current position, and less sensitive to movements occurring elsewhere.

If the target abruptly disappeared, the point in space where the neurons were most sensitive to movement continued to gradually move forward over time. Given that real-life targets typically disappear when they move behind other objects, this suggests that the brain is predicting where the target is most likely to reappear. The STMD neurons became more sensitive to movement by increasing their ability to detect differences in brightness between the target and its background. In some cases, the neurons increased their sensitivity more than five-fold.

Insects and mammals last shared a common ancestor more than 500 million years ago, and, in many respects, mammalian brains are substantially more complex than insect brains. Nevertheless, the results of Wiederman, Fabian et al. show that the insect brain can perform visual tasks that were previously associated only with mammals. Neuroscientists and engineers have used the insect brain for decades to study the circuits that support biological processes. In the coming years, insects such as the dragonfly may enable us to explore how visual regions of the brain predict future events. This knowledge could ultimately be applied to artificial vision systems, such as those in self-driving cars.

DOI:http://dx.doi.org/10.7554/eLife.26478.002

Performance of an insect-inspired target tracker in natural conditions

Robust and efficient target-tracking algorithms embedded on moving platforms, are a requirement for many computer vision and robotic applications. However, deployment of a real-time system is challenging, even with the computational power of modern hardware. As inspiration, we look to biological lightweight solutions-lightweight and low-powered flying insects. For example, dragonflies pursue prey and mates within cluttered, natural environments, deftly selecting their target amidst swarms. In our laboratory, we study the physiology and morphology of dragonfly 'small target motion detector' neurons likely to underlie this pursuit behaviour. Here we describe our insect-inspired tracking model derived from these data and compare its efficacy and efficiency with state-of-the-art engineering models. For model inputs, we use both publicly available video sequences, as well as our own task-specific dataset (small targets embedded within natural scenes). In the context of the tracking problem, we describe differences in object statistics within the video sequences. For the general dataset, our model often locks on to small components of larger objects, tracking these moving features. When input imagery includes small moving targets, for which our highly nonlinear filtering is matched, the robustness outperforms state-of-the-art trackers. In all scenarios, our insect-inspired tracker runs at least twice the speed of the comparison algorithms.

Neural mechanisms of insect navigation

We know more about the ethology of insect navigation than the neural substrates. Few studies have shown direct effects of brain manipulation on navigational behaviour; or measure brain responses that clearly relate to the animal's current location or spatial target, independently of specific sensory cues. This is partly due to the methodological problems of obtaining neural data in a naturally behaving animal. However, substantial indirect evidence, such as comparative anatomy and knowledge of the neural circuits that provide relevant sensory inputs provide converging arguments for the role of some specific brain areas: the mushroom bodies; and the central complex. Finally, modelling can help bridge the gap by relating the computational requirements of a given navigational task to the type of computation offered by different brain areas.

Properties of neuronal facilitation that improve target tracking in natural pursuit simulations

Although flying insects have limited visual acuity (approx. 1°) and relatively small brains, many species pursue tiny targets against cluttered backgrounds with high success. Our previous computational model, inspired by electrophysiological recordings from insect 'small target motion detector' (STMD) neurons, did not account for several key properties described from the biological system. These include the recent observations of response 'facilitation' (a slow build-up of response to targets that move on long, continuous trajectories) and 'selective attention', a competitive mechanism that selects one target from alternatives. Here, we present an elaborated STMD-inspired model, implemented in a closed loop target-tracking system that uses an active saccadic gaze fixation strategy inspired by insect pursuit. We test this system against heavily cluttered natural scenes. Inclusion of facilitation not only substantially improves success for even short-duration pursuits, but it also enhances the ability to 'attend' to one target in the presence of distracters. Our model predicts optimal facilitation parameters that are static in space and dynamic in time, changing with respect to the amount of background clutter and the intended purpose of the pursuit. Our results provide insights into insect neurophysiology and show the potential of this algorithm for implementation in artificial visual systems and robotic applications.

Distinct representation and distribution of visual information by specific cell types in mouse superficial superior colliculus

The superficial superior colliculus (sSC) occupies a critical node in the mammalian visual system; it is one of two major retinorecipient areas, receives visual cortical input, and innervates visual thalamocortical circuits. Nonetheless, the contribution of sSC neurons to downstream neural activity and visually guided behavior is unknown and frequently neglected. Here we identified the visual stimuli to which specific classes of sSC neurons respond, the downstream regions they target, and transgenic mice enabling class-specific manipulations. One class responds to small, slowly moving stimuli and projects exclusively to lateral posterior thalamus; another, comprising GABAergic neurons, responds to the sudden appearance or rapid movement of large stimuli and projects to multiple areas, including the lateral geniculate nucleus. A third class exhibits direction-selective responses and targets deeper SC layers. Together, our results show how specific sSC neurons represent and distribute diverse information and enable direct tests of their functional role.

Contrast sensitivity and the detection of moving patterns and features

Theories based on optimal sampling by the retina have been widely applied to visual ecology at the level of the optics of the eye, supported by visual behaviour. This leads to speculation about the additional processing that must lie in between-in the brain itself. But fewer studies have adopted a quantitative approach to evaluating the detectability of specific features in these neural pathways. We briefly review this approach with a focus on contrast sensitivity of two parallel pathways for motion processing in insects, one used for analysis of wide-field optic flow, the other for detection of small features. We further use a combination of optical modelling of image blur and physiological recording from both photoreceptors and higher-order small target motion detector neurons sensitive to small targets to show that such neurons operate right at the limits imposed by the optics of the eye and the noise level of single photoreceptors. Despite this, and the limitation of only being able to use information from adjacent receptors to detect target motion, they achieve a contrast sensitivity that rivals that of wide-field motion sensitive pathways in either insects or vertebrates-among the highest in absolute terms seen in any animal.

Correlation between OFF and ON channels underlies dark target selectivity in an insect visual system

In both vertebrates and invertebrates, evidence supports separation of luminance increments and decrements (ON and OFF channels) in early stages of visual processing (Hartline, 1938; Joesch et al., 2010); however, less is known about how these parallel pathways are recombined to encode form and motion. In Drosophila, genetic knockdown of inputs to putative ON and OFF pathways and direct recording from downstream neurons in the wide-field motion pathway reveal that local elementary motion detectors exist in pairs that separately correlate contrast polarity channels, ON with ON and OFF with OFF (Joesch et al., 2013). However, behavioral responses to reverse-phi motion of discrete features reveal additional correlations of the opposite signs (Clark et al., 2011). We here present intracellular recordings from feature detecting neurons in the dragonfly that provide direct physiological evidence for the correlation of OFF and ON pathways. These neurons show clear polarity selectivity for feature contrast, responding strongly to targets that are darker than the background and only weakly to dark contrasting edges. These dark target responses are much stronger than the linear combination of responses to ON and OFF edges. We compare these data with output from elementary motion detector-based models (Eichner et al., 2011; Clark et al., 2011), with and without stages of strong center-surround antagonism. Our data support an alternative elementary small target motion detector model, which derives dark target selectivity from the correlation of a delayed OFF with an un-delayed ON signal at each individual visual processing unit (Wiederman et al., 2008, 2009).

Novel flicker-sensitive visual circuit neurons inhibited by stationary patterns

Many animals use visual motion cues for navigating within their surroundings. Both flies and vertebrates compute local motion by temporal correlation of neighboring photoreceptors, via so-called elementary motion detectors (EMDs). In the fly lobula plate and the vertebrate visual cortex the output from many EMDs is pooled in neurons sensitive to wide-field optic flow. Although the EMD has been the preferred model for more than 50 years, recent work has highlighted its limitations in describing some visual behaviors, such as responses to higher-order motion stimuli. Non-EMD motion processing may therefore serve an important function in vision. Here, we describe a novel neuron class in the fly lobula plate that clearly does not derive its input from classic EMDs. The centrifugal stationary inhibited flicker excited (cSIFE) neuron is strongly excited by flicker, up to very high temporal frequencies. The non-EMD driven flicker sensitivity leads to strong, nondirectional responses to high-speed, wide-field motion. Furthermore, cSIFE is strongly inhibited by stationary patterns, within a narrow wavelength band. cSIFE's outputs overlap with the inputs of well described optic flow-sensitive lobula plate tangential cells (LPTCs). Driving cSIFE affects the active dendrites of LPTCs, and cSIFE may therefore play a large role in motion vision.

Visual attention: a cell that focuses on one object at a time

A new study has identified a remarkable neuron in the dragonfly brain that chooses and faithfully follows one and only one prey-like visual target, completely ignoring another, thereby demonstrating a form of competitive selection required for visual attention.

Facilitation of dragonfly target-detecting neurons by slow moving features on continuous paths

Dragonflies detect and pursue targets such as other insects for feeding and conspecific interaction. They have a class of neurons highly specialized for this task in their lobula, the “small target motion detecting” (STMD) neurons. One such neuron, CSTMD1, reaches maximum response slowly over hundreds of milliseconds of target motion. Recording the intracellular response from CSTMD1 and a second neuron in this system, BSTMD1, we determined that for the neurons to reach maximum response levels, target motion must produce sequential local activation of elementary motion detecting elements. This facilitation effect is most pronounced when targets move at velocities slower than what was previously thought to be optimal. It is completely disrupted if targets are instantaneously displaced a few degrees from their current location. Additionally, we utilize a simple computational model to discount the parsimonious hypothesis that CSTMD1's slow build-up to maximum response is due to it incorporating a sluggish neural delay filter. Whilst the observed facilitation may be too slow to play a role in prey pursuit flights, which are typically rapidly resolved, we hypothesize that it helps maintain elevated sensitivity during prolonged, aerobatically intricate conspecific pursuits. Since the effect seems to be localized, it most likely enhances the relative salience of the most recently “seen” locations during such pursuit flights.

Neuronal representation of visual motion and orientation in the fly medulla

In insects, the first extraction of motion and direction clues from local brightness modulations is thought to take place in the medulla. However, whether and how these computations are represented in the medulla stills remain widely unknown, because electrical recording of the neurons in the medulla is difficult. As an effort to overcome this difficulty, we employed local electroporation in vivo in the medulla of the blowfly (Calliphora vicina) to stain small ensembles of neurons with a calcium-sensitive dye. We studied the responses of these neuronal ensembles to spatial and temporal brightness modulations and found selectivity for grating orientation. In contrast, the responses to the two opposite directions of motion of a grating with the same orientation were similar in magnitude, indicating that strong directional selectivity is either not present in the types of neurons covered by our data set, or that direction-selective signals are too closely spaced to be distinguished by our calcium imaging. The calcium responses also showed a bell-shaped dependency on the temporal frequency of drifting gratings, with an optimum higher than that observed in one of the subsequent processing stages, i.e., the lobula plate. Medulla responses were elicited by on- as well as off-stimuli with some spatial heterogeneity in the sensitivity for “on” and “off”, and in the polarity of the responses. Medulla neurons thus show similarities to some established principles of motion and edge detection in the vertebrate visual system.

Joint control of Drosophila male courtship behavior by motion cues and activation of male-specific P1 neurons

Sexual behaviors in animals are governed by inputs from multiple external sensory modalities. However, how these inputs are integrated to jointly control animal behavior is still poorly understood. Whereas visual information alone is not sufficient to induce courtship behavior in Drosophila melanogaster males, when a subset of male-specific fruitless (fru)- and doublesex (dsx)-expressing neurons that respond to chemosensory cues (P1 neurons) were artificially activated via a temperature-sensitive cation channel (dTRPA1), males followed and extended their wing toward moving objects (even a moving piece of rubber band) intensively. When stationary, these objects were not courted. Our results indicate that motion input and activation of P1 neurons are individually necessary, and under our assay conditions, jointly sufficient to elicit early courtship behaviors, and provide insights into how courtship decisions are made via sensory integration.

Neural specializations for small target detection in insects

Despite being equipped with low-resolution eyes and tiny brains, many insects show exquisite abilities to detect and pursue targets even in highly textured surrounds. Target tracking behavior is subserved by neurons that are sharply tuned to the motion of small high-contrast targets. These neurons respond robustly to target motion, even against self-generated optic flow. A recent model, supported by neurophysiology, generates target selectivity by being sharply tuned to the unique spatiotemporal profile associated with target motion. Target neurons are likely connected in a complex network where some provide more direct output to behavior, whereas others serve an inter-regulatory role. These interactions may regulate attention and aid in the robust detection of targets in clutter observed in behavior.

Defining the computational structure of the motion detector in Drosophila

Many animals rely on visual motion detection for survival. Motion information is extracted from spatiotemporal intensity patterns on the retina, a paradigmatic neural computation. A phenomenological model, the Hassenstein-Reichardt correlator (HRC), relates visual inputs to neural activity and behavioral responses to motion, but the circuits that implement this computation remain unknown. By using cell-type specific genetic silencing, minimal motion stimuli, and in vivo calcium imaging, we examine two critical HRC inputs. These two pathways respond preferentially to light and dark moving edges. We demonstrate that these pathways perform overlapping but complementary subsets of the computations underlying the HRC. A numerical model implementing differential weighting of these operations displays the observed edge preferences. Intriguingly, these pathways are distinguished by their sensitivities to a stimulus correlation that corresponds to an illusory percept, "reverse phi," that affects many species. Thus, this computational architecture may be widely used to achieve edge selectivity in motion detection.

Discrimination of features in natural scenes by a dragonfly neuron

Flying insects engage in spectacular high-speed pursuit of targets, requiring visual discrimination of moving objects against cluttered backgrounds. As a first step toward understanding the neural basis for this complex task, we used computational modeling of insect small target motion detector (STMD) neurons to predict responses to features within natural scenes and then compared this with responses recorded from an identified STMD neuron in the dragonfly brain (Hemicordulia tau). A surprising model prediction confirmed by our electrophysiological recordings is that even heavily cluttered scenes contain very few features that excite these neurons, due largely to their exquisite tuning for small features. We also show that very subtle manipulations of the image cause dramatic changes in the response of this neuron, because of the complex inhibitory and facilitatory interactions within the receptive field.

An automated paradigm for Drosophila visual psychophysics

BACKGROUND

Mutations that cause learning and memory defects in Drosophila melanogaster have been found to also compromise visual responsiveness and attention. A better understanding of attention-like defects in such Drosophila mutants therefore requires a more detailed characterization of visual responsiveness across a range of visual parameters.

METHODOLOGY/PRINCIPAL FINDINGS

We designed an automated behavioral paradigm for efficiently dissecting visual responsiveness in Drosophila. Populations of flies walk through multiplexed serial choice mazes while being exposed to moving visuals displayed on computer monitors, and infra-red fly counters at the end of each maze automatically score the responsiveness of a strain. To test our new design, we performed a detailed comparison between wild-type flies and a learning and memory mutant, dunce(1). We first confirmed that the learning mutant dunce(1) displays increased responsiveness to a black/green moving grating compared to wild type in this new design. We then extended this result to explore responses to a wide range of psychophysical parameters for moving gratings (e.g., luminosity, contrast, spatial frequency, velocity) as well as to a different stimulus, moving dots. Finally, we combined these visuals (gratings versus dots) in competition to investigate how dunce(1) and wild-type flies respond to more complex and conflicting motion effects.

CONCLUSIONS/SIGNIFICANCE

We found that dunce(1) responds more strongly than wild type to high contrast and highly structured motion. This effect was found for simple gratings, dots, and combinations of both stimuli presented in competition.

Spatial facilitation by a high-performance dragonfly target-detecting neuron

Many animals visualize and track small moving targets at long distances—be they prey, approaching predators or conspecifics. Insects are an excellent model system for investigating the neural mechanisms that have evolved for this challenging task. Specialized small target motion detector (STMD) neurons in the optic lobes of the insect brain respond strongly even when the target size is below the resolution limit of the eye. Many STMDs also respond robustly to small targets against complex stationary or moving backgrounds. We hypothesized that this requires a complex mechanism to avoid breakthrough responses by background features, and yet to adequately amplify the weak signal of tiny targets. We compared responses of dragonfly STMD neurons to small targets that begin moving within the receptive field with responses to targets that approach the same location along longer trajectories. We find that responses along longer trajectories are strongly facilitated by a mechanism that builds up slowly over several hundred milliseconds. This allows the neurons to give sustained responses to continuous target motion, thus providing a possible explanation for their extraordinary sensitivity.

Vision and visual navigation in nocturnal insects

With their highly sensitive visual systems, nocturnal insects have evolved a remarkable capacity to discriminate colors, orient themselves using faint celestial cues, fly unimpeded through a complicated habitat, and navigate to and from a nest using learned visual landmarks. Even though the compound eyes of nocturnal insects are significantly more sensitive to light than those of their closely related diurnal relatives, their photoreceptors absorb photons at very low rates in dim light, even during demanding nocturnal visual tasks. To explain this apparent paradox, it is hypothesized that the necessary bridge between retinal signaling and visual behavior is a neural strategy of spatial and temporal summation at a higher level in the visual system. Exactly where in the visual system this summation takes place, and the nature of the neural circuitry that is involved, is currently unknown but provides a promising avenue for future research.

Intrinsic activity in the fly brain gates visual information during behavioral choices

The small insect brain is often described as an input/output system that executes reflex-like behaviors. It can also initiate neural activity and behaviors intrinsically, seen as spontaneous behaviors, different arousal states and sleep. However, less is known about how intrinsic activity in neural circuits affects sensory information processing in the insect brain and variability in behavior. Here, by simultaneously monitoring Drosophila's behavioral choices and brain activity in a flight simulator system, we identify intrinsic activity that is associated with the act of selecting between visual stimuli. We recorded neural output (multiunit action potentials and local field potentials) in the left and right optic lobes of a tethered flying Drosophila, while its attempts to follow visual motion (yaw torque) were measured by a torque meter. We show that when facing competing motion stimuli on its left and right, Drosophila typically generate large torque responses that flip from side to side. The delayed onset (0.1–1 s) and spontaneous switch-like dynamics of these responses, and the fact that the flies sometimes oppose the stimuli by flying straight, make this behavior different from the classic steering reflexes. Drosophila, thus, seem to choose one stimulus at a time and attempt to rotate toward its direction. With this behavior, the neural output of the optic lobes alternates; being augmented on the side chosen for body rotation and suppressed on the opposite side, even though the visual input to the fly eyes stays the same. Thus, the flow of information from the fly eyes is gated intrinsically. Such modulation can be noise-induced or intentional; with one possibility being that the fly brain highlights chosen information while ignoring the irrelevant, similar to what we know to occur in higher animals.

Local and large-range inhibition in feature detection

Lateral inhibition is perhaps the most ubiquitous of neuronal mechanisms, having been demonstrated in early stages of processing in many different sensory pathways of both mammals and invertebrates. Recent work challenges the long-standing view that assumes that similar mechanisms operate to tune neuronal responses to higher order properties. Scant evidence for lateral inhibition exists beyond the level of the most peripheral stages of visual processing, leading to suggestions that many features of the tuning of higher order visual neurons can be accounted for by the receptive field and other intrinsic coding properties of visual neurons. Using insect target neurons as a model, we present unequivocal evidence that feature tuning is shaped not by intrinsic properties but by potent spatial lateral inhibition operating well beyond the first stages of visual processing. In addition, we present evidence for a second form of higher-order spatial inhibition--a long-range interocular transfer of information that we argue serves a role in establishing interocular rivalry and thus potentially a neural substrate for directing attention to single targets in the presence of distracters. In so doing, we demonstrate not just one, but two levels of spatial inhibition acting beyond the level of peripheral processing.