Flies see second-order motion

Multisensory integration in Anopheles mosquito swarms: The role of visual and acoustic information in mate tracking and collision avoidance

Male mosquitoes form aerial aggregations, known as swarms, to attract females and maximize their chances of finding a mate. Within these swarms, individuals must be able to recognize potential mates and navigate the dynamic social environment to successfully intercept a mating partner. Prior research has almost exclusively focused on the role of acoustic cues in mediating the male mosquito's ability to recognize and pursue flying females. However, the role of other sensory modalities in this behavior has not been explored. Moreover, how males avoid collisions with one another in the dense swarm while pursuing females remains poorly understood. In this study, we combined free-flight and tethered flight simulator experiments to demonstrate that swarming Anopheles coluzzii mosquitoes integrate visual and acoustic information to track conspecifics and avoid collisions. Our tethered experiments revealed that acoustic stimuli gated mosquito steering responses to visual objects simulating nearby mosquitoes, especially in males that exhibited attraction to visual objects in the presence of female flight tones. Additionally, we observed that visual cues alone could trigger changes in mosquitoes' wingbeat amplitude and frequency. These findings were corroborated by our free-flight experiments, which revealed that mosquitoes modulate their flight responses to nearby conspecifics in a similar manner to tethered animals, allowing for collision avoidance within swarms. Together, these results demonstrate that both males and females integrate multiple sensory inputs to mediate swarming behavior, and for males, the change in flight kinematics in response to multimodal cues allows them to simultaneously track females while avoiding collisions.

Neural mechanisms to incorporate visual counterevidence in self-movement estimation

In selecting appropriate behaviors, animals should weigh sensory evidence both for and against specific beliefs about the world. For instance, animals measure optic flow to estimate and control their own rotation. However, existing models of flow detection can be spuriously triggered by visual motion created by objects moving in the world. Here, we show that stationary patterns on the retina, which constitute evidence against observer rotation, suppress inappropriate stabilizing rotational behavior in the fruit fly Drosophila. In silico experiments show that artificial neural networks (ANNs) that are optimized to distinguish observer movement from external object motion similarly detect stationarity and incorporate negative evidence. Employing neural measurements and genetic manipulations, we identified components of the circuitry for stationary pattern detection, which runs parallel to the fly's local motion and optic-flow detectors. Our results show how the fly brain incorporates negative evidence to improve heading stability, exemplifying how a compact brain exploits geometrical constraints of the visual world.

Zebrafish optomotor response to second-order motion illustrates that age-related changes in motion detection depend on the activated motion system

Various aspects of visual functioning, including motion perception, change with age. Yet, there is a lack of comprehensive understanding of age-related alterations at different stages of motion processing and in each motion system. To understand the effects of aging on second-order motion processing, we investigated optomotor responses (OMR) in younger and older wild-type (AB-strain) and acetylcholinesterase (achesb55/+) mutant zebrafish. The mutant fish with decreased levels of acetylcholinesterase have been shown to have delayed age-related cognitive decline. Compared to previous results on first-order motion, we found distinct changes in OMR to second-order motion. The polarity of OMR was dependent on age, such that second-order stimulation led to mainly negative OMR in the younger group while older zebrafish had positive responses. Hence, these findings revealed an overall aging effect on the detection of second-order motion. Moreover, neither the genotype of zebrafish nor the spatial frequency of motion significantly changed the response magnitude. Our findings support the view that age-related changes in motion detection depend on the activated motion system.

Neural mechanisms to incorporate visual counterevidence in self motion estimation

In selecting appropriate behaviors, animals should weigh sensory evidence both for and against specific beliefs about the world. For instance, animals measure optic flow to estimate and control their own rotation. However, existing models of flow detection can confuse the movement of external objects with genuine self motion. Here, we show that stationary patterns on the retina, which constitute negative evidence against self rotation, are used by the fruit fly Drosophila to suppress inappropriate stabilizing rotational behavior. In silico experiments show that artificial neural networks optimized to distinguish self and world motion similarly detect stationarity and incorporate negative evidence. Employing neural measurements and genetic manipulations, we identified components of the circuitry for stationary pattern detection, which runs parallel to the fly's motion- and optic flow-detectors. Our results exemplify how the compact brain of the fly incorporates negative evidence to improve heading stability, exploiting geometrical constraints of the visual world.

Proprioception gates visual object fixation in flying flies

Visual object tracking in animals as diverse as felines, frogs, and fish supports behaviors including predation, predator avoidance, and landscape navigation. Decades of experimental results show that a rigidly body-fixed tethered fly in a "virtual reality" visual flight simulator steers to follow the motion of a vertical bar, thereby "fixating" it on visual midline. This behavior likely reflects a desire to seek natural features such as plant stalks and has inspired algorithms for visual object tracking predicated on robust responses to bar velocity, particularly near visual midline. Using a modified flight simulator equipped with a magnetic pivot to allow frictionless turns about the yaw axis, we discovered that bar fixation as well as smooth steering responses to bar velocity are attenuated or eliminated in yaw-free conditions. Body-fixed Drosophila melanogaster respond to bar oscillation on a stationary ground with frequency-matched wing kinematics and fixate the bar on midline. Yaw-free flies respond to the same stimulus by ignoring the bar and maintaining their original heading. These differences are driven by proprioceptive signals, rather than visual signals, as artificially "clamping" a bar in the periphery of a yaw-free fly has no effect. When presented with a bar and ground oscillating at different frequencies, a yaw-free fly follows the frequency of the ground only, whereas a body-fixed fly robustly steers at the frequencies of both the bar and ground. Our findings support a model in which proprioceptive feedback promote active damping of high-gain optomotor responses to object motion.

Bioinspired figure-ground discrimination via visual motion smoothing

Flies detect and track moving targets among visual clutter, and this process mainly relies on visual motion. Visual motion is analyzed or computed with the pathway from the retina to T4/T5 cells. The computation of local directional motion was formulated as an elementary movement detector (EMD) model more than half a century ago. Solving target detection or figure-ground discrimination problems can be equivalent to extracting boundaries between a target and the background based on the motion discontinuities in the output of a retinotopic array of EMDs. Individual EMDs cannot measure true velocities, however, due to their sensitivity to pattern properties such as luminance contrast and spatial frequency content. It remains unclear how local directional motion signals are further integrated to enable figure-ground discrimination. Here, we present a computational model inspired by fly motion vision. Simulations suggest that the heavily fluctuating output of an EMD array is naturally surmounted by a lobula network, which is hypothesized to be downstream of the local motion detectors and have parallel pathways with distinct directional selectivity. The lobula network carries out a spatiotemporal smoothing operation for visual motion, especially across time, enabling the segmentation of moving figures from the background. The model qualitatively reproduces experimental observations in the visually evoked response characteristics of one type of lobula columnar (LC) cell. The model is further shown to be robust to natural scene variability. Our results suggest that the lobula is involved in local motion-based target detection.

Long timescale anti-directional rotation in Drosophila optomotor behavior

Locomotor movements cause visual images to be displaced across the eye, a retinal slip that is counteracted by stabilizing reflexes in many animals. In insects, optomotor turning causes the animal to turn in the direction of rotating visual stimuli, thereby reducing retinal slip and stabilizing trajectories through the world. This behavior has formed the basis for extensive dissections of motion vision. Here, we report that under certain stimulus conditions, two Drosophila species, including the widely studied D. melanogaster, can suppress and even reverse the optomotor turning response over several seconds. Such "anti-directional turning" is most strongly evoked by long-lasting, high-contrast, slow-moving visual stimuli that are distinct from those that promote syn-directional optomotor turning. Anti-directional turning, like the syn-directional optomotor response, requires the local motion detecting neurons T4 and T5. A subset of lobula plate tangential cells, CH cells, show involvement in these responses. Imaging from a variety of direction-selective cells in the lobula plate shows no evidence of dynamics that match the behavior, suggesting that the observed inversion in turning direction emerges downstream of the lobula plate. Further, anti-directional turning declines with age and exposure to light. These results show that Drosophila optomotor turning behaviors contain rich, stimulus-dependent dynamics that are inconsistent with simple reflexive stabilization responses.

A Neural Representation of Naturalistic Motion-Guided Behavior in the Zebrafish Brain

All animals must transform ambiguous sensory data into successful behavior. This requires sensory representations that accurately reflect the statistics of natural stimuli and behavior. Multiple studies show that visual motion processing is tuned for accuracy under naturalistic conditions, but the sensorimotor circuits extracting these cues and implementing motion-guided behavior remain unclear. Here we show that the larval zebrafish retina extracts a diversity of naturalistic motion cues, and the retinorecipient pretectum organizes these cues around the elements of behavior. We find that higher-order motion stimuli, gliders, induce optomotor behavior matching expectations from natural scene analyses. We then image activity of retinal ganglion cell terminals and pretectal neurons. The retina exhibits direction-selective responses across glider stimuli, and anatomically clustered pretectal neurons respond with magnitudes matching behavior. Peripheral computations thus reflect natural input statistics, whereas central brain activity precisely codes information needed for behavior. This general principle could organize sensorimotor transformations across animal species.

Differentiation of Theta Visual Motion from Fourier Motion Requires LC16 and R18C12 Neurons in Drosophila

Many animals perceive features of higher-order visual motion that are beyond the spatiotemporal correlations of luminance defined in first-order motion. Although the neural mechanisms of first-order motion detection have become understood in recent years, those underlying higher-order motion perception remain unclear. Here, we established a paradigm to assess the detection of theta motion—a type of higher-order motion—in freely walking Drosophila. Behavioral screening using this paradigm identified two clusters of neurons in the central brain, designated as R18C12, which were required for perception of theta motion but not for first-order motion. Furthermore, theta motion-activated R18C12 neurons were structurally and functionally located downstream of visual projection neurons in lobula, lobula columnar cells LC16, which activated R18C12 neurons via interactions of acetylcholine (ACh) and muscarinic acetylcholine receptors (mAChRs). The current study provides new insights into LC neurons and the neuronal mechanisms underlying visual information processing in complex natural scenes.

•

Perception of theta motion requires LC16 and R18C12 neurons

•

R18C12 neurons are activated by theta motion

•

R18C12 neurons form synaptic connections with LC16 neurons

•

LC16 neurons activate R18C12 neurons through ACh acting on mAChR

Second-order cues to figure motion enable object detection during prey capture by praying mantises

Detecting motion is essential for animals to perform a wide variety of functions. In order to do so, animals could exploit motion cues, including both first-order cues-such as luminance correlation over time-and second-order cues, by correlating higher-order visual statistics. Since first-order motion cues are typically sufficient for motion detection, it is unclear why sensitivity to second-order motion has evolved in animals, including insects. Here, we investigate the role of second-order motion in prey capture by praying mantises. We show that prey detection uses second-order motion cues to detect figure motion. We further present a model of prey detection based on second-order motion sensitivity, resulting from a layer of position detectors feeding into a second layer of elementary-motion detectors. Mantis stereopsis, in contrast, does not require figure motion and is explained by a simpler model that uses only the first layer in both eyes. Second-order motion cues thus enable prey motion to be detected, even when perfectly matching the average background luminance and independent of the elementary motion of any parts of the prey. Subsequent to prey detection, processes such as stereopsis could work to determine the distance to the prey. We thus demonstrate how second-order motion mechanisms enable ecologically relevant behavior such as detecting camouflaged targets for other visual functions including stereopsis and target tracking.

Neural Network Model for Detection of Edges Defined by Image Dynamics

Insects can detect the presence of discrete objects in their visual fields based on a range of differences in spatiotemporal characteristics between the images of object and background. This includes but is not limited to relative motion. Evidence suggests that edge detection is an integral part of this capability, and this study examines the ability of a bio-inspired processing model to detect the presence of boundaries between two regions of a one-dimensional visual field, based on general differences in image dynamics. The model consists of two parts. The first is an early vision module inspired by insect visual processing, which implements adaptive photoreception, ON and OFF channels with transient and sustained characteristics, and delayed and undelayed signal paths. This is replicated for a number of photoreceptors in a small linear array. It is followed by an artificial neural network trained to discriminate the presence vs. absence of an edge based on the array output signals. Input data are derived from natural imagery and feature both static and moving edges between regions with moving texture, flickering texture, and static patterns in all possible combinations. The model can discriminate the presence of edges, stationary or moving, at rates far higher than chance. The resources required (numbers of neurons and visual signals) are realistic relative to those available in the insect second optic ganglion, where the bulk of such processing would be likely to take place.

Pigeons integrate visual motion signals differently than humans

Perceiving motion is a fundamental ability for animals. Primates integrate local 1D motion across orientation and space to compute a rigid 2D motion. It is unknown whether the rule of 2D motion integration is universal within the vertebrate clade; comparative studies of animals with different ecological backgrounds from primates may help answer that question. Here we investigated 2D motion integration in pigeons, using hierarchically structured motion stimuli, namely a barber-pole illusion and plaid motion. The pigeons were trained to report the direction of motion of random dots. When a barber-pole or plaid stimulus was presented, they reported the direction perpendicular to the grating orientation for barber-pole and the vector average of two component gratings for plaid motion. These results demonstrate that pigeons perceive different directions than humans from the same motion stimuli, and suggest that the 2D integrating rules in the primate brain has been elaborated through phylogenetic or ecological factors specific to the clade.

General Visual and Contingent Thermal Cues Interact to Elicit Attraction in Female Aedes aegypti Mosquitoes

Female Aedes aegypti mosquitoes use multiple sensory modalities to hunt human hosts and obtain a blood meal for egg production. Attractive cues include carbon dioxide (CO₂), a major component of exhaled breath [1, 2]; heat elevated above ambient temperature, signifying warm-blooded skin [3, 4]; and dark visual contrast [5, 6], proposed to bridge long-range olfactory and short-range thermal cues [7]. Any of these sensory cues in isolation is an incomplete signal of a human host, and so a mosquito must integrate multimodal sensory information before committing to approaching and biting a person [8]. Here, we study the interaction of visual cues, heat, and CO₂ to investigate the contributions of human-associated stimuli to host-seeking decisions. We show that tethered flying mosquitoes strongly orient toward dark visual contrast, regardless of CO₂ stimulation and internal host-seeking status. This suggests that attraction to visual contrast is general and not contingent on other host cues. In free-flight experiments with CO₂, adding a dark contrasting visual cue to a warmed surface enhanced attraction. Moderate warmth became more attractive to mosquitoes, and mosquitoes aggregated on the cue at all non-noxious temperatures. Gr3 mutants, unable to detect CO₂, were lured to the visual cue at ambient temperatures but fled and did not return when the surface was warmed to host-like temperatures. This suggests that attraction to thermal cues is contingent on the presence of the additional sensory cue CO₂. Our results illustrate that mosquitoes integrate general attractive visual stimuli with context-dependent thermal stimuli to seek promising sites for blood feeding.

Olfactory and Neuromodulatory Signals Reverse Visual Object Avoidance to Approach in Drosophila

Behavioral reactions of animals to environmental sensory stimuli are sometimes reflexive and stereotyped but can also vary depending on contextual conditions. Engaging in active foraging or flight provokes a reversal in the valence of carbon dioxide responses from aversion to approach in Drosophila [1, 2], whereas mosquitoes encountering this same chemical cue show enhanced approach toward a small visual object [3]. Sensory plasticity in insects has been broadly attributed to the action of biogenic amines, which modulate behaviors such as olfactory learning, aggression, feeding, and egg laying [4-14]. Octopamine acts rapidly upon the onset of flight to modulate the response gain of directionally selective motion-detecting neurons in Drosophila [15]. How the action of biogenic amines might couple sensory modalities to each other or to locomotive states remains poorly understood. Here, we use a visual flight simulator [16] equipped for odor delivery [17] to confirm that flies avoid a small contrasting visual object in odorless air [18] but that the same animals reverse their preference to approach in the presence of attractive food odor. An aversive odor does not reverse object aversion. Optogenetic activation of either octopaminergic neurons or directionally selective motion-detecting neurons that express octopamine receptors elicits visual valence reversal in the absence of odor. Our results suggest a parsimonious model in which odor-activated octopamine release excites the motion detection pathway to increase the saliency of either a small object or a bar, eliciting tracking responses by both visual features.

Visuomotor strategies for object approach and aversion in Drosophila melanogaster

Animals classify stimuli to generate appropriate motor actions. In flight, Drosophila melanogaster classify equidistant large and small objects with categorically different behaviors: a tall object evokes approach whereas a small object elicits avoidance. We studied visuomotor behavior in rigidly and magnetically tethered D. melanogaster to reveal strategies that generate aversion to a small object. We discovered that small-object aversion in tethered flight is enabled by aversive saccades and smooth movement, which vary with the stimulus type. Aversive saccades in response to a short bar had different dynamics from approach saccades in response to a tall bar and the distribution of pre-saccade error angles was more stochastic for a short bar. Taken together, we show that aversive responses in D. melanogaster are driven in part by processes that elicit signed saccades with distinct dynamics and trigger mechanisms. Our work generates new hypotheses to study brain circuits that underlie classification of objects in D. melanogaster.

Object features and T4/T5 motion detectors modulate the dynamics of bar tracking by Drosophila

Visual objects can be discriminated by static spatial features such as luminance or dynamic features such as relative movement. Flies track a solid dark vertical bar moving on a bright background, a behavioral reaction so strong that for a rigidly tethered fly, the steering trajectory is phase advanced relative to the moving bar, apparently in anticipation of its future position. By contrast, flickering bars that generate no coherent motion or have a surface texture that moves in the direction opposite to the bar generate steering responses that lag behind the stimulus. It remains unclear how the spatial properties of a bar influence behavioral response dynamics. Here,  we show that a dark bar defined by its luminance contrast to the uniform background drives a co-directional steering response that is phase advanced relative to the response to a textured bar defined only by its motion relative to a stationary textured background. The textured bar drives an initial contra-directional turn and phase-locked tracking. The qualitatively distinct response dynamics could indicate parallel visual processing of a luminance versus motion-defined object. Calcium imaging shows that T4/T5 motion-detecting neurons are more responsive to a solid dark bar than a motion-defined bar. Genetically blocking T4/T5 neurons eliminates the phase-advanced co-directional response to the luminance-defined bar, leaving the orientation response largely intact. We conclude that T4/T5 neurons mediate a co-directional optomotor response to a luminance-defined bar, thereby driving phase-advanced wing kinematics, whereas separate unknown visual pathways elicit the contra-directional orientation response.

Insect Neurobiology: Optical Illusions at the Cellular Level

The reverse-phi illusion has been one key to understanding the use of correlation for motion perception in humans and many other animals. A new study finds the source of this illusion at the cellular level in fruit flies.

Spiking Elementary Motion Detector in Neuromorphic Systems

Apparent motion of the surroundings on an agent's retina can be used to navigate through cluttered environments, avoid collisions with obstacles, or track targets of interest. The pattern of apparent motion of objects, (i.e., the optic flow), contains spatial information about the surrounding environment. For a small, fast-moving agent, as used in search and rescue missions, it is crucial to estimate the distance to close-by objects to avoid collisions quickly. This estimation cannot be done by conventional methods, such as frame-based optic flow estimation, given the size, power, and latency constraints of the necessary hardware. A practical alternative makes use of event-based vision sensors. Contrary to the frame-based approach, they produce so-called events only when there are changes in the visual scene. We propose a novel asynchronous circuit, the spiking elementary motion detector (sEMD), composed of a single silicon neuron and synapse, to detect elementary motion from an event-based vision sensor. The sEMD encodes the time an object's image needs to travel across the retina into a burst of spikes. The number of spikes within the burst is proportional to the speed of events across the retina. A fast but imprecise estimate of the time-to-travel can already be obtained from the first two spikes of a burst and refined by subsequent interspike intervals. The latter encoding scheme is possible due to an adaptive nonlinear synaptic efficacy scaling. We show that the sEMD can be used to compute a collision avoidance direction in the context of robotic navigation in a cluttered outdoor environment and compared the collision avoidance direction to a frame-based algorithm. The proposed computational principle constitutes a generic spiking temporal correlation detector that can be applied to other sensory modalities (e.g., sound localization), and it provides a novel perspective to gating information in spiking neural networks.

Elementary Motion Detection in Drosophila: Algorithms and Mechanisms

Motion in the visual world provides critical information to guide the behavior of sighted animals. Furthermore, as visual motion estimation requires comparisons of signals across inputs and over time, it represents a paradigmatic and generalizable neural computation. Focusing on the Drosophila visual system, where an explosion of technological advances has recently accelerated experimental progress, we review our understanding of how, algorithmically and mechanistically, motion signals are first computed.

Multi-stability with ambiguous visual stimuli in Drosophila orientation behavior

It is widely accepted for humans and higher animals that vision is an active process in which the organism interprets the stimulus. To find out whether this also holds for lower animals, we designed an ambiguous motion stimulus, which serves as something like a multi-stable perception paradigm in Drosophila behavior. Confronted with a uniform panoramic texture in a closed-loop situation in stationary flight, the flies adjust their yaw torque to stabilize their virtual self-rotation. To make the visual input ambiguous, we added a second texture. Both textures got a rotatory bias to move into opposite directions at a constant relative angular velocity. The results indicate that the fly now had three possible frames of reference for self-rotation: either of the two motion components as well as the integrated motion vector of the two. In this ambiguous stimulus situation, the flies generated a continuous sequence of behaviors, each one adjusted to one or another of the three references.

Vision is considered an active process in humans and higher animals in which the stimulus is interpreted by the subject and can be perceived in different ways if it is ambiguous. We aimed to find out whether this also holds for lower animals, such as the fruit fly Drosophila melanogaster. To provide ambiguity, we exposed flies to transparent motion stimuli in a flight simulator and found their behavior to be multi-stable. These results show that the visual system of the fly can separate the individual components of a transparent motion stimulus, and that this kind of stimulus is ambiguous to the fly. The extent to which the fly shows component selectivity in its behavior depends on several properties of the stimulus, like pattern contrast and element density. The alternations between the different behaviors exhibit a stochasticity reminiscent of the temporal dynamics in human multi-stable perception.

A Novel Form of Stereo Vision in the Praying Mantis

Stereopsis is the ability to estimate distance based on the different views seen in the two eyes [1-5]. It is an important model perceptual system in neuroscience and a major area of machine vision. Mammalian, avian, and almost all machine stereo algorithms look for similarities between the luminance-defined images in the two eyes, using a series of computations to produce a map showing how depth varies across the scene [3, 4, 6-14]. Stereopsis has also evolved in at least one invertebrate, the praying mantis [15-17]. Mantis stereopsis is presumed to be simpler than vertebrates' [15, 18], but little is currently known about the underlying computations. Here, we show that mantis stereopsis uses a fundamentally different computational algorithm from vertebrate stereopsis-rather than comparing luminance in the two eyes' images directly, mantis stereopsis looks for regions of the images where luminance is changing. Thus, while there is no evidence that mantis stereopsis works at all with static images, it successfully reveals the distance to a moving target even in complex visual scenes with targets that are perfectly camouflaged against the background in terms of texture. Strikingly, these insects outperform human observers at judging stereoscopic distance when the pattern of luminance in the two eyes does not match. Insect stereopsis has thus evolved to be computationally efficient while being robust to poor image resolution and to discrepancies in the pattern of luminance between the two eyes. VIDEO ABSTRACT.

Drosophila Spatiotemporally Integrates Visual Signals to Control Saccades

Like many visually active animals, including humans, flies generate both smooth and rapid saccadic movements to stabilize their gaze. How rapid body saccades and smooth movement interact for simultaneous object pursuit and gaze stabilization is not understood. We directly observed these interactions in magnetically tethered Drosophila free to rotate about the yaw axis. A moving bar elicited sustained bouts of saccades following the bar, with surprisingly little smooth movement. By contrast, a moving panorama elicited robust smooth movement interspersed with occasional optomotor saccades. The amplitude, angular velocity, and torque transients of bar-fixation saccades were finely tuned to the speed of bar motion and were triggered by a threshold in the temporal integral of the bar error angle rather than its absolute retinal position error. Optomotor saccades were tuned to the dynamics of panoramic image motion and were triggered by a threshold in the integral of velocity over time. A hybrid control model based on integrated motion cues simulates saccade trigger and dynamics. We propose a novel algorithm for tuning fixation saccades in flies.

Asymmetry of Drosophila ON and OFF motion detectors enhances real-world velocity estimation

The reliable estimation of motion across varied surroundings represents a survival-critical task for sighted animals. How neural circuits have adapted to the particular demands of natural environments, however, is not well understood. We explored this question in the visual system of Drosophila melanogaster. Here, as in many mammalian retinas, motion is computed in parallel streams for brightness increments (ON) and decrements (OFF). When genetically isolated, ON and OFF pathways proved equally capable of accurately matching walking responses to realistic motion. To our surprise, detailed characterization of their functional tuning properties through in vivo calcium imaging and electrophysiology revealed stark differences in temporal tuning between ON and OFF channels. We trained an in silico motion estimation model on natural scenes and discovered that our optimized detector exhibited differences similar to those of the biological system. Thus, functional ON-OFF asymmetries in fly visual circuitry may reflect ON-OFF asymmetries in natural environments.

Neurons forming optic glomeruli compute figure-ground discriminations in Drosophila

Many animals rely on visual figure-ground discrimination to aid in navigation, and to draw attention to salient features like conspecifics or predators. Even figures that are similar in pattern and luminance to the visual surroundings can be distinguished by the optical disparity generated by their relative motion against the ground, and yet the neural mechanisms underlying these visual discriminations are not well understood. We show in flies that a diverse array of figure-ground stimuli containing a motion-defined edge elicit statistically similar behavioral responses to one another, and statistically distinct behavioral responses from ground motion alone. From studies in larger flies and other insect species, we hypothesized that the circuitry of the lobula--one of the four, primary neuropiles of the fly optic lobe--performs this visual discrimination. Using calcium imaging of input dendrites, we then show that information encoded in cells projecting from the lobula to discrete optic glomeruli in the central brain group these sets of figure-ground stimuli in a homologous manner to the behavior; "figure-like" stimuli are coded similar to one another and "ground-like" stimuli are encoded differently. One cell class responds to the leading edge of a figure and is suppressed by ground motion. Two other classes cluster any figure-like stimuli, including a figure moving opposite the ground, distinctly from ground alone. This evidence demonstrates that lobula outputs provide a diverse basis set encoding visual features necessary for figure detection.

Spatio-temporal dynamics of impulse responses to figure motion in optic flow neurons

White noise techniques have been used widely to investigate sensory systems in both vertebrates and invertebrates. White noise stimuli are powerful in their ability to rapidly generate data that help the experimenter decipher the spatio-temporal dynamics of neural and behavioral responses. One type of white noise stimuli, maximal length shift register sequences (m-sequences), have recently become particularly popular for extracting response kernels in insect motion vision. We here use such m-sequences to extract the impulse responses to figure motion in hoverfly lobula plate tangential cells (LPTCs). Figure motion is behaviorally important and many visually guided animals orient towards salient features in the surround. We show that LPTCs respond robustly to figure motion in the receptive field. The impulse response is scaled down in amplitude when the figure size is reduced, but its time course remains unaltered. However, a low contrast stimulus generates a slower response with a significantly longer time-to-peak and half-width. Impulse responses in females have a slower time-to-peak than males, but are otherwise similar. Finally we show that the shapes of the impulse response to a figure and a widefield stimulus are very similar, suggesting that the figure response could be coded by the same input as the widefield response.

Method and software for using m-sequences to characterize parallel components of higher-order visual tracking behavior in Drosophila

A moving visual figure may contain first-order signals defined by variation in mean luminance, as well as second-order signals defined by constant mean luminance and variation in luminance envelope, or higher-order signals that cannot be estimated by taking higher moments of the luminance distribution. Separating these properties of a moving figure to experimentally probe the visual subsystems that encode them is technically challenging and has resulted in debated mechanisms of visual object detection by flies. Our prior work took a white noise systems identification approach using a commercially available electronic display system to characterize the spatial variation in the temporal dynamics of two distinct subsystems for first- and higher-order components of visual figure tracking. The method relied on the use of single pixel displacements of two visual stimuli according to two binary maximum length shift register sequences (m-sequences) and cross-correlation of each m-sequence with time-varying flight steering measurements. The resultant spatio-temporal action fields represent temporal impulse responses parameterized by the azimuthal location of the visual figure, one STAF for first-order and another for higher-order components of compound stimuli. Here we review m-sequence and reverse correlation procedures, then describe our application in detail, provide Matlab code, validate the STAFs, and demonstrate the utility and robustness of STAFs by predicting the results of other published experimental procedures. This method has demonstrated how two relatively modest innovations on classical white noise analysis—the inclusion of space as a way to organize response kernels and the use of linear decoupling to measure the response to two channels of visual information simultaneously—could substantially improve our basic understanding of visual processing in the fly.

What can fish brains tell us about visual perception?

Fish are a complex taxonomic group, whose diversity and distance from other vertebrates well suits the comparative investigation of brain and behavior: in fish species we observe substantial differences with respect to the telencephalic organization of other vertebrates and an astonishing variety in the development and complexity of pallial structures. We will concentrate on the contribution of research on fish behavioral biology for the understanding of the evolution of the visual system. We shall review evidence concerning perceptual effects that reflect fundamental principles of the visual system functioning, highlighting the similarities and differences between distant fish groups and with other vertebrates. We will focus on perceptual effects reflecting some of the main tasks that the visual system must attain. In particular, we will deal with subjective contours and optical illusions, invariance effects, second order motion and biological motion and, finally, perceptual binding of object properties in a unified higher level representation.

Higher-order figure discrimination in fly and human vision

Visually-guided animals rely on their ability to stabilize the panorama and simultaneously track salient objects, or figures, that are distinct from the background in order to avoid predators, pursue food resources and mates, and navigate spatially. Visual figures are distinguished by luminance signals that produce coherent motion cues as well as more enigmatic 'higher-order' statistical features. Figure discrimination is thus a complex form of motion vision requiring specialized neural processing. In this minireview, we will highlight recent advances in understanding the perceptual, behavioral, and neurophysiological basis of higher-order figure detection in flies, much of which is grounded in the historical perspective and mechanistic underpinnings of human psychophysics.

Flies and humans share a motion estimation strategy that exploits natural scene statistics

Sighted animals extract motion information from visual scenes by processing spatiotemporal patterns of light falling on the retina. The dominant models for motion estimation exploit intensity correlations only between pairs of points in space and time. Moving natural scenes, however, contain more complex correlations. We found that fly and human visual systems encode the combined direction and contrast polarity of moving edges using triple correlations that enhance motion estimation in natural environments. Both species extracted triple correlations with neural substrates tuned for light or dark edges, and sensitivity to specific triple correlations was retained even as light and dark edge motion signals were combined. Thus, both species separately process light and dark image contrasts to capture motion signatures that can improve estimation accuracy. This convergence argues that statistical structures in natural scenes have greatly affected visual processing, driving a common computational strategy over 500 million years of evolution.

Figure-ground discrimination behavior in Drosophila. II. Visual influences on head movement behavior

Visual identification of small moving targets is a challenge for all moving animals. Their own motion generates displacement of the visual surroundings, inducing wide-field optic flow across the retina. Wide-field optic flow is used to sense perturbations in the flight course. Both ego-motion and corrective optomotor responses confound any attempt to track a salient target moving independently of the visual surroundings. What are the strategies that flying animals use to discriminate small-field figure motion from superimposed wide-field background motion? We examined how fruit flies adjust their gaze in response to a compound visual stimulus comprising a small moving figure against an independently moving wide-field ground, which they do by re-orienting their head or their flight trajectory. We found that fixing the head in place impairs object fixation in the presence of ground motion, and that head movements are necessary for stabilizing wing steering responses to wide-field ground motion when a figure is present. When a figure is moving relative to a moving ground, wing steering responses follow components of both the figure and ground trajectories, but head movements follow only the ground motion. To our knowledge, this is the first demonstration that wing responses can be uncoupled from head responses and that the two follow distinct trajectories in the case of simultaneous figure and ground motion. These results suggest that whereas figure tracking by wing kinematics is independent of head movements, head movements are important for stabilizing ground motion during active figure tracking.

Figure-ground discrimination behavior in Drosophila. I. Spatial organization of wing-steering responses

The behavioral algorithms and neural subsystems for visual figure–ground discrimination are not sufficiently described in any model system. The fly visual system shares structural and functional similarity with that of vertebrates and, like vertebrates, flies robustly track visual figures in the face of ground motion. This computation is crucial for animals that pursue salient objects under the high performance requirements imposed by flight behavior. Flies smoothly track small objects and use wide-field optic flow to maintain flight-stabilizing optomotor reflexes. The spatial and temporal properties of visual figure tracking and wide-field stabilization have been characterized in flies, but how the two systems interact spatially to allow flies to actively track figures against a moving ground has not. We took a systems identification approach in flying Drosophila and measured wing-steering responses to velocity impulses of figure and ground motion independently. We constructed a spatiotemporal action field (STAF) – the behavioral analog of a spatiotemporal receptive field – revealing how the behavioral impulse responses to figure tracking and concurrent ground stabilization vary for figure motion centered at each location across the visual azimuth. The figure tracking and ground stabilization STAFs show distinct spatial tuning and temporal dynamics, confirming the independence of the two systems. When the figure tracking system is activated by a narrow vertical bar moving within the frontal field of view, ground motion is essentially ignored despite comprising over 90% of the total visual input.

Shifting mirrors: adaptive changes in retinal reflections to winter darkness in Arctic reindeer

Arctic reindeer experience extreme changes in environmental light from continuous summer daylight to continuous winter darkness. Here, we show that they may have a unique mechanism to cope with winter darkness by changing the wavelength reflection from their tapetum lucidum (TL). In summer, it is golden with most light reflected back directly through the retina, whereas in winter it is deep blue with less light reflected out of the eye. The blue reflection in winter is associated with significantly increased retinal sensitivity compared with summer animals. The wavelength of reflection depends on TL collagen spacing, with reduced spacing resulting in shorter wavelengths, which we confirmed in summer and winter animals. Winter animals have significantly increased intra-ocular pressure, probably produced by permanent pupil dilation blocking ocular drainage. This may explain the collagen compression. The resulting shift to a blue reflection may scatter light through photoreceptors rather than directly reflecting it, resulting in elevated retinal sensitivity via increased photon capture. This is, to our knowledge, the first description of a retinal structural adaptation to seasonal changes in environmental light. Increased sensitivity occurs at the cost of reduced acuity, but may be an important adaptation in reindeer to detect moving predators in the dark Arctic winter.

Novel flicker-sensitive visual circuit neurons inhibited by stationary patterns

Many animals use visual motion cues for navigating within their surroundings. Both flies and vertebrates compute local motion by temporal correlation of neighboring photoreceptors, via so-called elementary motion detectors (EMDs). In the fly lobula plate and the vertebrate visual cortex the output from many EMDs is pooled in neurons sensitive to wide-field optic flow. Although the EMD has been the preferred model for more than 50 years, recent work has highlighted its limitations in describing some visual behaviors, such as responses to higher-order motion stimuli. Non-EMD motion processing may therefore serve an important function in vision. Here, we describe a novel neuron class in the fly lobula plate that clearly does not derive its input from classic EMDs. The centrifugal stationary inhibited flicker excited (cSIFE) neuron is strongly excited by flicker, up to very high temporal frequencies. The non-EMD driven flicker sensitivity leads to strong, nondirectional responses to high-speed, wide-field motion. Furthermore, cSIFE is strongly inhibited by stationary patterns, within a narrow wavelength band. cSIFE's outputs overlap with the inputs of well described optic flow-sensitive lobula plate tangential cells (LPTCs). Driving cSIFE affects the active dendrites of LPTCs, and cSIFE may therefore play a large role in motion vision.

The Drosophila visual system: From neural circuits to behavior

A compact genome and a tiny brain make Drosophila the prime model to understand the neural substrate of behavior. The neurogenetic efforts to reveal neural circuits underlying Drosophila vision started about half a century ago, and now the field is booming with sophisticated genetic tools, rich behavioral assays, and importantly, a greater number of scientists joining from different backgrounds. This review will briefly cover the structural anatomy of the Drosophila visual system, the animal’s visual behaviors, the genes involved in assembling these circuits, the new and powerful techniques, and the challenges ahead for ultimately identifying the general principles of biological computation in the brain.

 

A typical brain utilizes a great many compact neural circuits to collect and process information from the internal biological and external environmental worlds and generates motor commands for observable behaviors. The fruit fly Drosophila melanogaster, despite of its miniature body and tiny brain, can survive in almost any corner of the world.¹ It can find food, court mate, fight rival conspecific, avoid predators, and amazingly fly without crashing into trees. Drosophila vision and its underlying neuronal machinery has been a key research model for at least half century for neurogeneticists.² Given the efforts invested on the visual system, this animal model is likely to offer the first full understanding of how visual information is computed by a multi-cellular organism. Furthermore, research in Drosophila has revealed many genes that play crucial roles in the formation of functional brains across species. The architectural similarities between the visual systems of Drosophila and vertebrate at the molecular, cellular, and network levels suggest new principles discovered at the circuit level on the relationship between neurons and behavior in Drosophila shall also contribute greatly to our understanding of the general principles for how bigger brains work.³ I start with the anatomy of Drosophila visual system, which surprisingly still contains many uncharted areas.

Distinct acute zones for visual stimuli in different visual tasks in Drosophila

The fruit fly Drosophila melanogaster has a sophisticated visual system and exhibits complex visual behaviors. Visual responses, vision processing and higher cognitive processes in Drosophila have been studied extensively. However, little is known about whether the retinal location of visual stimuli can affect fruit fly performance in various visual tasks. We tested the response of wild-type Berlin flies to visual stimuli at several vertical locations. Three paradigms were used in our study: visual operant conditioning, visual object fixation and optomotor response. We observed an acute zone for visual feature memorization in the upper visual field when visual patterns were presented with a black background. However, when a white background was used, the acute zone was in the lower visual field. Similar to visual feature memorization, the best locations for visual object fixation and optomotor response to a single moving stripe were in the lower visual field with a white background and the upper visual field with a black background. The preferred location for the optomotor response to moving gratings was around the equator of the visual field. Our results suggest that different visual processing pathways are involved in different visual tasks and that there is a certain degree of overlap between the pathways for visual feature memorization, visual object fixation and optomotor response.

Facilitation of dragonfly target-detecting neurons by slow moving features on continuous paths

Dragonflies detect and pursue targets such as other insects for feeding and conspecific interaction. They have a class of neurons highly specialized for this task in their lobula, the “small target motion detecting” (STMD) neurons. One such neuron, CSTMD1, reaches maximum response slowly over hundreds of milliseconds of target motion. Recording the intracellular response from CSTMD1 and a second neuron in this system, BSTMD1, we determined that for the neurons to reach maximum response levels, target motion must produce sequential local activation of elementary motion detecting elements. This facilitation effect is most pronounced when targets move at velocities slower than what was previously thought to be optimal. It is completely disrupted if targets are instantaneously displaced a few degrees from their current location. Additionally, we utilize a simple computational model to discount the parsimonious hypothesis that CSTMD1's slow build-up to maximum response is due to it incorporating a sluggish neural delay filter. Whilst the observed facilitation may be too slow to play a role in prey pursuit flights, which are typically rapidly resolved, we hypothesize that it helps maintain elevated sensitivity during prolonged, aerobatically intricate conspecific pursuits. Since the effect seems to be localized, it most likely enhances the relative salience of the most recently “seen” locations during such pursuit flights.

Lobula-specific visual projection neurons are involved in perception of motion-defined second-order motion in Drosophila

A wide variety of animal species including humans and fruit flies see second-order motion although they lack coherent spatiotemporal correlations in luminance. Recent electrophysiological recordings, together with intensive psychophysical studies, are bringing to light the neural underpinnings of second-order motion perception in mammals. However, where and how the higher-order motion signals are processed in the fly brain is poorly understood. Using the rich genetic tools available in Drosophila and examining optomotor responses in fruit flies to several stimuli, we revealed that two lobula-specific visual projection neurons, specifically connecting the lobula and the central brain, are involved in the perception of motion-defined second-order motion, independent of whether the second-order feature is moving perpendicular or opposite to the local first-order motion. By contrast, blocking these neurons has no effect on first-order and flicker-defined second-order stimuli in terms of response delay. Our results suggest that visual neuropils deep in the optic lobe and the central brain, whose functional roles in motion processing were previously unclear, may be specifically required for motion-defined motion processing.

Higher-order motion sensitivity in fly visual circuits

In higher-order motion stimuli, the direction of object motion does not follow the direction of luminance change. Such stimuli could be generated by the wing movements of a flying butterfly and further complicated by its motion in and out of shadows. Human subjects readily perceive the direction of higher-order motion, although this stands in stark contrast to prevailing motion vision models. Flies and humans compute motion in similar ways, and because flies behaviorally track bars containing higher-order motion cues, they become an attractive model system for investigating the neurophysiology underlying higher-order motion sensitivity. We here use intracellular electrophysiology of motion-vision-sensitive neurons in the hoverfly lobula plate to quantify responses to stimuli containing higher-order motion. We show that motion sensitivity can be broken down into two separate streams, directionally coding for elementary motion and figure motion, respectively, and that responses to Fourier and theta motion can be predicted from these. The sensitivity is affected both by the stimulus' time course and by the neuron's underlying receptive field. Responses to preferred-direction theta motion are sexually dimorphic and particularly robust along the visual midline.

Figure tracking by flies is supported by parallel visual streams

Visual figures may be distinguished based on elementary motion or higher-order non-Fourier features, and flies track both. The canonical elementary motion detector, a compact computation for Fourier motion direction and amplitude, can also encode higher-order signals provided elaborate preprocessing. However, the way in which a fly tracks a moving figure containing both elementary and higher-order signals has not been investigated. Using a novel white noise approach, we demonstrate that (1) the composite response to an object containing both elementary motion (EM) and uncorrelated higher-order figure motion (FM) reflects the linear superposition of each component; (2) the EM-driven component is velocity-dependent, whereas the FM component is driven by retinal position; (3) retinotopic variation in EM and FM responses are different from one another; (4) the FM subsystem superimposes saccadic turns upon smooth pursuit; and (5) the two systems in combination are necessary and sufficient to predict the full range of figure tracking behaviors, including those that generate no EM cues at all. This analysis requires an extension of the model that fly motion vision is based on simple elementary motion detectors and provides a novel method to characterize the subsystems responsible for the pursuit of visual figures.

Bimodal optomotor response to plaids in blowflies: mechanisms of component selectivity and evidence for pattern selectivity

Many animals estimate their self-motion and the movement of external objects by exploiting panoramic patterns of visual motion. To probe how visual systems process compound motion patterns, superimposed visual gratings moving in different directions, plaid stimuli, have been successfully used in vertebrates. Surprisingly, nothing is known about how visually guided insects process plaids. Here, we explored in the blowfly how the well characterized yaw optomotor reflex and the activity of identified visual interneurons depend on plaid stimuli. We show that contrary to previous expectations, the yaw optomotor reflex shows a bimodal directional tuning for certain plaid stimuli. To understand the neural correlates of this behavior, we recorded the responses of a visual interneuron supporting the reflex, the H1 cell, which was also bimodally tuned to the plaid direction. Using a computational model, we identified the essential neural processing steps required to capture the observed response properties. These processing steps have functional parallels with mechanisms found in the primate visual system, despite different biophysical implementations. By characterizing other visual neurons supporting visually guided behaviors, we found responses that ranged from being bimodally tuned to the stimulus direction (component-selective), to responses that appear to be tuned to the direction of the global pattern (pattern-selective). Our results extend the current understanding of neural mechanisms of motion processing in insects, and indicate that the fly employs a wider range of behavioral responses to multiple motion cues than previously reported.

Symmetries in stimulus statistics shape the form of visual motion estimators

The estimation of visual motion has long been studied as a paradigmatic neural computation, and multiple models have been advanced to explain behavioral and neural responses to motion signals. A broad class of models, originating with the Reichardt correlator model, proposes that animals estimate motion by computing a temporal cross-correlation of light intensities from two neighboring points in visual space. These models provide a good description of experimental data in specific contexts but cannot explain motion percepts in stimuli lacking pairwise correlations. Here, we develop a theoretical formalism that can accommodate diverse stimuli and behavioral goals. To achieve this, we treat motion estimation as a problem of Bayesian inference. Pairwise models emerge as one component of the generalized strategy for motion estimation. However, correlation functions beyond second order enable more accurate motion estimation. Prior expectations that are asymmetric with respect to bright and dark contrast use correlations of both even and odd orders, and we show that psychophysical experiments using visual stimuli with symmetric probability distributions for contrast cannot reveal whether the subject uses odd-order correlators for motion estimation. This result highlights a gap in previous experiments, which have largely relied on symmetric contrast distributions. Our theoretical treatment provides a natural interpretation of many visual motion percepts, indicates that motion estimation should be revisited using a broader class of stimuli, demonstrates how correlation-based motion estimation is related to stimulus statistics, and provides multiple experimentally testable predictions.

Automated 3D trajectory measuring of large numbers of moving particles

Complex dynamics of natural particle systems, such as insect swarms, bird flocks, fish schools, has attracted great attention of scientists for years. Measuring 3D trajectory of each individual in a group is vital for quantitative study of their dynamic properties, yet such empirical data is rare mainly due to the challenges of maintaining the identities of large numbers of individuals with similar visual features and frequent occlusions. We here present an automatic and efficient algorithm to track 3D motion trajectories of large numbers of moving particles using two video cameras. Our method solves this problem by formulating it as three linear assignment problems (LAP). For each video sequence, the first LAP obtains 2D tracks of moving targets and is able to maintain target identities in the presence of occlusions; the second one matches the visually similar targets across two views via a novel technique named maximum epipolar co-motion length (MECL), which is not only able to effectively reduce matching ambiguity but also further diminish the influence of frequent occlusions; the last one links 3D track segments into complete trajectories via computing a globally optimal assignment based on temporal and kinematic cues. Experiment results on simulated particle swarms with various particle densities validated the accuracy and robustness of the proposed method. As real-world case, our method successfully acquired 3D flight paths of fruit fly (Drosophila melanogaster) group comprising hundreds of freely flying individuals.

Spatial facilitation by a high-performance dragonfly target-detecting neuron

Many animals visualize and track small moving targets at long distances—be they prey, approaching predators or conspecifics. Insects are an excellent model system for investigating the neural mechanisms that have evolved for this challenging task. Specialized small target motion detector (STMD) neurons in the optic lobes of the insect brain respond strongly even when the target size is below the resolution limit of the eye. Many STMDs also respond robustly to small targets against complex stationary or moving backgrounds. We hypothesized that this requires a complex mechanism to avoid breakthrough responses by background features, and yet to adequately amplify the weak signal of tiny targets. We compared responses of dragonfly STMD neurons to small targets that begin moving within the receptive field with responses to targets that approach the same location along longer trajectories. We find that responses along longer trajectories are strongly facilitated by a mechanism that builds up slowly over several hundred milliseconds. This allows the neurons to give sustained responses to continuous target motion, thus providing a possible explanation for their extraordinary sensitivity.

Neural heterogeneities influence envelope and temporal coding at the sensory periphery

Peripheral sensory neurons respond to stimuli containing a wide range of spatio-temporal frequencies. We investigated electroreceptor neuron coding in the gymnotiform wave-type weakly electric fish Apteronotus leptorhynchus. Previous studies used low to mid temporal frequencies (<256 Hz) and showed that electroreceptor neuron responses to sensory stimuli could be almost exclusively accounted for by linear models, thereby implying a rate code. We instead used temporal frequencies up to 425 Hz, which is in the upper behaviorally relevant range for this species. We show that electroreceptors can: (A) respond up to the highest frequencies tested and (B) display strong nonlinearities in their responses to such stimuli. These nonlinearities were manifested by the fact that the responses to repeated presentations of the same stimulus were coherent at temporal frequencies outside of those contained in the stimulus waveform. Specifically, these consisted of low frequencies corresponding to the time varying contrast or envelope of the stimulus as well as higher harmonics of the frequencies contained in the stimulus. Heterogeneities in the afferent population influenced nonlinear coding as afferents with the lowest baseline firing rates tended to display the strongest nonlinear responses. To understand the link between afferent heterogeneity and nonlinear responsiveness, we used a phenomenological mathematical model of electrosensory afferents. Varying a single parameter in the model was sufficient to account for the variability seen in our experimental data and yielded a prediction: nonlinear responses to the envelope and at higher harmonics are both due to afferents with lower baseline firing rates displaying greater degrees of rectification in their responses. This prediction was verified experimentally as we found that the coherence between the half-wave rectified stimulus and the response resembled the coherence between the responses to repeated presentations of the stimulus in our dataset. This result shows that rectification cannot only give rise to responses to low frequency envelopes but also at frequencies that are higher than those contained in the stimulus. The latter result implies that information is contained in the fine temporal structure of electroreceptor afferent spike trains. Our results show that heterogeneities in peripheral neuronal populations can have dramatic consequences on the nature of the neural code.

Theta motion processing in fruit flies

The tiny brains of insects presumably impose significant computational limitations on algorithms controlling their behavior. Nevertheless, they perform fast and sophisticated visual maneuvers. This includes tracking features composed of second-order motion, in which the feature is defined by higher-order image statistics, but not simple correlations in luminance. Flies can track the true direction of even theta motions, in which the first-order (luminance) motion is directed opposite the second-order moving feature. We exploited this paradoxical feature tracking response to dissect the particular image properties that flies use to track moving objects. We find that theta motion detection is not simply a result of steering toward any spatially restricted flicker. Rather, our results show that fly high-order feature tracking responses can be broken down into positional and velocity components – in other words, the responses can be modeled as a superposition of two independent steering efforts. We isolate these elements to show that each has differing influence on phase and amplitude of steering responses, and together they explain the time course of second-order motion tracking responses during flight. These observations are relevant to natural scenes, where moving features can be much more complex.

Multi-camera real-time three-dimensional tracking of multiple flying animals

Automated tracking of animal movement allows analyses that would not otherwise be possible by providing great quantities of data. The additional capability of tracking in real time--with minimal latency--opens up the experimental possibility of manipulating sensory feedback, thus allowing detailed explorations of the neural basis for control of behaviour. Here, we describe a system capable of tracking the three-dimensional position and body orientation of animals such as flies and birds. The system operates with less than 40 ms latency and can track multiple animals simultaneously. To achieve these results, a multi-target tracking algorithm was developed based on the extended Kalman filter and the nearest neighbour standard filter data association algorithm. In one implementation, an 11-camera system is capable of tracking three flies simultaneously at 60 frames per second using a gigabit network of nine standard Intel Pentium 4 and Core 2 Duo computers. This manuscript presents the rationale and details of the algorithms employed and shows three implementations of the system. An experiment was performed using the tracking system to measure the effect of visual contrast on the flight speed of Drosophila melanogaster. At low contrasts, speed is more variable and faster on average than at high contrasts. Thus, the system is already a useful tool to study the neurobiology and behaviour of freely flying animals. If combined with other techniques, such as 'virtual reality'-type computer graphics or genetic manipulation, the tracking system would offer a powerful new way to investigate the biology of flying animals.

Dynamics of optomotor responses in Drosophila to perturbations in optic flow

For a small flying insect, correcting unplanned course perturbations is essential for navigating through the world. Visual course control relies on estimating optic flow patterns which, in flies, are encoded by interneurons of the third optic ganglion. However, the rules that translate optic flow into flight motor commands remain poorly understood. Here, we measured the temporal dynamics of optomotor responses in tethered flies to optic flow fields about three cardinal axes. For each condition, we used white noise analysis to determine the optimal linear filters linking optic flow to the sum and difference of left and right wing beat amplitudes. The estimated filters indicate that flies react very quickly to perturbations of the motion field, with pure delays in the order of approximately 20 ms and time-to-peak of approximately 100 ms. By convolution the filters also predict responses to arbitrary stimulus sequences, accounting for over half the variance in 5 of our 6 stimulus types, demonstrating the approximate linearity of the system with respect to optic flow variables. In the remaining case of yaw optic flow we improved predictability by measuring individual flies, which also allowed us to analyze the variability of optomotor responses within a population. Finally, the linear filters at least partly explain the optomotor responses to superimposed and decomposed compound flow fields.