The spatial, temporal and contrast properties of expansion and rotation flight optomotor responses inDrosophila

Drosophila flying in augmented reality reveals the vision-based control autonomy of the optomotor response

For walking, swimming, and flying animals, the optomotor response is essential to stabilize gaze. How flexible is the optomotor response? Classic work in Drosophila has argued that flies adapt flight control under augmented visual feedback conditions during goal-directed bar fixation. However, whether the lower-level, reflexive optomotor response can similarly adapt to augmented visual feedback (partially autonomous) or not (autonomous) over long timescales is poorly understood. To address this question, we developed an augmented reality paradigm to study the vision-based control autonomy of the yaw optomotor response of flying fruit flies (Drosophila). Flies were placed in a flight simulator, which permitted free body rotation about the yaw axis. By feeding back body movements in real time to a visual display, we augmented and inverted visual feedback. Thus, this experimental paradigm caused a constant visual error between expected and actual visual feedback to study potential adaptive visuomotor control. By combining experiments with control theory, we demonstrate that the optomotor response is autonomous during augmented reality flight bouts of up to 30 min, which exceeds the reported learning epoch during bar fixation. Agreement between predictions from linear systems theory and experimental data supports the notion that the optomotor response is approximately linear and time invariant within our experimental assay. Even under positive visual feedback, which revealed the stability limit of flies in augmented reality, the optomotor response was autonomous. Our results support a hierarchical motor control architecture in flies with fast and autonomous reflexes at the bottom and more flexible behavior at higher levels.

Contrast sensitivity, visual acuity and the effect of behavioural state on optokinetic gain in fiddler crabs

Most animals rely on visual information for a variety of everyday tasks. The information available to a visual system depends in part on its spatial resolving power and contrast sensitivity. Because of their competing demands for physical space within an eye, these traits cannot simultaneously be improved without increasing overall eye size. The contrast sensitivity function is an integrated measure of visual performance that measures both resolution and contrast sensitivity. Its measurement helps us identify how different species have made a trade-off between contrast sensitivity and spatial resolution. It further allows us to identify the evolutionary drivers of sensory processing and visually mediated behaviour. Here, we measured the contrast sensitivity function of the fiddler crab Gelasimus dampieri using its optokinetic responses to wide-field moving sinusoidal intensity gratings of different orientations, spatial frequencies, contrasts and speeds. We further tested whether the behavioural state of the crabs (i.e. whether crabs are actively walking or not) affects their optokinetic gain and contrast sensitivity. Our results from a group of five crabs suggest a minimum perceived contrast of 6% and a horizontal and vertical visual acuity of 0.4 cyc deg-1 and 0.28 cyc deg-1, respectively, in the crabs' region of maximum optomotor sensitivity. Optokinetic gain increased in moving crabs compared with restrained crabs, adding another example of the importance of naturalistic approaches when studying the performance of animals.

Long-timescale anti-directional rotation in Drosophila optomotor behavior

Locomotor movements cause visual images to be displaced across the eye, a retinal slip that is counteracted by stabilizing reflexes in many animals. In insects, optomotor turning causes the animal to turn in the direction of rotating visual stimuli, thereby reducing retinal slip and stabilizing trajectories through the world. This behavior has formed the basis for extensive dissections of motion vision. Here, we report that under certain stimulus conditions, two Drosophila species, including the widely studied Drosophila melanogaster, can suppress and even reverse the optomotor turning response over several seconds. Such 'anti-directional turning' is most strongly evoked by long-lasting, high-contrast, slow-moving visual stimuli that are distinct from those that promote syn-directional optomotor turning. Anti-directional turning, like the syn-directional optomotor response, requires the local motion detecting neurons T4 and T5. A subset of lobula plate tangential cells, CH cells, show involvement in these responses. Imaging from a variety of direction-selective cells in the lobula plate shows no evidence of dynamics that match the behavior, suggesting that the observed inversion in turning direction emerges downstream of the lobula plate. Further, anti-directional turning declines with age and exposure to light. These results show that Drosophila optomotor turning behaviors contain rich, stimulus-dependent dynamics that are inconsistent with simple reflexive stabilization responses.

Long timescale anti-directional rotation in Drosophila optomotor behavior

Locomotor movements cause visual images to be displaced across the eye, a retinal slip that is counteracted by stabilizing reflexes in many animals. In insects, optomotor turning causes the animal to turn in the direction of rotating visual stimuli, thereby reducing retinal slip and stabilizing trajectories through the world. This behavior has formed the basis for extensive dissections of motion vision. Here, we report that under certain stimulus conditions, two Drosophila species, including the widely studied D. melanogaster, can suppress and even reverse the optomotor turning response over several seconds. Such "anti-directional turning" is most strongly evoked by long-lasting, high-contrast, slow-moving visual stimuli that are distinct from those that promote syn-directional optomotor turning. Anti-directional turning, like the syn-directional optomotor response, requires the local motion detecting neurons T4 and T5. A subset of lobula plate tangential cells, CH cells, show involvement in these responses. Imaging from a variety of direction-selective cells in the lobula plate shows no evidence of dynamics that match the behavior, suggesting that the observed inversion in turning direction emerges downstream of the lobula plate. Further, anti-directional turning declines with age and exposure to light. These results show that Drosophila optomotor turning behaviors contain rich, stimulus-dependent dynamics that are inconsistent with simple reflexive stabilization responses.

Automatic tracking of free-flying insects using a cable-driven robot

Flying insects have evolved to develop efficient strategies to navigate in natural environments. Yet, studying them experimentally is difficult because of their small size and high speed of motion. Consequently, previous studies were limited to tethered flights, hovering flights, or restricted flights within confined laboratory chambers. Here, we report the development of a cable-driven parallel robot, named lab-on-cables, for tracking and interacting with a free-flying insect. In this approach, cameras are mounted on cables, so as to move automatically with the insect. We designed a reactive controller that minimizes the online tracking error between the position of the flying insect, provided by an embedded stereo-vision system, and the position of the moving lab, computed from the cable lengths. We validated the lab-on-cables with Agrotis ipsilon moths (ca. 2 centimeters long) flying freely up to 3 meters per second. We further demonstrated, using prerecorded trajectories, the possibility to track other insects such as fruit flies or mosquitoes. The lab-on-cables is relevant to free-flight studies and may be used in combination with stimulus delivery to assess sensory modulation of flight behavior (e.g., pheromone-controlled anemotaxis in moths).

Ventral motion parallax enhances fruit fly steering to visual sideslip

Flies and other insects use incoherent motion (parallax) to the front and sides to measure distances and identify obstacles during translation. Although additional depth information could be drawn from below, there is no experimental proof that they use it. The finding that blowflies encode motion disparities in their ventral visual fields suggests this may be an important region for depth information. We used a virtual flight arena to measure fruit fly responses to optic flow. The stimuli appeared below (n = 51) or above the fly (n = 44), at different speeds, with or without parallax cues. Dorsal parallax does not affect responses, and similar motion disparities in rotation have no effect anywhere in the visual field. But responses to strong ventral sideslip (206° s⁻¹) change drastically depending on the presence or absence of parallax. Ventral parallax could help resolve ambiguities in cluttered motion fields, and enhance corrective responses to nearby objects.

Luminance Information Is Required for the Accurate Estimation of Contrast in Rapidly Changing Visual Contexts

Visual perception scales with changes in the visual stimulus, or contrast, irrespective of background illumination. However, visual perception is challenged when adaptation is not fast enough to deal with sudden declines in overall illumination, for example, when gaze follows a moving object from bright sunlight into a shaded area. Here, we show that the visual system of the fly employs a solution by propagating a corrective luminance-sensitive signal. We use in vivo 2-photon imaging and behavioral analyses to demonstrate that distinct OFF-pathway inputs encode contrast and luminance. Predictions of contrast-sensitive neuronal responses show that contrast information alone cannot explain behavioral responses in sudden dim light. The luminance-sensitive pathway via the L3 neuron is required for visual processing in such rapidly changing light conditions, ensuring contrast constancy when pure contrast sensitivity underestimates a stimulus. Thus, retaining a peripheral feature, luminance, in visual processing is required for robust behavioral responses.

The effect of vertical extent of stimuli on cockroach optomotor response

Using tethered American cockroaches walking on a trackball in a spherical virtual reality environment, we tested optomotor responses to horizontally moving black-and-white gratings of different vertical extent under six different light intensities. We found that shortening the vertical extent of the wide-field stimulus grating within a light level weakened response strength, reduced average velocity, and decreased angular walking distance. Optomotor responses with the vertically shortened stimuli persisted down to light intensity levels of 0.05 lx. Response latency seems to be independent of both the height of the stimulus and light intensity. The optomotor response started saturating at the light intensity of 5 lx, where the shortest behaviourally significant stimulus was 1°. This indicates that the number of vertical ommatidial rows needed to elicit an optomotor response at 5 lx and above is in the single digits, maybe even just one. Our behavioural results encourage further inquiry into the interplay of light intensity and stimulus size in insect dim-light vision.

Sensory processing by motoneurons: a numerical model for low-level flight control in flies

Rhythmic locomotor behaviour in animals requires exact timing of muscle activation within the locomotor cycle. In rapidly oscillating motor systems, conventional control strategies may be affected by neural delays, making these strategies inappropriate for precise timing control. In flies, wing control thus requires sensory processing within the peripheral nervous system, circumventing the central brain. The underlying mechanism, with which flies integrate graded depolarization of visual interneurons and spiking proprioceptive feedback for precise muscle activation, is under debate. Based on physiological parameters, we developed a numerical model of spike initiation in flight muscles of a blowfly. The simulated Hodgkin–Huxley neuron reproduces multiple experimental findings and explains on the cellular level how vision might control wing kinematics. Sensory processing by single motoneurons appears to be sufficient for control of muscle power during flight in flies and potentially other flying insects, reducing computational load on the central brain during body posture reflexes and manoeuvring flight.

Landing maneuvers of houseflies on vertical and inverted surfaces

Landing maneuvers of flies are complex behaviors which can be conceptually decomposed into sequences of modular actions, including body-deceleration, leg-extension, and body rotations. These behavioral ‘modules’ must be coordinated to ensure well-controlled landing. The composite nature of these behaviors induces kinematic variability, making it difficult to identify the central rules that govern landing. Many previous studies have relied on tethered preparations to study landing behaviors, but tethering induces experimental artefacts by forcing some behaviors to operate in open-feedback control loop while others remain closed-loop. On the other hand, it is harder for the experimenter to control the stimuli experienced by freely-flying insects. One approach towards understanding general mechanisms of landing is to determine the common elements of their kinematics on surfaces of different orientations. We conducted a series of experiments in which the houseflies, Musca domestica, were lured to land on vertical (wall landings) or inverted (ceiling landings) substrates, while their flight was recorded with multiple high-speed cameras. We observed that, in both cases, well-controlled landings occurred when the distance at which flies initiated deceleration was proportional to flight velocity component in the direction of substrate. The ratio of substrate distance and velocity at onset of deceleration (tau) was conserved, despite substantial differences in mechanics of vertical vs. ceiling landings. When these conditions were not satisfied, their landing performance was compromised, causing their heads to collide into the substrate. Unlike body-deceleration, leg-extension in flies was independent of substrate distance or approach velocity. Thus, the robust reflexive visual initiation of deceleration is independent of substrate orientation, and combines with a more variable initiation of leg-extension which depends on surface orientation. Together, these combinations of behaviors enable flies to land in a versatile manner on substrates of various orientations.

Representation of Haltere Oscillations and Integration with Visual Inputs in the Fly Central Complex

The reduced hindwings of flies, known as halteres, are specialized mechanosensory organs that detect body rotations during flight. Primary afferents of the haltere encode its oscillation frequency linearly over a wide bandwidth and with precise phase-dependent spiking. However, it is not currently known whether information from haltere primary afferent neurons is sent to higher brain centers where sensory information about body position could be used in decision making, or whether precise spike timing is useful beyond the peripheral circuits that drive wing movements. We show that in cells in the central brain, the timing and rates of neural spiking can be modulated by sensory input from experimental haltere movements (driven by a servomotor). Using multichannel extracellular recording in restrained flesh flies (Sarcophaga bullata of both sexes), we examined responses of central complex cells to a range of haltere oscillation frequencies alone, and in combination with visual motion speeds and directions. Haltere-responsive units fell into multiple response classes, including those responding to any haltere motion and others with firing rates linearly related to the haltere frequency. Cells with multisensory responses showed higher firing rates than the sum of the unisensory responses at higher haltere frequencies. They also maintained visual properties, such as directional selectivity, while increasing response gain nonlinearly with haltere frequency. Although haltere inputs have been described extensively in the context of rapid locomotion control, we find haltere sensory information in a brain region known to be involved in slower, higher-order behaviors, such as navigation.SIGNIFICANCE STATEMENT Many animals use vision for navigation; however, these cues must be interpreted in the context of the body's position. In mammalian brains, hippocampal cells combine visual and vestibular information to encode head direction. A region of the arthropod brain, known as the central complex (CX), similarly encodes heading information, but it is unknown whether proprioceptive information is integrated here as well. We show that CX neurons respond to input from halteres, specialized proprioceptors in flies that detect body rotations. These neurons also respond to visual input, providing one of the few examples of multiple sensory modalities represented in individual CX cells. Haltere stimulation modifies neural responses to visual signals, providing a mechanism for integrating vision with proprioception.

Spike Burst Coding of Translatory Optic Flow and Depth from Motion in the Fly Visual System

Many animals use the visual motion generated by traveling straight-the translatory optic flow-to successfully navigate obstacles: near objects appear larger and to move more quickly than distant objects. Flies are expert at navigating cluttered environments, and while their visual processing of rotatory optic flow is understood in exquisite detail, how they process translatory optic flow remains a mystery. We present novel cell types that have local motion receptive fields matched to translation self-motion, the vertical translation (VT) cells. One of these, the VT1 cell, encodes self-motion in the forward-sideslip direction and fires action potentials in spike bursts as well as single spikes. We show that the spike burst coding is size and speed-tuned and is selectively modulated by motion parallax-the relative motion experienced during translation. These properties are spatially organized, so that the cell is most excited by clutter rather than isolated objects. When the fly is presented with a simulation of flying past an elevated object, the spike burst activity is modulated by the height of the object, and the rate of single spikes is unaffected. When the moving object alone is experienced, the cell is weakly driven. Meanwhile, the VT2-3 cells have motion receptive fields matched to the lift axis. In conjunction with previously described horizontal cells, the VT cells have properties well suited to the visual navigation of clutter and to encode the fly's movements along near cardinal axes of thrust, lift, and forward sideslip.

High contrast sensitivity for visually guided flight control in bumblebees

Many insects rely on vision to find food, to return to their nest and to carefully control their flight between these two locations. The amount of information available to support these tasks is, in part, dictated by the spatial resolution and contrast sensitivity of their visual systems. Here, we investigate the absolute limits of these visual properties for visually guided position and speed control in Bombus terrestris. Our results indicate that the limit of spatial vision in the translational motion detection system of B. terrestris lies at 0.21 cycles deg⁻¹ with a peak contrast sensitivity of at least 33. In the perspective of earlier findings, these results indicate that bumblebees have higher contrast sensitivity in the motion detection system underlying position control than in their object discrimination system. This suggests that bumblebees, and most likely also other insects, have different visual thresholds depending on the behavioral context.

The optomotor response of the praying mantis is driven predominantly by the central visual field

The optomotor response has been widely used to investigate insect sensitivity to contrast and motion. Several studies have revealed the sensitivity of this response to frequency and contrast, but we know less about the spatial integration underlying this response. Specifically, few studies have investigated how the horizontal angular extent of stimuli influences the optomotor response. We presented mantises with moving gratings of varying horizontal extents at three different contrasts in the central or peripheral regions of their visual fields. We assessed the relative effectivity of different regions to elicit the optomotor response and modelled the dependency of the response on the angular extent subtended by stimuli at these different regions. Our results show that the optomotor response is governed by stimuli in the central visual field and not in the periphery. The model also shows that in the central region, the probability of response increases linearly with increase in horizontal extent up to a saturation point. Furthermore, the dependency of the optomotor response on the angular extent of the stimulus is modulated by contrast. We discuss the implications of our results for different modes of stimulus presentation and for models of the underlying mechanisms of motion detection in the mantis.

A Model for an Angular Velocity-Tuned Motion Detector Accounting for Deviations in the Corridor-Centering Response of the Bee

We present a novel neurally based model for estimating angular velocity (AV) in the bee brain, capable of quantitatively reproducing experimental observations of visual odometry and corridor-centering in free-flying honeybees, including previously unaccounted for manipulations of behaviour. The model is fitted using electrophysiological data, and tested using behavioural data. Based on our model we suggest that the AV response can be considered as an evolutionary extension to the optomotor response. The detector is tested behaviourally in silico with the corridor-centering paradigm, where bees navigate down a corridor with gratings (square wave or sinusoidal) on the walls. When combined with an existing flight control algorithm the detector reproduces the invariance of the average flight path to the spatial frequency and contrast of the gratings, including deviations from perfect centering behaviour as found in the real bee's behaviour. In addition, the summed response of the detector to a unit distance movement along the corridor is constant for a large range of grating spatial frequencies, demonstrating that the detector can be used as a visual odometer.

A higher order visual neuron tuned to the spatial amplitude spectra of natural scenes

Animal sensory systems are optimally adapted to those features typically encountered in natural surrounds, thus allowing neurons with limited bandwidth to encode challengingly large input ranges. Natural scenes are not random, and peripheral visual systems in vertebrates and insects have evolved to respond efficiently to their typical spatial statistics. The mammalian visual cortex is also tuned to natural spatial statistics, but less is known about coding in higher order neurons in insects. To redress this we here record intracellularly from a higher order visual neuron in the hoverfly. We show that the cSIFE neuron, which is inhibited by stationary images, is maximally inhibited when the slope constant of the amplitude spectrum is close to the mean in natural scenes. The behavioural optomotor response is also strongest to images with naturalistic image statistics. Our results thus reveal a close coupling between the inherent statistics of natural scenes and higher order visual processing in insects.

A visual horizon affects steering responses during flight in fruit flies

To navigate well through three-dimensional environments, animals must in some way gauge the distances to objects and features around them. Humans use a variety of visual cues to do this, but insects, with their small size and rigid eyes, are constrained to a more limited range of possible depth cues. For example, insects attend to relative image motion when they move, but cannot change the optical power of their eyes to estimate distance. On clear days, the horizon is one of the most salient visual features in nature, offering clues about orientation, altitude and, for humans, distance to objects. We set out to determine whether flying fruit flies treat moving features as farther off when they are near the horizon. Tethered flies respond strongly to moving images they perceive as close. We measured the strength of steering responses while independently varying the elevation of moving stimuli and the elevation of a virtual horizon. We found responses to vertical bars are increased by negative elevations of their bases relative to the horizon, closely correlated with the inverse of apparent distance. In other words, a bar that dips far below the horizon elicits a strong response, consistent with using the horizon as a depth cue. Wide-field motion also had an enhanced effect below the horizon, but this was only prevalent when flies were additionally motivated with hunger. These responses may help flies tune behaviors to nearby objects and features when they are too far off for motion parallax.

Noise-robust recognition of wide-field motion direction and the underlying neural mechanisms in Drosophila melanogaster

Appropriate and robust behavioral control in a noisy environment is important for the survival of most organisms. Understanding such robust behavioral control has been an attractive subject in neuroscience research. Here, we investigated the processing of wide-field motion with random dot noise at both the behavioral and neuronal level in Drosophila melanogaster. We measured the head yaw optomotor response (OMR) and the activity of motion-sensitive neurons, horizontal system (HS) cells, with in vivo whole-cell patch clamp recordings at various levels of noise intensity. We found that flies had a robust sensation of motion direction under noisy conditions, while membrane potential changes of HS cells were not correlated with behavioral responses. By applying signal classification theory to the distributions of HS cell responses, however, we found that motion direction under noise can be clearly discriminated by HS cells, and that this discrimination performance was quantitatively similar to that of OMR. Furthermore, we successfully reproduced HS cell activity in response to noisy motion stimuli with a local motion detector model including a spatial filter and threshold function. This study provides evidence for the physiological basis of noise-robust behavior in a tiny insect brain.

Fly visual course control: behaviour, algorithms and circuits

Understanding how the brain controls behaviour is undisputedly one of the grand goals of neuroscience research, and the pursuit of this goal has a long tradition in insect neuroscience. However, appropriate techniques were lacking for a long time. Recent advances in genetic and recording techniques now allow the participation of identified neurons in the execution of specific behaviours to be interrogated. By focusing on fly visual course control, I highlight what has been learned about the neuronal circuit modules that control visual guidance in Drosophila melanogaster through the use of these techniques.

Cellular mechanisms for integral feedback in visually guided behavior

Sensory feedback is a ubiquitous feature of guidance systems in both animals and engineered vehicles. For example, a common strategy for moving along a straight path is to turn such that the measured rate of rotation is zero. This task can be accomplished by using a feedback signal that is proportional to the instantaneous value of the measured sensory signal. In such a system, the addition of an integral term depending on past values of the sensory input is needed to eliminate steady-state error [proportional-integral (PI) control]. However, the means by which nervous systems implement such a computation are poorly understood. Here, we show that the optomotor responses of flying Drosophila follow a time course consistent with temporal integration of horizontal motion input. To investigate the cellular basis of this effect, we performed whole-cell patch-clamp recordings from the set of identified visual interneurons [horizontal system (HS) cells] thought to control this reflex during tethered flight. At high stimulus speeds, HS cells exhibit steady-state responses during flight that are absent during quiescence, a state-dependent difference in physiology that is explained by changes in their presynaptic inputs. However, even during flight, the membrane potential of the large-field interneurons exhibits no evidence for integration that could explain the behavioral responses. However, using a genetically encoded indicator, we found that calcium accumulates in the terminals of the interneurons along a time course consistent with the behavior and propose that this accumulation provides a mechanism for temporal integration of sensory feedback consistent with PI control.

Contributions of the 12 neuron classes in the fly lamina to motion vision

Motion detection is a fundamental neural computation performed by many sensory systems. In the fly, local motion computation is thought to occur within the first two layers of the visual system, the lamina and medulla. We constructed specific genetic driver lines for each of the 12 neuron classes in the lamina. We then depolarized and hyperpolarized each neuron type and quantified fly behavioral responses to a diverse set of motion stimuli. We found that only a small number of lamina output neurons are essential for motion detection, while most neurons serve to sculpt and enhance these feedforward pathways. Two classes of feedback neurons (C2 and C3), and lamina output neurons (L2 and L4), are required for normal detection of directional motion stimuli. Our results reveal a prominent role for feedback and lateral interactions in motion processing and demonstrate that motion-dependent behaviors rely on contributions from nearly all lamina neuron classes.

Modular use of peripheral input channels tunes motion-detecting circuitry

In the visual system, peripheral processing circuits are often tuned to specific stimulus features. How this selectivity arises and how these circuits are organized to inform specific visual behaviors is incompletely understood. Using forward genetics and quantitative behavioral studies, we uncover an input channel to motion detecting circuitry in Drosophila. The second-order neuron L3 acts combinatorially with two previously known inputs, L1 and L2, to inform circuits specialized to detect moving light and dark edges. In vivo calcium imaging of L3, combined with neuronal silencing experiments, suggests a neural mechanism to achieve selectivity for moving dark edges. We further demonstrate that different innate behaviors, turning and forward movement, can be independently modulated by visual motion. These two behaviors make use of different combinations of input channels. Such modular use of input channels to achieve feature extraction and behavioral specialization likely represents a general principle in sensory systems.

Visual motion speed determines a behavioral switch from forward flight to expansion avoidance in Drosophila

As an animal translates through the world, its eyes will experience a radiating pattern of optic flow in which there is a focus of expansion directly in front and a focus of contraction behind. For flying fruit flies, recent experiments indicate that flies actively steer away from patterns of expansion. Whereas such a reflex makes sense for avoiding obstacles, it presents a paradox of sorts because an insect could not navigate stably through a visual scene unless it tolerated flight towards a focus of expansion during episodes of forward translation. One possible solution to this paradox is that a fly's behavior might change such that it steers away from strong expansion, but actively steers towards weak expansion. In this study, we use a tethered flight arena to investigate the influence of stimulus strength on the magnitude and direction of turning responses to visual expansion in flies. These experiments indicate that the expansion-avoidance behavior is speed dependent. At slower speeds of expansion, flies exhibit an attraction to the focus of expansion, whereas the behavior transforms to expansion avoidance at higher speeds. Open-loop experiments indicate that this inversion of the expansion-avoidance response depends on whether or not the head is fixed to the thorax. The inversion of the expansion-avoidance response with stimulus strength has a clear manifestation under closed-loop conditions. Flies will actively orient towards a focus of expansion at low temporal frequency but steer away from it at high temporal frequency. The change in the response with temporal frequency does not require motion stimuli directly in front or behind the fly. Animals in which the stimulus was presented within 120 deg sectors on each side consistently steered towards expansion at low temporal frequency and steered towards contraction at high temporal frequency. A simple model based on an array of Hassenstein-Reichardt type elementary movement detectors suggests that the inversion of the expansion-avoidance reflex can explain the spatial distribution of straight flight segments and collision-avoidance saccades when flies fly freely within an open circular arena.

Binocular interactions underlying the classic optomotor responses of flying flies

In response to imposed course deviations, the optomotor reactions of animals reduce motion blur and facilitate the maintenance of stable body posture. In flies, many anatomical and electrophysiological studies suggest that disparate motion cues stimulating the left and right eyes are not processed in isolation but rather are integrated in the brain to produce a cohesive panoramic percept. To investigate the strength of such inter-ocular interactions and their role in compensatory sensory–motor transformations, we utilize a virtual reality flight simulator to record wing and head optomotor reactions by tethered flying flies in response to imposed binocular rotation and monocular front-to-back and back-to-front motion. Within a narrow range of stimulus parameters that generates large contrast insensitive optomotor responses to binocular rotation, we find that responses to monocular front-to-back motion are larger than those to panoramic rotation, but are contrast sensitive. Conversely, responses to monocular back-to-front motion are slower than those to rotation and peak at the lowest tested contrast. Together our results suggest that optomotor responses to binocular rotation result from the influence of non-additive contralateral inhibitory as well as excitatory circuit interactions that serve to confer contrast insensitivity to flight behaviors influenced by rotatory optic flow.

Bio-inspired visual ego-rotation sensor for MAVs

Flies are capable of extraordinary flight maneuvers at very high speeds largely due to their highly elaborate visual system. In this work we present a fly-inspired FPGA based sensor system able to visually sense rotations around different body axes, for use on board micro aerial vehicles (MAVs). Rotation sensing is performed analogously to the fly's VS cell network using zero-crossing detection. An additional key feature of our system is the ease of adding new functionalities akin to the different tasks attributed to the fly's lobula plate tangential cell network, such as object avoidance or collision detection. Our implementation consists of a modified eneo SC-MVC01 SmartCam module and a custom built circuit board, weighing less than 200 g and consuming less than 4 W while featuring 57,600 individual two-dimensional elementary motion detectors, a 185° field of view and a frame rate of 350 frames per second. This makes our sensor system compact in terms of size, weight and power requirements for easy incorporation into MAV platforms, while autonomously performing all sensing and processing on-board and in real time.

An olfactory circuit increases the fidelity of visual behavior

Multimodal integration allows neural circuits to be activated in a behaviorally context-specific manner. In the case of odor plume tracking by Drosophila, an attractive odorant increases the influence of yaw-optic flow on steering behavior in flight, which enhances visual stability reflexes, resulting in straighter flight trajectories within an odor plume. However, it is not well understood whether context-specific changes in optomotor behavior are the result of an increased sensitivity to motion inputs (e.g., through increased visual attention) or direct scaling of motor outputs (i.e., increased steering gain). We address this question by examining the optomotor behavior of Drosophila melanogaster in a tethered flight assay and demonstrate that whereas olfactory cues decrease the gain of the optomotor response to sideslip optic flow, they concomitantly increase the gain of the yaw optomotor response by enhancing the animal's ability to follow transient visual perturbations. Furthermore, ablating the mushroom bodies (MBs) of the fly brain via larval hydroxyurea (HU) treatment results in a loss of olfaction-dependent increase in yaw optomotor fidelity. By expressing either tetanus toxin light chain or diphtheria toxin in gal4-defined neural circuits, we were able to replicate the loss of function observed in the HU treatment within the lines expressing broadly in the mushroom bodies, but not within specific mushroom body lobes. Finally, we were able to genetically separate the yaw responses and sideslip responses in our behavioral assay. Together, our results implicate the MBs in a fast-acting, memory-independent olfactory modification of a visual reflex that is critical for flight control.

Flight activity alters velocity tuning of fly motion-sensitive neurons

Sensory neurons are mostly studied in fixed animals, but their response properties might change when the animal is free to move. Indeed, recent studies found differences between responses of sensory neurons in resting versus moving insects. Since the dynamic range of visual motion stimuli strongly depends on the speed at which an animal is moving, we investigated whether the visual system adapts to such changes in stimulus dynamics as induced by self-motion. Lobula plate tangential cells of flies lend themselves well to study this question because they are known to code for ego-motion based on optic-flow. We recorded the responses of the lobula plate tangential cell H1 to a visual pattern moving at different velocities under three different conditions: fixed flies before and after application of the octopamine agonist chlordimeform (CDM) and tethered flying flies. CDM has been previously shown to induce arousal in flies. We found that flying as well as the application of CDM significantly broadens the velocity tuning of H1 toward higher velocities.

Defining the computational structure of the motion detector in Drosophila

Many animals rely on visual motion detection for survival. Motion information is extracted from spatiotemporal intensity patterns on the retina, a paradigmatic neural computation. A phenomenological model, the Hassenstein-Reichardt correlator (HRC), relates visual inputs to neural activity and behavioral responses to motion, but the circuits that implement this computation remain unknown. By using cell-type specific genetic silencing, minimal motion stimuli, and in vivo calcium imaging, we examine two critical HRC inputs. These two pathways respond preferentially to light and dark moving edges. We demonstrate that these pathways perform overlapping but complementary subsets of the computations underlying the HRC. A numerical model implementing differential weighting of these operations displays the observed edge preferences. Intriguingly, these pathways are distinguished by their sensitivities to a stimulus correlation that corresponds to an illusory percept, "reverse phi," that affects many species. Thus, this computational architecture may be widely used to achieve edge selectivity in motion detection.

Active and passive antennal movements during visually guided steering in flying Drosophila

Insects use feedback from a variety of sensory modalities, including mechanoreceptors on their antennae, to stabilize the direction and speed of flight. Like all arthropod appendages, antennae not only supply sensory information but may also be actively positioned by control muscles. However, how flying insects move their antennae during active turns and how such movements might influence steering responses are currently unknown. Here we examined the antennal movements of flying Drosophila during visually induced turns in a tethered flight arena. In response to both rotational and translational patterns of visual motion, Drosophila actively moved their antennae in a direction opposite to that of the visual motion. We also observed two types of passive antennal movements: small tonic deflections of the antenna and rapid oscillations at wing beat frequency. These passive movements are likely the result of wing-induced airflow and increased in magnitude when the angular distance between the wing and the antenna decreased. In response to rotational visual motion, increases in passive antennal movements appear to trigger a reflex that reduces the stroke amplitude of the contralateral wing, thereby enhancing the visually induced turn. Although the active antennal movements significantly increased antennal oscillation by bringing the arista closer to the wings, it did not significantly affect the turning response in our head-fixed, tethered flies. These results are consistent with the hypothesis that flying Drosophila use mechanosensory feedback to detect changes in the wing induced airflow during visually induced turns and that this feedback plays a role in regulating the magnitude of steering responses.

Neural correlates of illusory motion perception in Drosophila

When the contrast of an image flickers as it moves, humans perceive an illusory reversal in the direction of motion. This classic illusion, called reverse-phi motion, has been well-characterized using psychophysics, and several models have been proposed to account for its effects. Here, we show that Drosophila melanogaster also respond behaviorally to the reverse-phi illusion and that the illusion is present in dendritic calcium signals of motion-sensitive neurons in the fly lobula plate. These results closely match the predictions of the predominant model of fly motion detection. However, high flicker rates cause an inversion of the reverse-phi behavioral response that is also present in calcium signals of lobula plate tangential cell dendrites but not predicted by the model. The fly's behavioral and neural responses to the reverse-phi illusion reveal unexpected interactions between motion and flicker signals in the fly visual system and suggest that a similar correlation-based mechanism underlies visual motion detection across the animal kingdom.

A model of visual-olfactory integration for odour localisation in free-flying fruit flies

Flying fruit flies (Drosophila melanogaster) locate a concealed appetitive odour source most accurately in environments containing vertical visual contrasts. To investigate how visuomotor and olfactory responses may be integrated, we examine the free-flight behaviour of flies in three visual conditions, with and without food odour present. While odour localisation is facilitated by uniformly distributed vertical contrast as compared with purely horizontal contrast, localised vertical contrast also facilitates odour localisation, but only if the odour source is situated close to it. We implement a model of visuomotor control consisting of three parallel subsystems: an optomotor response stabilising the model fly's yaw orientation; a collision avoidance system to saccade away from looming obstacles; and a speed regulation system. This model reproduces many of the behaviours we observe in flies, including visually mediated 'rebound' turns following saccades. Using recordings of real odour plumes, we simulate the presence of an odorant in the arena, and investigate ways in which the olfactory input could modulate visuomotor control. We reproduce the experimental results by using the change in odour intensity to regulate the sensitivity of collision avoidance, resulting in visually mediated chemokinesis. Additionally, it is necessary to amplify the optomotor response whenever odour is present, increasing the model fly's tendency to steer towards features of the visual environment. We conclude that visual and olfactory responses of Drosophila are not independent, but that relatively simple interaction between these modalities can account for the observed visual dependence of odour source localisation.

Drosophila fly straight by fixating objects in the face of expanding optic flow

Flies, like all animals that depend on vision to navigate through the world, must integrate the optic flow created by self-motion with the images generated by prominent features in their environment. Although much is known about the responses of Drosophila melanogaster to rotating flow fields, their reactions to the more complex patterns of motion that occur as they translate through the world are not well understood. In the present study we explore the interactions between two visual reflexes in Drosophila: object fixation and expansion avoidance. As a fly flies forward, it encounters an expanding visual flow field. However, recent results have demonstrated that Drosophila strongly turn away from patterns of expansion. Given the strength of this reflex, it is difficult to explain how flies make forward progress through a visual landscape. This paradox is partially resolved by the finding reported here that when undergoing flight directed towards a conspicuous object, Drosophila will tolerate a level of expansion that would otherwise induce avoidance. This navigation strategy allows flies to fly straight when orienting towards prominent visual features.

Walking modulates speed sensitivity in Drosophila motion vision

Changes in behavioral state modify neural activity in many systems. In some vertebrates such modulation has been observed and interpreted in the context of attention and sensorimotor coordinate transformations. Here we report state-dependent activity modulations during walking in a visual-motor pathway of Drosophila. We used two-photon imaging to monitor intracellular calcium activity in motion-sensitive lobula plate tangential cells (LPTCs) in head-fixed Drosophila walking on an air-supported ball. Cells of the horizontal system (HS)--a subgroup of LPTCs--showed stronger calcium transients in response to visual motion when flies were walking rather than resting. The amplified responses were also correlated with walking speed. Moreover, HS neurons showed a relatively higher gain in response strength at higher temporal frequencies, and their optimum temporal frequency was shifted toward higher motion speeds. Walking-dependent modulation of HS neurons in the Drosophila visual system may constitute a mechanism to facilitate processing of higher image speeds in behavioral contexts where these speeds of visual motion are relevant for course stabilization.

Dynamics of optomotor responses in Drosophila to perturbations in optic flow

For a small flying insect, correcting unplanned course perturbations is essential for navigating through the world. Visual course control relies on estimating optic flow patterns which, in flies, are encoded by interneurons of the third optic ganglion. However, the rules that translate optic flow into flight motor commands remain poorly understood. Here, we measured the temporal dynamics of optomotor responses in tethered flies to optic flow fields about three cardinal axes. For each condition, we used white noise analysis to determine the optimal linear filters linking optic flow to the sum and difference of left and right wing beat amplitudes. The estimated filters indicate that flies react very quickly to perturbations of the motion field, with pure delays in the order of approximately 20 ms and time-to-peak of approximately 100 ms. By convolution the filters also predict responses to arbitrary stimulus sequences, accounting for over half the variance in 5 of our 6 stimulus types, demonstrating the approximate linearity of the system with respect to optic flow variables. In the remaining case of yaw optic flow we improved predictability by measuring individual flies, which also allowed us to analyze the variability of optomotor responses within a population. Finally, the linear filters at least partly explain the optomotor responses to superimposed and decomposed compound flow fields.

Multisensory systems integration for high-performance motor control in flies

Engineered tracking systems 'fuse' data from disparate sensor platforms, such as radar and video, to synthesize information that is more reliable than any single input. The mammalian brain registers visual and auditory inputs to directionally localize an interesting environmental feature. For a fly, sensory perception is challenged by the extreme performance demands of high speed flight. Yet even a fruit fly can robustly track a fragmented odor plume through varying visual environments, outperforming any human engineered robot. Flies integrate disparate modalities, such as vision and olfaction, which are neither related by spatiotemporal spectra nor processed by registered neural tissue maps. Thus, the fly is motivating new conceptual frameworks for how low-level multisensory circuits and functional algorithms produce high-performance motor control.

Processing of horizontal optic flow in three visual interneurons of the Drosophila brain

Motion vision is essential for navigating through the environment. Due to its genetic amenability, the fruit fly Drosophila has been serving for a lengthy period as a model organism for studying optomotor behavior as elicited by large-field horizontal motion. However, the neurons underlying the control of this behavior have not been studied in Drosophila so far. Here we report the first whole cell recordings from three cells of the horizontal system (HSN, HSE, and HSS) in the lobula plate of Drosophila. All three HS cells are tuned to large-field horizontal motion in a direction-selective way; they become excited by front-to-back motion and inhibited by back-to-front motion in the ipsilateral field of view. The response properties of HS cells such as contrast and velocity dependence are in accordance with the correlation-type model of motion detection. Neurobiotin injection suggests extensive coupling among ipsilateral HS cells and additional coupling to tangential cells that have their dendrites in the contralateral hemisphere of the brain. This connectivity scheme accounts for the complex layout of their receptive fields and explains their sensitivity both to ipsilateral and to contralateral motion. Thus the main response properties of Drosophila HS cells are strikingly similar to the responses of their counterparts in the blowfly Calliphora, although we found substantial differences with respect to their dendritic structure and connectivity. This long-awaited functional characterization of HS cells in Drosophila provides the basis for the future dissection of optomotor behavior and the underlying neural circuitry by combining genetics, physiology, and behavior.

Visual stabilization dynamics are enhanced by standing flight velocity

A flying insect must travel to find food, mates and sites for oviposition, but for a small animal in a turbulent world this means dealing with frequent unplanned deviations from course. We measured a fly's sensory-motor impulse response to perturbations in optic flow. After an abrupt change in its apparent visual position, a fly generates a compensatory dynamical steering response in the opposite direction. The response dynamics, however, may be influenced by superimposed background velocity generated by the animal's flight direction. Here we show that constant forward velocity has no effect on the steering responses to orthogonal sideslip perturbations, whereas constant parallel sideslip substantially shortens the lags and relaxation times of the linear dynamical responses. This implies that for flies stabilizing in sideslip, the control effort is strongly affected by the direction of background motion.

The role of experience in flight behaviour ofDrosophila

Experience plays a key role in the acquisition of complex motor skills in running and flight of many vertebrates. To evaluate the significance of previous experience for the efficiency of motor behaviour in an insect, we investigated the flight behaviour of the fruit fly Drosophila. We reared flies in chambers in which the animals could freely walk and extend their wings, but could not gain any flight experience. These naive animals were compared with control flies under both open- and closed-loop tethered flight conditions in a flight simulator as well as in a free-flight arena. The data suggest that the overall flight behaviour in Drosophila seems to be predetermined because both groups exhibited similar mean stroke amplitude and stroke frequency, similar open-loop responses to visual stimulation and the immediate ability to track visual objects under closed-loop feedback conditions. In short free flight bouts, peak saccadic turning rate, angular acceleration, peak horizontal speed and flight altitude were also similar in naive and control flies. However, we found significant changes in other key parameters in naive animals such as a reduction in mean horizontal speed(–23%) and subtle changes in mean turning rate (–48%). Naive flies produced 25% less yaw torque-equivalent stroke amplitudes than the controls in response to a visual stripe rotating in open loop around the tethered animal,potentially suggesting a flight-dependent adaptation of the visuo-motor gain in the control group. This change ceased after the animals experienced visual closed-loop feedback. During closed-loop flight conditions, naive flies had 53% larger differences in left and right stroke amplitude when fixating a visual object, thus steering control was less precise. We discuss two alternative hypotheses to explain our results: the `neuronal experience'hypothesis, suggesting that there are some elements of learning and fine-tuning involved during the first flight experiences in Drosophila and the `muscular exercise' hypothesis. Our experiments support the first hypothesis because maximum locomotor capacity seems not to be significantly impaired in the naive group. Although this study primarily confirms the genetic pre-disposition for flight in Drosophila,previous experience may apparently adjust locomotor fine control and aerial performance, although this effect seems to be small compared with vertebrates.

The generation of forces and moments during visual-evoked steering maneuvers in flying Drosophila

Sideslip force, longitudinal force, rolling moment, and pitching moment generated by tethered fruit flies, Drosophila melanogaster, were measured during optomotor reactions within an electronic flight simulator. Forces and torques were acquired by optically measuring the angular deflections of the beam to which the flies were tethered using a laser and a photodiode. Our results indicate that fruit flies actively generate both sideslip and roll in response to a lateral focus of expansion (FOE). The polarity of this behavior was such that the animal's aerodynamic response would carry it away from the expanding pattern, suggesting that it constitutes an avoidance reflex or centering response. Sideslip forces and rolling moments were sinusoidal functions of FOE position, whereas longitudinal force was proportional to the absolute value of the sine of FOE position. Pitching moments remained nearly constant irrespective of stimulus position or strength, with a direction indicating a tonic nose-down pitch under tethered conditions. These experiments expand our understanding of the degrees of freedom that a fruit fly can actually control in flight.

Context-dependent olfactory enhancement of optomotor flight control in Drosophila

Sensing and following the chemical plume of food odors is a fundamental challenge faced by many organisms. For flying insects, the task is complicated by wind that distorts the plume and buffets the fly. To maintain an upwind heading, and thus stabilize their orientation in a plume, insects such as flies and moths make use of strong context-specific visual equilibrium reflexes. For example, flying straight requires the regulation of image rotation across the eye, whereas minimizing side-slip and avoiding a collision require regulation of image expansion. In flies, visual rotation stabilizes plume tracking, but rotation and expansion optomotor responses are controlled by separate visual pathways. Are olfactory signals integrated with optomotor responses in a manner dependent upon visual context? We addressed this question by investigating the effect of an attractive food odor on active optomotor flight control. Odorant caused flies both to increase aerodynamic power output and to steer straighter. However, when challenged with wide-field optic flow, odor resulted in enhanced amplitude rotation responses but reduced amplitude expansion responses. For both visual conditions, flies tracked motion signals more closely in odor, an indication of increased saliency. These results suggest a simple search algorithm by which olfactory signals improve the salience of visual stimuli and modify optomotor control in a context-dependent manner, thereby enabling an animal to fly straight up a plume and approach odiferous objects.

The free-flight response of Drosophila to motion of the visual environment

In the present study we investigated the behavioural strategies with which freely flying fruit flies (Drosophila) control their flight trajectories during active optomotor stimulation in a free-flight arena. We measured forward, turning and climbing velocities of single flies using high-speed video analysis and estimated the output of a 'Hassenstein-Reichardt' elementary motion detector (EMD) array and the fly's gaze to evaluate flight behaviour in response to a rotating visual panorama. In a stationary visual environment, flight is characterized by flight saccades during which the animals turn on average 120 degrees within 130 ms. In a rotating environment, the fly's behaviour typically changes towards distinct, concentric circular flight paths where the radius of the paths increases with increasing arena velocity. The EMD simulation suggests that this behaviour is driven by a rotation-sensitive EMD detector system that minimizes retinal slip on each compound eye, whereas an expansion-sensitive EMD system with a laterally centred visual focus potentially helps to achieve centring response on the circular flight path. We developed a numerical model based on force balance between horizontal, vertical and lateral forces that allows predictions of flight path curvature at a given locomotor capacity of the fly. The model suggests that turning flight in Drosophila is constrained by the production of centripetal forces needed to avoid side-slip movements. At maximum horizontal velocity this force may account for up to 70% of the fly's body weight during yaw turning. Altogether, our analyses are widely consistent with previous studies on Drosophila free flight and those on the optomotor response under tethered flight conditions.