Flying fruit flies correct for visual sideslip depending on relative speed of forward optic flow

Environmental impact on visual perception modulates behavioral responses of schooling fish to looming predators

Aggregation in social fishes has evolved to improve safety from predators. The individual interaction mechanisms that govern collective behavior are determined by the sensory systems that translate environmental information into behavior. In dynamic environments, shifts in conditions impede effective visual sensory perception in fish schools, and may induce changes in the collective response. Here, we consider whether environmental conditions that affect visual contrast modulate the collective response of schools to looming predators. By using a virtual environment to simulate four contrast levels, we tested whether the collective state of minnow fish schools was modified in response to a looming optical stimulus. Our results indicate that fish swam slower and were less polarized in lower contrast conditions. Additionally, schooling metrics known to be regulated by non-visual sensory systems tended to correlate better when contrast decreased. Over the course of the escape response, schools remained tightly formed and retained the capability of transferring social information. We propose that when visual perception is compromised, the interaction rules governing collective behavior are likely to be modified to prioritize ancillary sensory information crucial to maximizing chance of escape. Our results imply that multiple sensory systems can integrate to control collective behavior in environments with unreliable visual information.

Proprioception gates visual object fixation in flying flies

Visual object tracking in animals as diverse as felines, frogs, and fish supports behaviors including predation, predator avoidance, and landscape navigation. Decades of experimental results show that a rigidly body-fixed tethered fly in a "virtual reality" visual flight simulator steers to follow the motion of a vertical bar, thereby "fixating" it on visual midline. This behavior likely reflects a desire to seek natural features such as plant stalks and has inspired algorithms for visual object tracking predicated on robust responses to bar velocity, particularly near visual midline. Using a modified flight simulator equipped with a magnetic pivot to allow frictionless turns about the yaw axis, we discovered that bar fixation as well as smooth steering responses to bar velocity are attenuated or eliminated in yaw-free conditions. Body-fixed Drosophila melanogaster respond to bar oscillation on a stationary ground with frequency-matched wing kinematics and fixate the bar on midline. Yaw-free flies respond to the same stimulus by ignoring the bar and maintaining their original heading. These differences are driven by proprioceptive signals, rather than visual signals, as artificially "clamping" a bar in the periphery of a yaw-free fly has no effect. When presented with a bar and ground oscillating at different frequencies, a yaw-free fly follows the frequency of the ground only, whereas a body-fixed fly robustly steers at the frequencies of both the bar and ground. Our findings support a model in which proprioceptive feedback promote active damping of high-gain optomotor responses to object motion.

Carbon dioxide and blood-feeding shift visual cue tracking during navigation in Aedes aegypti mosquitoes

Haematophagous mosquitoes need a blood meal to complete their reproductive cycle. To accomplish this, female mosquitoes seek vertebrate hosts, land on them and bite. As their eggs mature, they shift attention away from hosts and towards finding sites to lay eggs. We asked whether females were more tuned to visual cues when a host-related signal, carbon dioxide, was present, and further examined the effect of a blood meal, which shifts behaviour to ovipositing. Using a custom, tethered-flight arena that records wing stroke changes while displaying visual cues, we found the presence of carbon dioxide enhances visual attention towards discrete stimuli and improves contrast sensitivity for host-seeking Aedes aegypti mosquitoes. Conversely, intake of a blood meal reverses vertical bar tracking, a stimulus that non-fed females readily follow. This switch in behaviour suggests that having a blood meal modulates visual attention in mosquitoes, a phenomenon that has been described before in olfaction but not in visually driven behaviours.

Acuity and summation strategies differ in vinegar and desert fruit flies

An animal's vision depends on terrain features that limit the amount and distribution of available light. Approximately 10,000 years ago, vinegar flies (Drosophila melanogaster) transitioned from a single plant specialist into a cosmopolitan generalist. Much earlier, desert flies (D. mojavensis) colonized the New World, specializing on rotting cactuses in southwest North America. Their desert habitats are characteristically flat, bright, and barren, implying environmental differences in light availability. Here, we demonstrate differences in eye morphology and visual motion perception under three ambient light levels. Reducing ambient light from 35 to 18 cd/m2 causes sensitivity loss in desert but not vinegar flies. However, at 3 cd/m2, desert flies sacrifice spatial and temporal acuity more severely than vinegar flies to maintain contrast sensitivity. These visual differences help vinegar flies navigate under variably lit habitats around the world and desert flies brave the harsh desert while accommodating their crepuscular lifestyle.

Facilitation of neural responses to targets moving against optic flow

Target detection in visual clutter is a difficult computational task that insects, with their poor spatial resolution compound eyes and small brains, do successfully and with extremely short behavioral delays. We here show that the responses of target selective descending neurons are attenuated by background motion in the same direction as target motion but facilitated by background motion in the opposite direction. This finding is important for understanding how target pursuit can occur in tandem with gaze stabilization. Indeed, the neural facilitation would come into effect if the hoverfly is subjected to background motion in one direction but the target it is pursuing moves in the opposite direction and could therefore be used to override gaze stabilizing corrective turns.

For the human observer, it can be difficult to follow the motion of small objects, especially when they move against background clutter. In contrast, insects efficiently do this, as evidenced by their ability to capture prey, pursue conspecifics, or defend territories, even in highly textured surrounds. We here recorded from target selective descending neurons (TSDNs), which likely subserve these impressive behaviors. To simulate the type of optic flow that would be generated by the pursuer’s own movements through the world, we used the motion of a perspective corrected sparse dot field. We show that hoverfly TSDN responses to target motion are suppressed when such optic flow moves syn-directional to the target. Indeed, neural responses are strongly suppressed when targets move over either translational sideslip or rotational yaw. More strikingly, we show that TSDNs are facilitated by optic flow moving counterdirectional to the target, if the target moves horizontally. Furthermore, we show that a small, frontal spatial window of optic flow is enough to fully facilitate or suppress TSDN responses to target motion. We argue that such TSDN response facilitation could be beneficial in modulating corrective turns during target pursuit.

Stabilizing responses to sideslip disturbances in Drosophila melanogaster are modulated by the density of moving elements on the ground

Stabilizing responses to sideslip disturbances are a critical part of the flight control system in flies. While strongly mediated by mechanoreception, much of the final response results from the wide-field motion detection system associated with vision. In order to be effective, these responses must match the disturbance they are aimed to correct. To do this, flies must estimate the velocity of the disturbance, although it is not known how they accomplish this task when presented with natural images or dot fields. The recent finding, that motion parallax in dot fields can modulate stabilizing responses only if perceived below the fly, raises the question of whether other image statistics are also processed differently between eye regions. One such parameter is the density of elements moving in translational optic flow. Depending on the habitat, there might be strong differences in the density of elements providing information about self-motion above and below the fly, which in turn could act as selective pressures tuning the visual system to process this parameter on a regional basis. By presenting laterally moving dot fields of different densities we found that, in Drosophila melanogaster, the amplitude of the stabilizing response is significantly affected by the number of elements in the field of view. Flies countersteer strongly within a relatively low and narrow range of element densities. But this effect is exclusive to the ventral region of the eye, and dorsal stimuli elicit an unaltered and stereotypical response regardless of the density of elements in the flow. This highlights local specialization of the eye and suggests the lower region may play a more critical role in translational flight stabilization.

Using virtual worlds to understand insect navigation for bio-inspired systems

Insects perform a wide array of intricate behaviors over large spatial and temporal scales in complex natural environments. A mechanistic understanding of insect cognition has direct implications on how brains integrate multimodal information and can inspire bio-based solutions for autonomous robots. Virtual Reality (VR) offers an opportunity assess insect neuroethology while presenting complex, yet controlled, stimuli. Here, we discuss the use of insects as inspiration for artificial systems, recent advances in different VR technologies, current knowledge gaps, and the potential for application of insect VR research to bio-inspired robots. Finally, we advocate the need to diversify our model organisms, behavioral paradigms, and embrace the complexity of the natural world. This will help us to uncover the proximate and ultimate basis of brain and behavior and extract general principles for common challenging problems.

Ventral motion parallax enhances fruit fly steering to visual sideslip

Flies and other insects use incoherent motion (parallax) to the front and sides to measure distances and identify obstacles during translation. Although additional depth information could be drawn from below, there is no experimental proof that they use it. The finding that blowflies encode motion disparities in their ventral visual fields suggests this may be an important region for depth information. We used a virtual flight arena to measure fruit fly responses to optic flow. The stimuli appeared below (n = 51) or above the fly (n = 44), at different speeds, with or without parallax cues. Dorsal parallax does not affect responses, and similar motion disparities in rotation have no effect anywhere in the visual field. But responses to strong ventral sideslip (206° s⁻¹) change drastically depending on the presence or absence of parallax. Ventral parallax could help resolve ambiguities in cluttered motion fields, and enhance corrective responses to nearby objects.

A flexible geometry for panoramic visual and optogenetic stimulation during behavior and physiology

BACKGROUND

To study visual processing, it is necessary to precisely control visual stimuli while recording neural and behavioral responses. It can be important to present stimuli over a broad area of the visual field, which can be technically difficult.

NEW METHOD

We present a simple geometry that can be used to display panoramic stimuli. A single digital light projector generates images that are reflected by mirrors onto flat screens that surround an animal. It can be used for behavioral and neurophysiological measurements, so virtually identical stimuli can be presented. Moreover, this geometry permits light from the projector to be used to activate optogenetic tools.

RESULTS

Using this geometry, we presented panoramic visual stimulation to Drosophila in three paradigms. We presented drifting contrast gratings while recording walking and turning speed. We used the same projector to activate optogenetic channels during visual stimulation. Finally, we used two-photon microscopy to record responses in direction-selective cells to drifting gratings.

COMPARISON WITH EXISTING METHOD(S)

Existing methods have typically required custom hardware or curved screens, while this method requires only flat back projection screens and a digital light projector. The projector generates images in real time and does not require pre-generated images. Finally, while many setups are large, this geometry occupies a 30 × 20 cm footprint with a 25 cm height.

CONCLUSIONS

This flexible geometry enables measurements of behavioral and neural responses to panoramic stimuli. This allows moderate throughput behavioral experiments with simultaneous optogenetic manipulation, with easy comparisons between behavior and neural activity using virtually identical stimuli.

Bi-directional Control of Walking Behavior by Horizontal Optic Flow Sensors

Moving animals experience constant sensory feedback, such as panoramic image shifts on the retina, termed optic flow. Underlying neuronal signals are thought to be important for exploratory behavior by signaling unintended course deviations and by providing spatial information about the environment [1, 2]. Particularly in insects, the encoding of self-motion-related optic flow is well understood [1-5]. However, a gap remains in understanding how the associated neuronal activity controls locomotor trajectories. In flies, visual projection neurons belonging to two groups encode panoramic horizontal motion: horizontal system (HS) cells respond with depolarization to front-to-back motion and hyperpolarization to the opposite direction [6, 7], and other neurons have the mirror-symmetrical response profile [6, 8, 9]. With primarily monocular sensitivity, the neurons' responses are ambiguous for different rotational and translational self-movement components. Such ambiguities can be greatly reduced by combining signals from both eyes [10-12] to determine turning and movement speed [13-16]. Here, we explore the underlying functional logic by optogenetic HS cell manipulation in tethered walking Drosophila. We show that de- and hyperpolarization evoke opposite turning behavior, indicating that both direction-selective signals are transmitted to descending pathways for course control. Further experiments reveal a negative effect of bilaterally symmetric de- and hyperpolarization on walking velocity. Our results are therefore consistent with a functional architecture in which the HS cells' membrane potential influences walking behavior bi-directionally via two decelerating pathways.

A high-density EEG study of differentiation between two speeds and directions of simulated optic flow in adults and infants

A high-density EEG study was carried out to investigate cortical activity in response to forward and backward visual motion at two different driving speeds, simulated through optic flow. Participants were prelocomotor infants at the age of 4-5 months and infants with at least 3 weeks of crawling experience at the age of 8-11 months, and adults. Adults displayed shorter N2 latencies in response to forward as opposed to backward visual motion and differentiated significantly between low and high speeds, with shorter latencies for low speeds. Only infants at 8-11 months displayed similar latency differences between motion directions, and exclusively in response to low speed. The developmental differences in latency between infant groups are interpreted in terms of a combination of increased experience with self-produced locomotion and neurobiological development. Analyses of temporal spectral evolution (TSE, time-dependent amplitude changes) were also performed to investigate nonphase-locked changes at lower frequencies in underlying neuronal networks. TSE showed event-related desynchronization activity in response to visual motion for infants compared to adults. The poorer responses in infants are probably related to immaturity of the dorsal visual stream specialized in the processing of visual motion and could explain the observed problems in infants with differentiating high speeds of up to 50 km/h.

Optic flow induces spatial filtering in fruit flies

The discrete and random nature of light absorption dictates that, when photons are scarce, images suffer from low signal-to-noise ratios [1]. Nocturnal animals combat this by increasing photon catch, either optically, with large pupils, photoreceptors, and ever larger eyes [2], or neurally, with summation strategies, which sacrifice acuity and leave images blurry and slow [3]. Fast image motion produces the same problems as dim light: photon noise and low signal-to-noise ratio. As a result, fast animals often exhibit optical adaptations comparable to nocturnal animals, but pronounced in lateral regions of the eye where images move quickly [4]. For example, flying insects typically increase equatorial curvature between the front and side of the eye, which increases interommatidial angle and improves light capture. Drosophila, curiously, show only minor changes in interommatidial angle [5]; but even without optical adaptations, they could potentially achieve a similar effect with neural summation. The optimal strategy would manifest as acuity loss, specifically in regions of fast motion, and only parallel to motion. Photoreceptors need quick temporal dynamics to cope with fast motion [6], so the ideal filtering would additionally be spatial. To determine if fruit flies implement filtering driven by background optic flow, I tested their frequency-dependent steering and found that flow transiently eliminates high spatial frequency responses. This effect increases from forward to lateral visual regions, acts only parallel to flow direction, and filters only high spatial, not temporal, responses. Facultative summation may maximize visual information by improving sensitivity during fast flight, but limiting acuity loss when the fly is still.

Small fruit flies sacrifice temporal acuity to maintain contrast sensitivity

Holometabolous insects, like fruit flies, grow primarily during larval development. Scarce larval feeding is common in nature and generates smaller adults. Despite the importance of vision to flies, eye size scales proportionately with body size, and smaller eyes confer poorer vision due to smaller optics. Variable larval feeding, therefore, causes within-species differences in visual processing, which have gone largely unnoticed due to ad libitum feeding in the lab that results in generally large adults. Do smaller eyes have smaller ommatidial lenses, reducing sensitivity, or broader inter-ommatidial angles, reducing acuity? And to what extent might neural processes adapt to these optical challenges with temporal and spatial summation? To understand this in the fruit fly, we generated a distribution of body lengths (1.67-2.34 mm; n = 24) and eye lengths (0.33-0.44 mm; n = 24), resembling the distribution of wild-caught flies, by removing larvae from food during their third instar. We find smaller eyes (0.19 vs.0.07 mm²) have substantially fewer (978 vs. 540, n = 45) and smaller ommatidia (222 vs. 121 μm²;n = 45) separated by slightly wider inter-ommatidial angles (4.5 vs.5.5°; n = 34). This corresponds to a greater loss in contrast sensitivity (<50%) than spatial acuity (<20%). Using a flight arena and psychophysics paradigm, we find that smaller flies lose little spatial acuity (0.126 vs. 0.118CPD; n = 45), and recover contrast sensitivity (2.22 for both; n = 65) by sacrificing temporal acuity (26.3 vs. 10.8Hz; n = 112) at the neural level. Therefore, smaller flies sacrifice contrast sensitivity to maintain spatial acuity optically, but recover contrast sensitivity, almost completely, by sacrificing temporal acuity neurally.

Fruit flies are multistable geniuses

Our sensory systems have evolved to provide us with information about the external world. Such information is useful only insofar as it leads to actions that enhance fitness, and thus, the link between sensation and action has been thoroughly studied in many species. In insects, for example, specific visual stimuli lead to highly stereotyped responses. In contrast, humans can exhibit a wide range of responses to the same stimulus, as occurs most notably in the phenomenon of multistable perception. On this basis, one might think that humans have a fundamentally different way of generating actions from sensory inputs, but Toepfer et al. show that flies show evidence of multistable perception as well. Specifically, when confronted with a sensory stimulus that can yield different motor responses, flies switch from one response to another with temporal dynamics that are similar to those of humans and other animals. This suggests that the mechanisms that give rise to the rich repertoire of sensory experience in humans have correlates in much simpler nervous systems.

Spike Burst Coding of Translatory Optic Flow and Depth from Motion in the Fly Visual System

Many animals use the visual motion generated by traveling straight-the translatory optic flow-to successfully navigate obstacles: near objects appear larger and to move more quickly than distant objects. Flies are expert at navigating cluttered environments, and while their visual processing of rotatory optic flow is understood in exquisite detail, how they process translatory optic flow remains a mystery. We present novel cell types that have local motion receptive fields matched to translation self-motion, the vertical translation (VT) cells. One of these, the VT1 cell, encodes self-motion in the forward-sideslip direction and fires action potentials in spike bursts as well as single spikes. We show that the spike burst coding is size and speed-tuned and is selectively modulated by motion parallax-the relative motion experienced during translation. These properties are spatially organized, so that the cell is most excited by clutter rather than isolated objects. When the fly is presented with a simulation of flying past an elevated object, the spike burst activity is modulated by the height of the object, and the rate of single spikes is unaffected. When the moving object alone is experienced, the cell is weakly driven. Meanwhile, the VT2-3 cells have motion receptive fields matched to the lift axis. In conjunction with previously described horizontal cells, the VT cells have properties well suited to the visual navigation of clutter and to encode the fly's movements along near cardinal axes of thrust, lift, and forward sideslip.

Sensorimotor Transformations Underlying Variability in Song Intensity during Drosophila Courtship

Diverse animal species, from insects to humans, utilize acoustic signals for communication. Studies of the neural basis for song or speech production have focused almost exclusively on the generation of spectral and temporal patterns, but animals can also adjust acoustic signal intensity when communicating. For example, humans naturally regulate the loudness of speech in accord with a visual estimate of receiver distance. The underlying mechanisms for this ability remain uncharacterized in any system. Here, we show that Drosophila males modulate courtship song amplitude with female distance, and we investigate each stage of the sensorimotor transformation underlying this behavior, from the detection of particular visual stimulus features and the timescales of sensory processing to the modulation of neural and muscle activity that generates song. Our results demonstrate an unanticipated level of control in insect acoustic communication and uncover novel computations and mechanisms underlying the regulation of acoustic signal intensity.

A visual horizon affects steering responses during flight in fruit flies

To navigate well through three-dimensional environments, animals must in some way gauge the distances to objects and features around them. Humans use a variety of visual cues to do this, but insects, with their small size and rigid eyes, are constrained to a more limited range of possible depth cues. For example, insects attend to relative image motion when they move, but cannot change the optical power of their eyes to estimate distance. On clear days, the horizon is one of the most salient visual features in nature, offering clues about orientation, altitude and, for humans, distance to objects. We set out to determine whether flying fruit flies treat moving features as farther off when they are near the horizon. Tethered flies respond strongly to moving images they perceive as close. We measured the strength of steering responses while independently varying the elevation of moving stimuli and the elevation of a virtual horizon. We found responses to vertical bars are increased by negative elevations of their bases relative to the horizon, closely correlated with the inverse of apparent distance. In other words, a bar that dips far below the horizon elicits a strong response, consistent with using the horizon as a depth cue. Wide-field motion also had an enhanced effect below the horizon, but this was only prevalent when flies were additionally motivated with hunger. These responses may help flies tune behaviors to nearby objects and features when they are too far off for motion parallax.

Bumblebees measure optic flow for position and speed control flexibly within the frontal visual field

When flying through narrow spaces, insects control their position by balancing the magnitude of apparent image motion (optic flow) experienced in each eye and their speed by holding this value about a desired set point. Previously, it has been shown that when bumblebees encounter sudden changes in the proximity to nearby surfaces – as indicated by a change in the magnitude of optic flow on each side of the visual field – they adjust their flight speed well before the change, suggesting that they measure optic flow for speed control at low visual angles in the frontal visual field. Here, we investigated the effect that sudden changes in the magnitude of translational optic flow have on both position and speed control in bumblebees if these changes are asymmetrical; that is, if they occur only on one side of the visual field. Our results reveal that the visual region over which bumblebees respond to optic flow cues for flight control is not dictated by a set viewing angle. Instead, bumblebees appear to use the maximum magnitude of translational optic flow experienced in the frontal visual field. This strategy ensures that bumblebees use the translational optic flow generated by the nearest obstacles – that is, those with which they have the highest risk of colliding – to control flight.

To keep on track during flight, fruitflies discount the skyward view

When small flying insects go off their intended course, they use the resulting pattern of motion on their eye, or optic flow, to guide corrective steering. A change in heading generates a unique, rotational motion pattern and a change in position generates a translational motion pattern, and each produces corrective responses in the wingbeats. Any image in the flow field can signal rotation, but owing to parallax, only the images of nearby objects can signal translation. Insects that fly near the ground might therefore respond more strongly to translational optic flow that occurs beneath them, as the nearby ground will produce strong optic flow. In these experiments, rigidly tethered fruitflies steered in response to computer-generated flow fields. When correcting for unintended rotations, flies weight the motion in their upper and lower visual fields equally. However, when correcting for unintended translations, flies weight the motion in the lower visual fields more strongly. These results are consistent with the interpretation that fruitflies stabilize by attending to visual areas likely to contain the strongest signals during natural flight conditions.