A hull reconstruction-reprojection method for pose estimation of free-flying fruit flies

Understanding the mechanisms of insect flight requires high-quality data of free-flight kinematics, e.g. for comparative studies or genetic screens. Although recent improvements in high-speed videography allow us to acquire large amounts of free-flight data, a significant bottleneck is automatically extracting accurate body and wing kinematics. Here, we present an experimental system and a hull reconstruction-reprojection algorithm for measuring the flight kinematics of fruit flies. The experimental system can automatically record hundreds of flight events per day. Our algorithm resolves a significant portion of the occlusions in this system by a reconstruction-reprojection scheme that integrates information from all cameras. Wing and body kinematics, including wing deformation, are then extracted from the hulls of the wing boundaries and body. This model-free method is fully automatic, accurate and open source, and can be readily adjusted for different camera configurations or insect species.

Sensory fusion in the hoverfly righting reflex

We study how falling hoverflies use sensory cues to trigger appropriate roll righting behavior. Before being released in a free fall, flies were placed upside-down with their legs contacting the substrate. The prior leg proprioceptive information about their initial orientation sufficed for the flies to right themselves properly. However, flies also use visual and antennal cues to recover faster and disambiguate sensory conflicts. Surprisingly, in one of the experimental conditions tested, hoverflies flew upside-down while still actively flapping their wings. In all the other conditions, flies were able to right themselves using two roll dynamics: fast ([Formula: see text]50ms) and slow ([Formula: see text]110ms) in the presence of consistent and conflicting cues, respectively. These findings suggest that a nonlinear sensory integration of the three types of sensory cues occurred. A ring attractor model was developed and discussed to account for this cue integration process.

Model-Based Tracking of Fruit Flies in Free Flight

Insect flight is a complex interdisciplinary phenomenon. Understanding its multiple aspects, such as flight control, sensory integration, physiology and genetics, often requires the analysis of large amounts of free flight kinematic data. Yet, one of the main bottlenecks in this field is automatically and accurately extracting such data from multi-view videos. Here, we present a model-based method for the pose estimation of free-flying fruit flies from multi-view high-speed videos. To obtain a faithful representation of the fly with minimum free parameters, our method uses a 3D model that includes two new aspects of wing deformation: A non-fixed wing hinge and a twisting wing surface. The method is demonstrated for free and perturbed flight. Our method does not use prior assumptions on the kinematics apart from the continuity of the wing pitch angle. Hence, this method can be readily adjusted for other insect species.

Elastic storage enables robustness of flapping wing dynamics

Flying insects could perform robust flapping-wing dynamics under various environments while minimizing the high energetic cost by using elastic flight muscles and motors. Here we propose a fluid-structure interaction model that couples unsteady flapping aerodynamics and three-torsional-spring-based elastic wing-hinge dynamics to determine passive and active mechanisms (PAM) in bumblebee hovering. The results show that a strategy of active-controlled stroke, passive-controlled wing pitch and deviation enables an optimal elastic storage. The flapping-wing dynamics is robust, which is characterized by dynamics-based passive elevation-rotation and aerodynamics-based passive feathering-rotation, capable of producing aerodynamic force while achieving high power efficiency over a broad range of wing-hinge stiffness. A force-impulse model further confirms the capability of external perturbation robustness under the PAM-based strategy.

Distinct forms of resonant optimality within insect indirect flight motors

Insect flight motors are extraordinary natural structures that operate efficiently at high frequencies. Structural resonance is thought to play a role in ensuring efficient motor operation, but the details of this role are elusive. While the efficiency benefits associated with resonance may be significant, a range of counterintuitive behaviours are observed. In particular, the relationship between insect wingbeat frequencies and thoracic natural frequencies is uncertain, with insects showing wingbeat frequency modulation over both short and long time scales. Here, we offer new explanations for this modulation. We show how, in linear and nonlinear models of an indirect flight motor, resonance is not a unitary state at a single frequency, but a complex cluster of distinct and mutually exclusive states, each representing a different form of resonant optimality. Additionally, by characterizing the relationship between resonance and the state of negative work absorption within the motor, we demonstrate how near-perfect resonant energetic optimality can be maintained over significant wingbeat frequency ranges. Our analysis leads to a new conceptual model of flight motor operation: one in which insects are not energetically restricted to a precise wingbeat frequency, but instead are robust to changes in thoracic and environmental properties-an illustration of the extraordinary robustness of these natural motors.

A Cyborg Insect Reveals a Function of a Muscle in Free Flight

While engineers put lots of effort, resources, and time in building insect scale micro aerial vehicles (MAVs) that fly like insects, insects themselves are the real masters of flight. What if we would use living insect as platform for MAV instead? Here, we reported a flight control via electrical stimulation of a flight muscle of an insect-computer hybrid robot, which is the interface of a mountable wireless backpack controller and a living beetle. The beetle uses indirect flight muscles to drive wing flapping and three major direct flight muscles (basalar, subalar, and third axilliary (3Ax) muscles) to control the kinematics of the wings for flight maneuver. While turning control was already achieved by stimulating basalar and 3Ax muscles, electrical stimulation of subalar muscles resulted in braking and elevation control in flight. We also demonstrated around 20 degrees of contralateral yaw and roll by stimulating individual subalar muscle. Stimulating both subalar muscles lead to an increase of 20 degrees in pitch and decelerate the flight by 1.5 m/s² as well as an induce in elevation of 2 m/s².

Using Drosophila to uncover molecular and physiological functions of circRNAs

Circular RNAs (circRNAs) are a class of covalently closed RNA molecules generated by backsplicing. circRNAs are expressed in a tissue-specific manner, accumulate with age in neural tissues, and are highly stable. In many cases, circRNAs are generated at the expense of a linear transcript as back-splicing competes with linear splicing. Some circRNAs regulate gene expression in cis, and some circRNAs can be translated into protein. The advent of deep sequencing and new bioinformatic tools has allowed detection of thousands of circRNAs in eukaryotes. Studying the functions of circRNAs is done using a combination of molecular and genetic methods. The unique genetic tools that can be used in studies of Drosophila melanogaster are ideal for deciphering the functions of circRNAs in vivo. These tools include the GAL4-UAS system, which can be used to manipulate the levels of circRNAs with exquisite temporal and spatial control, and genetic interaction screening, which could be used to identify pathways regulated by circRNAs. Research performed in Drosophila has revealed circRNAs production mechanisms, details of their translation, and their physiological functions. Due to their short lifecycle and the existence of excellent neurodegeneration models, Drosophila can also be used to study the role of circRNAs in aging and age-related disorders. Here, we review molecular and genetic tools and methods for detecting, manipulating, and studying circRNAs in Drosophila.

Timing precision in fly flight control: integrating mechanosensory input with muscle physiology

Animals rapidly collect and act on incoming information to navigate complex environments, making the precise timing of sensory feedback critical in the context of neural circuit function. Moreover, the timing of sensory input determines the biomechanical properties of muscles that undergo cyclic length changes, as during locomotion. Both of these issues come to a head in the case of flying insects, as these animals execute steering manoeuvres at timescales approaching the upper limits of performance for neuromechanical systems. Among insects, flies stand out as especially adept given their ability to execute manoeuvres that require sub-millisecond control of steering muscles. Although vision is critical, here I review the role of rapid, wingbeat-synchronous mechanosensory feedback from the wings and structures unique to flies, the halteres. The visual system and descending interneurons of the brain employ a spike rate coding scheme to relay commands to the wing steering system. By contrast, mechanosensory feedback operates at faster timescales and in the language of motor neurons, i.e. spike timing, allowing wing and haltere input to dynamically structure the output of the wing steering system. Although the halteres have been long known to provide essential input to the wing steering system as gyroscopic sensors, recent evidence suggests that the feedback from these vestigial hindwings is under active control. Thus, flies may accomplish manoeuvres through a conserved hindwing circuit, regulating the firing phase-and thus, the mechanical power output-of the wing steering muscles.

Intermittent control strategy can enhance stabilization robustness in bumblebee hovering

Active flight control plays a crucial role in stabilizing the body posture of insects to stay aloft under a complex natural environment. Insects can achieve a closed-loop flight control by integrating the external mechanical system and the internal working system through manipulating wing kinematics according to feedback information from multiple sensors. While studies of proportional derivative/proportional integral derivative-based algorithms are the main subject to explore the continuous flight control mechanisms associated with insect flights, it is normally observed that insects achieve an intermittent spike firing in steering muscles to manipulate wings in flight control discontinuously. Here we proposed a novel intermittent control strategy for a 3 degree of freedom (DoF) pitch-control and explored its stabilization robustness in bumblebee hovering. An integrated computational model was established and validated, which comprises an insect-inspired dynamic flight simulator and a novel discrete feedback controller as well as a simplified free-flight dynamic model. We found that the intermittent control model can achieve an angular-dominant flight control, whereas the continuous control model corresponds to an angular-velocity-dominant one. Given the biological constraints in sensorimotor neurobiology and musculoskeletal mechanics, the intermittent control strategy was examined capable of enhancing the stabilization robustness in terms of sensory latency, stroke derivation, spike interval, and damping strength. Our results indicate that the intermittent control strategy is likely a sophisticated flight control mechanism in insect flights while providing a bioinspired flight-control design for insect size flapping-wing micro air vehicles.

Wing rapid responses and aerodynamics of fruit flies during headwind gust perturbations

Insects are the main source of inspiration for flapping-wing micro air vehicles (FWMAVs). They frequently encounter wind gust perturbations in natural environments, and effectively cope with these perturbations. Here, we investigated the rapid gust response of flies to instruct the gust stability design of FWMAVs. A novel method to produce impulsive wind gusts that lasted less than 30 ms was designed to observe flies' rapid responses. Headwind gust perturbations were imposed on 14 tethered fruit flies, and the corresponding wing motions during perturbations were recorded by three high-speed cameras. The numerical simulation method was then applied to analyze aerodynamic forces and moments induced by the changes in wing kinematics. Results shows that flies mainly utilize three strategies against headwind gust perturbations, including decreasing the magnitude of stroke positional angle at ventral stroke reversal, delayed rotation and making the deviation angles in upstroke and downstroke closer (i.e. the wing tip trajectories of upstroke and downstroke tend be closer). Consequently, flies resist increments in lift and drag induced by the headwind gusts. However, flies seem to care little about changes in pitch moment in tethered conditions. These results provide useful suggestions for the stability control of FWMAVs during headwind gust perturbations.

Tuneable reflexes control antennal positioning in flying hawkmoths

Complex behaviours may be viewed as sequences of modular actions, each elicited by specific sensory cues in their characteristic timescales. From this perspective, we can construct models in which unitary behavioural modules are hierarchically placed in context of related actions. Here, we analyse antennal positioning reflex in hawkmoths as a tuneable behavioural unit. Mechanosensory feedback from two antennal structures, Böhm’s bristles (BB) and Johnston’s organs (JO), determines antennal position. At flight onset, antennae attain a specific position, which is maintained by feedback from BB. Simultaneously, JO senses deflections in flagellum-pedicel joint due to frontal airflow, to modulate its steady-state position. Restricting JO abolishes positional modulation but maintains stability against perturbations. Linear feedback models are sufficient to predict antennal dynamics at various set-points. We modelled antennal positioning as a hierarchical neural-circuit in which fast BB feedback maintains instantaneous set-point, but slow JO feedback modulates it, thereby elucidating mechanisms underlying its robustness and flexibility.

Flying insects position their antennae by integrating multisensory inputs across different timescales. This study describes an underlying hierarchical neural circuit that maintains antennal position in a fast and robust manner, whilst retaining flexibility to incorporate slower feedback to modulate position.

Fluid-structure interaction with the entropic lattice Boltzmann method

We propose a fluid-structure interaction (FSI) scheme using the entropic multi-relaxation time lattice Boltzmann (KBC) model for the fluid domain in combination with a nonlinear finite element solver for the structural part. We show the validity of the proposed scheme for various challenging setups by comparison to literature data. Beyond validation, we extend the KBC model to multiphase flows and couple it with a finite element method (FEM) solver. Robustness and viability of the entropic multi-relaxation time model for complex FSI applications is shown by simulations of droplet impact on elastic superhydrophobic surfaces.

Modeling visual-based pitch, lift and speed control strategies in hoverflies

To avoid crashing onto the floor, a free falling fly needs to trigger its wingbeats quickly and control the orientation of its thrust accurately and swiftly to stabilize its pitch and hence its speed. Behavioural data have suggested that the vertical optic flow produced by the fall and crossing the visual field plays a key role in this anti-crash response. Free fall behavior analyses have also suggested that flying insect may not rely on graviception to stabilize their flight. Based on these two assumptions, we have developed a model which accounts for hoverflies´ position and pitch orientation recorded in 3D with a fast stereo camera during experimental free falls. Our dynamic model shows that optic flow-based control combined with closed-loop control of the pitch suffice to stabilize the flight properly. In addition, our model sheds a new light on the visual-based feedback control of fly´s pitch, lift and thrust. Since graviceptive cues are possibly not used by flying insects, the use of a vertical reference to control the pitch is discussed, based on the results obtained on a complete dynamic model of a virtual fly falling in a textured corridor. This model would provide a useful tool for understanding more clearly how insects may or not estimate their absolute attitude.

On the basis of vision-based feedback control of optic flow occurring during insects’ flight, we developed a dynamic model that accounts for the pitch orientation and speed in plummeting flies. We compared the hoverflies’ responses with our model and showed that an optic-flow based control strategy can be used to correct the initial pitch misorientation caused by the free fall situation. To complete the model, we combined the closed-loop control of the vertical optic flow with an additional feedback control loop based on the value of the absolute pitch orientation. The need for this measurement to stabilize the pitch orientation raises the question as whether this is also the case in dipterans. After ruling out the possibility that insects may use gravity acceleration cues to control their flight, for which no experimental evidence has been found so far, we discussed the three main sensory processes possibly involved in in their ability to control their attitude. Our model provides a useful tool for studying the various sensory processes possibly involved in dipterans’ flight stabilization abilities as well as the interactions between these processes.

Asymmetries in wing inertial and aerodynamic torques contribute to steering in flying insects

Maneuvering in both natural and artificial miniature flying systems is assumed to be dominated by aerodynamic phenomena. To explore this, we develop a flapping wing model integrating aero and inertial dynamics. The model is applied to an elliptical wing similar to the forewing of the Hawkmoth Manduca sexta and realistic kinematics are prescribed. We scrutinize the stroke deviation phase, as it relates to firing latency in airborne insect steering muscles which has been correlated to various aerial maneuvers. We show that the average resultant force production acting on the body largely arises from wing pitch and roll and is insensitive to the phase and amplitude of stroke deviation. Inclusion of stroke deviation can generate significant averaged aerodynamic torques at steady-state and adjustment of its phase can facilitate body attitude control. Moreover, averaged wing angular momentum varies with stroke deviation phase, implying a non-zero impulse during a time-dependent phase shift. Simulations show wing inertial and aerodynamic impulses are of similar magnitude during short transients whereas aerodynamic impulses dominate during longer transients. Additionally, inertial effects become less significant for smaller flying insects. Body yaw rates arising from these impulses are consistent with biologically measured values. Thus, we conclude (1) modest changes in stroke deviation can significantly affect steering and (2) both aerodynamic and inertial torques are critical to maneuverability, the latter of which has not widely been considered. Therefore, the addition of a control actuator modulating stroke deviation may decouple lift/thrust production from steering mechanisms in flapping wing micro aerial vehicles and increase vehicle dexterity through inertial trajectory shaping.

Generation of the pitch moment during the controlled flight after takeoff of fruitflies

In the present paper, the controlled flight of fruitflies after voluntary takeoff is studied. Wing and body kinematics of the insects after takeoff are measured using high-speed video techniques, and the aerodynamic force and moment are calculated by the computational fluid dynamics method based on the measured data. How the control moments are generated is analyzed by correlating the computed moments with the wing kinematics. A fruit-fly has a large pitch-up angular velocity owing to the takeoff jump and the fly controls its body attitude by producing pitching moments. It is found that the pitching moment is produced by changes in both the aerodynamic force and the moment arm. The change in the aerodynamic force is mainly due to the change in angle of attack. The change in the moment arm is mainly due to the change in the mean stroke angle and deviation angle, and the deviation angle plays a more important role than the mean stroke angle in changing the moment arm (note that change in deviation angle implies variation in the position of the aerodynamic stroke plane with respect to the anatomical stroke plane). This is unlike the case of fruitflies correcting pitch perturbations in steady free flight, where they produce pitching moment mainly by changes in mean stroke angle.

Experimental optimization of wing shape for a hummingbird-like flapping wing micro air vehicle

Flapping wing micro air vehicles (MAVs) take inspiration from natural fliers, such as insects and hummingbirds. Existing designs manage to mimic the wing motion of natural fliers to a certain extent; nevertheless, differences will always exist due to completely different building blocks of biological and man-made systems. The same holds true for the design of the wings themselves, as biological and engineering materials differ significantly. This paper presents results of experimental optimization of wing shape of a flexible wing for a hummingbird-sized flapping wing MAV. During the experiments we varied the wing 'slackness' (defined by a camber angle), the wing shape (determined by the aspect and taper ratios) and the surface area. Apart from the generated lift, we also evaluated the overall power efficiency of the flapping wing MAV achieved with the various wing design. The results indicate that especially the camber angle and aspect ratio have a critical impact on the force production and efficiency. The best performance was obtained with a wing of trapezoidal shape with a straight leading edge and an aspect ratio of 9.3, both parameters being very similar to a typical hummingbird wing. Finally, the wing performance was demonstrated by a lift-off of a 17.2 g flapping wing robot.

The Function and Organization of the Motor System Controlling Flight Maneuvers in Flies

Animals face the daunting task of controlling their limbs using a small set of highly constrained actuators. This problem is particularly demanding for insects such as Drosophila, which must adjust wing motion for both quick voluntary maneuvers and slow compensatory reflexes using only a dozen pairs of muscles. To identify strategies by which animals execute precise actions using sparse motor networks, we imaged the activity of a complete ensemble of wing control muscles in intact, flying flies. Our experiments uncovered a remarkably efficient logic in which each of the four skeletal elements at the base of the wing are equipped with both large phasically active muscles capable of executing large changes and smaller tonically active muscles specialized for continuous fine-scaled adjustments. Based on the responses to a broad panel of visual motion stimuli, we have developed a model by which the motor array regulates aerodynamically functional features of wing motion. VIDEO ABSTRACT.

Flight control of fruit flies: dynamic response to optic-flow and headwind

Insects are magnificent fliers that are capable of performing many complex tasks such as speed regulation, smooth landings, and collision avoidance, even though their computational abilities are limited by their small brain. To investigate how flying insects respond to changes in wind speed and surrounding optic flow, the open-loop sensorimotor response of female Queensland fruit flies (Bactrocera tryoni) was examined. 136 flies were exposed to stimuli comprising sinusoidally varying optic flow and air flow (simulating forward movement) under tethered conditions in a virtual reality arena. Two responses were measured: the thrust, and the abdomen pitch. The dynamics of the responses to optic flow and air flow were measured at various frequencies, and modelled as a multicompartment linear system, which accurately captures the fruit flies' behavioural responses. The results indicate that these two behavioural responses are concurrently sensitive to changes of optic flow as well as wind. The abdomen pitch showed a streamlining response, where the abdomen was raised higher as the magnitude of either stimulus was increased. The thrust, on the other hand, exhibited a counter-phase response where maximum thrust occurred when the optic flow or wind flow was at a minimum, indicating that the flies were attempting to maintain an ideal flight speed. When the changes in the wind and optic flow were in phase (i.e. did not contradict each other), the net responses (thrust and abdomen pitch) were well approximated by an equally weighted sum of the responses to the individual stimuli. However, when the optic flow and wind stimuli were presented in counterphase, the flies seemed to respond to only one stimulus or the other, demonstrating a form of ‘selective attention’.