New experimental approaches to the biology of flight control systems

Through Hawks’ Eyes: Synthetically Reconstructing the Visual Field of a Bird in Flight

Birds of prey rely on vision to execute flight manoeuvres that are key to their survival, such as intercepting fast-moving targets or navigating through clutter. A better understanding of the role played by vision during these manoeuvres is not only relevant within the field of animal behaviour, but could also have applications for autonomous drones. In this paper, we present a novel method that uses computer vision tools to analyse the role of active vision in bird flight, and demonstrate its use to answer behavioural questions. Combining motion capture data from Harris’ hawks with a hybrid 3D model of the environment, we render RGB images, semantic maps, depth information and optic flow outputs that characterise the visual experience of the bird in flight. In contrast with previous approaches, our method allows us to consider different camera models and alternative gaze strategies for the purposes of hypothesis testing, allows us to consider visual input over the complete visual field of the bird, and is not limited by the technical specifications and performance of a head-mounted camera light enough to attach to a bird’s head in flight. We present pilot data from three sample flights: a pursuit flight, in which a hawk intercepts a moving target, and two obstacle avoidance flights. With this approach, we provide a reproducible method that facilitates the collection of large volumes of data across many individuals, opening up new avenues for data-driven models of animal behaviour.

Bumblebees land rapidly by intermittently accelerating and decelerating toward the surface during visually guided landings

Many flying animals parse visual information to control their landing, whereby they can decelerate smoothly by flying at a constant radial optic expansion rate. Here, we studied how bumblebees (Bombus terrestris) use optic expansion information to control their landing, by analyzing 10,005 landing maneuvers on vertical platforms with various optic information, and at three dim light conditions. We showed that bumblebees both decelerate and accelerate during these landings. Bumblebees decelerate by flying at a constant optic expansion rate, but they mostly accelerate toward the surface each time they switched to a new, often higher, optic expansion rate set-point. These transient acceleration phases allow bumblebees to increase their approach speed, and thereby land rapidly and robustly, even in dim twilight conditions. This helps explain why bumblebees are such robust foragers in challenging environmental conditions. The here-proposed sensorimotor landing control system can serve as bio-inspiration for landing control in unmanned aerial vehicles.

•

Bumblebees land by intermittently decelerating and accelerating toward a surface

•

Acceleration and deceleration phases result from a single visual-motor controller

•

The accelerations toward the surface allow bees to maximize their landing speed

•

Bumblebees adjust their sensorimotor control response to fly slower in dim light

Acoustotactic response of mosquitoes in untethered flight to incidental sound

Mosquitoes are vectors for some of the most devastating diseases on the planet. Given the centrality of acoustic sensing in the precopulatory behavior of these vectors, the use of an exogenous acoustic stimulus offers the potential of interfering with the courtship behavior of these insects. Previous research on the acoustotactic response of mosquitoes has been conducted on tethered preparations using low-intensity sound stimuli. To quantify differences in acoustotactic responses between mosquitos of distinct sex and species, we examined the effects of incidental sound stimuli on the flight behavior of free-flying male vs. female Aedes aegypti and Anopheles gambiae mosquitoes. The key variables were sound frequency (100–1000 Hz) and intensity (67–103 dB, measured at 12.5 cm from the source), and the acoustotactic response was measured in terms of the relative increase in flight speed in response to the stimulus. The data show, for the first time, significant sex- and species-specific differences in acoustotactic responses. A. aegypti exhibited a greater response to sound stimulus compared to An. gambiae, and the response also extended over a larger range of frequencies. Furthermore, the males of both species displayed a greater acoustotactic response than females, with An. gambiae females exhibiting minimal response to sound.

Wing rapid responses and aerodynamics of fruit flies during headwind gust perturbations

Insects are the main source of inspiration for flapping-wing micro air vehicles (FWMAVs). They frequently encounter wind gust perturbations in natural environments, and effectively cope with these perturbations. Here, we investigated the rapid gust response of flies to instruct the gust stability design of FWMAVs. A novel method to produce impulsive wind gusts that lasted less than 30 ms was designed to observe flies' rapid responses. Headwind gust perturbations were imposed on 14 tethered fruit flies, and the corresponding wing motions during perturbations were recorded by three high-speed cameras. The numerical simulation method was then applied to analyze aerodynamic forces and moments induced by the changes in wing kinematics. Results shows that flies mainly utilize three strategies against headwind gust perturbations, including decreasing the magnitude of stroke positional angle at ventral stroke reversal, delayed rotation and making the deviation angles in upstroke and downstroke closer (i.e. the wing tip trajectories of upstroke and downstroke tend be closer). Consequently, flies resist increments in lift and drag induced by the headwind gusts. However, flies seem to care little about changes in pitch moment in tethered conditions. These results provide useful suggestions for the stability control of FWMAVs during headwind gust perturbations.

Animals in Virtual Environments

The core idea in an XR (VR/MR/AR) application is to digitally stimulate one or more sensory systems (e.g. visual, auditory, olfactory) of the human user in an interactive way to achieve an immersive experience. Since the early 2000s biologists have been using Virtual Environments (VE) to investigate the mechanisms of behavior in non-human animals including insects, fish, and mammals. VEs have become reliable tools for studying vision, cognition, and sensory-motor control in animals. In turn, the knowledge gained from studying such behaviors can be harnessed by researchers designing biologically inspired robots, smart sensors, and rnulti-agent artificial intelligence. VE for animals is becoming a widely used application of XR technology but such applications have not previously been reported in the technical literature related to XR. Biologists and computer scientists can benefit greatly from deepening interdisciplinary research in this emerging field and together we can develop new methods for conducting fundamental research in behavioral sciences and engineering. To support our argument we present this review which provides an overview of animal behavior experiments conducted in virtual environments.

A Bird's-Eye View of Regulatory, Animal Care, and Training Considerations Regarding Avian Flight Research

A thorough understanding of how animals fly is a central goal of many scientific disciplines. Birds are a commonly used model organism for flight research. The success of this model requires studying healthy and naturally flying birds in a laboratory setting. This use of a nontraditional laboratory animal species presents unique challenges to animal care staff and researchers alike. Here we review regulatory, animal care, and training considerations associated with avian flight research.

Insect and insect-inspired aerodynamics: unsteadiness, structural mechanics and flight control

Flying insects impress by their versatility and have been a recurrent source of inspiration for engineering devices. A large body of literature has focused on various aspects of insect flight, with an essential part dedicated to the dynamics of flapping wings and their intrinsically unsteady aerodynamic mechanisms. Insect wings flex during flight and a better understanding of structural mechanics and aeroelasticity is emerging. Most recently, insights from solid and fluid mechanics have been integrated with physiological measurements from visual and mechanosensors in the context of flight control in steady airs and through turbulent conditions. We review the key recent advances concerning flight in unsteady environments and how the multi-body mechanics of the insect structure-wings and body-are at the core of the flight control question. The issues herein should be considered when applying bio-informed design principles to robotic flapping wings.

Biomechanics and biomimetics in insect-inspired flight systems

Insect- and bird-size drones-micro air vehicles (MAV) that can perform autonomous flight in natural and man-made environments are now an active and well-integrated research area. MAVs normally operate at a low speed in a Reynolds number regime of 10(4)-10(5) or lower, in which most flying animals of insects, birds and bats fly, and encounter unconventional challenges in generating sufficient aerodynamic forces to stay airborne and in controlling flight autonomy to achieve complex manoeuvres. Flying insects that power and control flight by flapping wings are capable of sophisticated aerodynamic force production and precise, agile manoeuvring, through an integrated system consisting of wings to generate aerodynamic force, muscles to move the wings and a control system to modulate power output from the muscles. In this article, we give a selective review on the state of the art of biomechanics in bioinspired flight systems in terms of flapping and flexible wing aerodynamics, flight dynamics and stability, passive and active mechanisms in stabilization and control, as well as flapping flight in unsteady environments. We further highlight recent advances in biomimetics of flapping-wing MAVs with a specific focus on insect-inspired wing design and fabrication, as well as sensing systems.This article is part of the themed issue 'Moving in a moving medium: new perspectives on flight'.

Proprioceptive feedback determines visuomotor gain in Drosophila

Multisensory integration is a prerequisite for effective locomotor control in most animals. Especially, the impressive aerial performance of insects relies on rapid and precise integration of multiple sensory modalities that provide feedback on different time scales. In flies, continuous visual signalling from the compound eyes is fused with phasic proprioceptive feedback to ensure precise neural activation of wing steering muscles (WSM) within narrow temporal phase bands of the stroke cycle. This phase-locked activation relies on mechanoreceptors distributed over wings and gyroscopic halteres. Here we investigate visual steering performance of tethered flying fruit flies with reduced haltere and wing feedback signalling. Using a flight simulator, we evaluated visual object fixation behaviour, optomotor altitude control and saccadic escape reflexes. The behavioural assays show an antagonistic effect of wing and haltere signalling on visuomotor gain during flight. Compared with controls, suppression of haltere feedback attenuates while suppression of wing feedback enhances the animal's wing steering range. Our results suggest that the generation of motor commands owing to visual perception is dynamically controlled by proprioception. We outline a potential physiological mechanism based on the biomechanical properties of WSM and sensory integration processes at the level of motoneurons. Collectively, the findings contribute to our general understanding how moving animals integrate sensory information with dynamically changing temporal structure.

How animals glide: from trajectory to morphology

Animals that glide produce aerodynamic forces that enable transit through the air in both arboreal and aquatic environments. The relative ease of gliding compared with flapping flight has led to a large diversity of taxa that have evolved some degree of flight capability. Glide paths are curved, reflecting the changing forces on the animal as it progresses through its aerial trajectory. These changing forces can be under control of the glider, which uses specific aspects of anatomy to modulate lift, drag, and rotational moments on the body. However, gliders share no single anatomical or behavioral feature, and some species are unspecialized for gliding, producing aerodynamic forces using posture and orientation alone. Animals use gliding in a broad range of ecological roles, suggesting that multiple performance metrics are relevant for consideration, but we are only beginning to understand how gliders produce and control their flight from takeoff to landing. In this review, we focus on the physical aspects of how glide trajectories are produced, and additionally discuss the range of morphologies and postures that are used to control aerial movements across the broad diversity of animal gliders.

Recent developments in the study of insect flight

Here we review recent contributions to the study of insect flight, in particular those brought about by advances in experimental techniques. We focus particularly on the following areas: wing flexibility and deformation, the physiology and biophysics of asynchronous insect flight muscle, the aerodynamics of flight, and stability and maneuverability. This recent research reveals the importance of wing flexibility to insect flight, provides a detailed model of how asynchronous flight muscle functions and how it may have evolved, synthesizes many recent studies of insect flight aerodynamics into a broad-reaching summary of unsteady flight aerodynamics, and highlights new insights into the sources of flight stability in insects. The focus on experimental techniques and recently developed apparatus shows how these advancements have occurred and point the way towards future experiments.

Insects modify their behaviour depending on the feedback sensor used when walking on a trackball in virtual reality

When using virtual-reality paradigms to study animal behaviour, careful attention must be paid to how the animal's actions are detected. This is particularly relevant in closed-loop experiments where the animal interacts with a stimulus. Many different sensor types have been used to measure aspects of behaviour, and although some sensors may be more accurate than others, few studies have examined whether, and how, such differences affect an animal's behaviour in a closed-loop experiment. To investigate this issue, we conducted experiments with tethered honeybees walking on an air-supported trackball and fixating a visual object in closed-loop. Bees walked faster and along straighter paths when the motion of the trackball was measured in the classical fashion - using optical motion sensors repurposed from computer mice - than when measured more accurately using a computer vision algorithm called 'FicTrac'. When computer mouse sensors were used to measure bees' behaviour, the bees modified their behaviour and achieved improved control of the stimulus. This behavioural change appears to be a response to a systematic error in the computer mouse sensor that reduces the sensitivity of this sensor system under certain conditions. Although the large perceived inertia and mass of the trackball relative to the honeybee is a limitation of tethered walking paradigms, observing differences depending on the sensor system used to measure bee behaviour was not expected. This study suggests that bees are capable of fine-tuning their motor control to improve the outcome of the task they are performing. Further, our findings show that caution is required when designing virtual-reality experiments, as animals can potentially respond to the artificial scenario in unexpected and unintended ways.

Markerless 3D motion capture for animal locomotion studies

Obtaining quantitative data describing the movements of animals is an essential step in understanding their locomotor biology. Outside the laboratory, measuring animal locomotion often relies on video-based approaches and analysis is hampered because of difficulties in calibration and often the limited availability of possible camera positions. It is also usually restricted to two dimensions, which is often an undesirable over-simplification given the essentially three-dimensional nature of many locomotor performances. In this paper we demonstrate a fully three-dimensional approach based on 3D photogrammetric reconstruction using multiple, synchronised video cameras. This approach allows full calibration based on the separation of the individual cameras and will work fully automatically with completely unmarked and undisturbed animals. As such it has the potential to revolutionise work carried out on free-ranging animals in sanctuaries and zoological gardens where ad hoc approaches are essential and access within enclosures often severely restricted. The paper demonstrates the effectiveness of video-based 3D photogrammetry with examples from primates and birds, as well as discussing the current limitations of this technique and illustrating the accuracies that can be obtained. All the software required is open source so this can be a very cost effective approach and provides a methodology of obtaining data in situations where other approaches would be completely ineffective.

How insect flight steering muscles work

Insights into how exactly a fly powers and controls flight have been hindered by the need to unpick the dynamic complexity of the muscles involved. The wingbeats of insects are driven by two antagonistic groups of power muscles and the force is funneled to the wing via a very complex hinge mechanism. The hinge consists of several hardened and articulated cuticle elements called sclerites. This articulation is controlled by a great number of small steering muscles, whose function has been studied by means of kinematics and muscle activity. The details and partly novel function of some of these steering muscles and their tendons have now been revealed in research published in this issue of PLOS Biology. The new study from Graham Taylor and colleagues applies time-resolved X-ray microtomography to obtain a three-dimensional view of the blowfly wingbeat. Asymmetric power output is achieved by differential wingbeat amplitude on the left and right wing, which is mediated by muscular control of the hinge elements to mechanically block the wing stroke and by absorption of work by steering muscles on one of the sides. This new approach permits visualization of the motion of the thorax, wing muscles, and the hinge mechanism. This very promising line of work will help to reveal the complete picture of the flight motor of a fly. It also holds great potential for novel bio-inspired designs of fly-like micro air vehicles.

Vision-based flight control in the hawkmoth Hyles lineata

Vision is a key sensory modality for flying insects, playing an important role in guidance, navigation and control. Here, we use a virtual-reality flight simulator to measure the optomotor responses of the hawkmoth Hyles lineata, and use a published linear-time invariant model of the flight dynamics to interpret the function of the measured responses in flight stabilization and control. We recorded the forces and moments produced during oscillation of the visual field in roll, pitch and yaw, varying the temporal frequency, amplitude or spatial frequency of the stimulus. The moths' responses were strongly dependent upon contrast frequency, as expected if the optomotor system uses correlation-type motion detectors to sense self-motion. The flight dynamics model predicts that roll angle feedback is needed to stabilize the lateral dynamics, and that a combination of pitch angle and pitch rate feedback is most effective in stabilizing the longitudinal dynamics. The moths' responses to roll and pitch stimuli coincided qualitatively with these functional predictions. The moths produced coupled roll and yaw moments in response to yaw stimuli, which could help to reduce the energetic cost of correcting heading. Our results emphasize the close relationship between physics and physiology in the stabilization of insect flight.

Multiple leading edge vortices of unexpected strength in freely flying hawkmoth

The Leading Edge Vortex (LEV) is a universal mechanism enhancing lift in flying organisms. LEVs, generally illustrated as a single vortex attached to the wing throughout the downstroke, have not been studied quantitatively in freely flying insects. Previous findings are either qualitative or from flappers and tethered insects. We measure the flow above the wing of freely flying hawkmoths and find multiple simultaneous LEVs of varying strength and structure along the wingspan. At the inner wing there is a single, attached LEV, while at mid wing there are multiple LEVs, and towards the wingtip flow separates. At mid wing the LEV circulation is ~40% higher than in the wake, implying that the circulation unrelated to the LEV may reduce lift. The strong and complex LEV suggests relatively high flight power in hawmoths. The variable LEV structure may result in variable force production, influencing flight control in the animals.

Interpretation of body-mounted accelerometry in flying animals and estimation of biomechanical power

An idealized energy fluctuation model of a bird's body undergoing horizontal flapping flight is developed, focusing on the biomechanical power discernible to a body-mounted accelerometer. Expressions for flight body power constructed from root mean square dynamic body accelerations and wingstroke frequency are derived from first principles and presented in dimensionally appropriate units. As wingstroke frequency increases, the model generally predicts a gradual transition in power from a linear to an asymptotically cubic relationship. However, the onset of this transition and the degree to which this occurs depends upon whether and how forward vibrations are exploited for temporary energy storage and retrieval. While this may vary considerably between species and individual birds, it is found that a quadrature phase arrangement is generally advantageous during level flight. Gravity-aligned vertical acceleration always enters into the calculation of body power, but, whenever forward acceleration becomes relevant, its contribution is subtractive. Several novel kinematic measures descriptive of flapping flight are postulated, offering fresh insights into the processes involved in airborne locomotion. The limitations of the model are briefly discussed, and departures from its predictions during ascending and descending flight evaluated. These findings highlight how body-mounted accelerometers can offer a valuable, insightful and non-invasive technique for investigating the flight of free-ranging birds and bats.

A fast and flexible panoramic virtual reality system for behavioural and electrophysiological experiments

Ideally, neuronal functions would be studied by performing experiments with unconstrained animals whilst they behave in their natural environment. Although this is not feasible currently for most animal models, one can mimic the natural environment in the laboratory by using a virtual reality (VR) environment. Here we present a novel VR system based upon a spherical projection of computer generated images using a modified commercial data projector with an add-on fish-eye lens. This system provides equidistant visual stimulation with extensive coverage of the visual field, high spatio-temporal resolution and flexible stimulus generation using a standard computer. It also includes a track-ball system for closed-loop behavioural experiments with walking animals. We present a detailed description of the system and characterize it thoroughly. Finally, we demonstrate the VR system’s performance whilst operating in closed-loop conditions by showing the movement trajectories of the cockroaches during exploratory behaviour in a VR forest.

The influence of sensory delay on the yaw dynamics of a flapping insect

In closed-loop systems, sensor feedback delays may have disastrous implications for performance and stability. Flies have evolved multiple specializations to reduce this latency, but the fastest feedback during flight involves a delay that is still significant on the timescale of body dynamics. We explored the effect of sensor delay on flight stability and performance for yaw turns using a dynamically scaled robotic model of the fruitfly, Drosophila. The robot was equipped with a real-time feedback system that performed active turns in response to measured torque about the functional yaw axis. We performed system response experiments for a proportional controller in yaw velocity for a range of feedback delays, similar in dimensionless timescale to those experienced by a fly. The results show a fundamental trade-off between sensor delay and permissible feedback gain, and suggest that fast mechanosensory feedback in flies, and most probably in other insects, provide a source of active damping which compliments that contributed by passive effects. Presented in the context of these findings, a control architecture whereby a haltere-mediated inner-loop proportional controller provides damping for slower visually mediated feedback is consistent with tethered-flight measurements, free-flight observations and engineering design principles.

Behavioural system identification of visual flight speed control in Drosophila melanogaster

Behavioural control in many animals involves complex mechanisms with intricate sensory-motor feedback loops. Modelling allows functional aspects to be captured without relying on a description of the underlying complex, and often unknown, mechanisms. A wide range of engineering techniques are available for modelling, but their ability to describe time-continuous processes is rarely exploited to describe sensory-motor control mechanisms in biological systems. We performed a system identification of visual flight speed control in the fruitfly Drosophila, based on an extensive dataset of open-loop responses previously measured under free flight conditions. We identified a second-order under-damped control model with just six free parameters that well describes both the transient and steady-state characteristics of the open-loop data. We then used the identified control model to predict flight speed responses after a visual perturbation under closed-loop conditions and validated the model with behavioural measurements performed in free-flying flies under the same closed-loop conditions. Our system identification of the fruitfly's flight speed response uncovers the high-level control strategy of a fundamental flight control reflex without depending on assumptions about the underlying physiological mechanisms. The results are relevant for future investigations of the underlying neuromotor processing mechanisms, as well as for the design of biomimetic robots, such as micro-air vehicles.

Visual control of flight speed in Drosophila melanogaster

Flight control in insects depends on self-induced image motion (optic flow), which the visual system must process to generate appropriate corrective steering maneuvers. Classic experiments in tethered insects applied rigorous system identification techniques for the analysis of turning reactions in the presence of rotating pattern stimuli delivered in open-loop. However, the functional relevance of these measurements for visual free-flight control remains equivocal due to the largely unknown effects of the highly constrained experimental conditions. To perform a systems analysis of the visual flight speed response under free-flight conditions, we implemented a 'one-parameter open-loop' paradigm using 'TrackFly' in a wind tunnel equipped with real-time tracking and virtual reality display technology. Upwind flying flies were stimulated with sine gratings of varying temporal and spatial frequencies, and the resulting speed responses were measured from the resulting flight speed reactions. To control flight speed, the visual system of the fruit fly extracts linear pattern velocity robustly over a broad range of spatio-temporal frequencies. The speed signal is used for a proportional control of flight speed within locomotor limits. The extraction of pattern velocity over a broad spatio-temporal frequency range may require more sophisticated motion processing mechanisms than those identified in flies so far. In Drosophila, the neuromotor pathways underlying flight speed control may be suitably explored by applying advanced genetic techniques, for which our data can serve as a baseline. Finally, the high-level control principles identified in the fly can be meaningfully transferred into a robotic context, such as for the robust and efficient control of autonomous flying micro air vehicles.

Wingbeat time and the scaling of passive rotational damping in flapping flight

Flying animals exhibit remarkable capabilities for both generating maneuvers and stabilizing their course and orientation after perturbation. Here we show that flapping fliers ranging in size from fruit flies to large birds benefit from substantial damping of angular velocity through a passive mechanism termed flapping counter-torque (FCT). Our FCT model predicts that isometrically scaled animals experience similar damping on a per-wingbeat time scale, resulting in similar turning dynamics in wingbeat time regardless of body size. The model also shows how animals may simultaneously specialize in both maneuverability and stability (at the cost of efficiency) and provides a framework for linking morphology, wing kinematics, maneuverability, and flight dynamics across a wide range of flying animals spanning insects, bats, and birds.