1
|
Vo-Doan TT, Titov VV, Harrap MJM, Lochner S, Straw AD. High-resolution outdoor videography of insects using Fast Lock-On tracking. Sci Robot 2024; 9:eadm7689. [PMID: 39413167 DOI: 10.1126/scirobotics.adm7689] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Accepted: 09/18/2024] [Indexed: 10/18/2024]
Abstract
Insects have important roles globally in ecology, economy, and health, yet our understanding of their behavior remains limited. Bees, for example, use vision and a tiny brain to find flowers and return home, but understanding how they perform these impressive tasks has been hampered by limitations in recording technology. Here, we present Fast Lock-On (FLO) tracking. This method moves an image sensor to remain focused on a retroreflective marker affixed to an insect. Using paraxial infrared illumination, simple image processing can localize the sensor location of the insect in a few milliseconds. When coupled with a feedback system to steer a high-magnification optical system to remain focused on the insect, a high-spatiotemporal resolution trajectory can be gathered over a large region. As the basis for several robotic systems, we show that FLO is a versatile idea that can be used in combination with other components. We demonstrate that the optical path can be split and used for recording high-speed video. Furthermore, by flying an FLO system on a quadcopter drone, we track a flying honey bee and anticipate tracking insects in the wild over kilometer scales. Such systems have the capability to provide higher-resolution information about insects behaving in natural environments and as such will be helpful in revealing the biomechanical and neuroethological mechanisms used by insects in natural settings.
Collapse
Affiliation(s)
- T Thang Vo-Doan
- Institute of Biology I, Faculty of Biology, Albert-Ludwigs-Universität Freiburg, Freiburg, Germany
| | - Victor V Titov
- Institute of Biology I, Faculty of Biology, Albert-Ludwigs-Universität Freiburg, Freiburg, Germany
| | - Michael J M Harrap
- Institute of Biology I, Faculty of Biology, Albert-Ludwigs-Universität Freiburg, Freiburg, Germany
| | - Stephan Lochner
- Institute of Biology I, Faculty of Biology, Albert-Ludwigs-Universität Freiburg, Freiburg, Germany
| | - Andrew D Straw
- Institute of Biology I, Faculty of Biology, Albert-Ludwigs-Universität Freiburg, Freiburg, Germany
- Bernstein Center Freiburg, Albert-Ludwigs-Universität Freiburg, Freiburg, Germany
| |
Collapse
|
2
|
Goyal P, van Leeuwen JL, Muijres FT. Bumblebees compensate for the adverse effects of sidewind during visually guided landings. J Exp Biol 2024; 227:jeb245432. [PMID: 38506223 PMCID: PMC11112349 DOI: 10.1242/jeb.245432] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2022] [Accepted: 02/26/2024] [Indexed: 03/21/2024]
Abstract
Flying animals often encounter winds during visually guided landings. However, how winds affect their flight control strategy during landing is unknown. Here, we investigated how sidewind affects the landing performance and sensorimotor control of foraging bumblebees (Bombus terrestris). We trained bumblebees to forage in a wind tunnel, and used high-speed stereoscopic videography to record 19,421 landing maneuvers in six sidewind speeds (0 to 3.4 m s-1), which correspond to winds encountered in nature. Bumblebees landed less often in higher windspeeds, but the landing durations from free flight were not increased by wind. By testing how bumblebees adjusted their landing control to compensate for adverse effects of sidewind on landing, we showed that the landing strategy in sidewind resembled that in still air, but with important adaptations. Bumblebees landing in a sidewind tended to drift downwind, which they controlled for by performing more hover maneuvers. Surprisingly, the increased hover prevalence did not increase the duration of free-flight landing maneuvers, as these bumblebees flew faster towards the landing platform outside the hover phases. Hence, by alternating these two flight modes along their flight path, free-flying bumblebees negated the adverse effects of high windspeeds on landing duration. Using control theory, we hypothesize that bumblebees achieve this by integrating a combination of direct aerodynamic feedback and a wind-mediated mechanosensory feedback control, with their vision-based sensorimotor control loop. The revealed landing strategy may be commonly used by insects landing in windy conditions, and may inspire the development of landing control strategies onboard autonomously flying robots.
Collapse
Affiliation(s)
- Pulkit Goyal
- Experimental Zoology Group, Wageningen University and Research, 6708 WD Wageningen, The Netherlands
| | - Johan L. van Leeuwen
- Experimental Zoology Group, Wageningen University and Research, 6708 WD Wageningen, The Netherlands
| | - Florian T. Muijres
- Experimental Zoology Group, Wageningen University and Research, 6708 WD Wageningen, The Netherlands
| |
Collapse
|
3
|
Goyal P, Baird E, Srinivasan MV, Muijres FT. Visual guidance of honeybees approaching a vertical landing surface. J Exp Biol 2023; 226:jeb245956. [PMID: 37589414 PMCID: PMC10482386 DOI: 10.1242/jeb.245956] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Accepted: 08/08/2023] [Indexed: 08/18/2023]
Abstract
Landing is a critical phase for flying animals, whereby many rely on visual cues to perform controlled touchdown. Foraging honeybees rely on regular landings on flowers to collect food crucial for colony survival and reproduction. Here, we explored how honeybees utilize optical expansion cues to regulate approach flight speed when landing on vertical surfaces. Three sensory-motor control models have been proposed for landings of natural flyers. Landing honeybees maintain a constant optical expansion rate set-point, resulting in a gradual decrease in approach velocity and gentile touchdown. Bumblebees exhibit a similar strategy, but they regularly switch to a new constant optical expansion rate set-point. In contrast, landing birds fly at a constant time to contact to achieve faster landings. Here, we re-examined the landing strategy of honeybees by fitting the three models to individual approach flights of honeybees landing on platforms with varying optical expansion cues. Surprisingly, the landing model identified in bumblebees proved to be the most suitable for these honeybees. This reveals that honeybees adjust their optical expansion rate in a stepwise manner. Bees flying at low optical expansion rates tend to increase their set-point stepwise, while those flying at high optical expansion rates tend to decrease it stepwise. This modular landing control system enables honeybees to land rapidly and reliably under a wide range of initial flight conditions and visual landing platform patterns. The remarkable similarity between the landing strategies of honeybees and bumblebees suggests that this may also be prevalent among other flying insects. Furthermore, these findings hold promising potential for bioinspired guidance systems in flying robots.
Collapse
Affiliation(s)
- Pulkit Goyal
- Experimental Zoology Group, Wageningen University & Research, 6708WD Wageningen, The Netherlands
| | - Emily Baird
- Department of Zoology, Stockholm University, 114 18 Stockholm, Sweden
| | - Mandyam V. Srinivasan
- Queensland Brain Institute, University of Queensland, St. Lucia, QLD 4072, Australia
| | - Florian T. Muijres
- Experimental Zoology Group, Wageningen University & Research, 6708WD Wageningen, The Netherlands
| |
Collapse
|
4
|
Lopez-Reyes K, Lankheet MJ, van Tol RWHM, Butler RC, Teulon DAJ, Armstrong KF. Tracking the flight and landing behaviour of western flower thrips in response to single and two-colour cues. Sci Rep 2023; 13:14178. [PMID: 37648681 PMCID: PMC10469208 DOI: 10.1038/s41598-023-37400-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 06/21/2023] [Indexed: 09/01/2023] Open
Abstract
Real-time 3D tracking and high-speed videography was used to examine the behaviour of a worldwide greenhouse pest, the western flower thrips (WFT), in response to different colours in the context of improving trap design. Measurements were taken of the number of landings on, and flight activity near, a lamp containing two LEDs of either the same colour or a combination of two colours presented side by side. Main findings show that landing patterns of WFT are different between colours, with landings on UV(+ red) as highly attractive stimulus being mostly distributed at the bottom half of the lamp, while for yellow also as very attractive and green as a 'neutral' stimulus, landings were clearly on the upper rim of the lamp. Additionally, a positive interaction with the UV-A(+ red) and yellow combination elicited the highest number of landings and flight time in front of the LED lamp. Conversely, a negative interaction was observed with decreased landings and flight time found for yellow when blue was present as the adjacent colour. Overall, differences between treatments were less obvious for flight times compared to number of landings, with tracking data suggesting that WFT might use different colours to orientate at different distances as they approach a visual stimulus.
Collapse
Affiliation(s)
- Karla Lopez-Reyes
- Department of Pest-Management and Conservation, Lincoln University, Lincoln, 7647, New Zealand.
| | - Martin J Lankheet
- Experimental Zoology, Animal Sciences, Wageningen University and Research, PO Box 338, Wageningen, 6700AH, The Netherlands
| | - Robert W H M van Tol
- Plant and Health Systems, Wageningen University and Research, PO Box 69, Wageningen, 6700AB, The Netherlands
- Bug Research Consultancy, Herendaal 1, Maastricht, 6228GV, The Netherlands
| | - Ruth C Butler
- StatsWork 2022 Limited, 48 Verdeco Boulevard, Lincoln, 7608, New Zealand
| | - David A J Teulon
- The New Zealand Institute for Plant and Food Research Limited, Private Bag 4704, Christchurch, New Zealand
- Better Border Biosecurity, Lincoln, New Zealand
| | - Karen F Armstrong
- Department of Pest-Management and Conservation, Lincoln University, Lincoln, 7647, New Zealand
- Better Border Biosecurity, Lincoln, New Zealand
| |
Collapse
|
5
|
Egelhaaf M. Optic flow based spatial vision in insects. J Comp Physiol A Neuroethol Sens Neural Behav Physiol 2023:10.1007/s00359-022-01610-w. [PMID: 36609568 DOI: 10.1007/s00359-022-01610-w] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Revised: 12/06/2022] [Accepted: 12/24/2022] [Indexed: 01/09/2023]
Abstract
The optic flow, i.e., the displacement of retinal images of objects in the environment induced by self-motion, is an important source of spatial information, especially for fast-flying insects. Spatial information over a wide range of distances, from the animal's immediate surroundings over several hundred metres to kilometres, is necessary for mediating behaviours, such as landing manoeuvres, collision avoidance in spatially complex environments, learning environmental object constellations and path integration in spatial navigation. To facilitate the processing of spatial information, the complexity of the optic flow is often reduced by active vision strategies. These result in translations and rotations being largely separated by a saccadic flight and gaze mode. Only the translational components of the optic flow contain spatial information. In the first step of optic flow processing, an array of local motion detectors provides a retinotopic spatial proximity map of the environment. This local motion information is then processed in parallel neural pathways in a task-specific manner and used to control the different components of spatial behaviour. A particular challenge here is that the distance information extracted from the optic flow does not represent the distances unambiguously, but these are scaled by the animal's speed of locomotion. Possible ways of coping with this ambiguity are discussed.
Collapse
Affiliation(s)
- Martin Egelhaaf
- Neurobiology and Center for Cognitive Interaction Technology (CITEC), Bielefeld University, Universitätsstraße 25, 33615, Bielefeld, Germany.
| |
Collapse
|
6
|
Goyal P, van Leeuwen JL, Muijres FT. Bumblebees land rapidly by intermittently accelerating and decelerating toward the surface during visually guided landings. iScience 2022; 25:104265. [PMID: 35521517 PMCID: PMC9065724 DOI: 10.1016/j.isci.2022.104265] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2022] [Revised: 03/19/2022] [Accepted: 04/12/2022] [Indexed: 11/18/2022] Open
Abstract
Many flying animals parse visual information to control their landing, whereby they can decelerate smoothly by flying at a constant radial optic expansion rate. Here, we studied how bumblebees (Bombus terrestris) use optic expansion information to control their landing, by analyzing 10,005 landing maneuvers on vertical platforms with various optic information, and at three dim light conditions. We showed that bumblebees both decelerate and accelerate during these landings. Bumblebees decelerate by flying at a constant optic expansion rate, but they mostly accelerate toward the surface each time they switched to a new, often higher, optic expansion rate set-point. These transient acceleration phases allow bumblebees to increase their approach speed, and thereby land rapidly and robustly, even in dim twilight conditions. This helps explain why bumblebees are such robust foragers in challenging environmental conditions. The here-proposed sensorimotor landing control system can serve as bio-inspiration for landing control in unmanned aerial vehicles. Bumblebees land by intermittently decelerating and accelerating toward a surface Acceleration and deceleration phases result from a single visual-motor controller The accelerations toward the surface allow bees to maximize their landing speed Bumblebees adjust their sensorimotor control response to fly slower in dim light
Collapse
Affiliation(s)
- Pulkit Goyal
- Experimental Zoology Group, Wageningen University and Research, P.O. Box 338, 6700 AH, Wageningen, the Netherlands
| | - Johan L van Leeuwen
- Experimental Zoology Group, Wageningen University and Research, P.O. Box 338, 6700 AH, Wageningen, the Netherlands
| | - Florian T Muijres
- Experimental Zoology Group, Wageningen University and Research, P.O. Box 338, 6700 AH, Wageningen, the Netherlands
| |
Collapse
|