Steering a virtual blowfly: simulation of visual pursuit

Visual versus visual-inertial guidance in hawks pursuing terrestrial targets

The aerial interception behaviour of falcons is well modelled by a guidance law called proportional navigation, which commands steering at a rate proportional to the angular rate of the line-of-sight from predator to prey. Because the line-of-sight rate is defined in an inertial frame of reference, proportional navigation must be implemented using visual-inertial sensor fusion. By contrast, the aerial pursuit behaviour of hawks chasing terrestrial targets is better modelled by a mixed guidance law combining information on the line-of-sight rate with information on the deviation angle between the attacker's velocity and the line-of-sight. Here we ask whether this behaviour may be controlled using visual information alone. We use high-speed motion capture to record n = 228 flights from N = 4 Harris' hawks Parabuteo unicinctus, and show that proportional navigation and mixed guidance both model their trajectories well. The mixed guidance law also models the data closely when visual-inertial information on the line-of-sight rate is replaced by visual information on the motion of the target relative to its background. Although the visual-inertial form of the mixed guidance law provides the closest fit, all three guidance laws provide an adequate phenomenological model of the behavioural data, whilst making different predictions on the physiological pathways involved.

Gap selection and steering during obstacle avoidance in pigeons

The ability of birds to fly through cluttered environments has inspired biologists interested in understanding its underlying mechanisms, and engineers interested in applying its underpinning principles. To analyse this problem empirically, we break it down into two distinct, but related, questions: How do birds select which gaps to aim for? And how do they steer through them? We answered these questions using a combined experimental and modelling approach, in which we released pigeons (Columbia livia domestica) inside a large hall with an open exit separated from the release point by a curtain creating two vertical gaps - one of which was obstructed by an obstacle. We tracked the birds using a high-speed motion capture system, and found that their gap choice seemed to be biased by their intrinsic handedness, rather than determined by extrinsic cues such as the size of the gap or its alignment with the destination. We modelled the pigeons' steering behaviour algorithmically by simulating their flight trajectories under a set of six candidate guidance laws, including those used previously to model target-oriented flight behaviours in birds. We found that their flights were best modelled by delayed proportional navigation commanding turning in proportion to the angular rate of the line-of-sight from the pigeon to the midpoint of the gap. Our results are consistent with this being a two-phase behaviour, in which the pigeon heads forward from the release point before steering towards the midpoint of whichever gap it chooses to aim for under closed-loop guidance. Our findings have implications for the sensorimotor mechanisms that underlie clutter negotiation in birds, uniting this with other kinds of target-oriented behaviours including aerial pursuit.

Characterisation of visual guidance of steering to intercept targets following curving trajectories using Qualitative Inconsistency Detection

This study explored the informational variables guiding steering behaviour in a locomotor interception task with targets moving along circular trajectories. Using a new method of analysis focussing on the temporal co-evolution of steering behaviour and the potential information sources driving it, we set out to invalidate reliance on plausible informational candidates. Applied to individual trials rather than ensemble averages, this Qualitative Inconsistency Detection (QuID) method revealed that steering behaviour was not compatible with reliance on information grounded in any type of change in the agent-centred target-heading angle. First-order changes in the environment-centred target's bearing angle could also not adequately account for the variations in behaviour observed under the different experimental conditions. Capturing the observed timing of unfolding steering behaviour ultimately required a combination of (velocity-based) first-order and (acceleration-based) second-order changes in bearing angle. While this result may point to reliance on fractional-order based changes in bearing angle, the overall importance of the present findings resides in the demonstration of the necessity to break away from the existing practice of trying to fit behaviour into a priori postulated functional strategies based on categorical differences between operative heuristic rules or control laws.

Two pursuit strategies for a single sensorimotor control task in blowfly

Effective visuomotor coordination is a necessary requirement for the survival of many terrestrial, aquatic, and aerial animal species. We studied the kinematics of aerial pursuit in the blowfly Lucilia sericata using an actuated dummy as target for freely flying males. We found that the flies perform target tracking in the horizontal plane and target interception in the vertical plane. Our behavioural data suggest that the flies’ trajectory changes are a controlled combination of target heading angle and of the rate of change of the bearing angle. We implemented control laws in kinematic models and found that the contributions of proportional navigation strategy are negligible. We concluded that the difference between horizontal and vertical control relates to the difference in target heading angle the fly keeps constant: 0° in azimuth and 23° in elevation. Our work suggests that male Lucilia control both horizontal and vertical steerings by employing proportional controllers to the error angles. In horizontal plane, this controller operates at time delays as small as 10 ms, the fastest steering response observed in any flying animal, so far.

Human Navigational Strategy for Intercepting an Erratically Moving Target in Chase and Escape Interactions

Pursuit and interception of moving targets are fundamental skills of many animal species. Although previous studies in human interception behaviors have proposed several navigational strategies for intercepting a moving target, it is still unknown which navigational strategy humans use in chase-and-escape interactions. In the present experimental study, by using two one-on-one tasks as seen in ball sports, we showed that human interception behaviors were statistically consistent with a time-optimal model. Our results provide the insight about the navigational strategy for intercepting a moving target in chase-and-escape interactions, which may be common across species.

A novel setup for 3D chasing behavior analysis in free flying flies

BACKGROUND

Insects catching prey or mates on the wing perform one of the fastest behaviours observed in nature. Some dipteran flies are aerial acrobats specialized to detect, chase and capture their targets within the blink of an eye. Studies of aerial pursuits and its underlying sensorimotor control have been a long-standing subject of interest in neuroethology research.

NEW METHOD

We designed an actuated dummy target to trigger chasing flights in male blowflies. Our setup generates arbitrary 2D target trajectories in the horizontal plane combining translation up to 1 m/s and angular rotation up to 720°/s.

RESULTS

Using stereovision methods we reconstructed target and pursuer positions every 5 ms with a maximum 3D error of 5 mm. The pursuer's body pitch and yaw angles were resolved within an error range of 6deg. An embedded observation point provides a close-up view of the pursuer's final approach and enables us to measure its body roll angle. We observed banked turns and sideslip which have not been reported for chasing blowflies in the past.

COMPARISON WITH EXISTING METHOD(S)

Previous studies focused on pursuit along circular paths or interception of translating targets while our method allows us to generate more complex target trajectories. Measurements of body orientation in earlier accounts were limited to the heading direction while we extended the analysis to include the full body orientation during pursuit.

CONCLUSIONS

Our setup offers an opportunity to investigate kinematics and governing visual parameters of chasing behaviour in species up to the size of blowflies under a large variety of experimental conditions.

Photoreceptor signalling is sufficient to explain the detectability threshold of insect aerial pursuers

An essential biological task for many flying insects is the detection of small, moving targets, such as when pursuing prey or conspecifics. Neural pathways underlying such 'target-detecting' behaviours have been investigated for their sensitivity and tuning properties (size, velocity). However, which stage of neuronal processing limits target detection is not yet known. Here, we investigated several skilled, aerial pursuers (males of four insect species), measuring the target-detection limit (signal-to-noise ratio) of light-adapted photoreceptors. We recorded intracellular responses to moving targets of varying size, extended well below the nominal resolution of single ommatidia. We found that the signal detection limit (2× photoreceptor noise) matches physiological or behavioural target-detection thresholds observed in each species. Thus, across a diverse range of flying insects, individual photoreceptor responses to changes in light intensity establish the sensitivity of the feature detection pathway, indicating later stages of processing are dedicated to feature tuning, tracking and selection.

The Role of Motion Extrapolation in Amphibian Prey Capture

Sensorimotor delays decouple behaviors from the events that drive them. The brain compensates for these delays with predictive mechanisms, but the efficacy and timescale over which these mechanisms operate remain poorly understood. Here, we assess how prediction is used to compensate for prey movement that occurs during visuomotor processing. We obtained high-speed video records of freely moving, tongue-projecting salamanders catching walking prey, emulating natural foraging conditions. We found that tongue projections were preceded by a rapid head turn lasting ∼ 130 ms. This motor lag, combined with the ∼ 100 ms phototransduction delay at photopic light levels, gave a ∼ 230 ms visuomotor response delay during which prey typically moved approximately one body length. Tongue projections, however, did not significantly lag prey position but were highly accurate instead. Angular errors in tongue projection accuracy were consistent with a linear extrapolation model that predicted prey position at the time of tongue contact using the average prey motion during a ∼ 175 ms period one visual latency before the head movement. The model explained successful strikes where the tongue hit the fly, and unsuccessful strikes where the fly turned and the tongue hit a phantom location consistent with the fly's earlier trajectory. The model parameters, obtained from the data, agree with the temporal integration and latency of retinal responses proposed to contribute to motion extrapolation. These results show that the salamander predicts future prey position and that prediction significantly improves prey capture success over a broad range of prey speeds and light levels.

SIGNIFICANCE STATEMENT

Neural processing delays cause actions to lag behind the events that elicit them. To cope with these delays, the brain predicts what will happen in the future. While neural circuits in the retina and beyond have been suggested to participate in such predictions, few behaviors have been explored sufficiently to constrain circuit function. Here we show that salamanders aim their tongues by using extrapolation to estimate future prey position, thereby compensating for internal delays from both visual and motor processing. Predictions made just before a prey turn resulted in the tongue being projected to a position consistent with the prey's pre-turn trajectory. These results define the computations and operating regimen for neural circuits that predict target motion.

A systematic analysis of the Braitenberg vehicle 2b for point-like stimulus sources

Braitenberg vehicles have been used experimentally for decades in robotics with limited empirical understanding. This paper presents the first mathematical model of the vehicle 2b, displaying so-called aggression behaviour, and analyses the possible trajectories for point-like smooth stimulus sources. This sensory-motor steering control mechanism is used to implement biologically grounded target approach, target-seeking or obstacle-avoidance behaviour. However, the analysis of the resulting model reveals that complex and unexpected trajectories can result even for point-like stimuli. We also prove how the implementation of the controller and the vehicle morphology interact to affect the behaviour of the vehicle. This work provides a better understanding of Braitenberg vehicle 2b, explains experimental results and paves the way for a formally grounded application on robotics as well as for a new way of understanding target seeking in biology.

An optimal control strategy for two-dimensional motion camouflage with non-holonimic constraints

Motion camouflage is a stealth behaviour observed both in hover-flies and in dragonflies. Existing controllers for mimicking motion camouflage generate this behaviour on an empirical basis or without considering the kinematic motion restrictions present in animal trajectories. This study summarises our formal contributions to solve the generation of motion camouflage as a non-linear optimal control problem. The dynamics of the system capture the kinematic restrictions to motion of the agents, while the performance index ensures camouflage trajectories. An extensive set of simulations support the technique, and a novel analysis of the obtained trajectories contributes to our understanding of possible mechanisms to obtain sensor based motion camouflage, for instance, in mobile robots.

Chasing behavior and optomotor following in free-flying male blowflies: flight performance and interactions of the underlying control systems

The chasing behavior of male blowflies after small targets belongs to the most rapid and virtuosic visually guided behaviors found in nature. Since in a structured environment any turn towards a target inevitably leads to a displacement of the entire retinal image in the opposite direction, it might evoke optomotor following responses counteracting the turn. To analyze potential interactions between the control systems underlying chasing behavior and optomotor following, respectively, we performed behavioral experiments on male blowflies and examined the characteristics of the two flight control systems in isolation and in combination. Three findings are particularly striking. (i) The characteristic saccadic flight and gaze style - a distinctive feature of blowfly cruising flights - is largely abandoned when the entire visual surroundings move around the fly; in this case flies tend to follow the moving pattern in a relatively continuous and smooth way. (ii) When male flies engage in following a small target, they also employ a smooth pursuit strategy. (iii) Although blowflies are reluctant to fly at high background velocities, the performance and dynamical characteristics of the chasing system are not much affected when the background moves in either the same or in the opposite direction as the target. Hence, the optomotor following response is largely suppressed by the chasing system and does not much impair chasing performance.

A Mobile Sensor Network Forming Concentric Circles Through Local Interaction and Consensus Building

We address the problem of a swarm of autonomous mobile robotic sensors generating geometric shapes to build wireless ad hoc surveillance sensor networks. Robot swarms with limited observation are required to form different shapes under different task conditions. To do this, we propose decentralized coordination enabling a robot swarm dispersed across an area to form a desired shape. Our approach has emphn robots generate a circumscribed circle of a regular emphn-polygon based on local interaction with neighboring robots. The approach also enables a large robot swarm to form concentric circles through consensus. We mathematically demonstrate convergence confirming the feasibility using extensive simulation. Our results indicate that our approach is applicable to mobile sensor network surveillance and security networks.

Saccadic flight strategy facilitates collision avoidance: closed-loop performance of a cyberfly

Behavioural and electrophysiological experiments suggest that blowflies employ an active saccadic strategy of flight and gaze control to separate the rotational from the translational optic flow components. As a consequence, this allows motion sensitive neurons to encode during translatory intersaccadic phases of locomotion information about the spatial layout of the environment. So far, it has not been clear whether and how a motor controller could decode the responses of these neurons to prevent a blowfly from colliding with obstacles. Here we propose a simple model of the blowfly visual course control system, named cyberfly, and investigate its performance and limitations. The sensory input module of the cyberfly emulates a pair of output neurons subserving the two eyes of the blowfly visual motion pathway. We analyse two sensory-motor interfaces (SMI). An SMI coupling the differential signal of the sensory neurons proportionally to the yaw rotation fails to avoid obstacles. A more plausible SMI is based on a saccadic controller. Even with sideward drift after saccades as is characteristic of real blowflies, the cyberfly is able to successfully avoid collisions with obstacles. The relative distance information contained in the optic flow during translatory movements between saccades is provided to the SMI by the responses of the visual output neurons. An obvious limitation of this simple mechanism is its strong dependence on the textural properties of the environment.

Order in spontaneous behavior

Brains are usually described as input/output systems: they transform sensory input into motor output. However, the motor output of brains (behavior) is notoriously variable, even under identical sensory conditions. The question of whether this behavioral variability merely reflects residual deviations due to extrinsic random noise in such otherwise deterministic systems or an intrinsic, adaptive indeterminacy trait is central for the basic understanding of brain function. Instead of random noise, we find a fractal order (resembling Lévy flights) in the temporal structure of spontaneous flight maneuvers in tethered Drosophila fruit flies. Lévy-like probabilistic behavior patterns are evolutionarily conserved, suggesting a general neural mechanism underlying spontaneous behavior. Drosophila can produce these patterns endogenously, without any external cues. The fly's behavior is controlled by brain circuits which operate as a nonlinear system with unstable dynamics far from equilibrium. These findings suggest that both general models of brain function and autonomous agents ought to include biologically relevant nonlinear, endogenous behavior-initiating mechanisms if they strive to realistically simulate biological brains or out-compete other agents.

Characterisation of a blowfly male-specific neuron using behaviourally generated visual stimuli

The pursuit system controlling chasing behaviour in male blowflies has to cope with extremely fast and dynamically changing visual input. An identified male-specific visual neuron called Male Lobula Giant 1 (MLG1) is presumably one major element of this pursuit system. Previous behavioural and modelling analyses have indicated that angular target size, retinal target position and target velocity are relevant input variables of the pursuit system. To investigate whether MLG1 specifically represents any of these visual parameters we obtained in vivo intracellular recordings while replaying optical stimuli that simulate the visual signals received by a male fly during chasing manoeuvres. On the basis of these naturalistic stimuli we find that MLG1 shows distinct direction sensitivity and is depolarised if the target motion contains an upward component. The responses of MLG1 are jointly determined by the retinal position, the speed and direction, and the duration of target motimotion. Coherence analysis reveals that although retinal target size and position are in some way inherent in the responses of MLG1, we find no confirmation of the hypothesis that MLG1 encodes any of these parameters exclusively.

Instability helps virtual flies to mate

In this paper we perform a bifurcation analysis for a discrete time dynamical system, describing the behavior of a virtual fly, developed by Böddeker and Egelhaaf (2003). Like real blowflies, the virtual counterparts exhibit a dichotomous behavior: they catch small targets but follow big objects at a constant distance. We consider this model for targets on linear and on circular trajectories. Then we transform the system into a ''frozen'' form, such that the position of the target is fixed. It turns out that the loss of stability of a fixed point in the frozen system due to a Neimark-Sacker bifurcation, explains the dichotomous behavior of the virtual fly.

A single control system for smooth and saccade-like pursuit in blowflies

During courtship, male blowflies perform aerobatic pursuits that rank among the fastest visual behaviours that can be observed in nature. The viewing strategies during pursuit behaviour of blowflies appear to be very similar to eye movements during pursuit in primates: a combination of smooth pursuit and catch-up saccades. Whereas in primates these two components of pursuit eye movements are thought to be controlled by distinct oculomotor subsystems, we present evidence that in blowflies both types of pursuit responses can be produced by a single control system. In numerical simulations of chasing behaviour the proposed control system generates qualitatively the same behaviour as with real blowflies. As a consequence of time constants in the control system, mimicking neuronal processing times, muscular dynamics and inertia, saccade-like changes in gaze direction are generated if the target is displaced rapidly on the pursuing fly's retina. In the behavioural context of visual pursuit, saccade-like changes of the fly's gaze direction can thus be parsimoniously explained as an emergent property of a smooth pursuit system without assuming a priori different mechanisms underlying smooth and saccadic tracking behaviour.

Visual motor computations in insects

With their relatively simple nervous systems and purpose-designed behaviors and reflexes, insects are an excellent organism in which to investigate how visual information is acquired and processed to guide locomotion and navigation. Flies maintain a straight course and monitor their motion through the environment by sensing the patterns of optic flow induced in the eyes. Bees negotiate narrow gaps by balancing the speeds of the images in their two eyes, and they control flight speed by holding constant the average image velocity as seen with their two eyes. Bees achieve a smooth landing on a horizontal surface by holding the image velocity of the surface constant during approach, thus ensuring that flight speed is automatically close to zero at touchdown. Foraging bees estimate the distance that they have traveled to reach a food source by integrating the optic flow experienced en route; this integration gives them a visually driven "odometer." Insects have also evolved sophisticated visuomotor mechanisms for pursuing prey or mates and possibly for concealing their own motion while shadowing objects of interest.

Chasing a dummy target: smooth pursuit and velocity control in male blowflies

Male blowflies chase and catch other flies in fast acrobatic flights. To unravel the underlying control system, we presented a black moving sphere instead of a real fly as a pursuit target. By varying the size and speed of the target, we were able to systematically analyse the decisive visual determinants that guide chasing behaviour. Flies pursue targets of a wide range of sizes and velocities. The percentage of pursuits resulting in target capture decreases with increasing target size and speed. Chasing male flies adjust their forward velocity depending on the retinal size of the target, indicating that retinal size is a relevant input variable of the control system. The chasing fly focuses the target with great accuracy in the frontal part of its visual field by means of a smooth pursuit control system using the retinal position of the target to determine the flight direction. We conclude that for a comprehensive understanding of chasing control different time lags in the control systems of angular and forward velocity together with the impact of inertia on fly movements need to be taken into account.