Neural mechanisms for prediction: do insects have forward models?

Useful and Useless Misnomers in Motor Control

This article addresses the issue of using terms and concepts in motor control that are ill-defined, undefined, and/or imported from nonbiological fields. In many of such cases, the discourse turns nonscientific and unproductive. Some of such terms are potentially useful but need to be properly and exactly defined. Other terms seem to be misleading and nonfixable. There is also an intermediate group with terms that may or may not be useful if defined properly. The paper presents three examples per group: "reflex," "synergy," and "posture" versus "motor program," "efference copy," and "internal model" versus "muscle tone," "stiffness and impedance," and "redundancy." These terms are analyzed assuming that motor control is a branch of natural science, which must be analyzed using laws of nature, not a subfield of the control theory. In the discussion, we also accept the framework of the theory of movement control with spatial referent coordinates as the only example built on laws of nature with clearly formulated physical and physiological nature of the control parameters.

Navigating the unknown: Leveraging self-information and diversity in partially observable environments

Reinforcement learning algorithms often struggle to learn in partially observable environments, where different states of the environment may appear identical. However, not all partially observable environments pose the same level of difficulty for learning. This work introduces the concept of dissonance distance, a metric that can estimate the difficulty of learning in such environments. We demonstrate that self-information, such as internal oscillations or memory of previous actions, can increase the dissonance distance and make learning easier in partially observable environments. Additionally, sensory occlusion may occur after learning was completed, leading to a lack of sufficient information and catastrophic failure. To address this, we propose a spatially layered architecture (SLA) inspired by the brain, which trains multiple policies in parallel for the same task. SLA can change the amount of external information processed at each timestep, providing an adaptive approach to handle the changing information in the environment state-space. We evaluate the effectiveness of our SLA method showing learnability and robustness against realistic noise and occlusion in sensory inputs for the partially observable Continuous Mountain Car environment. We hypothesize that multi-policy approaches like SLA might explain the complex dopamine dynamics in the brain that cannot be explained with the state of the art scalar Temporal Difference error.

Animal flight: Fly gyros get a new spin

A new study exploits genetic approaches available in Drosophila to record the neural activity within the specialized mechanosensory fields of halteres, the unique equilibrium organs of flies. The results challenge the traditional explanation for how these rapidly oscillating structures encode angular velocity during flight.

Modulation of prey capture kinematics in relation to prey distance helps predict success

Predators are not perfect, as some of their prey capture attempts result in failure. Successful attempts may be partly due to predators modulating their capture kinematics in relation to variation in the visual cues of the prey to increase the probability of success. In praying mantises, which have been suggested to possess stereoscopic vision, variation in prey distance has been shown to elicit variation in the probability of an attempt. However, it remains to be examined whether variation in prey distance results in mantises modulating their attempt to successfully capture prey. The goals of this study were to examine these relationships using the praying mantis system. Using 11 adult female Sphodromantis lineola, we recorded 192 prey capture attempts at 1000 Hz with two cameras to examine the 3D kinematics of successful and unsuccessful prey capture attempts. Using a combination of principal component analysis (PCA) and logistic regression, our results show that as prey distance increases, mantises adjust through greater and faster expansion of the forelegs and body (PC1), which significantly predicts capture success. However, PC1 only explains 22% of the variation in all prey capture attempts, suggesting that the other components may be related to additional aspects of the prey. Our results suggest that the distances at which mantises prefer to attempt to capture prey may be the result of their greater probability of successfully capturing the prey. These results highlight the range of motions mantises use when attempting to capture prey, suggesting flexibility in their prey capture attempts in relation to prey position.

Interchangeable Role of Motor Cortex and Reafference for the Stable Execution of an Orofacial Action

Animals interact with their environment through mechanically active, mobile sensors. The efficient use of these sensory organs implies the ability to track their position; otherwise, perceptual stability or prehension would be profoundly impeded. The nervous system may keep track of the position of a sensorimotor organ via two complementary feedback mechanisms-peripheral reafference (external, sensory feedback) and efference copy (internal feedback). Yet, the potential contributions of these mechanisms remain largely unexplored. By training male rats to place one of their vibrissae within a predetermined angular range without contact, a task that depends on knowledge of vibrissa position relative to their face, we found that peripheral reafference is not required. The presence of motor cortex is not required either, except in the absence of peripheral reafference to maintain motor stability. Finally, the red nucleus, which receives descending inputs from motor cortex and cerebellum and projects to facial motoneurons, is critically involved in the execution of the vibrissa positioning task. All told, our results point toward the existence of an internal model that requires either peripheral reafference or motor cortex to optimally drive voluntary motion.SIGNIFICANCE STATEMENT How does an animal know where a mechanically active, mobile sensor lies relative to its body? We address this basic question in sensorimotor integration using the motion of the vibrissae in rats. We show that rats can learn to reliably position their vibrissae in the absence of sensory feedback or in the absence of motor cortex. Yet, when both sensory feedback and motor cortex are absent, motor precision is degraded. This suggests the existence of an internal model able to operate in closed- and open-loop modes, requiring either motor cortex or sensory feedback to maintain motor stability.

Roles of muscle activities in foreleg movements during predatory strike of the mantis

Foreleg trajectory in the mantis strike varies depending on prey distance. To examine how muscle activities affect foreleg trajectory, we recorded strike behaviours of the Chinese mantis with a high-speed camera and electromyograms of the foreleg trochanteral extensor and flexor. At the approach phase of the mantis strike, the prothorax-coxa (P-C) joint elevated and the femur-tibia (F-T) joint extended. At the sweep phase, the coxa-trochanter (C-T) joint rapidly extended, then, the F-T joint rapidly flexed to capture the prey. At capture initiation, the C-T joint extended more with greater prey distance. After cutting the tendon of the trochanteral flexor, the C-T joint extended similarly to that of the intact foreleg but did not flex after it reached its peak angle. After cutting the tendon of the trochanteral extensor, the C-T joint did not extend as much as that of the intact foreleg. During rapid extension of the C-T joint, a burst of spikes from the coxal trochanteral extensor was observed in electromyograms. Among several parameters, burst duration was the best predictor of C-T joint angular change during strike. Unexpectedly, trochanteral flexor activity was also observed during rapid extension of the C-T joint. These results indicated that the coxal trochanteral extensor mainly contributed to the rapid C-T extension during strike, but other muscles also contributed at the beginning of extension. The trochanteral flexor appeared to contribute to C-T flexion by countering the rapid extension.

Accommodating unobservability to control flight attitude with optic flow

Attitude control is an essential flight capability. Whereas flying robots commonly rely on accelerometers¹ for estimating attitude, flying insects lack an unambiguous sense of gravity^2,3. Despite the established role of several sense organs in attitude stabilization^3-5, the dependence of flying insects on an internal gravity direction estimate remains unclear. Here we show how attitude can be extracted from optic flow when combined with a motion model that relates attitude to acceleration direction. Although there are conditions such as hover in which the attitude is unobservable, we prove that the ensuing control system is still stable, continuously moving into and out of these conditions. Flying robot experiments confirm that accommodating unobservability in this manner leads to stable, but slightly oscillatory, attitude control. Moreover, experiments with a bio-inspired flapping-wing robot show that residual, high-frequency attitude oscillations from flapping motion improve observability. The presented approach holds a promise for robotics, with accelerometer-less autopilots paving the road for insect-scale autonomous flying robots⁶. Finally, it forms a hypothesis on insect attitude estimation and control, with the potential to provide further insight into known biological phenomena^5,7,8 and to generate new predictions such as reduced head and body attitude variance at higher flight speeds⁹.

Increased upper-limb sensory attenuation with age

The pressure of our own finger on the arm feels differently than the same pressure exerted by an external agent: the latter involves just touch, whereas the former involves a combination of touch and predictive output from the internal model of the body. This internal model predicts the movement of our own finger and hence the intensity of the sensation of the finger press is decreased. A decrease in intensity of the self-produced stimulus is called sensory attenuation. It has been reported that, due to decreased proprioception with age and an increased reliance on the prediction of the internal model, sensory attenuation is increased in older adults. In this study, we used a force-matching paradigm to test if sensory attenuation is also present over the arm and if aging increases sensory attenuation. We demonstrated that, while both young and older adults overestimate a self-produced force, older adults overestimate it even more showing an increased sensory attenuation. In addition, we also found that both younger and older adults self-produce higher forces when activating the homologous muscles of the upper limb. While this is traditionally viewed as evidence for an increased reliance on internal model function in older adults because of decreased proprioception, proprioception appeared unimpaired in our older participants. This begs the question of whether an age-related decrease in proprioception is really responsible for the increased sensory attenuation observed in older people.

Flight-induced compass representation in the monarch butterfly heading network

For navigation, animals use a robust internal compass. Compass navigation is crucial for long-distance migrating animals like monarch butterflies, which use the sun to navigate over 4,000 km to their overwintering sites every fall. Sun-compass neurons of the central complex have only been recorded in immobile butterflies, and experimental evidence for encoding the animal's heading in these neurons is still missing. Although the activity of central-complex neurons exhibits a locomotor-dependent modulation in many insects, the function of such modulations remains unexplored. Here, we developed tetrode recordings from tethered flying monarch butterflies to reveal how flight modulates heading representation. We found that, during flight, heading-direction neurons change their tuning, transforming the central-complex network to function as a global compass. This compass is characterized by the dominance of processing steering feedback and allows for robust heading representation even under unreliable visual scenarios, an ideal strategy for maintaining a migratory heading over enormous distances.

Insect-Inspired Robots: Bridging Biological and Artificial Systems

This review article aims to address common research questions in hexapod robotics. How can we build intelligent autonomous hexapod robots that can exploit their biomechanics, morphology, and computational systems, to achieve autonomy, adaptability, and energy efficiency comparable to small living creatures, such as insects? Are insects good models for building such intelligent hexapod robots because they are the only animals with six legs? This review article is divided into three main sections to address these questions, as well as to assist roboticists in identifying relevant and future directions in the field of hexapod robotics over the next decade. After an introduction in section (1), the sections will respectively cover the following three key areas: (2) biomechanics focused on the design of smart legs; (3) locomotion control; and (4) high-level cognition control. These interconnected and interdependent areas are all crucial to improving the level of performance of hexapod robotics in terms of energy efficiency, terrain adaptability, autonomy, and operational range. We will also discuss how the next generation of bioroboticists will be able to transfer knowledge from biology to robotics and vice versa.

A connectome of the Drosophila central complex reveals network motifs suitable for flexible navigation and context-dependent action selection

Flexible behaviors over long timescales are thought to engage recurrent neural networks in deep brain regions, which are experimentally challenging to study. In insects, recurrent circuit dynamics in a brain region called the central complex (CX) enable directed locomotion, sleep, and context- and experience-dependent spatial navigation. We describe the first complete electron microscopy-based connectome of the Drosophila CX, including all its neurons and circuits at synaptic resolution. We identified new CX neuron types, novel sensory and motor pathways, and network motifs that likely enable the CX to extract the fly's head direction, maintain it with attractor dynamics, and combine it with other sensorimotor information to perform vector-based navigational computations. We also identified numerous pathways that may facilitate the selection of CX-driven behavioral patterns by context and internal state. The CX connectome provides a comprehensive blueprint necessary for a detailed understanding of network dynamics underlying sleep, flexible navigation, and state-dependent action selection.

Oscillations make a self-scaled model for honeybees' visual odometer reliable regardless of flight trajectory

Honeybees foraging and recruiting nest-mates by performing the waggle dance need to be able to gauge the flight distance to the food source regardless of the wind and terrain conditions. Previous authors have hypothesized that the foragers’ visual odometer mathematically integrates the angular velocity of the ground image sweeping backward across their ventral viewfield, known as translational optic flow. The question arises as to how mathematical integration of optic flow (usually expressed in radians/s) can reliably encode distances, regardless of the height and speed of flight. The vertical self-oscillatory movements observed in honeybees trigger expansions and contractions of the optic flow vector field, yielding an additional visual cue called optic flow divergence. We have developed a self-scaled model for the visual odometer in which the translational optic flow is scaled by the visually estimated current clearance from the ground. In simulation, this model, which we have called SOFIa, was found to be reliable in a large range of flight trajectories, terrains and wind conditions. It reduced the statistical dispersion of the estimated flight distances approximately 10-fold in comparison with the mathematically integrated raw optic flow model. The SOFIa model can be directly implemented in robotic applications based on minimalistic visual equipment.

A leg to stand on: computational models of proprioception

Dexterous motor control requires feedback from proprioceptors, internal mechanosensory neurons that sense the body's position and movement. An outstanding question in neuroscience is how diverse proprioceptive feedback signals contribute to flexible motor control. Genetic tools now enable targeted recording and perturbation of proprioceptive neurons in behaving animals; however, these experiments can be challenging to interpret, due to the tight coupling of proprioception and motor control. Here, we argue that understanding the role of proprioceptive feedback in controlling behavior will be aided by the development of multiscale models of sensorimotor loops. We review current phenomenological and structural models for proprioceptor encoding and discuss how they may be integrated with existing models of posture, movement, and body state estimation.

Sniff-synchronized, gradient-guided olfactory search by freely moving mice

For many organisms, searching for relevant targets such as food or mates entails active, strategic sampling of the environment. Finding odorous targets may be the most ancient search problem that motile organisms evolved to solve. While chemosensory navigation has been well characterized in microorganisms and invertebrates, spatial olfaction in vertebrates is poorly understood. We have established an olfactory search assay in which freely moving mice navigate noisy concentration gradients of airborne odor. Mice solve this task using concentration gradient cues and do not require stereo olfaction for performance. During task performance, respiration and nose movement are synchronized with tens of milliseconds precision. This synchrony is present during trials and largely absent during inter-trial intervals, suggesting that sniff-synchronized nose movement is a strategic behavioral state rather than simply a constant accompaniment to fast breathing. To reveal the spatiotemporal structure of these active sensing movements, we used machine learning methods to parse motion trajectories into elementary movement motifs. Motifs fall into two clusters, which correspond to investigation and approach states. Investigation motifs lock precisely to sniffing, such that the individual motifs preferentially occur at specific phases of the sniff cycle. The allocentric structure of investigation and approach indicates an advantage to sampling both sides of the sharpest part of the odor gradient, consistent with a serial-sniff strategy for gradient sensing. This work clarifies sensorimotor strategies for mouse olfactory search and guides ongoing work into the underlying neural mechanisms.

Insightful behaviour in arthropods?

Arthropod behaviour is usually explained through ‘hard-wired’ motor routines and learning abilities that have been favoured by natural selection. We describe observations in which two arthropods solved rare and perhaps completely novel problems, and consider four possible explanations for their behaviours: (i) the behaviour was a pre-programmed motor routine evolved to solve this particular problem, or evolved for other functions but pre-programmed to be recruited for this function under certain conditions; (ii) it was learned previously; (iii) it resulted by chance; or (iv) it was the result of insightful behaviour. Pre-programmed solutions can be favoured by natural selection if they provide solutions to common or crucial problems. Given the apparent rarity of the problems that these animals solved, the solutions they employed are unlikely to represent innate behaviour. Learning and random chance seem unlikely, although we cannot rule them out completely. Possibly these animals employed some degree of insight.

Efference copy in kinesthetic perception: a copy of what is it?

A number of notions in the fields of motor control and kinesthetic perception have been used without clear definitions. In this review, we consider definitions for efference copy, percept, and sense of effort based on recent studies within the physical approach, which assumes that the neural control of movement is based on principles of parametric control and involves defining time-varying profiles of spatial referent coordinates for the effectors. The apparent redundancy in both motor and perceptual processes is reconsidered based on the principle of abundance. Abundance of efferent and afferent signals is viewed as the means of stabilizing both salient action characteristics and salient percepts formalized as stable manifolds in high-dimensional spaces of relevant elemental variables. This theoretical scheme has led recently to a number of novel predictions and findings. These include, in particular, lower accuracy in perception of variables produced by elements involved in a multielement task compared with the same elements in single-element tasks, dissociation between motor and perceptual effects of muscle coactivation, force illusions induced by muscle vibration, and errors in perception of unintentional drifts in performance. Taken together, these results suggest that participation of efferent signals in perception frequently involves distorted copies of actual neural commands, particularly those to antagonist muscles. Sense of effort is associated with such distorted efferent signals. Distortions in efference copy happen spontaneously and can also be caused by changes in sensory signals, e.g., those produced by muscle vibration.

Distortions of the Efferent Copy during Force Perception: A Study of Force Drifts and Effects of Muscle Vibration

We used a finger force matching task to explore the role of efferent signals in force perception. Healthy, young participants performed accurate force production tasks at different force levels with the index and middle fingers of one hand (task-hand). They received visual feedback during an early part of each trial only. After the feedback was turned off, the force drifted toward lower magnitudes. After 5 s of the drift, the participants matched the force with the same finger pair of the other hand (match-hand). The match-hand consistently overshot the task-hand force by a magnitude invariant over the initial force levels. During force matching, both hands were lifted and lowered smoothly to estimate their referent coordinate (RC) and apparent stiffness values. These trials were performed without muscle vibration and under vibration applied to the finger/hand flexors or extensors of the task-hand or match-hand. Effects of vibration were seen in the match-hand only; they were the same during vibration of flexors and extensors. We interpret the vibration-induced effects as consequences of using distorted copies of the central commands to the task-hand during force matching. In particular, using distorted copies of the RC for the antagonist muscle group could account for the differences between the task-hand and match-hand. We conclude that efferent signals may be distorted before their participation in the perceptual process. Such distortions emerge spontaneously and may be amplified by the response of sensory endings to muscle vibration combined over both agonist and antagonist muscle groups.

Movement-Related Signals in Sensory Areas: Roles in Natural Behavior

Recent studies have demonstrated prominent and widespread movement-related signals in the brain of head-fixed mice, even in primary sensory areas. However, it is still unknown what role these signals play in sensory processing. Why are these sensory areas 'contaminated' by movement signals? During natural behavior, animals actively acquire sensory information as they move through the environment and use this information to guide ongoing actions. In this context, movement-related signals could allow sensory systems to predict self-induced sensory changes and extract additional information about the environment. In this review we summarize recent findings on the presence of movement-related signals in sensory areas and discuss how their study, in the context of natural freely moving behaviors, could advance models of sensory processing.

Tutorial Review of Bio-Inspired Approaches to Robotic Manipulation for Space Debris Salvage

We present a comprehensive tutorial review that explores the application of bio-inspired approaches to robot control systems for grappling and manipulating a wide range of space debris targets. Current robot manipulator control systems exploit limited techniques which can be supplemented by additional bio-inspired methods to provide a robust suite of robot manipulation technologies. In doing so, we review bio-inspired control methods because this will be the key to enabling such capabilities. In particular, force feedback control may be supplemented with predictive forward models and software emulation of viscoelastic preflexive joint behaviour. This models human manipulation capabilities as implemented by the cerebellum and muscles/joints respectively. In effect, we are proposing a three-level control strategy based on biomimetic forward models for predictive estimation, traditional feedback control and biomimetic muscle-like preflexes. We place emphasis on bio-inspired forward modelling suggesting that all roads lead to this solution for robust and adaptive manipulator control. This promises robust and adaptive manipulation for complex tasks in salvaging space debris.

Connecting brain to behaviour: a role for general purpose steering circuits in insect orientation?

The lateral accessory lobes (LALs), paired structures that are homologous among all insect species, have been well studied for their role in pheromone tracking in silkmoths and phonotaxis in crickets, where their outputs have been shown to correlate with observed motor activity. Further studies have shown more generally that the LALs are crucial both for an insect's ability to steer correctly and for organising the outputs of the descending pathways towards the motor centres. In this context, we propose a framework by which the LALs may be generally involved in generating steering commands across a variety of insects and behaviours. Across different behaviours, we see that the LAL is involved in generating two kinds of steering: (1) search behaviours and (2) targeted steering driven by direct sensory information. Search behaviours are generated when the current behaviourally relevant cues are not available, and a well-described LAL subnetwork produces activity which increases sampling of the environment. We propose that, when behaviourally relevant cues are available, the LALs may integrate orientation information from several sensory modalities, thus leading to a collective output for steering driven by those cues. These steering commands are then sent to the motor centres, and an additional efference copy is sent back to the orientation-computing areas. In summary, we have taken known aspects of the neurophysiology and function of the insect LALs and generated a speculative framework that suggests how LALs might be involved in steering control for a variety of complex real-world behaviours in insects.

Interactive rhythms across species: the evolutionary biology of animal chorusing and turn-taking

The study of human language is progressively moving toward comparative and interactive frameworks, extending the concept of turn-taking to animal communication. While such an endeavor will help us understand the interactive origins of language, any theoretical account for cross-species turn-taking should consider three key points. First, animal turn-taking must incorporate biological studies on animal chorusing, namely how different species coordinate their signals over time. Second, while concepts employed in human communication and turn-taking, such as intentionality, are still debated in animal behavior, lower level mechanisms with clear neurobiological bases can explain much of animal interactive behavior. Third, social behavior, interactivity, and cooperation can be orthogonal, and the alternation of animal signals need not be cooperative. Considering turn-taking a subset of chorusing in the rhythmic dimension may avoid overinterpretation and enhance the comparability of future empirical work.

How foresight might support the behavioral flexibility of arthropods

The small brains of insects and other invertebrates are often thought to constrain these animals to live entirely 'in the moment'. In this view, each one of their many seemingly hard-wired behavioral routines is triggered by a precisely defined environmental stimulus configuration, but there is no mental appreciation of the possible outcomes of one's actions, and therefore little flexibility. However, many studies show problem-solving behavior in various arthropod species that falls outside the range of fixed behavior routines. We propose that a basic form of foresight, the ability to predict the outcomes of one's own actions, is at the heart of such behavioral flexibility, and that the evolutionary roots of such outcome expectation are found in the need to disentangle sensory input that is predictable from self-generated motion versus input generated by changes in the outside world. Based on this, locusts, grasshoppers, dragonflies and flies seem to use internal models of the surrounding world to tailor their actions adaptively to predict the imminent future. Honeybees and orb-weaving spiders appear to act towards a desired outcome of their respective constructions, and the genetically pre-programmed routines that govern these constructions are subordinate to achieving the desired goal. Jumping spiders seem to preplan their route to prey suggesting they recognize the spatial challenge and actions necessary to obtain prey. Bumblebees and ants utilize objects not encountered in the wild as types of tools to solve problems in a manner that suggests an awareness of the desired outcome. Here we speculate that it may be simpler, in terms of the required evolutionary changes, computation and neural architecture, for arthropods to recognize their goal and predict the outcomes of their actions towards that goal, rather than having a large number of pre-programmed behaviors necessary to account for their observed behavioral flexibility.

Bee cognition

Maeterlinck did not mean to suggest that honeybees rival humans in intelligence - rather he saw in the bee a qualitatively different form of intelligence, tailored to the challenges of a profoundly different kind of society and lifestyle. Insects are strange "aliens from inner space", with sensory and cognitive worlds wholly different from our own. The 19^th century discovery that ants can detect ultraviolet light triggered a golden age in the exploration of the diversity of sensory systems of insects (and indeed other animals), identifying such abilities as magnetic compasses, electrosensitivity, polarization vision, and peculiar locations for sense organs such as the infrared sensors on the abdomens of some beetles or photoreceptors on the genitalia of some butterflies. Could insect minds be equally strange and diverse?

Transfer of Visual Learning Between a Virtual and a Real Environment in Honey Bees: The Role of Active Vision

To study visual learning in honey bees, we developed a virtual reality (VR) system in which the movements of a tethered bee walking stationary on a spherical treadmill update the visual panorama presented in front of it (closed-loop conditions), thus creating an experience of immersion within a virtual environment. In parallel, we developed a small Y-maze with interchangeable end-boxes, which allowed replacing repeatedly a freely walking bee into the starting point of the maze for repeated decision recording. Using conditioning and transfer experiments between the VR setup and the Y-maze, we studied the extent to which movement freedom and active vision are crucial for learning a simple color discrimination. Approximately 57% of the bees learned the visual discrimination in both conditions. Transfer from VR to the maze improved significantly the bees’ performances: 75% of bees having chosen the CS+ continued doing so and 100% of bees having chosen the CS− reverted their choice in favor of the CS+. In contrast, no improvement was seen for these two groups of bees during the reciprocal transfer from the Y-maze to VR. In this case, bees exhibited inconsistent choices in the VR setup. The asymmetric transfer between contexts indicates that the information learned in each environment may be different despite the similar learning success. Moreover, it shows that reducing the possibility of active vision and movement freedom in the passage from the maze to the VR impairs the expression of visual learning while increasing them in the reciprocal transfer improves it. Our results underline the active nature of visual processing in bees and allow discussing the developments required for immersive VR experiences in insects.

Cognitive Aspects of Comb-Building in the Honeybee?

The wax-made comb of the honeybee is a masterpiece of animal architecture. The highly regular, double-sided hexagonal structure is a near-optimal solution to storing food and housing larvae, economizing on building materials and space. Elaborate though they may seem, such animal constructions are often viewed as the result of 'just instinct,' governed by inflexible, pre-programmed, innate behavior routines. An inspection of the literature on honeybee comb construction, however, reveals a different picture. Workers have to learn, at least in part, certain elements of the technique, and there is considerable flexibility in terms of how the shape of the comb and its gradual manufacture is tailored to the circumstances, especially the available space. Moreover, we explore the 2-century old and now largely forgotten work by François Huber, where glass screens were placed between an expanding comb construction and the intended target wall. Bees took corrective action before reaching the glass obstacle, and altered the ongoing construction so as to reach the nearest wooden wall. Though further experiments will be necessary, these results suggest a form of spatial planning skills. We discuss these findings in the context of what is now known about insect cognition, and ask if it is possible that the production of hexagonal wax combs is the result of behavioral heuristics where a complex structure emerges as the result of simple behavioral rules applied by each individual, or whether prospective cognition might be involved.

Using insects to drive mobile robots - hybrid robots bridge the gap between biological and artificial systems

The use of mobile robots is an effective method of validating sensory-motor models of animals in a real environment. The well-identified insect sensory-motor systems have been the major targets for modeling. Furthermore, mobile robots implemented with such insect models attract engineers who aim to avail advantages from organisms. However, directly comparing the robots with real insects is still difficult, even if we successfully model the biological systems, because of the physical differences between them. We developed a hybrid robot to bridge the gap. This hybrid robot is an insect-controlled robot, in which a tethered male silkmoth (Bombyx mori) drives the robot in order to localize an odor source. This robot has the following three advantages: 1) from a biomimetic perspective, the robot enables us to evaluate the potential performance of future insect-mimetic robots; 2) from a biological perspective, the robot enables us to manipulate the closed-loop of an onboard insect for further understanding of its sensory-motor system; and 3) the robot enables comparison with insect models as a reference biological system. In this paper, we review the recent works regarding insect-controlled robots and discuss the significance for both engineering and biology.

ReaCog, a Minimal Cognitive Controller Based on Recruitment of Reactive Systems

It has often been stated that for a neuronal system to become a cognitive one, it has to be large enough. In contrast, we argue that a basic property of a cognitive system, namely the ability to plan ahead, can already be fulfilled by small neuronal systems. As a proof of concept, we propose an artificial neural network, termed reaCog, that, first, is able to deal with a specific domain of behavior (six-legged-walking). Second, we show how a minor expansion of this system enables the system to plan ahead and deploy existing behavioral elements in novel contexts in order to solve current problems. To this end, the system invents new solutions that are not possible for the reactive network. Rather these solutions result from new combinations of given memory elements. This faculty does not rely on a dedicated system being more or less independent of the reactive basis, but results from exploitation of the reactive basis by recruiting the lower-level control structures in a way that motor planning becomes possible as an internal simulation relying on internal representation being grounded in embodied experiences.

Adaptive and Energy Efficient Walking in a Hexapod Robot Under Neuromechanical Control and Sensorimotor Learning

The control of multilegged animal walking is a neuromechanical process, and to achieve this in an adaptive and energy efficient way is a difficult and challenging problem. This is due to the fact that this process needs in real time: 1) to coordinate very many degrees of freedom of jointed legs; 2) to generate the proper leg stiffness (i.e., compliance); and 3) to determine joint angles that give rise to particular positions at the endpoints of the legs. To tackle this problem for a robotic application, here we present a neuromechanical controller coupled with sensorimotor learning. The controller consists of a modular neural network for coordinating 18 joints and several virtual agonist-antagonist muscle mechanisms (VAAMs) for variable compliant joint motions. In addition, sensorimotor learning, including forward models and dual-rate learning processes, is introduced for predicting foot force feedback and for online tuning the VAAMs' stiffness parameters. The control and learning mechanisms enable the hexapod robot advanced mobility sensor driven-walking device (AMOS) to achieve variable compliant walking that accommodates different gaits and surfaces. As a consequence, AMOS can perform more energy efficient walking, compared to other small legged robots. In addition, this paper also shows that the tight combination of neural control with tunable muscle-like functions, guided by sensory feedback and coupled with sensorimotor learning, is a way forward to better understand and solve adaptive coordination problems in multilegged locomotion.

Spatiotopic updating facilitates perception immediately after saccades

As the neural representation of visual information is initially coded in retinotopic coordinates, eye movements (saccades) pose a major problem for visual stability. If no visual information were maintained across saccades, retinotopic representations would have to be rebuilt after each saccade. It is currently strongly debated what kind of information (if any at all) is accumulated across saccades, and when this information becomes available after a saccade. Here, we use a motion illusion to examine the accumulation of visual information across saccades. In this illusion, an annulus with a random texture slowly rotates, and is then replaced with a second texture (motion transient). With increasing rotation durations, observers consistently perceive the transient as large rotational jumps in the direction opposite to rotation direction (backward jumps). We first show that accumulated motion information is updated spatiotopically across saccades. Then, we show that this accumulated information is readily available after a saccade, immediately biasing postsaccadic perception. The current findings suggest that presaccadic information is used to facilitate postsaccadic perception and are in support of a forward model of transsaccadic perception, aiming at anticipating the consequences of eye movements and operating within the narrow perisaccadic time window.

The aerodynamics and control of free flight manoeuvres in Drosophila

A firm understanding of how fruit flies hover has emerged over the past two decades, and recent work has focused on the aerodynamic, biomechanical and neurobiological mechanisms that enable them to manoeuvre and resist perturbations. In this review, we describe how flies manipulate wing movement to control their body motion during active manoeuvres, and how these actions are regulated by sensory feedback. We also discuss how the application of control theory is providing new insight into the logic and structure of the circuitry that underlies flight stability.This article is part of the themed issue 'Moving in a moving medium: new perspectives on flight'.

Motor control: how dragonflies catch their prey

Detailed measurements of head and body motion have revealed previously unknown complexity in the predatory behavior of dragonflies. The new evidence suggests that the brains of these agile predators compute internal models of their own actions and those of their prey.

What insects can tell us about the origins of consciousness

How, why, and when consciousness evolved remain hotly debated topics. Addressing these issues requires considering the distribution of consciousness across the animal phylogenetic tree. Here we propose that at least one invertebrate clade, the insects, has a capacity for the most basic aspect of consciousness: subjective experience. In vertebrates the capacity for subjective experience is supported by integrated structures in the midbrain that create a neural simulation of the state of the mobile animal in space. This integrated and egocentric representation of the world from the animal's perspective is sufficient for subjective experience. Structures in the insect brain perform analogous functions. Therefore, we argue the insect brain also supports a capacity for subjective experience. In both vertebrates and insects this form of behavioral control system evolved as an efficient solution to basic problems of sensory reafference and true navigation. The brain structures that support subjective experience in vertebrates and insects are very different from each other, but in both cases they are basal to each clade. Hence we propose the origins of subjective experience can be traced to the Cambrian.

Distributed recurrent neural forward models with synaptic adaptation and CPG-based control for complex behaviors of walking robots

Walking animals, like stick insects, cockroaches or ants, demonstrate a fascinating range of locomotive abilities and complex behaviors. The locomotive behaviors can consist of a variety of walking patterns along with adaptation that allow the animals to deal with changes in environmental conditions, like uneven terrains, gaps, obstacles etc. Biological study has revealed that such complex behaviors are a result of a combination of biomechanics and neural mechanism thus representing the true nature of embodied interactions. While the biomechanics helps maintain flexibility and sustain a variety of movements, the neural mechanisms generate movements while making appropriate predictions crucial for achieving adaptation. Such predictions or planning ahead can be achieved by way of internal models that are grounded in the overall behavior of the animal. Inspired by these findings, we present here, an artificial bio-inspired walking system which effectively combines biomechanics (in terms of the body and leg structures) with the underlying neural mechanisms. The neural mechanisms consist of (1) central pattern generator based control for generating basic rhythmic patterns and coordinated movements, (2) distributed (at each leg) recurrent neural network based adaptive forward models with efference copies as internal models for sensory predictions and instantaneous state estimations, and (3) searching and elevation control for adapting the movement of an individual leg to deal with different environmental conditions. Using simulations we show that this bio-inspired approach with adaptive internal models allows the walking robot to perform complex locomotive behaviors as observed in insects, including walking on undulated terrains, crossing large gaps, leg damage adaptations, as well as climbing over high obstacles. Furthermore, we demonstrate that the newly developed recurrent network based approach to online forward models outperforms the adaptive neuron forward models, which have hitherto been the state of the art, to model a subset of similar walking behaviors in walking robots.

Bumblebee Homing: The Fine Structure of Head Turning Movements

Changes in flight direction in flying insects are largely due to roll, yaw and pitch rotations of their body. Head orientation is stabilized for most of the time by counter rotation. Here, we use high-speed video to analyse head- and body-movements of the bumblebee Bombus terrestris while approaching and departing from a food source located between three landmarks in an indoor flight-arena. The flight paths consist of almost straight flight segments that are interspersed with rapid turns. These short and fast yaw turns ("saccades") are usually accompanied by even faster head yaw turns that change gaze direction. Since a large part of image rotation is thereby reduced to brief instants of time, this behavioural pattern facilitates depth perception from visual motion parallax during the intersaccadic intervals. The detailed analysis of the fine structure of the bees' head turning movements shows that the time course of single head saccades is very stereotypical. We find a consistent relationship between the duration, peak velocity and amplitude of saccadic head movements, which in its main characteristics resembles the so-called "saccadic main sequence" in humans. The fact that bumblebee head saccades are highly stereotyped as in humans, may hint at a common principle, where fast and precise motor control is used to reliably reduce the time during which the retinal images moves.

Insects modify their behaviour depending on the feedback sensor used when walking on a trackball in virtual reality

When using virtual-reality paradigms to study animal behaviour, careful attention must be paid to how the animal's actions are detected. This is particularly relevant in closed-loop experiments where the animal interacts with a stimulus. Many different sensor types have been used to measure aspects of behaviour, and although some sensors may be more accurate than others, few studies have examined whether, and how, such differences affect an animal's behaviour in a closed-loop experiment. To investigate this issue, we conducted experiments with tethered honeybees walking on an air-supported trackball and fixating a visual object in closed-loop. Bees walked faster and along straighter paths when the motion of the trackball was measured in the classical fashion - using optical motion sensors repurposed from computer mice - than when measured more accurately using a computer vision algorithm called 'FicTrac'. When computer mouse sensors were used to measure bees' behaviour, the bees modified their behaviour and achieved improved control of the stimulus. This behavioural change appears to be a response to a systematic error in the computer mouse sensor that reduces the sensitivity of this sensor system under certain conditions. Although the large perceived inertia and mass of the trackball relative to the honeybee is a limitation of tethered walking paradigms, observing differences depending on the sensor system used to measure bee behaviour was not expected. This study suggests that bees are capable of fine-tuning their motor control to improve the outcome of the task they are performing. Further, our findings show that caution is required when designing virtual-reality experiments, as animals can potentially respond to the artificial scenario in unexpected and unintended ways.

Cellular evidence for efference copy in Drosophila visuomotor processing

Each time a locomoting fly turns, the visual image sweeps over the retina and generates a motion stimulus. Classic behavioral experiments suggested that flies use active neural-circuit mechanisms to suppress the perception of self-generated visual motion during intended turns. Direct electrophysiological evidence, however, has been lacking. We found that visual neurons in Drosophila receive motor-related inputs during rapid flight turns. These inputs arrived with a sign and latency appropriate for suppressing each targeted cell's visual response to the turn. Precise measurements of behavioral and neuronal response latencies supported the idea that motor-related inputs to optic flow-processing cells represent internal predictions of the expected visual drive induced by voluntary turns. Motor-related inputs to small object-selective visual neurons could reflect either proprioceptive feedback from the turn or internally generated signals. Our results in Drosophila echo the suppression of visual perception during rapid eye movements in primates, demonstrating common functional principles of sensorimotor processing across phyla.

A simple behaviour provides accuracy and flexibility in odour plume tracking—the robotic control of sensory-motor coupling in silkmoths

Odour plume tracking is an essential behaviour for animal survival. A fundamental strategy for this is moving upstream and then across-stream. Male silkmoths, Bombyx mori, display this strategy as a pre-programmed sequential behaviour. They walk forward (surge) in response to the female sex pheromone and perform a zigzagging “mating dance”. Though pre-programmed, the surge direction is modulated by the bilateral olfactory input and optic flow. However, the nature of interaction between these two sensory modalities and contribution of the resultant motor command to localizing an odour source are still unknown. We evaluated the ability of the silkmoth to localize the odour source under conditions of disturbed sensory-motor coupling, using a silkmoth-driven mobile robot. The significance of the bilateral olfaction of the moth was confirmed by inverting the robot's olfactory input that is acquired using two flexible air-suction tubes and delivered to the antennae of the moth, or its motor output. Inversion of the motor output induced consecutive circling, which was inhibited by covering the visual field of the moth. This suggests that the corollary discharge from the motor command and the reafference of self-generated optic flow generates compensatory signals to guide the surge accurately. Additionally, after inverting the olfactory input, the robot successfully tracked the odour plume by using a combination of behaviours. These results indicate that accurate guidance of the reflexive surge by integrating bilateral olfactory and visual information with innate pre-programmed behaviours increases the flexibility to track an odour plume even under disturbed circumstances.

Brain-Map Based Carangiform Swimming Behaviour Modeling and Control in a Robotic Fish Underwater Vehicle

Fish swimming demonstrates impressive speeds and exceptional characteristics in the fluid environment. The objective of this paper is to mimic undulatory swimming behaviour and its control of a body caudal fin (BCF) carangiform fish in a robotic counterpart. Based on fish biology kinematics study, a 2-level behavior based distributed control scheme is proposed. The high-level control is modeled by robotic fish swimming behavior. It uses a Lighthill (LH) body wave to generate desired joint trajectory patterns. Generated LH body wave is influenced by intrinsic kinematic parameters Tail-beat frequency (TBF) and Caudal amplitude (CA) which can be modulated to change the trajectory pattern. Parameter information is retrieved from a fish memory (cerebellum) inspired brain map. This map stores operating region information on TBF and CA parameters obtained from yellow fin tuna kinematics study. Based on an environment based error feedback signal, robotic fish map selects the right parameters value showing adaptive behaviour. A finite state machine methodology has been used to model this brain-kinematic-map control. The low-level control is implemented using inverse dynamics based computed torque method (CTM) with dynamic PD compensation. It tracks high-level generated and encoded patterns (trajectory) for fish-tail undulation. Three types of parameter adaptation for the two chosen parameters have been shown to successfully emulate robotic fish swimming behavior. Based on the proposed control strategy joint-position and velocity tracking results are discussed. They are found to be satisfactory with error magnitudes within permissible bounds.

A novel acoustic-vibratory multimodal duet

The communication strategy of most crickets and bushcrickets typically consists of males broadcasting loud acoustic calling songs, while females perform phonotaxis, moving towards the source of the call. Males of the pseudophylline bushcricket species Onomarchus uninotatus produce an unusually low-pitched call, and we found that the immediate and most robust response of females to the male acoustic call was a bodily vibration, or tremulation, following each syllable of the call. We hypothesized that these bodily oscillations might send out a vibrational signal along the substrate on which the female stands, which males could use to localize her position. We quantified these vibrational signals using a laser vibrometer and found a clear phase relationship of alternation between the chirps of the male acoustic call and the female vibrational response. This system therefore constitutes a novel multimodal duet with a reliable temporal structure. We also found that males could localize the source of vibration but only if both the acoustic and vibratory components of the duet were played back. This unique multimodal duetting system may have evolved in response to higher levels of bat predation on searching bushcricket females than calling males, shifting a part of the risks associated with partner localization onto the male. This is the first known example of bushcricket female tremulation in response to a long-range male acoustic signal and of a multimodal duet among animals.

Internal models direct dragonfly interception steering

Sensorimotor control in vertebrates relies on internal models. When extending an arm to reach for an object, the brain uses predictive models of both limb dynamics and target properties. Whether invertebrates use such models remains unclear. Here we examine to what extent prey interception by dragonflies (Plathemis lydia), a behaviour analogous to targeted reaching, requires internal models. By simultaneously tracking the position and orientation of a dragonfly's head and body during flight, we provide evidence that interception steering is driven by forward and inverse models of dragonfly body dynamics and by models of prey motion. Predictive rotations of the dragonfly's head continuously track the prey's angular position. The head-body angles established by prey tracking appear to guide systematic rotations of the dragonfly's body to align it with the prey's flight path. Model-driven control thus underlies the bulk of interception steering manoeuvres, while vision is used for reactions to unexpected prey movements. These findings illuminate the computational sophistication with which insects construct behaviour.

The Time of Perception and the Other Way Around

The world we perceive is delayed in relation to its flowing content, as well as the outcome of our actions on the world in relation to the moment we decide to act. This mosaic of different latencies permeating both perception and action has to be taken into account critically in order for us to cope with the temporal challenges constantly imposed by the environment. Fundamental notions, such as the sense of agency and causality, depend on the temporal relationship of events occurring in well-defined windows of time. Here, we offer a broad, yet abridged, historical view of some thought-provoking issues concerning the time of perception and action. From the pioneering work of Wundt, Titchener, and Libet to recent findings and ideas related to the employment of visual illusions as psychophysical probes (such as the flash-lag effect), we have tried to expose some problems inherent to the act of measuring the time of both perception and action, and devise possible solutions as well.

Evidence for a causal inverse model in an avian cortico-basal ganglia circuit

Learning by imitation is fundamental to both communication and social behavior and requires the conversion of complex, nonlinear sensory codes for perception into similarly complex motor codes for generating action. To understand the neural substrates underlying this conversion, we study sensorimotor transformations in songbird cortical output neurons of a basal-ganglia pathway involved in song learning. Despite the complexity of sensory and motor codes, we find a simple, temporally specific, causal correspondence between them. Sensory neural responses to song playback mirror motor-related activity recorded during singing, with a temporal offset of roughly 40 ms, in agreement with short feedback loop delays estimated using electrical and auditory stimulation. Such matching of mirroring offsets and loop delays is consistent with a recent Hebbian theory of motor learning and suggests that cortico-basal ganglia pathways could support motor control via causal inverse models that can invert the rich correspondence between motor exploration and sensory feedback.