Behaviour-based modelling of hexapod locomotion: linking biology and technical application

Mechanosensory bristles mediate avoidance behavior by triggering sustained local motor activity in Drosophila melanogaster

During locomotion, most vertebrates-and invertebrates such as Drosophila melanogaster-are able to quickly adapt to terrain irregularities or avoid physical threats by integrating sensory information along with motor commands. Key to this adaptability are leg mechanosensory structures, which assist in motor coordination by transmitting external cues and proprioceptive information to motor centers in the central nervous system. Nevertheless, how different mechanosensory structures engage these locomotor centers remains poorly understood. Here, we tested the role of mechanosensory structures in movement initiation by optogenetically stimulating specific classes of leg sensory structures. We found that stimulation of leg mechanosensory bristles (MsBs) and the femoral chordotonal organ (ChO) is sufficient to initiate forward movement in immobile animals. While the stimulation of the ChO required brain centers to induce forward movement, unexpectedly, brief stimulation of leg MsBs triggered a fast response and sustained motor activity dependent only on the ventral nerve cord (VNC). Moreover, this leg-MsB-mediated movement lacked inter- and intra-leg coordination but preserved antagonistic muscle activity within joints. Finally, we show that leg-MsB activation mediates strong avoidance behavior away from the stimulus source, which is preserved even in the absence of a central brain. Overall, our data show that mechanosensory stimulation can elicit a fast motor response, independently of central brain commands, to evade potentially harmful stimuli. In addition, it sheds light on how specific sensory circuits modulate motor control, including initiation of movement, allowing a better understanding of how different levels of coordination are controlled by the VNC and central brain locomotor circuits.

The neuronal innervation pattern of the subgenual organ complex in Peruphasma schultei (Insecta: Phasmatodea)

The proximal tibia of orthopteroid insects contains sensory organs, the subgenual organ complex, detecting mechanical stimuli including substrate vibration. In stick insects, two chordotonal organs occur in close proximity, the subgenual organ and the distal organ, which likely detect substrate vibrations. In most stick insects, both organs are innervated by separate nerve branches. To obtain more data on the neuroanatomy of the subgenual organ complex from the New World phasmids (Occidophasmata), the present study documents the neuronal innervation of sensory organs in the subgenual organ complex of Peruphasma schultei, the first species from Pseudophasmatinae investigated for this sensory complex. The innervation pattern shows a distinct nerve branch for the subgenual organ and for the distal organ in most cases. Some variability in the innervation, which generally occurs for these chordotonal organs, was noted for both organs in P. schultei. The most common innervation for both organs was by a single nerve branch for each organ. The innervation of the subgenual organ resembled the nerve pattern of another New World phasmid, but was simpler than in the Old World phasmids (Oriophasmata) studied so far. Therefore, the peripheral neuronal innervation of sensory organs could reflect phylogenetic relationships and provide phylogenetic information, while the overall neuroanatomy of the subgenual organ complex is similar in stick insects.

Flexible locomotion in complex environments: the influence of species, speed and sensory feedback on panarthropod inter-leg coordination

Panarthropods (a clade containing arthropods, tardigrades and onychophorans) can adeptly move across a wide range of challenging terrains and their ability to do so given their relatively simple nervous systems makes them compelling study organisms. Studies of forward walking on flat terrain excitingly point to key features in inter-leg coordination patterns that seem to be 'universally' shared across panarthropods. However, when movement through more complex, naturalistic terrain is considered, variability in coordination patterns - from the intra-individual to inter-species level - becomes more apparent. This variability is likely to be due to the interplay between sensory feedback and local pattern-generating activity, and depends crucially on species, walking speed and behavioral goal. Here, I gather data from the literature of panarthropod walking coordination on both flat ground and across more complex terrain. This Review aims to emphasize the value of: (1) designing experiments with an eye towards studying organisms in natural environments; (2) thoughtfully integrating results from various experimental techniques, such as neurophysiological and biomechanical studies; and (3) ensuring that data is collected and made available from a wider range of species for future comparative analyses.

neuroWalknet, a controller for hexapod walking allowing for context dependent behavior

Decentralized control has been established as a key control principle in insect walking and has been successfully leveraged to account for a wide range of walking behaviors in the proposed neuroWalknet architecture. This controller allows for walking patterns at different velocities in both, forward and backward direction-quite similar to the behavior shown in stick insects-, for negotiation of curves, and for robustly dealing with various disturbances. While these simulations focus on the cooperation of different, decentrally controlled legs, here we consider a set of biological experiments not yet been tested by neuroWalknet, that focus on the function of the individual leg and are context dependent. These intraleg studies deal with four groups of interjoint reflexes. The reflexes are elicited by stimulation of the femoral chordotonal organ (fCO) or groups of campaniform sensilla (CS). Motor output signals are recorded from the alpha-joint, the beta-joint or the gamma-joint of the leg. Furthermore, the influence of these sensory inputs to artificially induced oscillations by application of pilocarpine has been studied. Although these biological data represent results obtained from different local reflexes in different contexts, they fit with and are embedded into the behavior shown by the global structure of neuroWalknet. In particular, a specific and intensively studied behavior, active reaction, has since long been assumed to represent a separate behavioral element, from which it is not clear why it occurs in some situations, but not in others. This question could now be explained as an emergent property of the holistic structure of neuroWalknet which has shown to be able to produce artificially elicited pilocarpine-driven oscillation that can be controlled by sensory input without the need of explicit innate CPG structures. As the simulation data result from a holistic system, further results were obtained that could be used as predictions to be tested in further biological experiments.

Adaptive Interlimb Coordination Mechanism for Hexapod Locomotion Based on Active Load Sensing

Insects can flexibly coordinate their limbs to adapt to various locomotor conditions, e.g., complex environments, changes in locomotion speed, and leg amputation. An interesting aspect of insect locomotion is that the gait patterns are not necessarily stereotypical but are often highly variable, e.g., searching behavior to obtain stable footholds in complex environments. Several previous studies have focused on the mechanism for the emergence of variable limb coordination patterns. However, the proposed mechanisms are complicated and the essential mechanism underlying insect locomotion remains elusive. To address this issue, we proposed a simple mathematical model for the mechanism of variable interlimb coordination in insect locomotion. The key idea of the proposed model is “decentralized active load sensing,” wherein each limb actively moves and detects the reaction force from the ground to judge whether it plays a pivotal role in maintaining the steady support polygon. Based on active load sensing, each limb stays in the stance phase when the limb is necessary for body support. To evaluate the proposed model, we conducted simulation experiments using a hexapod robot. The results showed that the proposed simple mechanism allows the hexapod robot to exhibit typical gait patterns in response to the locomotion speed. Furthermore, the proposed mechanism improves the adaptability of the hexapod robot for leg amputations and lack of footholds by changing each limb's walking and searching behavior in a decentralized manner based on the physical interaction between the body and the environment.

Decentralized control and local information for robust and adaptive decentralized Deep Reinforcement Learning

Decentralization is a central characteristic of biological motor control that allows for fast responses relying on local sensory information. In contrast, the current trend of Deep Reinforcement Learning (DRL) based approaches to motor control follows a centralized paradigm using a single, holistic controller that has to untangle the whole input information space. This motivates to ask whether decentralization as seen in biological control architectures might also be beneficial for embodied sensori-motor control systems when using DRL. To answer this question, we provide an analysis and comparison of eight control architectures for adaptive locomotion that were derived for a four-legged agent, but with their degree of decentralization varying systematically between the extremes of fully centralized and fully decentralized. Our comparison shows that learning speed is significantly enhanced in distributed architectures-while still reaching the same high performance level of centralized architectures-due to smaller search spaces and local costs providing more focused information for learning. Second, we find an increased robustness of the learning process in the decentralized cases-it is less demanding to hyperparameter selection and less prone to becoming trapped in poor local minima. Finally, when examining generalization to uneven terrains-not used during training-we find best performance for an intermediate architecture that is decentralized, but integrates only local information from both neighboring legs. Together, these findings demonstrate beneficial effects of distributing control into decentralized units and relying on local information. This appears as a promising approach towards more robust DRL and better generalization towards adaptive behavior.

Tardigrades exhibit robust interlimb coordination across walking speeds and terrains

Tardigrades must negotiate heterogeneous, fluctuating environments and accordingly utilize locomotive strategies capable of dealing with variable terrain. We analyze the kinematics and interleg coordination of freely walking tardigrades (species: Hypsibius exemplaris). We find that tardigrade walking replicates several key features of walking in insects despite disparities in size, skeleton, and habitat. To test the effect of environmental changes on tardigrade locomotor control circuits we measure kinematics and interleg coordination during walking on two substrates of different stiffnesses. We find that the phase offset between contralateral leg pairs is flexible, while ipsilateral coordination is preserved across environmental conditions. This mirrors similar results in insects and crustaceans. We propose that these functional similarities in walking coordination between tardigrades and arthropods is either due to a generalized locomotor control circuit common to panarthropods or to independent convergence onto an optimal strategy for robust multilegged control in small animals with simple circuitry. Our results highlight the value of tardigrades as a comparative system toward understanding the mechanisms-neural and/or mechanical-underlying coordination in panarthropod locomotion.

Universal Features in Panarthropod Inter-Limb Coordination during Forward Walking

Terrestrial animals must often negotiate heterogeneous, varying environments. Accordingly, their locomotive strategies must adapt to a wide range of terrain, as well as to a range of speeds to accomplish different behavioral goals. Studies in Drosophila have found that inter-leg coordination patterns (ICPs) vary smoothly with walking speed, rather than switching between distinct gaits as in vertebrates (e.g., horses transitioning between trotting and galloping). Such a continuum of stepping patterns implies that separate neural controllers are not necessary for each observed ICP. Furthermore, the spectrum of Drosophila stepping patterns includes all canonical coordination patterns observed during forward walking in insects. This raises the exciting possibility that the controller in Drosophila is common to all insects, and perhaps more generally to panarthropod walkers. Here, we survey and collate data on leg kinematics and inter-leg coordination relationships during forward walking in a range of arthropod species, as well as include data from a recent behavioral investigation into the tardigrade Hypsibius exemplaris. Using this comparative dataset, we point to several functional and morphological features that are shared among panarthropods. The goal of the framework presented in this review is to emphasize the importance of comparative functional and morphological analyses in understanding the origins and diversification of walking in Panarthropoda. Introduction.

From Motor-Output to Connectivity: An In-Depth Study of in-vitro Rhythmic Patterns in the Cockroach Periplaneta americana

The cockroach is an established model in the study of locomotion control. While previous work has offered important insights into the interplay among brain commands, thoracic central pattern generators, and the sensory feedback that shapes their motor output, there remains a need for a detailed description of the central pattern generators' motor output and their underlying connectivity scheme. To this end, we monitored pilocarpine-induced activity of levator and depressor motoneurons in two types of novel in-vitro cockroach preparations: isolated thoracic ganglia and a whole-chain preparation comprising the thoracic ganglia and the subesophageal ganglion. Our data analyses focused on the motoneuron firing patterns and the coordination among motoneuron types in the network. The burstiness and rhythmicity of the motoneurons were monitored, and phase relations, coherence, coupling strength, and frequency-dependent variability were analyzed. These parameters were all measured and compared among network units both within each preparation and among the preparations. Here, we report differences among the isolated ganglia, including asymmetries in phase and coupling strength, which indicate that they are wired to serve different functions. We also describe the intrinsic default gait and a frequency-dependent coordination. The depressor motoneurons showed mostly similar characteristics throughout the network regardless of interganglia connectivity; whereas the characteristics of the levator motoneurons activity were mostly ganglion-dependent, and influenced by the presence of interganglia connectivity. Asymmetries were also found between the anterior and posterior homolog parts of the thoracic network, as well as between ascending and descending connections. Our analyses further discover a frequency-dependent inversion of the interganglia coordination from alternations between ipsilateral homolog oscillators to simultaneous activity. We present a detailed scheme of the network couplings, formulate coupling rules, and review a previously suggested model of connectivity in light of our new findings. Our data support the notion that the inter-hemiganglia coordination derives from the levator networks and their coupling with local depressor interneurons. Our findings also support a dominant role of the metathoracic ganglion and its ascending output in governing the anterior ganglia motor output during locomotion in the behaving animal.

Drosophila uses a tripod gait across all walking speeds, and the geometry of the tripod is important for speed control

Changes in walking speed are characterized by changes in both the animal's gait and the mechanics of its interaction with the ground. Here we study these changes in walking Drosophila. We measured the fly's center of mass movement with high spatial resolution and the position of its footprints. Flies predominantly employ a modified tripod gait that only changes marginally with speed. The mechanics of a tripod gait can be approximated with a simple model - angular and radial spring-loaded inverted pendulum (ARSLIP) - which is characterized by two springs of an effective leg that become stiffer as the speed increases. Surprisingly, the change in the stiffness of the spring is mediated by the change in tripod shape rather than a change in stiffness of individual legs. The effect of tripod shape on mechanics can also explain the large variation in kinematics among insects, and ARSLIP can model these variations.

Distributed control of motor circuits for backward walking in Drosophila

How do descending inputs from the brain control leg motor circuits to change how an animal walks? Conceptually, descending neurons are thought to function either as command-type neurons, in which a single type of descending neuron exerts a high-level control to elicit a coordinated change in motor output, or through a population coding mechanism, whereby a group of neurons, each with local effects, act in combination to elicit a global motor response. The Drosophila Moonwalker Descending Neurons (MDNs), which alter leg motor circuit dynamics so that the fly walks backwards, exemplify the command-type mechanism. Here, we identify several dozen MDN target neurons within the leg motor circuits, and show that two of them mediate distinct and highly-specific changes in leg muscle activity during backward walking: LBL40 neurons provide the hindleg power stroke during stance phase; LUL130 neurons lift the legs at the end of stance to initiate swing. Through these two effector neurons, MDN directly controls both the stance and swing phases of the backward stepping cycle. These findings suggest that command-type descending neurons can also operate through the distributed control of local motor circuits.

Decentralized control of insect walking: A simple neural network explains a wide range of behavioral and neurophysiological results

Controlling the six legs of an insect walking in an unpredictable environment is a challenging task, as many degrees of freedom have to be coordinated. Solutions proposed to deal with this task are usually based on the highly influential concept that (sensory-modulated) central pattern generators (CPG) are required to control the rhythmic movements of walking legs. Here, we investigate a different view. To this end, we introduce a sensor based controller operating on artificial neurons, being applied to a (simulated) insectoid robot required to exploit the "loop through the world" allowing for simplification of neural computation. We show that such a decentralized solution leads to adaptive behavior when facing uncertain environments which we demonstrate for a broad range of behaviors never dealt with in a single system by earlier approaches. This includes the ability to produce footfall patterns such as velocity dependent "tripod", "tetrapod", "pentapod" as well as various stable intermediate patterns as observed in stick insects and in Drosophila. These patterns are found to be stable against disturbances and when starting from various leg configurations. Our neuronal architecture easily allows for starting or interrupting a walk, all being difficult for CPG controlled solutions. Furthermore, negotiation of curves and walking on a treadmill with various treatments of individual legs is possible as well as backward walking and performing short steps. This approach can as well account for the neurophysiological results usually interpreted to support the idea that CPGs form the basis of walking, although our approach is not relying on explicit CPG-like structures. Application of CPGs may however be required for very fast walking. Our neuronal structure allows to pinpoint specific neurons known from various insect studies. Interestingly, specific common properties observed in both insects and crustaceans suggest a significance of our controller beyond the realm of insects.

Integrative Biomimetics of Autonomous Hexapedal Locomotion

Despite substantial advances in many different fields of neurorobotics in general, and biomimetic robots in particular, a key challenge is the integration of concepts: to collate and combine research on disparate and conceptually disjunct research areas in the neurosciences and engineering sciences. We claim that the development of suitable robotic integration platforms is of particular relevance to make such integration of concepts work in practice. Here, we provide an example for a hexapod robotic integration platform for autonomous locomotion. In a sequence of six focus sections dealing with aspects of intelligent, embodied motor control in insects and multipedal robots—ranging from compliant actuation, distributed proprioception and control of multiple legs, the formation of internal representations to the use of an internal body model—we introduce the walking robot HECTOR as a research platform for integrative biomimetics of hexapedal locomotion. Owing to its 18 highly sensorized, compliant actuators, light-weight exoskeleton, distributed and expandable hardware architecture, and an appropriate dynamic simulation framework, HECTOR offers many opportunities to integrate research effort across biomimetics research on actuation, sensory-motor feedback, inter-leg coordination, and cognitive abilities such as motion planning and learning of its own body size.

Analytically founded yaw control algorithm for walking on uneven terrain applied to a hexapod robot

This article presents a new control algorithm for the omnidirectional motion of a legged robot on uneven terrain based on an analytical kinematic solution without the use of Jacobians. In order to control the robot easily and efficiently in all situations, a simplified circle-based workspace approximation has been introduced. Foot trajectories for legged robot movement were generated on concentric circular paths around an analytically computed common centre of motion. This systematic motion model, together with new gait control variables that can be changed during legged robot motion, enabled the implementation of a new adaptive gait phase control algorithm, as well as the addition of algorithms for ground-level adaptation, 3-dimensional map-based step adjustment and fusion of all corrections to establish and/or maintain foot contact with the ground. The method being applicable to different legged robot designs was performed and tested on the laboratory prototype of a hexagonal hexapod robot, and the results of the experiments showed the practical value of the proposed adaptive yaw control method (available also as a video supplement).

A kinematic model of stick-insect walking

Animal, and insect walking (locomotion) in particular, have attracted much attention from scientists over many years up to now. The investigations included behavioral, electrophysiological experiments, as well as modeling studies. Despite the large amount of material collected, there are left many unanswered questions as to how walking and related activities are generated, maintained, and controlled. It is obvious that for them to take place, precise coordination within muscle groups of one leg and between the legs is required: intra- and interleg coordination. The nature, the details, and the interactions of these coordination mechanisms are not entirely clear. To help uncover them, we made use of modeling techniques, and succeeded in developing a six-leg model of stick-insect walking. Our main goal was to prove that the same model can mimic a variety of walking-related behavioral modes, as well as the most common coordination patterns of walking just by changing the values of a few input or internal variables. As a result, the model can reproduce the basic coordination patterns of walking: tetrapod and tripod and the transition between them. It can also mimic stop and restart, change from forward-to-backward walking and back. Finally, it can exhibit so-called search movements of the front legs both while walking or standing still. The mechanisms of the model that enable it to produce the aforementioned behavioral modes can hint at and prove helpful in uncovering further details of the biological mechanisms underlying walking.

Static stability predicts the continuum of interleg coordination patterns in Drosophila

During walking, insects must coordinate the movements of their six legs for efficient locomotion. This interleg coordination is speed dependent: fast walking in insects is associated with tripod coordination patterns, whereas slow walking is associated with more variable, tetrapod-like patterns. To date, however, there has been no comprehensive explanation as to why these speed-dependent shifts in interleg coordination should occur in insects. Tripod coordination would be sufficient at low walking speeds. The fact that insects use a different interleg coordination pattern at lower speeds suggests that it is more optimal or advantageous at these speeds. Furthermore, previous studies focused on discrete tripod and tetrapod coordination patterns. Experimental data, however, suggest that changes observed in interleg coordination are part of a speed-dependent spectrum. Here, we explore these issues in relation to static stability as an important aspect for interleg coordination in Drosophila We created a model that uses basic experimentally measured parameters in fruit flies to find the interleg phase relationships that maximize stability for a given walking speed. The model predicted a continuum of interleg coordination patterns spanning the complete range of walking speeds as well as an anteriorly directed swing phase progression. Furthermore, for low walking speeds, the model predicted tetrapod-like patterns to be most stable, whereas at high walking speeds, tripod coordination emerged as most optimal. Finally, we validated the basic assumption of a continuum of interleg coordination patterns in a large set of experimental data from walking fruit flies and compared these data with the model-based predictions.

A load-based mechanism for inter-leg coordination in insects

Animals rely on an adaptive coordination of legs during walking. However, which specific mechanisms underlie coordination during natural locomotion remains largely unknown. One hypothesis is that legs can be coordinated mechanically based on a transfer of body load from one leg to another. To test this hypothesis, we simultaneously recorded leg kinematics, ground reaction forces and muscle activity in freely walking stick insects (Carausius morosus). Based on torque calculations, we show that load sensors (campaniform sensilla) at the proximal leg joints are well suited to encode the unloading of the leg in individual steps. The unloading coincides with a switch from stance to swing muscle activity, consistent with a load reflex promoting the stance-to-swing transition. Moreover, a mechanical simulation reveals that the unloading can be ascribed to the loading of a specific neighbouring leg, making it exploitable for inter-leg coordination. We propose that mechanically mediated load-based coordination is used across insects analogously to mammals.

The role of phase shifts of sensory inputs in walking revealed by means of phase reduction

Detailed neural network models of animal locomotion are important means to understand the underlying mechanisms that control the coordinated movement of individual limbs. Daun-Gruhn and Tóth, Journal of Computational Neuroscience 31(2), 43-60 (2011) constructed an inter-segmental network model of stick insect locomotion consisting of three interconnected central pattern generators (CPGs) that are associated with the protraction-retraction movements of the front, middle and hind leg. This model could reproduce the basic locomotion coordination patterns, such as tri- and tetrapod, and the transitions between them. However, the analysis of such a system is a formidable task because of its large number of variables and parameters. In this study, we employed phase reduction and averaging theory to this large network model in order to reduce it to a system of coupled phase oscillators. This enabled us to analyze the complex behavior of the system in a reduced parameter space. In this paper, we show that the reduced model reproduces the results of the original model. By analyzing the interaction of just two coupled phase oscillators, we found that the neighboring CPGs could operate within distinct regimes, depending on the phase shift between the sensory inputs from the extremities and the phases of the individual CPGs. We demonstrate that this dependence is essential to produce different coordination patterns and the transition between them. Additionally, applying averaging theory to the system of all three phase oscillators, we calculate the stable fixed points - they correspond to stable tripod or tetrapod coordination patterns and identify two ways of transition between them.

Simple analytical model reveals the functional role of embodied sensorimotor interaction in hexapod gaits

Insects have various gaits with specific characteristics and can change their gaits smoothly in accordance with their speed. These gaits emerge from the embodied sensorimotor interactions that occur between the insect’s neural control and body dynamic systems through sensory feedback. Sensory feedback plays a critical role in coordinated movements such as locomotion, particularly in stick insects. While many previously developed insect models can generate different insect gaits, the functional role of embodied sensorimotor interactions in the interlimb coordination of insects remains unclear because of their complexity. In this study, we propose a simple physical model that is amenable to mathematical analysis to explain the functional role of these interactions clearly. We focus on a foot contact sensory feedback called phase resetting, which regulates leg retraction timing based on touchdown information. First, we used a hexapod robot to determine whether the distributed decoupled oscillators used for legs with the sensory feedback generate insect-like gaits through embodied sensorimotor interactions. The robot generated two different gaits and one had similar characteristics to insect gaits. Next, we proposed the simple model as a minimal model that allowed us to analyze and explain the gait mechanism through the embodied sensorimotor interactions. The simple model consists of a rigid body with massless springs acting as legs, where the legs are controlled using oscillator phases with phase resetting, and the governed equations are reduced such that they can be explained using only the oscillator phases with some approximations. This simplicity leads to analytical solutions for the hexapod gaits via perturbation analysis, despite the complexity of the embodied sensorimotor interactions. This is the first study to provide an analytical model for insect gaits under these interaction conditions. Our results clarified how this specific foot contact sensory feedback contributes to generation of insect-like ipsilateral interlimb coordination during hexapod locomotion.

A Synthetic Nervous System Controls a Simulated Cockroach

The purpose of this work is to better understand how animals control locomotion. This knowledge can then be applied to neuromechanical design to produce more capable and adaptable robot locomotion. To test hypotheses about animal motor control, we model animals and their nervous systems with dynamical simulations, which we call synthetic nervous systems (SNS). However, one major challenge is picking parameter values that produce the intended dynamics. This paper presents a design process that solves this problem without the need for global optimization. We test this method by selecting parameter values for SimRoach2, a dynamical model of a cockroach. Each leg joint is actuated by an antagonistic pair of Hill muscles. A distributed SNS was designed based on pathways known to exist in insects, as well as hypothetical pathways that produced insect-like motion. Each joint’s controller was designed to function as a proportional-integral (PI) feedback loop and tuned with numerical optimization. Once tuned, SimRoach2 walks through a simulated environment, with several cockroach-like features. A model with such reliable low-level performance is necessary to investigate more sophisticated locomotion patterns in the future.

Six-legged walking in insects: how CPGs, peripheral feedback, and descending signals generate coordinated and adaptive motor rhythms

Walking is a rhythmic locomotor behavior of legged animals, and its underlying mechanisms have been the subject of neurobiological research for more than 100 years. In this article, we review relevant historical aspects and contemporary studies in this field of research with a particular focus on the role of central pattern generating networks (CPGs) and their contribution to the generation of six-legged walking in insects. Aspects of importance are the generation of single-leg stepping, the generation of interleg coordination, and how descending signals influence walking. We first review how CPGs interact with sensory signals from the leg in the generation of leg stepping. Next, we summarize how these interactions are modified in the generation of motor flexibility for forward and backward walking, curve walking, and speed changes. We then review the present state of knowledge with regard to the role of CPGs in intersegmental coordination and how CPGs might be involved in mediating descending influences from the brain for the initiation, maintenance, modification, and cessation of the motor output for walking. Throughout, we aim to specifically address gaps in knowledge, and we describe potential future avenues and approaches, conceptual and methodological, with the latter emphasizing in particular options arising from the advent of neurogenetic approaches to this field of research and its combination with traditional approaches.

Adaptive Control Strategies for Interlimb Coordination in Legged Robots: A Review

Walking animals produce adaptive interlimb coordination during locomotion in accordance with their situation. Interlimb coordination is generated through the dynamic interactions of the neural system, the musculoskeletal system, and the environment, although the underlying mechanisms remain unclear. Recently, investigations of the adaptation mechanisms of living beings have attracted attention, and bio-inspired control systems based on neurophysiological findings regarding sensorimotor interactions are being developed for legged robots. In this review, we introduce adaptive interlimb coordination for legged robots induced by various factors (locomotion speed, environmental situation, body properties, and task). In addition, we show characteristic properties of adaptive interlimb coordination, such as gait hysteresis and different time-scale adaptations. We also discuss the underlying mechanisms and control strategies to achieve adaptive interlimb coordination and the design principle for the control system of legged robots.

Intra- and intersegmental influences among central pattern generating networks in the walking system of the stick insect

To efficiently move around, animals need to coordinate their limbs. Proper, context-dependent coupling among the neural networks underlying leg movement is necessary for generating intersegmental coordination. In the slow-walking stick insect, local sensory information is very important for shaping coordination. However, central coupling mechanisms among segmental central pattern generators (CPGs) may also contribute to this. Here, we analyzed the interactions between contralateral networks that drive the depressor trochanteris muscle of the legs in both isolated and interconnected deafferented thoracic ganglia of the stick insect on application of pilocarpine, a muscarinic acetylcholine receptor agonist. Our results show that depressor CPG activity is only weakly coupled between all segments. Intrasegmental phase relationships differ between the three isolated ganglia, and they are modified and stabilized when ganglia are interconnected. However, the coordination patterns that emerge do not resemble those observed during walking. Our findings are in line with recent studies and highlight the influence of sensory input on coordination in slowly walking insects. Finally, as a direct interaction between depressor CPG networks and contralateral motoneurons could not be observed, we hypothesize that coupling is based on interactions at the level of CPG interneurons.NEW & NOTEWORTHY Maintaining functional interleg coordination is vitally important as animals locomote through changing environments. The relative importance of central mechanisms vs. sensory feedback in this process is not well understood. We analyzed coordination among the neural networks generating leg movements in stick insect preparations lacking phasic sensory feedback. Under these conditions, the networks governing different legs were only weakly coupled. In stick insect, central connections alone are thus insufficient to produce the leg coordination observed behaviorally.

A Minimal Model Describing Hexapedal Interlimb Coordination: The Tegotae-Based Approach

Insects exhibit adaptive and versatile locomotion despite their minimal neural computing. Such locomotor patterns are generated via coordination between leg movements, i.e., an interlimb coordination, which is largely controlled in a distributed manner by neural circuits located in thoracic ganglia. However, the mechanism responsible for the interlimb coordination still remains elusive. Understanding this mechanism will help us to elucidate the fundamental control principle of animals' agile locomotion and to realize robots with legs that are truly adaptive and could not be developed solely by conventional control theories. This study aims at providing a “minimal" model of the interlimb coordination mechanism underlying hexapedal locomotion, in the hope that a single control principle could satisfactorily reproduce various aspects of insect locomotion. To this end, we introduce a novel concept we named “Tegotae,” a Japanese concept describing the extent to which a perceived reaction matches an expectation. By using the Tegotae-based approach, we show that a surprisingly systematic design of local sensory feedback mechanisms essential for the interlimb coordination can be realized. We also use a hexapod robot we developed to show that our mathematical model of the interlimb coordination mechanism satisfactorily reproduces various insects' gait patterns.

Predictive control of intersegmental tarsal movements in an insect

In many animals intersegmental reflexes are important for postural and movement control but are still poorly undesrtood. Mathematical methods can be used to model the responses to stimulation, and thus go beyond a simple description of responses to specific inputs. Here we analyse an intersegmental reflex of the foot (tarsus) of the locust hind leg, which raises the tarsus when the tibia is flexed and depresses it when the tibia is extended. A novel method is described to measure and quantify the intersegmental responses of the tarsus to a stimulus to the femoro-tibial chordotonal organ. An Artificial Neural Network, the Time Delay Neural Network, was applied to understand the properties and dynamics of the reflex responses. The aim of this study was twofold: first to develop an accurate method to record and analyse the movement of an appendage and second, to apply methods to model the responses using Artificial Neural Networks. The results show that Artificial Neural Networks provide accurate predictions of tarsal movement when trained with an average reflex response to Gaussian White Noise stimulation compared to linear models. Furthermore, the Artificial Neural Network model can predict the individual responses of each animal and responses to others inputs such as a sinusoid. A detailed understanding of such a reflex response could be included in the design of orthoses or functional electrical stimulation treatments to improve walking in patients with neurological disorders as well as the bio/inspired design of robots.

Effects of functional decoupling of a leg in a model of stick insect walking incorporating three ipsilateral legs

Legged locomotion is a fundamental form of activity of insects during which the legs perform coordinated movements. Sensory signals conveying position, velocity and load of a leg are sent between the thoracic ganglia where the local control networks of the leg muscles are situated. They affect the actual state of the local control networks, hence the stepping of the legs. Sensory coordination in stepping has been intensively studied but important details of its neuronal mechanisms are still unclear. One possibility to tackle this problem is to study what happens to the coordination if a leg is, reversibly or irreversibly, deprived of its normal function. There are numerous behavioral studies on this topic but they could not fully uncover the underlying neuronal mechanisms. Another promising approach to make further progress here can be the use of appropriate models that help elucidate those coordinating mechanisms. We constructed a model of three ipsilateral legs of a stick insect that can mimic coordinated stepping of these legs. We used this model to investigate the possible effects of decoupling a leg. We found that decoupling of the front or the hind leg did not disrupt the coordinated walking of the two remaining legs. However, decoupling of the middle leg yielded mixed results. Both disruption and continuation of coordinated stepping of the front and hind leg occurred. These results agree with the majority of corresponding experimental findings. The model suggests a number of possible mechanisms of decoupling that might bring about the changes.

ReaCog, a Minimal Cognitive Controller Based on Recruitment of Reactive Systems

It has often been stated that for a neuronal system to become a cognitive one, it has to be large enough. In contrast, we argue that a basic property of a cognitive system, namely the ability to plan ahead, can already be fulfilled by small neuronal systems. As a proof of concept, we propose an artificial neural network, termed reaCog, that, first, is able to deal with a specific domain of behavior (six-legged-walking). Second, we show how a minor expansion of this system enables the system to plan ahead and deploy existing behavioral elements in novel contexts in order to solve current problems. To this end, the system invents new solutions that are not possible for the reactive network. Rather these solutions result from new combinations of given memory elements. This faculty does not rely on a dedicated system being more or less independent of the reactive basis, but results from exploitation of the reactive basis by recruiting the lower-level control structures in a way that motor planning becomes possible as an internal simulation relying on internal representation being grounded in embodied experiences.

Joint torques in a freely walking insect reveal distinct functions of leg joints in propulsion and posture control

Determining the mechanical output of limb joints is critical for understanding the control of complex motor behaviours such as walking. In the case of insect walking, the neural infrastructure for single-joint control is well described. However, a detailed description of the motor output in form of time-varying joint torques is lacking. Here, we determine joint torques in the stick insect to identify leg joint function in the control of body height and propulsion. Torques were determined by measuring whole-body kinematics and ground reaction forces in freely walking animals. We demonstrate that despite strong differences in morphology and posture, stick insects show a functional division of joints similar to other insect model systems. Propulsion was generated by strong depression torques about the coxa-trochanter joint, not by retraction or flexion/extension torques. Torques about the respective thorax-coxa and femur-tibia joints were often directed opposite to fore-aft forces and joint movements. This suggests a posture-dependent mechanism that counteracts collapse of the leg under body load and directs the resultant force vector such that strong depression torques can control both body height and propulsion. Our findings parallel propulsive mechanisms described in other walking, jumping and flying insects, and challenge current control models of insect walking.

Endogenous rhythm and pattern-generating circuit interactions in cockroach motor centres

Cockroaches are rapid and stable runners whose gaits emerge from the intricate, and not fully resolved, interplay between endogenous oscillatory pattern-generating networks and sensory feedback that shapes their rhythmic output. Here we studied the endogenous motor output of a brainless, deafferented preparation. We monitored the pilocarpine-induced rhythmic activity of levator and depressor motor neurons in the mesothoracic and metathoracic segments in order to reveal the oscillatory networks’ architecture and interactions. Data analyses included phase relations, latencies between and overlaps of rhythmic bursts, spike frequencies, and the dependence of these parameters on cycle frequency. We found that, overall, ipsilateral connections are stronger than contralateral ones. Our findings revealed asymmetries in connectivity among the different ganglia, in which meta-to-mesothoracic ascending coupling is stronger than meso-to-metathoracic descending coupling. Within-ganglion coupling between the metathoracic hemiganglia is stronger than that in the mesothoracic ganglion. We also report differences in the role and mode of operation of homologue network units (manifested by levator and depressor nerve activity). Many observed characteristics are similar to those exhibited by intact animals, suggesting a dominant role for feedforward control in cockroach locomotion. Based on these data we posit a connectivity scheme among components of the locomotion pattern generating system.

Summary: Detailed analysis of fictive motor patterns unveils endogenous characteristics of the cockroach thoracic locomotion control networks and their interrelations and enables an explanatory connectivity model.

Modular Reactive Neurocontrol for Biologically Inspired Walking Machines

A neurocontroller is described which generates the basic locomotion and controls the sensor-driven behavior of a four-legged and a six-legged walking machine. The controller utilizes discrete-time neurodynamics, and is of modular structure. One module is for processing sensor signals, one is a neural oscillator network serving as a central pattern generator, and the third one is a so-called velocity regulating network. These modules are small and their structures and their functionalities are analyzable. In combination, they enable the machines to autonomously explore an unknown environment, to avoid obstacles, and to escape from corners or deadlock situations. The neurocontroller was developed and tested first using a physical simulation environment, and then it was successfully transferred to the physical walking machines. Locomotion is based on a gait where the diagonal legs are paired and move together, e.g. trot gait for the four-legged walking machine and tripod gait for the six-legged walking machine. The controller developed is universal in the sense that it can easily be adapted to different types of even-legged walking machines without changing the internal structure and its parameters.

Force Sensors in Hexapod Locomotion

Biological systems can provide useful insights into principles of design and control of locomotion that can be applied to legged robots. In this paper we review our work on cockroaches using finite element analysis to model how loads are sensed and regulated in walking and climbing. A number of biological studies have shown that sensors that detect forces in the legs of insects are of particular importance in controlling walking and adapting locomotion to non-horizontal terrains. Our analysis strongly suggests that (i) the system can detect specific force vectors (body load versus propulsion) via sensors located in the leg in positions close to the body and (ii) the system uses this information in positive load feedback to regulate walking movements. These principles and design elements provide examples that can be applied in legged locomotion in walking machines.

Speed-dependent interplay between local pattern-generating activity and sensory signals during walking in Drosophila

In insects, the coordinated motor output required for walking is based on the interaction between local pattern-generating networks providing basic rhythmicity and leg sensory signals which modulate this output on a cycle-to-cycle basis. How this interplay changes speed-dependently and thereby gives rise to the different coordination patterns observed at different speeds is understood insufficiently. Here, we used amputation to reduce sensory signals in single legs and decouple them mechanically during walking in Drosophila. This allowed for the dissociation between locally-generated motor output in the stump and coordinating influences from intact legs. Leg stumps were still rhythmically active during walking. While the oscillatory frequency in intact legs was dependent on walking speed, stumps showed a high and relatively constant oscillation frequency at all walking speeds. At low walking speeds we found no strict cycle-to-cycle coupling between stumps and intact legs. In contrast, at high walking speeds stump oscillations were strongly coupled to the movement of intact legs on a 1-to-1 basis. While during slow walking there was no preferred phase between stumps and intact legs, we nevertheless found a preferred time interval between touch-down or lift-off events in intact legs and levation or depression of stumps. Based on these findings, we hypothesize that, as in other insects, walking speed in Drosophila is predominantly controlled by indirect mechanisms and that direct modulation of basic pattern-generating circuits plays a subsidiary role. Furthermore, inter-leg coordination strength seems to be speed-dependent and greater coordination is evident at higher walking speeds.

A three-leg model producing tetrapod and tripod coordination patterns of ipsilateral legs in the stick insect

Insect locomotion requires the precise coordination of the movement of all six legs. Detailed investigations have revealed that the movement of the legs is controlled by local dedicated neuronal networks, which interact to produce walking of the animal. The stick insect is well suited to experimental investigations aimed at understanding the mechanisms of insect locomotion. Beside the experimental approach, models have also been constructed to elucidate those mechanisms. Here, we describe a model that replicates both the tetrapod and tripod coordination pattern of three ipsilateral legs. The model is based on an earlier insect leg model, which includes the three main leg joints, three antagonistic muscle pairs, and their local neuronal control networks. These networks are coupled via angular signals to establish intraleg coordination of the three neuromuscular systems during locomotion. In the present three-leg model, we coupled three such leg models, representing front, middle, and hind leg, in this way. The coupling was between the levator-depressor local control networks of the three legs. The model could successfully simulate tetrapod and tripod coordination patterns, as well as the transition between them. The simulations showed that for the interleg coordination during tripod, the position signals of the levator-depressor neuromuscular systems sent between the legs were sufficient, while in tetrapod, additional information on the angular velocities in the same system was necessary, and together with the position information also sufficient. We therefore suggest that, during stepping, the connections between the levator-depressor neuromuscular systems of the different legs are of primary importance.

A Computational Model of a Descending Mechanosensory Pathway Involved in Active Tactile Sensing

Many animals, including humans, rely on active tactile sensing to explore the environment and negotiate obstacles, especially in the dark. Here, we model a descending neural pathway that mediates short-latency proprioceptive information from a tactile sensor on the head to thoracic neural networks. We studied the nocturnal stick insect Carausius morosus, a model organism for the study of adaptive locomotion, including tactually mediated reaching movements. Like mammals, insects need to move their tactile sensors for probing the environment. Cues about sensor position and motion are therefore crucial for the spatial localization of tactile contacts and the coordination of fast, adaptive motor responses. Our model explains how proprioceptive information about motion and position of the antennae, the main tactile sensors in insects, can be encoded by a single type of mechanosensory afferents. Moreover, it explains how this information is integrated and mediated to thoracic neural networks by a diverse population of descending interneurons (DINs). First, we quantified responses of a DIN population to changes in antennal position, motion and direction of movement. Using principal component (PC) analysis, we find that only two PCs account for a large fraction of the variance in the DIN response properties. We call the two-dimensional space spanned by these PCs ‘coding-space’ because it captures essential features of the entire DIN population. Second, we model the mechanoreceptive input elements of this descending pathway, a population of proprioceptive mechanosensory hairs monitoring deflection of the antennal joints. Finally, we propose a computational framework that can model the response properties of all important DIN types, using the hair field model as its only input. This DIN model is validated by comparison of tuning characteristics, and by mapping the modelled neurons into the two-dimensional coding-space of the real DIN population. This reveals the versatility of the framework for modelling a complete descending neural pathway.

Many nocturnal and burrowing animals rely on their tactile sense to explore the surrounding space, and tactile cues are often used to adapt locomotion to a structurally complex environment. Most mammals use facial whiskers for active tactile exploration, while most insects use their antennae. Since whiskers and antennae are long, thin, cylindrical structures, they must be moved to probe the surrounding space. The nervous system therefore has to keep track of tactile sensor movement by encoding sensor position and motion in order to locate tactile contacts. Here, we model a descending neural pathway of the stick insect, which transfers information about tactile sensor movement to thoracic neural networks with short latency. We show that information about sensor position and motion can be derived from a single class of proprioceptors at the antennal joints, and present a computational model that explains the activity of four previously described groups of descending interneurons during antennal stimulation. Our model is validated against electrophysiological data on antennal mechanoreceptors and descending interneurons.

Controlling legs for locomotion-insights from robotics and neurobiology

Walking is the most common terrestrial form of locomotion in animals. Its great versatility and flexibility has led to many attempts at building walking machines with similar capabilities. The control of walking is an active research area both in neurobiology and robotics, with a large and growing body of work. This paper gives an overview of the current knowledge on the control of legged locomotion in animals and machines and attempts to give walking control researchers from biology and robotics an overview of the current knowledge in both fields. We try to summarize the knowledge on the neurobiological basis of walking control in animals, emphasizing common principles seen in different species. In a section on walking robots, we review common approaches to walking controller design with a slight emphasis on biped walking control. We show where parallels between robotic and neurobiological walking controllers exist and how robotics and biology may benefit from each other. Finally, we discuss where research in the two fields diverges and suggest ways to bridge these gaps.

A network model comprising 4 segmental, interconnected ganglia, and its application to simulate multi-legged locomotion in crustaceans

Inter-segmental coordination is crucial for the locomotion of animals. Arthropods show high variability of leg numbers, from 6 in insects up to 750 legs in millipedes. Despite this fact, the anatomical and functional organization of their nervous systems show basic similarities. The main similarities are the segmental organization, and the way the function of the segmental units is coordinated. We set out to construct a model that could describe locomotion (walking) in animals with more than 6 legs, as well as in 6-legged animals (insects). To this end, we extended a network model by Daun-Gruhn and Tóth (Journal of Computational Neuroscience, doi: 10.1007/s10827-010-0300-1 , 2011). This model describes inter-segmental coordination of the ipsilateral legs in the stick insect during walking. Including an additional segment (local network) into the original model, we could simulate coordination patterns that occur in animals walking on eight legs (e.g., crayfish). We could improve the model by modifying its original cyclic connection topology. In all model variants, the phase relations between the afferent segmental excitatory sensory signals and the oscillatory activity of the segmental networks played a crucial role. Our results stress the importance of this sensory input on the generation of different stable coordination patterns. The simulations confirmed that using the modified connection topology, the flexibility of the model behaviour increased, meaning that changing a single phase parameter, i.e., gating properties of just one afferent sensory signal was sufficient to reproduce all coordination patterns seen in the experiments.

Intersegmental coupling and recovery from perturbations in freely running cockroaches

Cockroaches are remarkably stable runners, exhibiting rapid recovery from external perturbations. To uncover the mechanisms behind this important behavioral trait, we recorded leg kinematics of freely running animals in both undisturbed and perturbed trials. Functional coupling underlying inter-leg coordination was monitored before and during localized perturbations, which were applied to single legs via magnetic impulses. The resulting transient effects on all legs and the recovery times to normal pre-perturbation kinematics were studied. We estimated coupling architecture and strength by fitting experimental data to a six-leg-unit phase oscillator model. Using maximum-likelihood techniques, we found that a network with nearest-neighbor inter-leg coupling best fitted the data and that, although coupling strengths vary among preparations, the overall inputs entering each leg are approximately balanced and consistent. Simulations of models with different coupling strengths encountering perturbations suggest that the coupling schemes estimated from our experiments allow animals relatively fast and uniform recoveries from perturbations.

Comparative whole-body kinematics of closely related insect species with different body morphology

Legged locomotion through natural environments is very complex and variable. For example, leg kinematics may differ strongly between species, but even within the same species it is adaptive and context-dependent. Inter-species differences in locomotion are often difficult to interpret, because both morphological and ecological differences among species may be strong and, as a consequence, confound each other's effects. In order to understand better how body morphology affects legged locomotion, we compare unrestrained whole-body kinematics of three stick insect species with different body proportions, but similar feeding ecology: Carausius morosus, Aretaon asperrimus and Medauroidea extradentata (=Cuniculina impigra). In order to co-vary locomotory context, we introduced a gradually increasing demand for climbing by varying the height of stairs in the setup. The species were similar in many aspects, for example in using distinct classes of steps, with minor differences concerning the spread of corrective short steps. Major differences were related to antenna length, segment lengths of thorax and head, and the ratio of leg length to body length. Whereas all species continuously moved their antennae, only Medauroidea executed high swing movements with its front legs to search for obstacles in the near-range environment. Although all species adjusted their body inclination, the range in which body segments moved differed considerably, with longer thorax segments tending to be moved more. Finally, leg posture, time courses of leg joint angles and intra-leg coordination differed most strongly in long-legged Medauroidea.

Kinematic responses to changes in walking orientation and gravitational load in Drosophila melanogaster

Walking behavior is context-dependent, resulting from the integration of internal and external influences by specialized motor and pre-motor centers. Neuronal programs must be sufficiently flexible to the locomotive challenges inherent in different environments. Although insect studies have contributed substantially to the identification of the components and rules that determine locomotion, we still lack an understanding of how multi-jointed walking insects respond to changes in walking orientation and direction and strength of the gravitational force. In order to answer these questions we measured with high temporal and spatial resolution the kinematic properties of untethered Drosophila during inverted and vertical walking. In addition, we also examined the kinematic responses to increases in gravitational load. We find that animals are capable of shifting their step, spatial and inter-leg parameters in order to cope with more challenging walking conditions. For example, flies walking in an inverted orientation decreased the duration of their swing phase leading to increased contact with the substrate and, as a result, greater stability. We also find that when flies carry additional weight, thereby increasing their gravitational load, some changes in step parameters vary over time, providing evidence for adaptation. However, above a threshold that is between 1 and 2 times their body weight flies display locomotion parameters that suggest they are no longer capable of walking in a coordinated manner. Finally, we find that functional chordotonal organs are required for flies to cope with additional weight, as animals deficient in these proprioceptors display increased sensitivity to load bearing as well as other locomotive defects.

Insects use two distinct classes of steps during unrestrained locomotion

Background

Adaptive, context-dependent control of locomotion requires modulation of centrally generated rhythmic motor patterns through peripheral control loops and postural reflexes. Thus assuming that the modulation of rhythmic motor patterns accounts for much of the behavioural variability observed in legged locomotion, investigating behavioural variability is a key to the understanding of context-dependent control mechanisms in locomotion. To date, the variability of unrestrained locomotion is poorly understood, and virtually nothing is known about the features that characterise the natural statistics of legged locomotion. In this study, we quantify the natural variability of hexapedal walking and climbing in insects, drawing from a database of several thousand steps recorded over two hours of walking time.

Results

We show that the range of step length used by unrestrained climbing stick insects is large, showing that step length can be changed substantially for adaptive locomotion. Step length distributions were always bimodal, irrespective of leg type and walking condition, suggesting the presence of two distinct classes of steps: short and long steps. Probability density of step length was well-described by a gamma distribution for short steps, and a logistic distribution for long steps. Major coefficients of these distributions remained largely unaffected by walking conditions. Short and long steps differed concerning their spatial occurrence on the walking substrate, their timing within the step sequence, and their prevalent swing direction. Finally, ablation of structures that serve to improve foothold increased the ratio of short to long steps, indicating a corrective function of short steps.

Conclusions

Statistical and functional differences suggest that short and long steps are physiologically distinct classes of leg movements that likely reflect distinct control mechanisms at work.

A bioinspired autonomous swimming robot as a tool for studying goal-directed locomotion

The bioinspired approach has been key in combining the disciplines of robotics with neuroscience in an effective and promising fashion. Indeed, certain aspects in the field of neuroscience, such as goal-directed locomotion and behaviour selection, can be validated through robotic artefacts. In particular, swimming is a functionally important behaviour where neuromuscular structures, neural control architecture and operation can be replicated artificially following models from biology and neuroscience. In this article, we present a biomimetic system inspired by the lamprey, an early vertebrate that locomotes using anguilliform swimming. The artefact possesses extra- and proprioceptive sensory receptors, muscle-like actuation, distributed embedded control and a vision system. Experiments on optimised swimming and on goal-directed locomotion are reported, as well as the assessment of the performance of the system, which shows high energy efficiency and adaptive behaviour. While the focus is on providing a robotic platform for testing biological models, the reported system can also be of major relevance for the development of engineering system applications.

A hexapod walker using a heterarchical architecture for action selection

Moving in a cluttered environment with a six-legged walking machine that has additional body actuators, therefore controlling 22 DoFs, is not a trivial task. Already simple forward walking on a flat plane requires the system to select between different internal states. The orchestration of these states depends on walking velocity and on external disturbances. Such disturbances occur continuously, for example due to irregular up-and-down movements of the body or slipping of the legs, even on flat surfaces, in particular when negotiating tight curves. The number of possible states is further increased when the system is allowed to walk backward or when front legs are used as grippers and cannot contribute to walking. Further states are necessary for expansion that allow for navigation. Here we demonstrate a solution for the selection and sequencing of different (attractor) states required to control different behaviors as are forward walking at different speeds, backward walking, as well as negotiation of tight curves. This selection is made by a recurrent neural network (RNN) of motivation units, controlling a bank of decentralized memory elements in combination with the feedback through the environment. The underlying heterarchical architecture of the network allows to select various combinations of these elements. This modular approach representing an example of neural reuse of a limited number of procedures allows for adaptation to different internal and external conditions. A way is sketched as to how this approach may be expanded to form a cognitive system being able to plan ahead. This architecture is characterized by different types of modules being arranged in layers and columns, but the complete network can also be considered as a holistic system showing emergent properties which cannot be attributed to a specific module.

Walknet, a bio-inspired controller for hexapod walking

Walknet comprises an artificial neural network that allows for the simulation of a considerable amount of behavioral data obtained from walking and standing stick insects. It has been tested by kinematic and dynamic simulations as well as on a number of six-legged robots. Over the years, various different expansions of this network have been provided leading to different versions of Walknet. This review summarizes the most important biological findings described by Walknet and how they can be simulated. Walknet shows how a number of properties observed in insects may emerge from a decentralized architecture. Examples are the continuum of so-called "gaits," coordination of up to 18 leg joints during stance when walking forward or backward over uneven surfaces and negotiation of curves, dealing with leg loss, as well as being able following motion trajectories without explicit precalculation. The different Walknet versions are compared to other approaches describing insect-inspired hexapod walking. Finally, we briefly address the ability of this decentralized reactive controller to form the basis for the simulation of higher-level cognitive faculties exceeding the capabilities of insects.

How and to what end may consciousness contribute to action? Attributing properties of consciousness to an embodied, minimally cognitive artificial neural network

An artificial neural network called reaCog is described which is based on a decentralized, reactive and embodied architecture developed to control non-trivial hexapod walking in an unpredictable environment (Walknet) while using insect-like navigation (Navinet). In reaCog, these basic networks are extended in such a way that the complete system, reaCog, adopts the capability of inventing new behaviors and - via internal simulation - of planning ahead. This cognitive expansion enables the reactive system to be enriched with additional procedures. Here, we focus on the question to what extent properties of phenomena to be characterized on a different level of description as for example consciousness can be found in this minimally cognitive system. Adopting a monist view, we argue that the phenomenal aspect of mental phenomena can be neglected when discussing the function of such a system. Under this condition, reaCog is discussed to be equipped with properties as are bottom-up and top-down attention, intentions, volition, and some aspects of Access Consciousness. These properties have not been explicitly implemented but emerge from the cooperation between the elements of the network. The aspects of Access Consciousness found in reaCog concern the above mentioned ability to plan ahead and to invent and guide (new) actions. Furthermore, global accessibility of memory elements, another aspect characterizing Access Consciousness is realized by this network. reaCog allows for both reactive/automatic control and (access-) conscious control of behavior. We discuss examples for interactions between both the reactive domain and the conscious domain. Metacognition or Reflexive Consciousness is not a property of reaCog. Possible expansions are discussed to allow for further properties of Access Consciousness, verbal report on internal states, and for Metacognition. In summary, we argue that already simple networks allow for properties of consciousness if leaving the phenomenal aspect aside.

A neuromechanical model for the neuronal basis of curve walking in the stick insect

The coordination of the movement of single and multiple limbs is essential for the generation of locomotion. Movement about single joints and the resulting stepping patterns are usually generated by the activity of antagonistic muscle pairs. In the stick insect, the three major muscle pairs of a leg are the protractor and retractor coxae, the levator and depressor trochanteris, and the flexor and extensor tibiae. The protractor and retractor move the coxa, and thereby the leg, forward and backward. The levator and depressor move the femur up and down. The flexor flexes, and the extensor extends the tibia about the femur-tibia joint. The underlying neuronal mechanisms for a forward stepping middle leg have been thoroughly investigated in experimental and theoretical studies. However, the details of the neuronal and mechanical mechanisms driving a stepping single leg in situations other than forward walking remain largely unknown. Here, we present a neuromechanical model of the coupled three joint control system of the stick insect's middle leg. The model can generate forward, backward, or sideward stepping. Switching between them is achieved by changing only a few central signals controlling the neuromechanical model. In kinematic simulations, we are able to generate curve walking with two different mechanisms. In the first, the inner middle leg is switched from forward to sideward and in the second to backward stepping. Both are observed in the behaving animal, and in the model and animal alike, backward stepping of the inner middle leg produces tighter turns than sideward stepping.

What's Next: Recruitment of a Grounded Predictive Body Model for Planning a Robot's Actions

Even comparatively simple, reactive systems are able to control complex motor tasks, such as hexapod walking on unpredictable substrate. The capability of such a controller can be improved by introducing internal models of the body and of parts of the environment. Such internal models can be applied as inverse models, as forward models or to solve the problem of sensor fusion. Usually, separate models are used for these functions. Furthermore, separate models are used to solve different tasks. Here we concentrate on internal models of the body as the brain considers its own body the most important part of the world. The model proposed is formed by a recurrent neural network with the property of pattern completion. The model shows a hierarchical structure but nonetheless comprises a holistic system. One and the same model can be used as a forward model, as an inverse model, for sensor fusion, and, with a simple expansion, as a model to internally simulate (new) behaviors to be used for prediction. The model embraces the geometrical constraints of a complex body with many redundant degrees of freedom, and allows finding geometrically possible solutions. To control behavior such as walking, climbing, or reaching, this body model is complemented by a number of simple reactive procedures together forming a procedural memory. In this article, we illustrate the functioning of this network. To this end we present examples for solutions of the forward function and the inverse function, and explain how the complete network might be used for predictive purposes. The model is assumed to be “innate,” so learning the parameters of the model is not (yet) considered.