Neural control and adaptive neural forward models for insect-like, energy-efficient, and adaptable locomotion of walking machines

Frictional Anisotropic Locomotion and Adaptive Neural Control for a Soft Crawling Robot

Crawling animals with bendable soft bodies use the friction anisotropy of their asymmetric body structures to traverse various substrates efficiently. Although the effect of friction anisotropy has been investigated and applied to robot locomotion, the dynamic interactions between soft body bending at different frequencies (low and high), soft asymmetric surface structures at various aspect ratios (low, medium, and high), and different substrates (rough and smooth) have not been studied comprehensively. To address this lack, we developed a simple soft robot model with a bioinspired asymmetric structure (sawtooth) facing the ground. The robot uses only a single source of pressure for its pneumatic actuation. The frequency, teeth aspect ratio, and substrate parameters and the corresponding dynamic interactions were systematically investigated and analyzed. The study findings indicate that the anterior and posterior parts of the structure deform differently during the interaction, generating different frictional forces. In addition, these parts switched their roles dynamically from push to pull and vice versa in various states, resulting in the robot's emergent locomotion. Finally, autonomous adaptive crawling behavior of the robot was demonstrated using sensor-driven neural control with a miniature laser sensor installed in the anterior part of the robot. The robot successfully adapted its actuation frequency to reduce body bending and crawl through a narrow space, such as a tunnel. The study serves as a stepping stone for developing simple soft crawling robots capable of navigating cluttered and confined spaces autonomously.

Robust and reusable self-organized locomotion of legged robots under adaptive physical and neural communications

Introduction

Animals such as cattle can achieve versatile and elegant behaviors through automatic sensorimotor coordination. Their self-organized movements convey an impression of adaptability, robustness, and motor memory. However, the adaptive mechanisms underlying such natural abilities of these animals have not been completely realized in artificial legged systems.

Methods

Hence, we propose adaptive neural control that can mimic these abilities through adaptive physical and neural communications. The control algorithm consists of distributed local central pattern generator (CPG)-based neural circuits for generating basic leg movements, an adaptive sensory feedback mechanism for generating self-organized phase relationships among the local CPG circuits, and an adaptive neural coupling mechanism for transferring and storing the formed phase relationships (a gait pattern) into the neural structure. The adaptive neural control was evaluated in experiments using a quadruped robot.

Results

The adaptive neural control enabled the robot to 1) rapidly and automatically form its gait (i.e., self-organized locomotion) within a few seconds, 2) memorize the gait for later recovery, and 3) robustly walk, even when a sensory feedback malfunction occurs. It also enabled maneuverability, with the robot being able to change its walking speed and direction. Moreover, implementing adaptive physical and neural communications provided an opportunity for understanding the mechanism of motor memory formation.

Discussion

Overall, this study demonstrates that the integration of the two forms of communications through adaptive neural control is a powerful way to achieve robust and reusable self-organized locomotion in legged robots.

A robotic leg inspired from an insect leg

While most insect-inspired robots come with a simple tarsus, such as a hemispherical foot tip, insect legs have complex tarsal structures and claws, which enable them to walk on complex terrain. Their sharp claws can smoothly attach and detach on plant surfaces by actuating a single muscle. Thus, installing an insect-inspired tarsus on legged robots would improve their locomotion on complex terrain. This paper shows that the tendon-driven ball-socket structure provides the tarsus with both flexibility and rigidity, which is necessary for the beetle to walk on a complex substrate such as a mesh surface. Disabling the tarsus' rigidity by removing the socket and elastic membrane of a tarsal joint, means that the claws could not attach to the mesh securely. Meanwhile, the beetle struggled to draw the claws out of the substrate when we turned the tarsus rigid by tubing. We then developed a cable-driven bio-inspired tarsus structure to validate the function of the tarsus as well as to show its potential application in the legged robot. With the tarsus, the robotic leg was able to attach and retract smoothly from the mesh substrate when performing a walking cycle.

Generic Mechanism for Waveform Regulation and Synchronization of Oscillators: An Application for Robot Behavior Diversity Generation

While nonlinear oscillators have been widely used for central pattern generators to produce basic rhythmic signals for robot locomotion control, methods to shape and regulate the signal waveform without changing the characteristics of the oscillators have not been fully investigated, especially during the network synchronization process. To illustrate the principle and process of waveform regulation of nonlinear oscillators in detail and ensure that the influence can be controlled, we present a method for waveform regulation and synchronization and analyze the relationship of different factors (e.g., initial conditions, network parameters, phase, and waveform regulation factors) in synchronization deviation. Then, the method is indicated to be effective in other commonly used nonlinear oscillators and neural oscillators. As an example application, a three-layer behavioral control architecture for a legged robot is constructed based on the proposed method. Modules for the body behavior, leg coordination, and single-leg adjustment are established to realize diverse robot behaviors. The effectiveness of the method is validated by a series of experiments. The results prove that the method performs well in terms of signal control accuracy, behavior pattern diversity, and smooth motion transition.

Continuous Online Adaptation of Bioinspired Adaptive Neuroendocrine Control for Autonomous Walking Robots

Walking animals can continuously adapt their locomotion to deal with unpredictable changing environments. They can also take proactive steps to avoid colliding with an obstacle. In this study, we aim to realize such features for autonomous walking robots so that they can efficiently traverse complex terrains. To achieve this, we propose novel bioinspired adaptive neuroendocrine control. In contrast to conventional locomotion control methods, this approach does not require robot and environmental models, exteroceptive feedback, or multiple learning trials. It integrates three main modular neural mechanisms, relying only on proprioceptive feedback and short-term memory, namely: 1) neural central pattern generator (CPG)-based control; 2) an artificial hormone network (AHN); and 3) unsupervised input correlation-based learning (ICO). The neural CPG-based control creates insect-like gaits, while the AHN can continuously adapt robot joint movement individually with respect to the terrain during the stance phase using only the torque feedback. In parallel, the ICO generates short-term memory for proactive obstacle negotiation during the swing phase, allowing the posterior legs to step over the obstacle before hitting it. The control approach is evaluated on a bioinspired hexapod robot walking on complex unpredictable terrains (e.g., gravel, grass, and extreme random stepfield). The results show that the robot can successfully perform energy-efficient autonomous locomotion and online continuous adaptation with proactivity to overcome such terrains. Since our adaptive neural control approach does not require a robot model, it is general and can be applied to other bioinspired walking robots to achieve a similar adaptive, autonomous, and versatile function.

Locomotion Control With Frequency and Motor Pattern Adaptations

Existing adaptive locomotion control mechanisms for legged robots are usually aimed at one specific type of adaptation and rarely combined with others. Adaptive mechanisms thus stay at a conceptual level without their coupling effect with other mechanisms being investigated. However, we hypothesize that the combination of adaptation mechanisms can be exploited for enhanced and more efficient locomotion control as in biological systems. Therefore, in this work, we present a central pattern generator (CPG) based locomotion controller integrating both a frequency and motor pattern adaptation mechanisms. We use the state-of-the-art Dual Integral Learner for frequency adaptation, which can automatically and quickly adapt the CPG frequency, enabling the entire motor pattern or output signal of the CPG to be followed at a proper high frequency with low tracking error. Consequently, the legged robot can move with high energy efficiency and perform the generated locomotion with high precision. The versatile state-of-the-art CPG-RBF network is used as a motor pattern adaptation mechanism. Using this network, the motor patterns or joint trajectories can be adapted to fit the robot's morphology and perform sensorimotor integration enabling online motor pattern adaptation based on sensory feedback. The results show that the two adaptation mechanisms can be combined for adaptive locomotion control of a hexapod robot in a complex environment. Using the CPG-RBF network for motor pattern adaptation, the hexapod learned basic straight forward walking, steering, and step climbing. In general, the frequency and motor pattern mechanisms complement each other well and their combination can be seen as an essential step toward further studies on adaptive locomotion control.

Insect-Inspired Robots: Bridging Biological and Artificial Systems

This review article aims to address common research questions in hexapod robotics. How can we build intelligent autonomous hexapod robots that can exploit their biomechanics, morphology, and computational systems, to achieve autonomy, adaptability, and energy efficiency comparable to small living creatures, such as insects? Are insects good models for building such intelligent hexapod robots because they are the only animals with six legs? This review article is divided into three main sections to address these questions, as well as to assist roboticists in identifying relevant and future directions in the field of hexapod robotics over the next decade. After an introduction in section (1), the sections will respectively cover the following three key areas: (2) biomechanics focused on the design of smart legs; (3) locomotion control; and (4) high-level cognition control. These interconnected and interdependent areas are all crucial to improving the level of performance of hexapod robotics in terms of energy efficiency, terrain adaptability, autonomy, and operational range. We will also discuss how the next generation of bioroboticists will be able to transfer knowledge from biology to robotics and vice versa.

Generic Neural Locomotion Control Framework for Legged Robots

In this article, we present a generic locomotion control framework for legged robots and a strategy for control policy optimization. The framework is based on neural control and black-box optimization. The neural control combines a central pattern generator (CPG) and a radial basis function (RBF) network to create a CPG-RBF network. The control network acts as a neural basis to produce arbitrary rhythmic trajectories for the joints of robots. The main features of the CPG-RBF network are: 1) it is generic since it can be applied to legged robots with different morphologies; 2) it has few control parameters, resulting in fast learning; 3) it is scalable, both in terms of policy/trajectory complexity and the number of legs that can be controlled using similar trajectories; 4) it does not rely heavily on sensory feedback to generate locomotion and is thus less prone to sensory faults; and 5) once trained, it is simple, minimal, and intuitive to use and analyze. These features will lead to an easy-to-use framework with fast convergence and the ability to encode complex locomotion control policies. In this work, we show that the framework can successfully be applied to three different simulated legged robots with varying morphologies and, even broken joints, to learn locomotion control policies. We also show that after learning, the control policies can also be successfully transferred to a real-world robot without any modifications. We, furthermore, show the scalability of the framework by implementing it as a central controller for all legs of a robot and as a decentralized controller for individual legs and leg pairs. By investigating the correlation between robot morphology and encoding type, we are able to present a strategy for control policy optimization. Finally, we show how sensory feedback can be integrated into the CPG-RBF network to enable online adaptation.

State-space aerodynamic model reveals high force control authority and predictability in flapping flight

Flying animals resort to fast, large-degree-of-freedom motion of flapping wings, a key feature that distinguishes them from rotary or fixed-winged robotic fliers with limited motion of aerodynamic surfaces. However, flapping-wing aerodynamics are characterized by highly unsteady and three-dimensional flows difficult to model or control, and accurate aerodynamic force predictions often rely on expensive computational or experimental methods. Here, we developed a computationally efficient and data-driven state-space model to dynamically map wing kinematics to aerodynamic forces/moments. This model was trained and tested with a total of 548 different flapping-wing motions and surpassed the accuracy and generality of the existing quasi-steady models. This model used 12 states to capture the unsteady and nonlinear fluid effects pertinent to force generation without explicit information of fluid flows. We also provided a comprehensive assessment of the control authority of key wing kinematic variables and found that instantaneous aerodynamic forces/moments were largely predictable by the wing motion history within a half-stroke cycle. Furthermore, the angle of attack, normal acceleration and pitching motion had the strongest effects on the aerodynamic force/moment generation. Our results show that flapping flight inherently offers high force control authority and predictability, which can be key to developing agile and stable aerial fliers.

Adaptive and Energy-Efficient Optimal Control in CPGs Through Tegotae-Based Feedback

To obtain biologically inspired robotic control, the architecture of central pattern generators (CPGs) has been extensively adopted to generate periodic patterns for locomotor control. This is attributed to the interesting properties of nonlinear oscillators. Although sensory feedback in CPGs is not necessary for the generation of patterns, it plays a central role in guaranteeing adaptivity to environmental conditions. Nonetheless, its inclusion significantly modifies the dynamics of the CPG architecture, which often leads to bifurcations. For instance, the force feedback can be exploited to derive information regarding the state of the system. In particular, the Tegotae approach can be adopted by coupling proprioceptive information with the state of the oscillation itself in the CPG model. This paper discusses this policy with respect to other types of feedback; it provides higher adaptivity and an optimal energy efficiency for reflex-like actuation. We believe this is the first attempt to analyse the optimal energy efficiency along with the adaptivity of the Tegotae approach.

Distributed-force-feedback-based reflex with online learning for adaptive quadruped motor control

Biological motor control mechanisms (e.g., central pattern generators (CPGs), sensory feedback, reflexes, and motor learning) play a crucial role in the adaptive locomotion of animals. However, the interaction and integration of these mechanisms - necessary for generating the efficient, adaptive locomotion responses of legged robots to diverse terrains - have not yet been fully realized. One issue is that of achieving adaptive motor control for fast postural adaptation across various terrains. To address this issue, this study proposes a novel distributed-force-feedback-based reflex with online learning (DFRL). It integrates force-sensory feedback, reflexes, and learning to cooperate with CPGs in producing adaptive motor commands. The DFRL is based on a simple neural network that uses plastic synapses modulated online by a fast dual integral learner. Experimental results on different quadruped robots show that the DFRL can (1) automatically and rapidly adapt the CPG patterns (motor commands) of the robots, enabling them to realize appropriate body postures during locomotion and (2) enable the robots to effectively accommodate themselves to various slope terrains, including steep ones. Consequently, the DFRL-controlled robots can achieve efficient adaptive locomotion, to tackle complex terrains with diverse slopes.

A Comparative Study of Adaptive Interlimb Coordination Mechanisms for Self-Organized Robot Locomotion

Walking animals demonstrate impressive self-organized locomotion and adaptation to body property changes by skillfully manipulating their complicated and redundant musculoskeletal systems. Adaptive interlimb coordination plays a crucial role in this achievement. It has been identified that interlimb coordination is generated through dynamical interactions between the neural system, musculoskeletal system, and environment. Based on this principle, two classical interlimb coordination mechanisms (continuous phase modulation and phase resetting) have been proposed independently. These mechanisms use decoupled central pattern generators (CPGs) with sensory feedback, such as ground reaction forces (GRFs), to generate robot locomotion autonomously without predefining it (i.e., self-organized locomotion). A comparative study was conducted on the two mechanisms under decoupled CPG-based control implemented on a quadruped robot in simulation. Their characteristics were compared by observing their CPG phase convergence processes at different control parameter values. Additionally, the mechanisms were investigated when the robot faced various unexpected situations, such as noisy feedback, leg motor damage, and carrying a load. The comparative study reveals that the phase modulation and resetting mechanisms demonstrate satisfactory performance when they are subjected to symmetric and asymmetric GRF distributions, respectively. This work also suggests a strategy for the appropriate selection of adaptive interlimb coordination mechanisms under different conditions and for the optimal setting of their control parameter values to enhance their control performance.

Sprawling Quadruped Robot Driven by Decentralized Control With Cross-Coupled Sensory Feedback Between Legs and Trunk

Quadruped animals achieve agile and highly adaptive locomotion owing to the coordination between their legs and other body parts, such as the trunk, head, and tail, that is, body–limb coordination. This study aims to understand the sensorimotor control underlying body–limb coordination. To this end, we adopted sprawling locomotion in vertebrate animals as a model behavior. This is a quadruped walking gait with lateral body bending used by many amphibians and lizards. Our previous simulation study demonstrated that cross-coupled sensory feedback between the legs and trunk helps to rapidly establish body–limb coordination and improve locomotion performance. This paper presented an experimental validation of the cross-coupled sensory feedback control using a newly developed quadruped robot. The results show similar tendencies to the simulation study. Sensory feedback provides rapid convergence to stable gait, robustness against leg failure, and morphological changes. Our study suggests that sensory feedback potentially plays an essential role in body–limb coordination and provides a robust, sensory-driven control principle for quadruped robots.

Framework for Developing Bio-Inspired Morphologies for Walking Robots

Morphology is a defining trait of any walking entity, animal or robot, and is crucial in obtaining movement versatility, dexterity and durability. Collaborations between biologist and engineers create opportunities for implementing bio-inspired morphologies in walking robots. However, there is little guidance for such interdisciplinary collaborations and what tools to use. We propose a development framework for transferring animal morphologies to robots and substantiate it with a replication of the ability of the dung beetle species Scarabaeus galenus to use the same morphology for both locomotion and object manipulation. As such, we demonstrate the advantages of a bio-inspired dung beetle-like robot, ALPHA, and how its morphology outperforms a conventional hexapod by increasing the (1) step length by 50.0%, (2) forward and upward reach by 95.5%, and by lowering the (3) overall motor acceleration by 7.9%, and (4) step frequency by 21.1% at the same walking speed. Thereby, the bio-inspired robot has longer and fewer steps that lower fatigue-inducing impulses, a greater variety of step patterns, and can potentially better utilise its workspace to overcome obstacles. Hence, we demonstrate how the framework can be used to develop legged robots with bio-inspired morphologies that embody greater movement versatility, dexterity and durability.

General Distributed Neural Control and Sensory Adaptation for Self-Organized Locomotion and Fast Adaptation to Damage of Walking Robots

Walking animals such as invertebrates can effectively perform self-organized and robust locomotion. They can also quickly adapt their gait to deal with injury or damage. Such a complex achievement is mainly performed via coordination between the legs, commonly known as interlimb coordination. Several components underlying the interlimb coordination process (like distributed neural control circuits, local sensory feedback, and body-environment interactions during movement) have been recently identified and applied to the control systems of walking robots. However, while the sensory pathways of biological systems are plastic and can be continuously readjusted (referred to as sensory adaptation), those implemented on robots are typically static. They first need to be manually adjusted or optimized offline to obtain stable locomotion. In this study, we introduce a fast learning mechanism for online sensory adaptation. It can continuously adjust the strength of sensory pathways, thereby introducing flexible plasticity into the connections between sensory feedback and neural control circuits. We combine the sensory adaptation mechanism with distributed neural control circuits to acquire the adaptive and robust interlimb coordination of walking robots. This novel approach is also general and flexible. It can automatically adapt to different walking robots and allow them to perform stable self-organized locomotion as well as quickly deal with damage within a few walking steps. The adaptation of plasticity after damage or injury is considered here as lesion-induced plasticity. We validated our adaptive interlimb coordination approach with continuous online sensory adaptation on simulated 4-, 6-, 8-, and 20-legged robots. This study not only proposes an adaptive neural control system for artificial walking systems but also offers a possibility of invertebrate nervous systems with flexible plasticity for locomotion and adaptation to injury.

Error-Based Learning Mechanism for Fast Online Adaptation in Robot Motor Control

Existing state-of-the-art frequency adaptation mechanisms of central pattern generators (CPGs) for robot locomotion control typically rely on correlation-based learning. They do not account for the tracking error that may occur between the actual system motion and CPG output, leading to the loss of precision, unwanted movement, inefficient energy locomotion, and in the worst cases, motor collapse. To overcome this problem, we developed online error-based learning for frequency adaptation of CPGs. The learning mechanism used for error reduction is a novel modification of the dual learner (DL) called dual integral learner (DIL). Being able to reduce tracking and steady-state errors, it can also perform fast and stable learning, adapting the CPG frequency to match the performance of robotic systems. Control parameters of the DIL are more straightforward for complex systems (like walking robots), compared to traditional correlation-based learning, since they correspond to error reduction. Due to its embedded memory, the DIL can relearn quickly and recover spontaneously from the previously learned parameters. All these features are not covered by the existing frequency adaptation mechanisms. We integrated the DIL into a neural CPG-based motor control system for use on different legged robots with various morphologies for evaluation. The results show that: 1) the DIL does not require precise adjustment of its parameters to fit specific robots; and 2) the DIL can automatically and quickly adapt the CPG frequency to the robots such that the entire trajectory of the CPG can be precisely followed with very low tracking and steady-state errors. Consequently, the robots can perform the desired movements with more energy-efficient locomotion compared to the state-of-the-art correlation-based learning mechanism called frequency adaptation through fast dynamical coupling (AFDC). In the future, the proposed error-based learning mechanism for fast online adaptation in robot motor control can be used as a basis for trajectory optimization, universal controllers, and other studies concerning the change of intrinsic or extrinsic parameters.

Decentralized control of insect walking: A simple neural network explains a wide range of behavioral and neurophysiological results

Controlling the six legs of an insect walking in an unpredictable environment is a challenging task, as many degrees of freedom have to be coordinated. Solutions proposed to deal with this task are usually based on the highly influential concept that (sensory-modulated) central pattern generators (CPG) are required to control the rhythmic movements of walking legs. Here, we investigate a different view. To this end, we introduce a sensor based controller operating on artificial neurons, being applied to a (simulated) insectoid robot required to exploit the "loop through the world" allowing for simplification of neural computation. We show that such a decentralized solution leads to adaptive behavior when facing uncertain environments which we demonstrate for a broad range of behaviors never dealt with in a single system by earlier approaches. This includes the ability to produce footfall patterns such as velocity dependent "tripod", "tetrapod", "pentapod" as well as various stable intermediate patterns as observed in stick insects and in Drosophila. These patterns are found to be stable against disturbances and when starting from various leg configurations. Our neuronal architecture easily allows for starting or interrupting a walk, all being difficult for CPG controlled solutions. Furthermore, negotiation of curves and walking on a treadmill with various treatments of individual legs is possible as well as backward walking and performing short steps. This approach can as well account for the neurophysiological results usually interpreted to support the idea that CPGs form the basis of walking, although our approach is not relying on explicit CPG-like structures. Application of CPGs may however be required for very fast walking. Our neuronal structure allows to pinpoint specific neurons known from various insect studies. Interestingly, specific common properties observed in both insects and crustaceans suggest a significance of our controller beyond the realm of insects.

Synchronization of Non-linear Oscillators for Neurobiologically Inspired Control on a Bionic Parallel Waist of Legged Robot

Synchronization of coupled non-linear oscillators inspired by a central pattern generator (CPG) can control the bionic robot and promote the coordination and diversity of locomotion. However, for a robot with a strong mutual coupled structure, such neurobiological control is still missing. In this contribution, we present a σ-Hopf harmonic oscillator with decoupled parameters to expand the solution space of the locomotion of the robot. Unlike the synchronization of original Hopf oscillators, which has been fully discussed, the asymmetric factor of σ-Hopf oscillator causes a deformation in oscillation waveform. Using the non-linear synchronization theory, we construct the transition state model of the synchronization process to analyze the asymmetrical distortion, period change and duty ratio inconsistency. Then a variable coupling strength is introduced to eliminate the waveform deformation and maintain the fast convergence rate. Finally, the approach is used for the locomotion control of a bionic parallel waist of legged robot, which is a highly coupled system. The effectiveness of the approach in both independent and synthesis behavior of four typical motion patterns are validated. The result proves the importance of controllability of the oscillation waveform and the instantaneous state of the synchronization, which benefits the transition and transformation of the locomotion and makes the coupling motion more flexible.

Analytically founded yaw control algorithm for walking on uneven terrain applied to a hexapod robot

This article presents a new control algorithm for the omnidirectional motion of a legged robot on uneven terrain based on an analytical kinematic solution without the use of Jacobians. In order to control the robot easily and efficiently in all situations, a simplified circle-based workspace approximation has been introduced. Foot trajectories for legged robot movement were generated on concentric circular paths around an analytically computed common centre of motion. This systematic motion model, together with new gait control variables that can be changed during legged robot motion, enabled the implementation of a new adaptive gait phase control algorithm, as well as the addition of algorithms for ground-level adaptation, 3-dimensional map-based step adjustment and fusion of all corrections to establish and/or maintain foot contact with the ground. The method being applicable to different legged robot designs was performed and tested on the laboratory prototype of a hexagonal hexapod robot, and the results of the experiments showed the practical value of the proposed adaptive yaw control method (available also as a video supplement).

Lateral undulation of the flexible spine of sprawling posture vertebrates

Sprawling posture vertebrates have a flexible spine that bends the trunk primarily in the horizontal plane during locomotion. By coordinating cyclical lateral trunk flexion and limb movements, these animals are very mobile and show extraordinary maneuverability. The dynamic and static stability displayed in complex and changing environments are highly correlated with such lateral bending patterns. The axial dynamics of their compliant body can also be critical for achieving energy-efficient locomotion at high velocities. In this paper, lateral undulation is used to characterize the bending pattern. The production of ground reaction forces (GRFs) and the related center of mass (COM) dynamics during locomotion are the fundamental mechanisms to be considered. Mainly based on research on geckos, which show unrestricted movement in three-dimensional space, we review current knowledge on the trunk flexibility and waveforms of lateral trunk movement. We investigate locomotion dynamics and mechanisms underlying the lateral undulation pattern. This paper also provides insights into the roles of this pattern in obtaining flexible and efficient walking, running, and climbing. Finally, we discuss the potential application of lateral undulation patterns to bio-inspired robotics.

Simple analytical model reveals the functional role of embodied sensorimotor interaction in hexapod gaits

Insects have various gaits with specific characteristics and can change their gaits smoothly in accordance with their speed. These gaits emerge from the embodied sensorimotor interactions that occur between the insect’s neural control and body dynamic systems through sensory feedback. Sensory feedback plays a critical role in coordinated movements such as locomotion, particularly in stick insects. While many previously developed insect models can generate different insect gaits, the functional role of embodied sensorimotor interactions in the interlimb coordination of insects remains unclear because of their complexity. In this study, we propose a simple physical model that is amenable to mathematical analysis to explain the functional role of these interactions clearly. We focus on a foot contact sensory feedback called phase resetting, which regulates leg retraction timing based on touchdown information. First, we used a hexapod robot to determine whether the distributed decoupled oscillators used for legs with the sensory feedback generate insect-like gaits through embodied sensorimotor interactions. The robot generated two different gaits and one had similar characteristics to insect gaits. Next, we proposed the simple model as a minimal model that allowed us to analyze and explain the gait mechanism through the embodied sensorimotor interactions. The simple model consists of a rigid body with massless springs acting as legs, where the legs are controlled using oscillator phases with phase resetting, and the governed equations are reduced such that they can be explained using only the oscillator phases with some approximations. This simplicity leads to analytical solutions for the hexapod gaits via perturbation analysis, despite the complexity of the embodied sensorimotor interactions. This is the first study to provide an analytical model for insect gaits under these interaction conditions. Our results clarified how this specific foot contact sensory feedback contributes to generation of insect-like ipsilateral interlimb coordination during hexapod locomotion.

Adaptive Control Strategies for Interlimb Coordination in Legged Robots: A Review

Walking animals produce adaptive interlimb coordination during locomotion in accordance with their situation. Interlimb coordination is generated through the dynamic interactions of the neural system, the musculoskeletal system, and the environment, although the underlying mechanisms remain unclear. Recently, investigations of the adaptation mechanisms of living beings have attracted attention, and bio-inspired control systems based on neurophysiological findings regarding sensorimotor interactions are being developed for legged robots. In this review, we introduce adaptive interlimb coordination for legged robots induced by various factors (locomotion speed, environmental situation, body properties, and task). In addition, we show characteristic properties of adaptive interlimb coordination, such as gait hysteresis and different time-scale adaptations. We also discuss the underlying mechanisms and control strategies to achieve adaptive interlimb coordination and the design principle for the control system of legged robots.

A Minimal Model Describing Hexapedal Interlimb Coordination: The Tegotae-Based Approach

Insects exhibit adaptive and versatile locomotion despite their minimal neural computing. Such locomotor patterns are generated via coordination between leg movements, i.e., an interlimb coordination, which is largely controlled in a distributed manner by neural circuits located in thoracic ganglia. However, the mechanism responsible for the interlimb coordination still remains elusive. Understanding this mechanism will help us to elucidate the fundamental control principle of animals' agile locomotion and to realize robots with legs that are truly adaptive and could not be developed solely by conventional control theories. This study aims at providing a “minimal" model of the interlimb coordination mechanism underlying hexapedal locomotion, in the hope that a single control principle could satisfactorily reproduce various aspects of insect locomotion. To this end, we introduce a novel concept we named “Tegotae,” a Japanese concept describing the extent to which a perceived reaction matches an expectation. By using the Tegotae-based approach, we show that a surprisingly systematic design of local sensory feedback mechanisms essential for the interlimb coordination can be realized. We also use a hexapod robot we developed to show that our mathematical model of the interlimb coordination mechanism satisfactorily reproduces various insects' gait patterns.

A Neurocomputational Model of Goal-Directed Navigation in Insect-Inspired Artificial Agents

Despite their small size, insect brains are able to produce robust and efficient navigation in complex environments. Specifically in social insects, such as ants and bees, these navigational capabilities are guided by orientation directing vectors generated by a process called path integration. During this process, they integrate compass and odometric cues to estimate their current location as a vector, called the home vector for guiding them back home on a straight path. They further acquire and retrieve path integration-based vector memories globally to the nest or based on visual landmarks. Although existing computational models reproduced similar behaviors, a neurocomputational model of vector navigation including the acquisition of vector representations has not been described before. Here we present a model of neural mechanisms in a modular closed-loop control—enabling vector navigation in artificial agents. The model consists of a path integration mechanism, reward-modulated global learning, random search, and action selection. The path integration mechanism integrates compass and odometric cues to compute a vectorial representation of the agent's current location as neural activity patterns in circular arrays. A reward-modulated learning rule enables the acquisition of vector memories by associating the local food reward with the path integration state. A motor output is computed based on the combination of vector memories and random exploration. In simulation, we show that the neural mechanisms enable robust homing and localization, even in the presence of external sensory noise. The proposed learning rules lead to goal-directed navigation and route formation performed under realistic conditions. Consequently, we provide a novel approach for vector learning and navigation in a simulated, situated agent linking behavioral observations to their possible underlying neural substrates.

Feed-Forwardness of Spinal Networks in Posture and Locomotion

We present a new perspective on the concept of feed-forward compared to feedback mechanisms for motor control. We propose that conceptually all sensory information in real time provided to the brain and spinal cord can be viewed as a feed-forward phenomenon. We also propose that the spinal cord continually adapts to a broad array of ongoing sensory information that is used to adjust the probability of making timely and predictable decisions of selected networks that will execute a given response. One interpretation of the term feedback historically entails responses with short delays. We propose that feed-forward mechanisms, however, range in timeframes of milliseconds to an evolutionary perspective, that is, "evolutionary learning." Continuously adapting events enable a high level of automaticity within the sensorimotor networks that mediate "planned" motor tasks. We emphasize that either a very small or a very large proportion of motor responses can be under some level of conscious vs automatic control. Furthermore, we make a case that a major component of automaticity of the neural control of movement in vertebrates is located within spinal cord networks. Even without brain input, the spinal cord routinely uses feed-forward processing of sensory information, particularly proprioceptive and cutaneous, to continuously make fundamental decisions that define motor responses. In effect, these spinal networks may be largely responsible for executing coordinated sensorimotor tasks, even those under normal "conscious" control.

Enhanced Locomotion Efficiency of a Bio-inspired Walking Robot using Contact Surfaces with Frictional Anisotropy

Based on the principles of morphological computation, we propose a novel approach that exploits the interaction between a passive anisotropic scale-like material (e.g., shark skin) and a non-smooth substrate to enhance locomotion efficiency of a robot walking on inclines. Real robot experiments show that passive tribologically-enhanced surfaces of the robot belly or foot allow the robot to grip on specific surfaces and move effectively with reduced energy consumption. Supplementing the robot experiments, we investigated tribological properties of the shark skin as well as its mechanical stability. It shows high frictional anisotropy due to an array of sloped denticles. The orientation of the denticles to the underlying collagenous material also strongly influences their mechanical interlocking with the substrate. This study not only opens up a new way of achieving energy-efficient legged robot locomotion but also provides a better understanding of the functionalities and mechanical properties of anisotropic surfaces. That understanding will assist developing new types of material for other real-world applications.

Adaptive and Energy Efficient Walking in a Hexapod Robot Under Neuromechanical Control and Sensorimotor Learning

The control of multilegged animal walking is a neuromechanical process, and to achieve this in an adaptive and energy efficient way is a difficult and challenging problem. This is due to the fact that this process needs in real time: 1) to coordinate very many degrees of freedom of jointed legs; 2) to generate the proper leg stiffness (i.e., compliance); and 3) to determine joint angles that give rise to particular positions at the endpoints of the legs. To tackle this problem for a robotic application, here we present a neuromechanical controller coupled with sensorimotor learning. The controller consists of a modular neural network for coordinating 18 joints and several virtual agonist-antagonist muscle mechanisms (VAAMs) for variable compliant joint motions. In addition, sensorimotor learning, including forward models and dual-rate learning processes, is introduced for predicting foot force feedback and for online tuning the VAAMs' stiffness parameters. The control and learning mechanisms enable the hexapod robot advanced mobility sensor driven-walking device (AMOS) to achieve variable compliant walking that accommodates different gaits and surfaces. As a consequence, AMOS can perform more energy efficient walking, compared to other small legged robots. In addition, this paper also shows that the tight combination of neural control with tunable muscle-like functions, guided by sensory feedback and coupled with sensorimotor learning, is a way forward to better understand and solve adaptive coordination problems in multilegged locomotion.

Trajectory Correction and Locomotion Analysis of a Hexapod Walking Robot with Semi-Round Rigid Feet

Aimed at solving the misplaced body trajectory problem caused by the rolling of semi-round rigid feet when a robot is walking, a legged kinematic trajectory correction methodology based on the Least Squares Support Vector Machine (LS-SVM) is proposed. The concept of ideal foothold is put forward for the three-dimensional kinematic model modification of a robot leg, and the deviation value between the ideal foothold and real foothold is analyzed. The forward/inverse kinematic solutions between the ideal foothold and joint angular vectors are formulated and the problem of direct/inverse kinematic nonlinear mapping is solved by using the LS-SVM. Compared with the previous approximation method, this correction methodology has better accuracy and faster calculation speed with regards to inverse kinematics solutions. Experiments on a leg platform and a hexapod walking robot are conducted with multi-sensors for the analysis of foot tip trajectory, base joint vibration, contact force impact, direction deviation, and power consumption, respectively. The comparative analysis shows that the trajectory correction methodology can effectively correct the joint trajectory, thus eliminating the contact force influence of semi-round rigid feet, significantly improving the locomotion of the walking robot and reducing the total power consumption of the system.

Synaptic plasticity in a recurrent neural network for versatile and adaptive behaviors of a walking robot

Walking animals, like insects, with little neural computing can effectively perform complex behaviors. For example, they can walk around their environment, escape from corners/deadlocks, and avoid or climb over obstacles. While performing all these behaviors, they can also adapt their movements to deal with an unknown situation. As a consequence, they successfully navigate through their complex environment. The versatile and adaptive abilities are the result of an integration of several ingredients embedded in their sensorimotor loop. Biological studies reveal that the ingredients include neural dynamics, plasticity, sensory feedback, and biomechanics. Generating such versatile and adaptive behaviors for a many degrees-of-freedom (DOFs) walking robot is a challenging task. Thus, in this study, we present a bio-inspired approach to solve this task. Specifically, the approach combines neural mechanisms with plasticity, exteroceptive sensory feedback, and biomechanics. The neural mechanisms consist of adaptive neural sensory processing and modular neural locomotion control. The sensory processing is based on a small recurrent neural network consisting of two fully connected neurons. Online correlation-based learning with synaptic scaling is applied to adequately change the connections of the network. By doing so, we can effectively exploit neural dynamics (i.e., hysteresis effects and single attractors) in the network to generate different turning angles with short-term memory for a walking robot. The turning information is transmitted as descending steering signals to the neural locomotion control which translates the signals into motor actions. As a result, the robot can walk around and adapt its turning angle for avoiding obstacles in different situations. The adaptation also enables the robot to effectively escape from sharp corners or deadlocks. Using backbone joint control embedded in the the locomotion control allows the robot to climb over small obstacles. Consequently, it can successfully explore and navigate in complex environments. We firstly tested our approach on a physical simulation environment and then applied it to our real biomechanical walking robot AMOSII with 19 DOFs to adaptively avoid obstacles and navigate in the real world.

Distributed recurrent neural forward models with synaptic adaptation and CPG-based control for complex behaviors of walking robots

Walking animals, like stick insects, cockroaches or ants, demonstrate a fascinating range of locomotive abilities and complex behaviors. The locomotive behaviors can consist of a variety of walking patterns along with adaptation that allow the animals to deal with changes in environmental conditions, like uneven terrains, gaps, obstacles etc. Biological study has revealed that such complex behaviors are a result of a combination of biomechanics and neural mechanism thus representing the true nature of embodied interactions. While the biomechanics helps maintain flexibility and sustain a variety of movements, the neural mechanisms generate movements while making appropriate predictions crucial for achieving adaptation. Such predictions or planning ahead can be achieved by way of internal models that are grounded in the overall behavior of the animal. Inspired by these findings, we present here, an artificial bio-inspired walking system which effectively combines biomechanics (in terms of the body and leg structures) with the underlying neural mechanisms. The neural mechanisms consist of (1) central pattern generator based control for generating basic rhythmic patterns and coordinated movements, (2) distributed (at each leg) recurrent neural network based adaptive forward models with efference copies as internal models for sensory predictions and instantaneous state estimations, and (3) searching and elevation control for adapting the movement of an individual leg to deal with different environmental conditions. Using simulations we show that this bio-inspired approach with adaptive internal models allows the walking robot to perform complex locomotive behaviors as observed in insects, including walking on undulated terrains, crossing large gaps, leg damage adaptations, as well as climbing over high obstacles. Furthermore, we demonstrate that the newly developed recurrent network based approach to online forward models outperforms the adaptive neuron forward models, which have hitherto been the state of the art, to model a subset of similar walking behaviors in walking robots.

A novel approach to locomotion learning: Actor-Critic architecture using central pattern generators and dynamic motor primitives

In this article, we propose an architecture of a bio-inspired controller that addresses the problem of learning different locomotion gaits for different robot morphologies. The modeling objective is split into two: baseline motion modeling and dynamics adaptation. Baseline motion modeling aims to achieve fundamental functions of a certain type of locomotion and dynamics adaptation provides a "reshaping" function for adapting the baseline motion to desired motion. Based on this assumption, a three-layer architecture is developed using central pattern generators (CPGs, a bio-inspired locomotor center for the baseline motion) and dynamic motor primitives (DMPs, a model with universal "reshaping" functions). In this article, we use this architecture with the actor-critic algorithms for finding a good "reshaping" function. In order to demonstrate the learning power of the actor-critic based architecture, we tested it on two experiments: (1) learning to crawl on a humanoid and, (2) learning to gallop on a puppy robot. Two types of actor-critic algorithms (policy search and policy gradient) are compared in order to evaluate the advantages and disadvantages of different actor-critic based learning algorithms for different morphologies. Finally, based on the analysis of the experimental results, a generic view/architecture for locomotion learning is discussed in the conclusion.

Visually guided gait modifications for stepping over an obstacle: a bio-inspired approach

There is an increasing interest in conceiving robotic systems that are able to move and act in an unstructured and not predefined environment, for which autonomy and adaptability are crucial features. In nature, animals are autonomous biological systems, which often serve as bio-inspiration models, not only for their physical and mechanical properties, but also their control structures that enable adaptability and autonomy-for which learning is (at least) partially responsible. This work proposes a system which seeks to enable a quadruped robot to online learn to detect and to avoid stumbling on an obstacle in its path. The detection relies in a forward internal model that estimates the robot's perceptive information by exploring the locomotion repetitive nature. The system adapts the locomotion in order to place the robot optimally before attempting to step over the obstacle, avoiding any stumbling. Locomotion adaptation is achieved by changing control parameters of a central pattern generator (CPG)-based locomotion controller. The mechanism learns the necessary alterations to the stride length in order to adapt the locomotion by changing the required CPG parameter. Both learning tasks occur online and together define a sensorimotor map, which enables the robot to learn to step over the obstacle in its path. Simulation results show the feasibility of the proposed approach.

Biologically-inspired adaptive obstacle negotiation behavior of hexapod robots

Neurobiological studies have shown that insects are able to adapt leg movements and posture for obstacle negotiation in changing environments. Moreover, the distance to an obstacle where an insect begins to climb is found to be a major parameter for successful obstacle negotiation. Inspired by these findings, we present an adaptive neural control mechanism for obstacle negotiation behavior in hexapod robots. It combines locomotion control, backbone joint control, local leg reflexes, and neural learning. While the first three components generate locomotion including walking and climbing, the neural learning mechanism allows the robot to adapt its behavior for obstacle negotiation with respect to changing conditions, e.g., variable obstacle heights and different walking gaits. By successfully learning the association of an early, predictive signal (conditioned stimulus, CS) and a late, reflex signal (unconditioned stimulus, UCS), both provided by ultrasonic sensors at the front of the robot, the robot can autonomously find an appropriate distance from an obstacle to initiate climbing. The adaptive neural control was developed and tested first on a physical robot simulation, and was then successfully transferred to a real hexapod robot, called AMOS II. The results show that the robot can efficiently negotiate obstacles with a height up to 85% of the robot's leg length in simulation and 75% in a real environment.