A survey on CPG-inspired control models and system implementation

Workspace trajectory generation with smooth gait transition using CPG-based locomotion control for hexapod robot

-This paper presents a new control methodology for achieving smooth gait transitions for a hexapod robot using Central Pattern Generators (CPGs). The approach involves modifying the Phase Oscillator within the CPG network to enable smooth transitions between different gaits in order to improve the adaptability to changing environmental conditions. The foot trajectory generator is designed based on the CPG output, allowing the possibility of online adjustment of foot trajectory parameters, such as step height and size, as well as the robot's speed and direction. Our simulation demonstrates the effectiveness of the modified oscillator in achieving smoother gait transitions with a transition time falls close to the output period of the CPG oscillators, and experiments on a real hexapod robot validate the feasibility and efficiency of our approach in considering online adjustability of trajectory parameters, confirming the potential of this methodology to enhance the locomotion capabilities of legged robots for navigating complex terrains.

Bioinspired Control Architecture for Adaptive and Resilient Navigation of Unmanned Underwater Vehicle in Monitoring Missions of Submerged Aquatic Vegetation Meadows

Submerged aquatic vegetation plays a fundamental role as a habitat for the biodiversity of marine species. To carry out the research and monitoring of submerged aquatic vegetation more efficiently and accurately, it is important to use advanced technologies such as underwater robots. However, when conducting underwater missions to capture photographs and videos near submerged aquatic vegetation meadows, algae can become entangled in the propellers and cause vehicle failure. In this context, a neurobiologically inspired control architecture is proposed for the control of unmanned underwater vehicles with redundant thrusters. The proposed control architecture learns to control the underwater robot in a non-stationary environment and combines the associative learning method and vector associative map learning to generate transformations between the spatial and velocity coordinates in the robot actuator. The experimental results obtained show that the proposed control architecture exhibits notable resilience capabilities while maintaining its operation in the face of thruster failures. In the discussion of the results obtained, the importance of the proposed control architecture is highlighted in the context of the monitoring and conservation of underwater vegetation meadows. Its resilience, robustness, and adaptability capabilities make it an effective tool to face challenges and meet mission objectives in such critical environments.

Acinonyx jubatus-Inspired Quadruped Robotics: Integrating Neural Oscillators for Enhanced Locomotion Control

This study presents the design, simulation, and prototype creation of a quadruped robot inspired by the Acinonyx jubatus (cheetah), specifically designed to replicate its distinctive walking, trotting, and galloping locomotion patterns. Following a detailed examination of the cheetah's skeletal muscle anatomy and biomechanics, a simplified model of the robot with 12 degrees of freedom was conducted. The mathematical transformation hierarchy model was established, and direct kinematics were simulated. A bio-inspired control approach was introduced, employing a Central Pattern Generator model based on Wilson-Cowan neural oscillators for each limb, interconnected by synaptic weights. This approach assisted in the simulation of oscillatory signals for relative phases corresponding to four distinct gaits in a system-level simulation platform. The design phase was conducted using CAD software (SolidWorks 2018), resulting in a 1:3-scale robot mirroring the cheetah's actual proportions. Movement simulations were performed in a virtual mechanics software environment, leading to the construction of a prototype measuring 35.5 cm in length, 21 cm in width, 27 cm in height (when standing), and weighing approximately 2.1 kg. The experimental validation of the prototype's limb angular positions and trajectories was achieved through the image processing of video-recorded movements, demonstrating a high correlation (0.9025 to 0.9560) in joint angular positions, except for the knee joint, where a correlation of 0.7071 was noted. This comprehensive approach from theoretical analysis to practical implementation showcases the potential of bio-inspired robotics in emulating complex biological locomotion.

Structural Design and Control Research of Multi-Segmented Biomimetic Millipede Robot

Due to their advantages of good stability, adaptability, and flexibility, multi-legged robots are increasingly important in fields such as rescue, military, and healthcare. This study focuses on the millipede, a multi-segmented organism, and designs a novel multi-segment biomimetic robot based on an in-depth investigation of the millipede's biological characteristics and locomotion mechanisms. Key leg joints of millipede locomotion are targeted, and a mathematical model of the biomimetic robot's leg joint structure is established for kinematic analysis. Furthermore, a central pattern generator (CPG) control strategy is studied for multi-jointed biomimetic millipede robots. Inspired by the millipede's neural system, a simplified single-loop CPG network model is constructed, reducing the number of oscillators from 48 to 16. Experimental trials are conducted using a prototype to test walking in a wave-like gait, walking with a leg removed, and walking on complex terrain. The results demonstrate that under CPG waveform input conditions, the robot can walk stably, and the impact of a leg failure on overall locomotion is acceptable, with minimal speed loss observed when walking on complex terrain. The research on the structure and motion control algorithms of multi-jointed biomimetic robots lays a technical foundation, expanding their potential applications in exploring unknown environments, rescue missions, agriculture, and other fields.

Viability leads to the emergence of gait transitions in learning agile quadrupedal locomotion on challenging terrains

Quadruped animals are capable of seamless transitions between different gaits. While energy efficiency appears to be one of the reasons for changing gaits, other determinant factors likely play a role too, including terrain properties. In this article, we propose that viability, i.e., the avoidance of falls, represents an important criterion for gait transitions. We investigate the emergence of gait transitions through the interaction between supraspinal drive (brain), the central pattern generator in the spinal cord, the body, and exteroceptive sensing by leveraging deep reinforcement learning and robotics tools. Consistent with quadruped animal data, we show that the walk-trot gait transition for quadruped robots on flat terrain improves both viability and energy efficiency. Furthermore, we investigate the effects of discrete terrain (i.e., crossing successive gaps) on imposing gait transitions, and find the emergence of trot-pronk transitions to avoid non-viable states. Viability is the only improved factor after gait transitions on both flat and discrete gap terrains, suggesting that viability could be a primary and universal objective of gait transitions, while other criteria are secondary objectives and/or a consequence of viability. Moreover, our experiments demonstrate state-of-the-art quadruped robot agility in challenging scenarios.

Optimization strategies to obtain smooth gait transitions through biologically plausible central pattern generators

Central pattern generators are small networks that contribute to generating animal locomotion. The models used to study gait generation and gait transition mechanisms often require biologically accurate neuron and synapse models, with high dimensionality and complex dynamics. Tuning the parameters of these models to elicit network dynamics compatible with gait features is not a trivial task, due to the impossibility of inferring a priori the effects of each parameter on the nonlinear system's emergent dynamics. In this paper we explore the use of global optimization strategies for parameter optimization in multigait central pattern generator (CPG) models with complex cell dynamics and minimal topology. We first consider an existing quadruped CPG model as a test bed for the objective function formulation, then proceed to optimize the parameters of a newly proposed multigait, interlimb hexapod CPG model. We successfully obtain hexapod gaits and prompt gait transitions by varying only control currents, while all CPG parameters, once optimized, are kept fixed. This mechanism of gait transitions is compatible with short-term synaptic plasticity.

Control of movement of underwater swimmers: Animals, simulated animates and swimming robots

The control of movement in living organisms represents a fundamental task that the brain has evolved to solve. One crucial aspect is how the nervous system organizes the transformation of sensory information into motor commands. These commands lead to muscle activation and subsequent animal movement, which can exhibit complex patterns. One example of such movement is locomotion, which involves the translation of the entire body through space. Central Pattern Generators (CPGs) are neuronal circuits that provide control signals for these movements. Compared to the intricate circuits found in the brain, CPGs can be simplified into networks of neurons that generate rhythmic activation, coordinating muscle movements. Since the 1990s, researchers have developed numerous models of locomotive circuits to simulate different types of animal movement, including walking, flying, and swimming. Initially, the primary goal of these studies was to construct biomimetic robots. However, it became apparent that simplified CPGs alone were not sufficient to replicate the diverse range of adaptive locomotive movements observed in living organisms. Factors such as sensory modulation, higher-level control, and cognitive components related to learning and memory needed to be considered. This necessitated the use of more complex, high-dimensional circuits, as well as novel materials and hardware, in both modeling and robotics. With advancements in high-power computing, artificial intelligence, big data processing, smart materials, and electronics, the possibility of designing a new generation of true bio-mimetic robots has emerged. These robots have the capability to imitate not only simple locomotion but also exhibit adaptive motor behavior and decision-making. This motivation serves as the foundation for the current review, which aims to analyze existing concepts and models of movement control systems. As an illustrative example, we focus on underwater movement and explore the fundamental biological concepts, as well as the mathematical and physical models that underlie locomotion and its various modulations.

Multi-constraint spatial coupling for the body joint quadruped robot and the CPG control method on rough terrain

Quadruped robots have frequently appeared in various situations, including wilderness rescue, planetary exploration, and nuclear power facility maintenance. The quadruped robot with an active body joint has better environmental adaptability than one without body joints. However, it is difficult to guarantee the stability of the body joint quadruped robot when walking on rough terrain. Given the above issues, this paper proposed a gait control method for the body joint quadruped robot based on multi-constraint spatial coupling (MCSC) algorithm. The body workspace of the robot is divided into three subspaces, which are solved for different gaits, and then coupled to obtain the stable workspace of the body. A multi-layer central pattern generator model based on the Hopf oscillator is built to realize the generation and switching of walk and trot gaits. Then, combined with the MCSC area of the body, the reflex adjustment strategy on different terrains is established to adjust the body's posture in real time and realize the robot's stable locomotion. Finally, the robot prototype is developed to verify the effectiveness of the control method. The simulation and experiment results show that the proposed method can reduce the offset of the swing legs and the fluctuation of the body attitude angle. Furthermore, the quadruped robot is ensured to maintain stability by dynamically modifying its body posture. The relevant result can offer a helpful reference for the control of quadruped robots in complex environments.

Central Pattern Generator (CPG)-Based Locomotion Control and Hydrodynamic Experiments of Synergistical Interaction between Pectoral Fins and Caudal Fin for Boxfish-like Robot

Locomotion control of synergistical interaction between fins has been one of the key problems in the field of robotic fish research owing to its contribution to improving and enhancing swimming performance. In this paper, the coordinated locomotion control of the boxfish-like robot with pectoral and caudal fins is studied, and the effects of different control parameters on the propulsion performance are quantitatively analyzed by using hydrodynamic experiments. First, an untethered boxfish-like robot with two pectoral fins and one caudal fin was designed. Second, a central pattern generator (CPG)-based controller is used to coordinate the motions of the pectoral and caudal fins to realize the bionic locomotion of the boxfish-like robot. Finally, extensive hydrodynamic experiments are conducted to explore the effects of different CPG parameters on the propulsion performance under the synergistic interaction of pectoral and caudal fins. Results show that the amplitude and frequency significantly affect the propulsion performance, and the propulsion ability is the best when the frequency is 1 Hz. Different phase lags and offset angles between twisting and flapping of the pectoral fin can generate positive and reverse forces, which realize the forward, backward, and pitching swimming by adjusting these parameters. This paper reveals for the first time the effects of different CPG parameters on the propulsion performance in the case of the synergistic interaction between the pectoral fins and the caudal fin using hydrodynamic experimental methods, which sheds light on the optimization of the design and control parameters of the robotic fish.

Design, Modeling, and Control of an Aurelia-Inspired Robot Based on SMA Artificial Muscles

This paper presented a flexible and easily fabricated untethered underwater robot inspired by Aurelia, which is named "Au-robot". The Au-robot is actuated by six radial fins made of shape memory alloy (SMA) artificial muscle modules, which can realize pulse jet propulsion motion. The thrust model of the Au-robot's underwater motion is developed and analyzed. To achieve a multimodal and smooth swimming transition for the Au-robot, a control method integrating a central pattern generator (CPG) and an adaptive regulation (AR) heating strategy is provided. The experimental results demonstrate that the Au-robot, with good bionic properties in structure and movement mode, can achieve a smooth transition from low-frequency swimming to high-frequency swimming with an average maximum instantaneous velocity of 12.61 cm/s. It shows that a robot designed and fabricated with artificial muscle can imitate biological structures and movement traits more realistically and has better motor performance.

A CPG-Based Versatile Control Framework for Metameric Earthworm-Like Robotic Locomotion

Annelids such as earthworms are considered to have central pattern generators (CPGs) that generate rhythms in neural circuits to coordinate the deformation of body segments for effective locomotion. At present, the states of earthworm-like robot segments are often assigned holistically and artificially by mimicking the earthworms' retrograde peristalsis wave, which is unable to adapt their gaits for variable environments and tasks. This motivates the authors to extend the bioinspired research from morphology to neurobiology by mimicking the CPG to build a versatile framework for spontaneous motion control. Here, the spatiotemporal dynamics is exploited from the coupled Hopf oscillators to not only unify the two existing gait generators for restoring temporal-symmetric phase-coordinated gaits and discrete gaits but also generate novel temporal-asymmetric phase-coordinated gaits. Theoretical and experimental tests consistently confirm that the introduction of temporal asymmetry improves the robot's locomotion performance. The CPG-based controller also enables seamless online switching of locomotion gaits to avoid abrupt changes, sharp stops, and starts, thus improving the robot's adaptability in variable working scenarios.

Robust and reusable self-organized locomotion of legged robots under adaptive physical and neural communications

Introduction

Animals such as cattle can achieve versatile and elegant behaviors through automatic sensorimotor coordination. Their self-organized movements convey an impression of adaptability, robustness, and motor memory. However, the adaptive mechanisms underlying such natural abilities of these animals have not been completely realized in artificial legged systems.

Methods

Hence, we propose adaptive neural control that can mimic these abilities through adaptive physical and neural communications. The control algorithm consists of distributed local central pattern generator (CPG)-based neural circuits for generating basic leg movements, an adaptive sensory feedback mechanism for generating self-organized phase relationships among the local CPG circuits, and an adaptive neural coupling mechanism for transferring and storing the formed phase relationships (a gait pattern) into the neural structure. The adaptive neural control was evaluated in experiments using a quadruped robot.

Results

The adaptive neural control enabled the robot to 1) rapidly and automatically form its gait (i.e., self-organized locomotion) within a few seconds, 2) memorize the gait for later recovery, and 3) robustly walk, even when a sensory feedback malfunction occurs. It also enabled maneuverability, with the robot being able to change its walking speed and direction. Moreover, implementing adaptive physical and neural communications provided an opportunity for understanding the mechanism of motor memory formation.

Discussion

Overall, this study demonstrates that the integration of the two forms of communications through adaptive neural control is a powerful way to achieve robust and reusable self-organized locomotion in legged robots.

Parameter search of a CPG network using a genetic algorithm for a snake robot with tactile sensors moving on a soft floor

When a snake robot explores a collapsed house as a rescue robot, it needs to move through various obstacles, some of which may be made of soft materials, such as mattresses. In this study, we call mattress-like environment as a soft floor, which deforms when some force is added to it. We focused on the central pattern generator (CPG) network as a control for the snake robot to propel itself on the soft floor and constructed a CPG network that feeds back contact information between the robot and the floor. A genetic algorithm was used to determine the parameters of the CPG network suitable for the soft floor. To verify the obtained parameters, comparative simulations were conducted using the parameters obtained for the soft and hard floor, and the parameters were confirmed to be appropriate for each environment. By observing the difference in snake robot’s propulsion depending on the presence or absence of the tactile sensor feedback signal, we confirmed the effectiveness of the tactile sensor considered in the parameter search.

Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning

In this article, the underwater target tracking control problem of a biomimetic underwater vehicle (BUV) is addressed. Since it is difficult to build an effective mathematic model of a BUV due to the uncertainty of hydrodynamics, target tracking control is converted into the Markov decision process and is further achieved via deep reinforcement learning. The system state and reward function of underwater target tracking control are described. Based on the actor-critic reinforcement learning framework, the deep deterministic policy gradient actor-critic algorithm with supervision controller is proposed. The training tricks, including prioritized experience replay, actor network indirect supervision training, target network updating with different periods, and expansion of exploration space by applying random noise, are presented. Indirect supervision training is designed to address the issues of low stability and slow convergence of reinforcement learning in the continuous state and action space. Comparative simulations are performed to show the effectiveness of the training tricks. Finally, the proposed actor-critic reinforcement learning algorithm with supervision controller is applied to the physical BUV. Swimming pool experiments of underwater object tracking of the BUV are conducted in multiple scenarios to verify the effectiveness and robustness of the proposed method.

Generic Mechanism for Waveform Regulation and Synchronization of Oscillators: An Application for Robot Behavior Diversity Generation

While nonlinear oscillators have been widely used for central pattern generators to produce basic rhythmic signals for robot locomotion control, methods to shape and regulate the signal waveform without changing the characteristics of the oscillators have not been fully investigated, especially during the network synchronization process. To illustrate the principle and process of waveform regulation of nonlinear oscillators in detail and ensure that the influence can be controlled, we present a method for waveform regulation and synchronization and analyze the relationship of different factors (e.g., initial conditions, network parameters, phase, and waveform regulation factors) in synchronization deviation. Then, the method is indicated to be effective in other commonly used nonlinear oscillators and neural oscillators. As an example application, a three-layer behavioral control architecture for a legged robot is constructed based on the proposed method. Modules for the body behavior, leg coordination, and single-leg adjustment are established to realize diverse robot behaviors. The effectiveness of the method is validated by a series of experiments. The results prove that the method performs well in terms of signal control accuracy, behavior pattern diversity, and smooth motion transition.

Bioinspired Central Pattern Generator and T-S Fuzzy Neural Network-Based Control of a Robotic Manta for Depth and Heading Tracking

Aiming at the difficult problem of motion control of robotic manta with pectoral fin flexible deformation, this paper proposes a control scheme that combines the bioinspired Central Pattern Generator (CPG) and T-S Fuzzy neural network(NN)-based control. An improved CPG drive network is presented for the multi-stage fin structure of the robotic manta. Considering the unknown dynamics and the external environmental disturbances, a sensor-based classic T-S Fuzzy NN controller is designed for heading and depth control. Finally, a pool test demonstrates the effectiveness and robustness of the proposed controller: the robotic manta can track the depth and heading with an error of ±6 cm and ±6°, satisfying accuracy requirements.

Continuous Online Adaptation of Bioinspired Adaptive Neuroendocrine Control for Autonomous Walking Robots

Walking animals can continuously adapt their locomotion to deal with unpredictable changing environments. They can also take proactive steps to avoid colliding with an obstacle. In this study, we aim to realize such features for autonomous walking robots so that they can efficiently traverse complex terrains. To achieve this, we propose novel bioinspired adaptive neuroendocrine control. In contrast to conventional locomotion control methods, this approach does not require robot and environmental models, exteroceptive feedback, or multiple learning trials. It integrates three main modular neural mechanisms, relying only on proprioceptive feedback and short-term memory, namely: 1) neural central pattern generator (CPG)-based control; 2) an artificial hormone network (AHN); and 3) unsupervised input correlation-based learning (ICO). The neural CPG-based control creates insect-like gaits, while the AHN can continuously adapt robot joint movement individually with respect to the terrain during the stance phase using only the torque feedback. In parallel, the ICO generates short-term memory for proactive obstacle negotiation during the swing phase, allowing the posterior legs to step over the obstacle before hitting it. The control approach is evaluated on a bioinspired hexapod robot walking on complex unpredictable terrains (e.g., gravel, grass, and extreme random stepfield). The results show that the robot can successfully perform energy-efficient autonomous locomotion and online continuous adaptation with proactivity to overcome such terrains. Since our adaptive neural control approach does not require a robot model, it is general and can be applied to other bioinspired walking robots to achieve a similar adaptive, autonomous, and versatile function.

Synchronization of relaxation oscillators with adaptive thresholds and application to automated guided vehicles

The present paper proposes an adaptive control law for inducing in-phase and antiphase synchronization in a pair of relaxation oscillators. We analytically show that the phase dynamics of the oscillators coupled by the control law is equivalent to that of Kuramoto phase oscillators and then extend the results for a pair of oscillators to three or more oscillators. We also provide a systematic procedure for designing the controller parameters for oscillator networks with all-to-all and ring topologies. Our numerical simulations demonstrate that these analytical results can be used to solve a dispatching problem encountered by automated guided vehicles (AGVs) in factories. AGV congestion can be avoided and the peak value of the amount of materials or parts in buffers can be suppressed.

Adapting Highly-Dynamic Compliant Movements to Changing Environments: A Benchmark Comparison of Reflex- vs. CPG-Based Control Strategies

To control highly-dynamic compliant motions such as running or hopping, vertebrates rely on reflexes and Central Pattern Generators (CPGs) as core strategies. However, decoding how much each strategy contributes to the control and how they are adjusted under different conditions is still a major challenge. To help solve this question, the present paper provides a comprehensive comparison of reflexes, CPGs and a commonly used combination of the two applied to a biomimetic robot. It leverages recent findings indicating that in mammals both control principles act within a low-dimensional control submanifold. This substantially reduces the search space of parameters and enables the quantifiable comparison of the different control strategies. The chosen metrics are motion stability and energy efficiency, both key aspects for the evolution of the central nervous system. We find that neither for stability nor energy efficiency it is favorable to apply the state-of-the-art approach of a continuously feedback-adapted CPG. In both aspects, a pure reflex is more effective, but the pure CPG allows easy signal alteration when needed. Additionally, the hardware experiments clearly show that the shape of a control signal has a strong influence on energy efficiency, while previous research usually only focused on frequency alignment. Both findings suggest that currently used methods to combine the advantages of reflexes and CPGs can be improved. In future research, possible combinations of the control strategies should be reconsidered, specifically including the modulation of the control signal's shape. For this endeavor, the presented setup provides a valuable benchmark framework to enable the quantitative comparison of different bioinspired control principles.

Locomotion Control With Frequency and Motor Pattern Adaptations

Existing adaptive locomotion control mechanisms for legged robots are usually aimed at one specific type of adaptation and rarely combined with others. Adaptive mechanisms thus stay at a conceptual level without their coupling effect with other mechanisms being investigated. However, we hypothesize that the combination of adaptation mechanisms can be exploited for enhanced and more efficient locomotion control as in biological systems. Therefore, in this work, we present a central pattern generator (CPG) based locomotion controller integrating both a frequency and motor pattern adaptation mechanisms. We use the state-of-the-art Dual Integral Learner for frequency adaptation, which can automatically and quickly adapt the CPG frequency, enabling the entire motor pattern or output signal of the CPG to be followed at a proper high frequency with low tracking error. Consequently, the legged robot can move with high energy efficiency and perform the generated locomotion with high precision. The versatile state-of-the-art CPG-RBF network is used as a motor pattern adaptation mechanism. Using this network, the motor patterns or joint trajectories can be adapted to fit the robot's morphology and perform sensorimotor integration enabling online motor pattern adaptation based on sensory feedback. The results show that the two adaptation mechanisms can be combined for adaptive locomotion control of a hexapod robot in a complex environment. Using the CPG-RBF network for motor pattern adaptation, the hexapod learned basic straight forward walking, steering, and step climbing. In general, the frequency and motor pattern mechanisms complement each other well and their combination can be seen as an essential step toward further studies on adaptive locomotion control.

Towards more biologically plausible central-pattern-generator models

Central pattern generators (CPGs) are relatively small neural networks that play a fundamental role in the control of animal locomotion. In this paper we define a method for the systematic design of CPG models able to exhibit biologically plausible gait transitions by implementing short-term synaptic plasticity mechanisms. As a case study, we focus on a simple CPG for quadruped locomotion. By applying the proposed method, three of four standard quadruped gaits were correctly reproduced by the obtained CPG model, not only in terms of the alternating sequence of the limbs but also in terms of frequency, duty cycle, and phase lags.

Parameters optimization of central pattern generators for hexapod robot based on multi-objective genetic algorithm

In this article, a network of central pattern generators is used for the motion planning of a hexapod robot. There are many parameters in the planning network, which determine the motion performance of the hexapod robot. On the other hand, the network is a highly nonlinear coupling network, which is difficult to obtain optimal parameters by an analytical method. Optimizing these parameters to make the robot walk well is a multi-objective optimization process. There is a certain mutual exclusion relationship among the targets. To find a well-performing network as soon as possible, a multi-objective genetic algorithm is used for the process of parameter tuning. The hexapod robot simulation model is performed in Webots, and the motion performance parameters of the robot are obtained through built-in sensors and are also considered as mean values of the optimization algorithm. The optimization algorithm is written and run with MATLAB. Finally, the optimization algorithm and simulation results are proven by an experiment.

Generic Neural Locomotion Control Framework for Legged Robots

In this article, we present a generic locomotion control framework for legged robots and a strategy for control policy optimization. The framework is based on neural control and black-box optimization. The neural control combines a central pattern generator (CPG) and a radial basis function (RBF) network to create a CPG-RBF network. The control network acts as a neural basis to produce arbitrary rhythmic trajectories for the joints of robots. The main features of the CPG-RBF network are: 1) it is generic since it can be applied to legged robots with different morphologies; 2) it has few control parameters, resulting in fast learning; 3) it is scalable, both in terms of policy/trajectory complexity and the number of legs that can be controlled using similar trajectories; 4) it does not rely heavily on sensory feedback to generate locomotion and is thus less prone to sensory faults; and 5) once trained, it is simple, minimal, and intuitive to use and analyze. These features will lead to an easy-to-use framework with fast convergence and the ability to encode complex locomotion control policies. In this work, we show that the framework can successfully be applied to three different simulated legged robots with varying morphologies and, even broken joints, to learn locomotion control policies. We also show that after learning, the control policies can also be successfully transferred to a real-world robot without any modifications. We, furthermore, show the scalability of the framework by implementing it as a central controller for all legs of a robot and as a decentralized controller for individual legs and leg pairs. By investigating the correlation between robot morphology and encoding type, we are able to present a strategy for control policy optimization. Finally, we show how sensory feedback can be integrated into the CPG-RBF network to enable online adaptation.

Fish can save energy via proprioceptive sensing

Fish have evolved diverse and robust locomotive strategies to swim efficiently in complex fluid environments. However, we know little, if anything, about how these strategies can be achieved. Although most studies suggest that fish rely on the lateral line system to sense local flow and optimise body undulation, recent work has shown that fish are still able to gain benefits from the local flow even with the lateral line impaired. In this paper, we hypothesise that fish can save energy by extracting vortices shed from their neighbours using only simple proprioceptive sensing with the caudal fin. We tested this hypothesis on both computational and robotic fish by synthesising a central pattern generator (CPG) with feedback, proprioceptive sensing, and reinforcement learning. The CPG controller adjusts the body undulation after receiving feedback from the proprioceptive sensing signal, decoded via reinforcement learning. In our study, we consider potential proprioceptive sensing inputs to consist of low-dimensional signals (e.g. perceived forces) detected from the flow. With simulations on a computational robot and experiments on a robotic fish swimming in unknown dynamic flows, we show that the simple proprioceptive sensing is sufficient to optimise the body undulation to save energy, without any input from the lateral line. Our results reveal a new sensory-motor mechanism in schooling fish and shed new light on the strategy of control for robotic fish swimming in complex flows with high efficiency.

Generalized half-center oscillators with short-term synaptic plasticity

How can we develop simple yet realistic models of the small neural circuits known as central pattern generators (CPGs), which contribute to generate complex multiphase locomotion in living animals? In this paper we introduce a new model (with design criteria) of a generalized half-center oscillator, (pools of) neurons reciprocally coupled by fast/slow inhibitory and excitatory synapses, to produce either alternating bursting or other rhythmic patterns, characterized by different phase lags, depending on the sensory or other external input. We also show how to calibrate its parameters, based on both physiological and functional criteria and on bifurcation analysis. This model accounts for short-term neuromodulation in a biophysically plausible way and is a building block to develop more realistic and functionally accurate CPG models. Examples and counterexamples are used to point out the generality and effectiveness of our design approach.

Humanoid adaptive locomotion control through a bioinspired CPG-based controller

To achieve adaptive gait planning of humanoid robots, a hierarchical central pattern generator (H-CPG) model with a basic rhythmic signal generation layer and a pattern formation layer is proposed to modulate the center of mass (CoM) and the online foot trajectory. The entrainment property of the CPG is exploited for adaptive walking in the absence of a priori knowledge of walking conditions, and the sensory feedback is applied to modulate the generated trajectories online to improve walking adaptability and stability. The developed control strategy is verified using a humanoid robot on sloped terrain and shows good performance.

Development of Modular Bio-Inspired Autonomous Underwater Vehicle for Close Subsea Asset Inspection

To reduce human risk and maintenance costs, Autonomous Underwater Vehicles (AUVs) are involved in subsea inspections and measurements for a wide range of marine industries such as offshore wind farms and other underwater infrastructure. Most of these inspections may require levels of manoeuvrability similar to what can be achieved by tethered vehicles, called Remotely Operated Vehicles (ROVs). To extend AUV intervention time and perform closer inspection in constrained spaces, AUVs need to be more efficient and flexible by being able to undulate around physical constraints. A biomimetic fish-like AUV known as RoboFish has been designed to mimic propulsion techniques observed in nature to provide high thrust efficiency and agility to navigate its way autonomously around complex underwater structures. Building upon advances in acoustic communications, computer vision, electronics and autonomy technologies, RoboFish aims to provide a solution to such critical inspections. This paper introduces the first RoboFish prototype that comprises cost-effective 3D printed modules joined together with innovative magnetic coupling joints and a modular software framework. Initial testing shows that the preliminary working prototype is functional in terms of water-tightness, propulsion, body control and communication using acoustics, with visual localisation and mapping capability.

From Motor-Output to Connectivity: An In-Depth Study of in-vitro Rhythmic Patterns in the Cockroach Periplaneta americana

The cockroach is an established model in the study of locomotion control. While previous work has offered important insights into the interplay among brain commands, thoracic central pattern generators, and the sensory feedback that shapes their motor output, there remains a need for a detailed description of the central pattern generators' motor output and their underlying connectivity scheme. To this end, we monitored pilocarpine-induced activity of levator and depressor motoneurons in two types of novel in-vitro cockroach preparations: isolated thoracic ganglia and a whole-chain preparation comprising the thoracic ganglia and the subesophageal ganglion. Our data analyses focused on the motoneuron firing patterns and the coordination among motoneuron types in the network. The burstiness and rhythmicity of the motoneurons were monitored, and phase relations, coherence, coupling strength, and frequency-dependent variability were analyzed. These parameters were all measured and compared among network units both within each preparation and among the preparations. Here, we report differences among the isolated ganglia, including asymmetries in phase and coupling strength, which indicate that they are wired to serve different functions. We also describe the intrinsic default gait and a frequency-dependent coordination. The depressor motoneurons showed mostly similar characteristics throughout the network regardless of interganglia connectivity; whereas the characteristics of the levator motoneurons activity were mostly ganglion-dependent, and influenced by the presence of interganglia connectivity. Asymmetries were also found between the anterior and posterior homolog parts of the thoracic network, as well as between ascending and descending connections. Our analyses further discover a frequency-dependent inversion of the interganglia coordination from alternations between ipsilateral homolog oscillators to simultaneous activity. We present a detailed scheme of the network couplings, formulate coupling rules, and review a previously suggested model of connectivity in light of our new findings. Our data support the notion that the inter-hemiganglia coordination derives from the levator networks and their coupling with local depressor interneurons. Our findings also support a dominant role of the metathoracic ganglion and its ascending output in governing the anterior ganglia motor output during locomotion in the behaving animal.

Spiking neural state machine for gait frequency entrainment in a flexible modular robot

We propose a modular architecture for neuromorphic closed-loop control based on bistable relaxation oscillator modules consisting of three spiking neurons each. Like its biological prototypes, this basic component is robust to parameter variation but can be modulated by external inputs. By combining these modules, we can construct a neural state machine capable of generating the cyclic or repetitive behaviors necessary for legged locomotion. A concrete case study for the approach is provided by a modular robot constructed from flexible plastic volumetric pixels, in which we produce a forward crawling gait entrained to the natural frequency of the robot by a minimal system of twelve neurons organized into four modules.

Design Principles for Central Pattern Generators With Preset Rhythms

This article is concerned with the design of synthetic central pattern generators (CPGs). Biological CPGs are neural circuits that determine a variety of rhythmic activities, including locomotion, in animals. A synthetic CPG is a network of dynamical elements (here called cells) properly coupled by various synapses to emulate rhythms produced by a biological CPG. We focus on CPGs for locomotion of quadrupeds and present our design approach, based on the principles of nonlinear dynamics, bifurcation theory, and parameter optimization. This approach lets us design the synthetic CPG with a set of desired rhythms and switch between them as the parameter representing the control actions from the brain is varied. The developed four-cell CPG can produce four distinct gaits: walk, trot, gallop, and bound, similar to the mouse locomotion. The robustness and adaptability of the network design principles are verified using different cell and synapse models.

The effect of sensory feedback topology on the entrainment of a neural oscillator with a compliant foil for swimming systems

The sensorimotor system of fish endows them with remarkable swimming performance that is unmatched by current underwater robotic vehicles. To close the gap between the capabilities of fish and the capabilities of underwater vehicles engineers are investigating how fish swim. In particular, engineers are exploring the sensorimotor systems of fish that control the motion of fins. It is generally accepted that specialized neural circuits (known as central pattern generators) within the sensorimotor system produce the periodic drive signal that is used to control the motion of fins. An important aspect of these circuits is that their output signal can be modified by sensory feedback. Specifically, the way in which sensory feedback signals are applied to a CPG (i.e. the sensory feedback topology) affects the CPG's entrainment characteristics. This has been shown in simulation but has not been investigated in a robot interacting in the real-world. Furthermore, CPG-based control has only limitedly been applied to fish like robots and many questions remain as to how it should be applied to these types of systems. In this work we examine the effect of sensory feedback topology on the entrainment characteristics of a CPG-based neural oscillator driving three different foils swimming in flow. Additionally, we investigate how sensory feedback should be acquired from a foil and applied to a neural oscillator to promote beneficial swimming characteristics.

Dynamics and coupling of fractional-order models of the motor cortex and central pattern generators

OBJECTIVE

Fractional calculus plays a key role in the analysis of neural dynamics. In particular, fractional calculus has been recently exploited for analyzing complex biological systems and capturing intrinsic phenomena. Also, artificial neural networks have been shown to have complex neuronal dynamics and characteristics that can be modeled by fractional calculus. Moreover, for a neural microcircuit placed on the spinal cord, fractional calculus can be employed to model the central pattern generator (CPG). However, the relation between the CPG and the motor cortex is still unclear.

APPROACH

In this paper, fractional-order models of the CPG and the motor cortex are built on the Van der Pol oscillator and the neural mass model (NMM), respectively. A self-consistent mean field approximation is used to construct the potential landscape of the Van der Pol oscillator. This landscape provides a useful tool to observe the 3D dynamics of the oscillator. To infer the relation of the motor cortex and CPG, the coupling model between the fractional-order Van der Pol oscillator and the NMM is built. As well, the influence of the coupling parameters on the CPG and the motor cortex is assessed.

MAIN RESULTS

Fractional-order NMM and coupling model of the motor cortex and the CPG are first established. The potential landscape is used to show 3D probabilistic evolution of the Van der Pol oscillator states. Detailed observations of the evolution of the system states can be made with fractional calculus. In particular, fractional calculus enables the observation of the creation of stable modes and switching between them.

SIGNIFICANCE

The results confirm that the motor cortex and CPG have associated modes or states that can be switched based on changes in the fractional order and the time delay. Fractional calculus and the potential landscape are helpful methods for better understanding of the working principles of locomotion systems.

Error-Based Learning Mechanism for Fast Online Adaptation in Robot Motor Control

Existing state-of-the-art frequency adaptation mechanisms of central pattern generators (CPGs) for robot locomotion control typically rely on correlation-based learning. They do not account for the tracking error that may occur between the actual system motion and CPG output, leading to the loss of precision, unwanted movement, inefficient energy locomotion, and in the worst cases, motor collapse. To overcome this problem, we developed online error-based learning for frequency adaptation of CPGs. The learning mechanism used for error reduction is a novel modification of the dual learner (DL) called dual integral learner (DIL). Being able to reduce tracking and steady-state errors, it can also perform fast and stable learning, adapting the CPG frequency to match the performance of robotic systems. Control parameters of the DIL are more straightforward for complex systems (like walking robots), compared to traditional correlation-based learning, since they correspond to error reduction. Due to its embedded memory, the DIL can relearn quickly and recover spontaneously from the previously learned parameters. All these features are not covered by the existing frequency adaptation mechanisms. We integrated the DIL into a neural CPG-based motor control system for use on different legged robots with various morphologies for evaluation. The results show that: 1) the DIL does not require precise adjustment of its parameters to fit specific robots; and 2) the DIL can automatically and quickly adapt the CPG frequency to the robots such that the entire trajectory of the CPG can be precisely followed with very low tracking and steady-state errors. Consequently, the robots can perform the desired movements with more energy-efficient locomotion compared to the state-of-the-art correlation-based learning mechanism called frequency adaptation through fast dynamical coupling (AFDC). In the future, the proposed error-based learning mechanism for fast online adaptation in robot motor control can be used as a basis for trajectory optimization, universal controllers, and other studies concerning the change of intrinsic or extrinsic parameters.

CFD based parameter tuning for motion control of robotic fish

After millions of years of evolution, fishes have been endowed with agile swimming ability to accomplish various behaviourally relevant tasks. In comparison, robotic fish are still quite poor swimmers. One of the unique challenges facing robotic fish is the difficulty in tuning the motion control parameters on the robot directly. This is mainly due to the complex fluid environment robotic fish need to contend with and endurance limitations (i.e. battery capacity limitations). To overcome these limitations, we propose a computational fluid dynamics (CFD) simulation platform to first tune the motion control parameters for the computational robotic fish and then refine the parameters by experiments on robotic fish. Within the simulation platform, the body morphology and gait control of the computational robotic fish are designed according to a robotic fish. The gait control is implemented by a central pattern generator (CPG); The CFD model is solved by using a hydrodynamic-kinematics strong-coupling method. We tested our simulation platform with three basic tasks under active disturbance rejection control (ADRC) and try-and-error-based parameter tuning. Trajectory comparisons between the computational robotic fish and robotic fish verify the effectiveness of our simulation platform. Moreover, power costs and swimming efficiency under the motion control are also analyzed based on the outputs from the simulation platform. Our results indicate that the CFD based simulation platform is powerful and robust, and shed new light on the efficient design and parameter optimization of the motion control of robotic fish.

Matsuoka's CPG With Desired Rhythmic Signals for Adaptive Walking of Humanoid Robots

The desired rhythmic signals for adaptive walking of humanoid robots should have proper frequencies, phases, and shapes. Matsuoka's central pattern generator (CPG) is able to generate rhythmic signals with reasonable frequencies and phases, and thus has been widely applied to control the movements of legged robots, such as walking of humanoid robots. However, it is difficult for this kind of CPG to generate rhythmic signals with desired shapes, which limits the adaptability of walking of humanoid robots in various environments. To address this issue, a new framework that can generate desired rhythmic signals for Matsuoka's CPG is presented. The proposed framework includes three main parts. First, feature processing is conducted to transform the Matsuoka's CPG outputs into a normalized limit cycle. Second, by combining the normalized limit cycle with robot feedback as the feature inputs and setting the required learning objective, the neural network (NN) learns to generate desired rhythmic signals. Finally, in order to ensure the continuity of the desired rhythmic signals, signal filtering is applied to the outputs of NN, with the aim of smoothing the discontinuous parts. Numerical experiments on the proposed framework suggest that it can not only generate a variety of rhythmic signals with desired shapes but also preserve the frequency and phase properties of Matsuoka's CPG. In addition, the proposed framework is embedded into a control system for adaptive omnidirectional walking of humanoid robot NAO. Extensive simulation and real experiments on this control system demonstrate that the proposed framework is able to generate desired rhythmic signals for adaptive walking of NAO on fixed and changing inclined surfaces. Furthermore, the comparison studies verify that the proposed framework can significantly improve the adaptability of NAO's walking compared with the other methods.

Comparative study of forced oscillators for the adaptive generation of rhythmic movements in robot controllers

The interest of central pattern generators in robot motor coordination is universally recognized so much so that a lot of possibilities on different scales of modeling are nowadays available. While each method obviously has its advantages and drawbacks, some could be more suitable for human-robot interactions. In this paper, we compare three oscillator models: Matsuoka, Hopf and Rowat-Selverston models. These models are integrated to a control architecture for a robotic arm and evaluated in simulation during a simplified handshaking interaction which involves constrained rhythmic movements. Furthermore, Hebbian plasticity mechanisms are integrated to the Hopf and Rowat-Selverston models which can incorporate such mechanisms, contrary to the Matsuoka. Results show that the Matsuoka oscillator is subpar in all aspects and for the two others, that plasticity improves synchronization and leads to a significant decrease in the power consumption.

CPG-Based Gait Generation of the Curved-Leg Hexapod Robot with Smooth Gait Transition

This paper presents a novel CPG-based gait generation of the curved-leg hexapod robot that can enable smooth gait transitions between multi-mode gaits. First, the locomotion of the curved leg and instability during the gait transitions are analyzed. Then, a modified Hopf oscillator is applied in the CPG control, which can realize multiple gaits by adjusting a simple parameter. In addition, a smooth gait switching method is also proposed via smooth gait transition functions and gait planning. Tripod gait, quadruped gait, and wave gait are planned for the hexapod robot to achieve quick and stable gait transitions smoothly and continuously. MATLAB and ADAMS simulations and corresponding practical experiments are conducted. The results show that the proposed method can achieve smooth and continuous mutual gait transitions, which proves the effectiveness of the proposed CPG-based hexapod robot control.