Fast Dynamical Coupling Enhances Frequency Adaptation of Oscillators for Robotic Locomotion Control

Hybrid learning mechanisms under a neural control network for various walking speed generation of a quadruped robot

Legged robots that can instantly change motor patterns at different walking speeds are useful and can accomplish various tasks efficiently. However, state-of-the-art control methods either are difficult to develop or require long training times. In this study, we present a comprehensible neural control framework to integrate probability-based black-box optimization (PIBB) and supervised learning for robot motor pattern generation at various walking speeds. The control framework structure is based on a combination of a central pattern generator (CPG), a radial basis function (RBF) -based premotor network and a hypernetwork, resulting in a so-called neural CPG-RBF-hyper control network. First, the CPG-driven RBF network, acting as a complex motor pattern generator, was trained to learn policies (multiple motor patterns) for different speeds using PIBB. We also introduce an incremental learning strategy to avoid local optima. Second, the hypernetwork, which acts as a task/behavior to control parameter mapping, was trained using supervised learning. It creates a mapping between the internal CPG frequency (reflecting the walking speed) and motor behavior. This map represents the prior knowledge of the robot, which contains the optimal motor joint patterns at various CPG frequencies. Finally, when a user-defined robot walking frequency or speed is provided, the hypernetwork generates the corresponding policy for the CPG-RBF network. The result is a versatile locomotion controller which enables a quadruped robot to perform stable and robust walking at different speeds without sensory feedback. The policy of the controller was trained in the simulation (less than 1 h) and capable of transferring to a real robot. The generalization ability of the controller was demonstrated by testing the CPG frequencies that were not encountered during training.

Frictional Anisotropic Locomotion and Adaptive Neural Control for a Soft Crawling Robot

Crawling animals with bendable soft bodies use the friction anisotropy of their asymmetric body structures to traverse various substrates efficiently. Although the effect of friction anisotropy has been investigated and applied to robot locomotion, the dynamic interactions between soft body bending at different frequencies (low and high), soft asymmetric surface structures at various aspect ratios (low, medium, and high), and different substrates (rough and smooth) have not been studied comprehensively. To address this lack, we developed a simple soft robot model with a bioinspired asymmetric structure (sawtooth) facing the ground. The robot uses only a single source of pressure for its pneumatic actuation. The frequency, teeth aspect ratio, and substrate parameters and the corresponding dynamic interactions were systematically investigated and analyzed. The study findings indicate that the anterior and posterior parts of the structure deform differently during the interaction, generating different frictional forces. In addition, these parts switched their roles dynamically from push to pull and vice versa in various states, resulting in the robot's emergent locomotion. Finally, autonomous adaptive crawling behavior of the robot was demonstrated using sensor-driven neural control with a miniature laser sensor installed in the anterior part of the robot. The robot successfully adapted its actuation frequency to reduce body bending and crawl through a narrow space, such as a tunnel. The study serves as a stepping stone for developing simple soft crawling robots capable of navigating cluttered and confined spaces autonomously.

A virtuous cycle between invertebrate and robotics research: perspective on a decade of Living Machines research

Many invertebrates are ideal model systems on which to base robot design principles due to their success in solving seemingly complex tasks across domains while possessing smaller nervous systems than vertebrates. Three areas are particularly relevant for robot designers: Research on flying and crawling invertebrates has inspired new materials and geometries from which robot bodies (their morphologies) can be constructed, enabling a new generation of softer, smaller, and lighter robots. Research on walking insects has informed the design of new systems for controlling robot bodies (their motion control) and adapting their motion to their environment without costly computational methods. And research combining wet and computational neuroscience with robotic validation methods has revealed the structure and function of core circuits in the insect brain responsible for the navigation and swarming capabilities (their mental faculties) displayed by foraging insects. The last decade has seen significant progress in the application of principles extracted from invertebrates, as well as the application of biomimetic robots to model and better understand how animals function. This Perspectives paper on the past 10 years of the Living Machines conference outlines some of the most exciting recent advances in each of these fields before outlining lessons gleaned and the outlook for the next decade of invertebrate robotic research.

Field-programmable analog array (FPAA) based four-state adaptive oscillator for analog frequency analysis

Adaptive oscillators are a subset of nonlinear oscillators that can learn and encode information in dynamic states. By appending additional states onto a classical Hopf oscillator, a four-state adaptive oscillator is created that can learn both the frequency and amplitude of an external forcing frequency. Analog circuit implementations of nonlinear differential systems are usually achieved by using operational amplifier-based integrator networks, in which redesign procedures of the system topology is time consuming. Here, an analog implementation of a four-state adaptive oscillator is presented for the first time as a field-programmable analog array (FPAA) circuit. The FPAA diagram is described, and the hardware performance is presented. This simple FPAA-based oscillator can be used as an analog frequency analyzer, as its frequency state will evolve to match the external forcing frequency. Notably, this is done without any analog-to-digital conversion or pre-processing, making it an ideal frequency analyzer for low-power and low-memory applications.

Reliable estimation of internal oscillator properties from a novel, fast-paced tapping paradigm

Rhythmic structure in speech, music, and other auditory signals helps us track, anticipate, and understand the sounds in our environment. The dynamic attending framework proposes that biological systems possess internal rhythms, generated via oscillatory mechanisms, that synchronize with (entrain to) rhythms in the external world. Here, we focused on two properties of internal oscillators: preferred rate, the default rate of an oscillator in the absence of any input, and flexibility, the oscillator's ability to adapt to changes in external rhythmic context. We aimed to develop methods that can reliably estimate preferred rate and flexibility on an individual basis. The experiment was a synchronization-continuation finger tapping paradigm with a unique design: the stimulus rates were finely sampled over a wide range of rates and were presented only once. Individuals tapped their finger to 5-event isochronous stimulus sequences and continued the rhythm at the same pace. Preferred rate was estimated by assessing the best-performance conditions where the difference between the stimulus rate and continuation tapping rate (tempo-matching error) was minimum. The results revealed harmonically related, multiple preferred rates for each individual. We maximized the differences in stimulus rate between consecutive trials to challenge individuals' flexibility, which was then estimated by how much tempo-matching errors in synchronization tapping increase with this manipulation. Both measures showed test-retest reliability. The findings demonstrate the influence of properties of the auditory context on rhythmic entrainment, and have implications for development of methods that can improve attentional synchronization and hearing.

Locomotion Control With Frequency and Motor Pattern Adaptations

Existing adaptive locomotion control mechanisms for legged robots are usually aimed at one specific type of adaptation and rarely combined with others. Adaptive mechanisms thus stay at a conceptual level without their coupling effect with other mechanisms being investigated. However, we hypothesize that the combination of adaptation mechanisms can be exploited for enhanced and more efficient locomotion control as in biological systems. Therefore, in this work, we present a central pattern generator (CPG) based locomotion controller integrating both a frequency and motor pattern adaptation mechanisms. We use the state-of-the-art Dual Integral Learner for frequency adaptation, which can automatically and quickly adapt the CPG frequency, enabling the entire motor pattern or output signal of the CPG to be followed at a proper high frequency with low tracking error. Consequently, the legged robot can move with high energy efficiency and perform the generated locomotion with high precision. The versatile state-of-the-art CPG-RBF network is used as a motor pattern adaptation mechanism. Using this network, the motor patterns or joint trajectories can be adapted to fit the robot's morphology and perform sensorimotor integration enabling online motor pattern adaptation based on sensory feedback. The results show that the two adaptation mechanisms can be combined for adaptive locomotion control of a hexapod robot in a complex environment. Using the CPG-RBF network for motor pattern adaptation, the hexapod learned basic straight forward walking, steering, and step climbing. In general, the frequency and motor pattern mechanisms complement each other well and their combination can be seen as an essential step toward further studies on adaptive locomotion control.

Insect-Inspired Robots: Bridging Biological and Artificial Systems

This review article aims to address common research questions in hexapod robotics. How can we build intelligent autonomous hexapod robots that can exploit their biomechanics, morphology, and computational systems, to achieve autonomy, adaptability, and energy efficiency comparable to small living creatures, such as insects? Are insects good models for building such intelligent hexapod robots because they are the only animals with six legs? This review article is divided into three main sections to address these questions, as well as to assist roboticists in identifying relevant and future directions in the field of hexapod robotics over the next decade. After an introduction in section (1), the sections will respectively cover the following three key areas: (2) biomechanics focused on the design of smart legs; (3) locomotion control; and (4) high-level cognition control. These interconnected and interdependent areas are all crucial to improving the level of performance of hexapod robotics in terms of energy efficiency, terrain adaptability, autonomy, and operational range. We will also discuss how the next generation of bioroboticists will be able to transfer knowledge from biology to robotics and vice versa.

Generic Neural Locomotion Control Framework for Legged Robots

In this article, we present a generic locomotion control framework for legged robots and a strategy for control policy optimization. The framework is based on neural control and black-box optimization. The neural control combines a central pattern generator (CPG) and a radial basis function (RBF) network to create a CPG-RBF network. The control network acts as a neural basis to produce arbitrary rhythmic trajectories for the joints of robots. The main features of the CPG-RBF network are: 1) it is generic since it can be applied to legged robots with different morphologies; 2) it has few control parameters, resulting in fast learning; 3) it is scalable, both in terms of policy/trajectory complexity and the number of legs that can be controlled using similar trajectories; 4) it does not rely heavily on sensory feedback to generate locomotion and is thus less prone to sensory faults; and 5) once trained, it is simple, minimal, and intuitive to use and analyze. These features will lead to an easy-to-use framework with fast convergence and the ability to encode complex locomotion control policies. In this work, we show that the framework can successfully be applied to three different simulated legged robots with varying morphologies and, even broken joints, to learn locomotion control policies. We also show that after learning, the control policies can also be successfully transferred to a real-world robot without any modifications. We, furthermore, show the scalability of the framework by implementing it as a central controller for all legs of a robot and as a decentralized controller for individual legs and leg pairs. By investigating the correlation between robot morphology and encoding type, we are able to present a strategy for control policy optimization. Finally, we show how sensory feedback can be integrated into the CPG-RBF network to enable online adaptation.

Gait Phase Estimation Based on User–Walker Interaction Force

Smart walkers have been developed for assistance and rehabilitation of elderly people and patients with physical health conditions. A force sensor mounted under the handle is widely used in smart walkers to establish a human–machine interface. The interaction force can be used to control the walker and estimate gait parameters using methods such as the Kalman filter for real-time estimation. However, the estimation performance decreases when the peaks of the interaction force are not captured. To improve the stability and accuracy of gait parameter estimation, we propose an online estimation method to continuously estimate the gait phase and cadence. A multiple model switching mechanism is introduced to improve the estimation performance when gait is asymmetric, and an adaptive rule is proposed to improve the estimation robustness and accuracy. Simulations and experiments demonstrate the effectiveness and accuracy of the proposed gait parameter estimation method. Here, the average estimation error for the gait phase is 0.691 rad when the gait is symmetric and 0.722 rad when it is asymmetric.

Adaptive and Energy-Efficient Optimal Control in CPGs Through Tegotae-Based Feedback

To obtain biologically inspired robotic control, the architecture of central pattern generators (CPGs) has been extensively adopted to generate periodic patterns for locomotor control. This is attributed to the interesting properties of nonlinear oscillators. Although sensory feedback in CPGs is not necessary for the generation of patterns, it plays a central role in guaranteeing adaptivity to environmental conditions. Nonetheless, its inclusion significantly modifies the dynamics of the CPG architecture, which often leads to bifurcations. For instance, the force feedback can be exploited to derive information regarding the state of the system. In particular, the Tegotae approach can be adopted by coupling proprioceptive information with the state of the oscillation itself in the CPG model. This paper discusses this policy with respect to other types of feedback; it provides higher adaptivity and an optimal energy efficiency for reflex-like actuation. We believe this is the first attempt to analyse the optimal energy efficiency along with the adaptivity of the Tegotae approach.

A four-state adaptive Hopf oscillator

Adaptive oscillators (AOs) are nonlinear oscillators with plastic states that encode information. Here, an analog implementation of a four-state adaptive oscillator, including design, fabrication, and verification through hardware measurement, is presented. The result is an oscillator that can learn the frequency and amplitude of an external stimulus over a large range. Notably, the adaptive oscillator learns parameters of external stimuli through its ability to completely synchronize without using any pre- or post-processing methods. Previously, Hopf oscillators have been built as two-state (a regular Hopf oscillator) and three-state (a Hopf oscillator with adaptive frequency) systems via VLSI and FPGA designs. Building on these important implementations, a continuous-time, analog circuit implementation of a Hopf oscillator with adaptive frequency and amplitude is achieved. The hardware measurements and SPICE simulation show good agreement. To demonstrate some of its functionality, the circuit’s response to several complex waveforms, including the response of a square wave, a sawtooth wave, strain gauge data of an impact of a nonlinear beam, and audio data of a noisy microphone recording, are reported. By learning both the frequency and amplitude, this circuit could be used to enhance applications of AOs for robotic gait, clock oscillators, analog frequency analyzers, and energy harvesting.

Error-Based Learning Mechanism for Fast Online Adaptation in Robot Motor Control

Existing state-of-the-art frequency adaptation mechanisms of central pattern generators (CPGs) for robot locomotion control typically rely on correlation-based learning. They do not account for the tracking error that may occur between the actual system motion and CPG output, leading to the loss of precision, unwanted movement, inefficient energy locomotion, and in the worst cases, motor collapse. To overcome this problem, we developed online error-based learning for frequency adaptation of CPGs. The learning mechanism used for error reduction is a novel modification of the dual learner (DL) called dual integral learner (DIL). Being able to reduce tracking and steady-state errors, it can also perform fast and stable learning, adapting the CPG frequency to match the performance of robotic systems. Control parameters of the DIL are more straightforward for complex systems (like walking robots), compared to traditional correlation-based learning, since they correspond to error reduction. Due to its embedded memory, the DIL can relearn quickly and recover spontaneously from the previously learned parameters. All these features are not covered by the existing frequency adaptation mechanisms. We integrated the DIL into a neural CPG-based motor control system for use on different legged robots with various morphologies for evaluation. The results show that: 1) the DIL does not require precise adjustment of its parameters to fit specific robots; and 2) the DIL can automatically and quickly adapt the CPG frequency to the robots such that the entire trajectory of the CPG can be precisely followed with very low tracking and steady-state errors. Consequently, the robots can perform the desired movements with more energy-efficient locomotion compared to the state-of-the-art correlation-based learning mechanism called frequency adaptation through fast dynamical coupling (AFDC). In the future, the proposed error-based learning mechanism for fast online adaptation in robot motor control can be used as a basis for trajectory optimization, universal controllers, and other studies concerning the change of intrinsic or extrinsic parameters.

Synchronization of Non-linear Oscillators for Neurobiologically Inspired Control on a Bionic Parallel Waist of Legged Robot

Synchronization of coupled non-linear oscillators inspired by a central pattern generator (CPG) can control the bionic robot and promote the coordination and diversity of locomotion. However, for a robot with a strong mutual coupled structure, such neurobiological control is still missing. In this contribution, we present a σ-Hopf harmonic oscillator with decoupled parameters to expand the solution space of the locomotion of the robot. Unlike the synchronization of original Hopf oscillators, which has been fully discussed, the asymmetric factor of σ-Hopf oscillator causes a deformation in oscillation waveform. Using the non-linear synchronization theory, we construct the transition state model of the synchronization process to analyze the asymmetrical distortion, period change and duty ratio inconsistency. Then a variable coupling strength is introduced to eliminate the waveform deformation and maintain the fast convergence rate. Finally, the approach is used for the locomotion control of a bionic parallel waist of legged robot, which is a highly coupled system. The effectiveness of the approach in both independent and synthesis behavior of four typical motion patterns are validated. The result proves the importance of controllability of the oscillation waveform and the instantaneous state of the synchronization, which benefits the transition and transformation of the locomotion and makes the coupling motion more flexible.

A Review on the Control of the Mechanical Properties of Ankle Foot Orthosis for Gait Assistance

In the past decade, advanced technologies in robotics have been explored to enhance the rehabilitation of post-stroke patients. Previous works have shown that gait assistance for post-stroke patients can be provided through the use of robotics technology in ancillary equipment, such as Ankle Foot Orthosis (AFO). An AFO is usually used to assist patients with spasticity or foot drop problems. There are several types of AFOs, depending on the flexibility of the joint, such as rigid, flexible rigid, and articulated AFOs. A rigid AFO has a fixed joint, and a flexible rigid AFO has a more flexible joint, while the articulated AFO has a freely rotating ankle joint, where the mechanical properties of the AFO are more controllable compared to the other two types of AFOs. This paper reviews the control of the mechanical properties of existing AFOs for gait assistance in post-stroke patients. Several aspects that affect the control of the mechanical properties of an AFO, such as the controller input, number of gait phases, controller output reference, and controller performance evaluation are discussed and compared. Thus, this paper will be of interest to AFO researchers or developers who would like to design their own AFOs with the most suitable mechanical properties based on their application. The controller input and the number of gait phases are discussed first. Then, the discussion moves forward to the methods of estimating the controller output reference, which is the main focus of this study. Based on the estimation method, the gait control strategies can be classified into subject-oriented estimations and phase-oriented estimations. Finally, suggestions for future studies are addressed, one of which is the application of the adaptive controller output reference to maximize the benefits of the AFO to users.

Hebbian Plasticity in CPG Controllers Facilitates Self-Synchronization for Human-Robot Handshaking

It is well-known that human social interactions generate synchrony phenomena which are often unconscious. If the interaction between individuals is based on rhythmic movements, synchronized and coordinated movements will emerge from the social synchrony. This paper proposes a plausible model of plastic neural controllers that allows the emergence of synchronized movements in physical and rhythmical interactions. The controller is designed with central pattern generators (CPG) based on rhythmic Rowat-Selverston neurons endowed with neuronal and synaptic Hebbian plasticity. To demonstrate the interest of the proposed model, the case of handshaking is considered because it is a very common, both physically and socially, but also, a very complex act in the point of view of robotics, neuroscience and psychology. Plastic CPGs controllers are implemented in the joints of a simulated robotic arm that has to learn the frequency and amplitude of an external force applied to its effector, thus reproducing the act of handshaking with a human. Results show that the neural and synaptic Hebbian plasticity are working together leading to a natural and autonomous synchronization between the arm and the external force even if the frequency is changing during the movement. Moreover, a power consumption analysis shows that, by offering emergence of synchronized and coordinated movements, the plasticity mechanisms lead to a significant decrease in the energy spend by the robot actuators thus generating a more adaptive and natural human/robot handshake.

Adaptive Control Strategies for Interlimb Coordination in Legged Robots: A Review

Walking animals produce adaptive interlimb coordination during locomotion in accordance with their situation. Interlimb coordination is generated through the dynamic interactions of the neural system, the musculoskeletal system, and the environment, although the underlying mechanisms remain unclear. Recently, investigations of the adaptation mechanisms of living beings have attracted attention, and bio-inspired control systems based on neurophysiological findings regarding sensorimotor interactions are being developed for legged robots. In this review, we introduce adaptive interlimb coordination for legged robots induced by various factors (locomotion speed, environmental situation, body properties, and task). In addition, we show characteristic properties of adaptive interlimb coordination, such as gait hysteresis and different time-scale adaptations. We also discuss the underlying mechanisms and control strategies to achieve adaptive interlimb coordination and the design principle for the control system of legged robots.