Cardioid oscillator-based pattern generator for imitating the time-ratio-asymmetrical behavior of the lower limb exoskeleton

Introduction

Periodicity, self-excitation, and time ratio asymmetry are the fundamental characteristics of the human gait. In order to imitate these mentioned characteristics, a pattern generator with four degrees of freedom is proposed based on cardioid oscillators developed by the authors.

Method

The proposed pattern generator is composed of four coupled cardioid oscillators, which are self-excited and have asymmetric time ratios. These oscillators are connected with other oscillators through coupled factors. The dynamic behaviors of the proposed oscillators, such as phase locking, time ratio, and self-excitation, are analyzed via simulations by employing the harmonic balance method. Moreover, for comparison, the simulated trajectories are compared with the natural joint trajectories measured in experiments.

Results and discussion

Simulation and experimental results show that the behaviors of the proposed pattern generator are similar to those of the natural lower limb. It means the simulated trajectories from the generator are self-excited without any additional inputs and have asymmetric time ratios. Their phases are locked with others. Moreover, the proposed pattern generator can be applied as the reference model for the lower limb exoskeleton controlling algorithm to produce self-adjusted reference trajectories.

Identification of a Step-And-Brake Controller of a Human Based on Prediction of Capturability

An explicit mathematical form of a human’s step-and-brake controller is identified through motion measurement of the human subject. The controller was originally designed for biped robots based on the reduced-order dynamics and the model predictive control scheme with the terminal capturability condition, and is compatible with both stand-still and stepping motions. The minimal number of parameters facilitates the identification from measured trajectories of the center of mass and the zero-moment point of the human subject. In spite of the minimality, the result only suited the human’s behaviors well with slight modifications of the model by taking direction-dependency of the natural falling speed and the inertial torque about the center of mass into account. Furthermore, the parameters are successfully identified even from the first half of motion sequence, which means that the proposed method is available in designing on-the-fly systems to evaluate balancing abilities of humans and to assist balances of humans in walking.

Identification of COM Controller of a Human in Stance Based on Motion Measurement and Phase-Space Analysis

This article proposes a process to identify the standing stabilizer, namely, the controller in humans to keep upright posture stable against perturbations. We model the controller as a piecewise-linear feedback system, where the state of the center of mass (COM) is regulated by coordinating the whole body so as to locate the zero-moment point (ZMP) at the desired position. This was developed for humanoid robots and is possibly able to elaborate the fundamental control scheme used by humans to stabilize themselves. Difficulties lie on how to collect motion trajectories in a wide area of the state space for reliable identification and how to identify the piecewise-affine dynamical system. For the former problem, a motion measurement protocol is devised based on the theoretical phase portrait of the system. Regarding the latter problem, some clustering techniques including K-means method and EM (Expectation-and-Maximization) algorithm were examined. We found that a modified K-means method produced the most accurate result in this study. The method was applied to the identification of a lateral standing controller of a human subject. The result of the identification quantitatively supported a hypothesis that the COM-ZMP regulator reasonably models the human’s controller when deviations of the angular momentum about the COM are limited.

Gyro-Sensor-Based Vibration Control for Dynamic Humanoid-Robot Walking on Inclined Surfaces

An efficient motor-control system for stable walking of the lightweight humanoid robot KONDO KHR-3HV on inclined surfaces is investigated. The motor-control system is based on the angular velocity of the pitch motion of the robot, which is detected by a gyro sensor attached to the robot torso and referred to as the angular-pitch velocity. The robot gait is analyzed for different downslopes with and without the motor-feedback control. A novel method of frequency-domain analysis of the angular-pitch velocity is proposed for explaining the reasons behind the instabilities of dynamic humanoid-robot walking on inclined surfaces. The results show, that a nonlinear nature of the motor torque, due to a force induced by the slope, gives rise to harmonics of the fundamental walking frequency of 1.73 Hz. These harmonics are the origin of the unstable robot walking. Additionally, the feedback-gain parameters K_A and K_H affect the amplitudes of the harmonics, which give rise to vibrations at a higher surface inclination. Increased surface friction allows a reduction of the feedback gain, which reduces this specific contribution to the harmonics and thus stabilizes the robot. To improve the walking stability on inclined surfaces, it is found that the damped natural frequency of the motor-control system must be kept lower than the fundamental walking frequency.

Phase portraits as movement primitives for fast humanoid robot control

Currently, usual approaches for fast robot control are largely reliant on solving online optimal control problems. Such methods are known to be computationally intensive and sensitive to model accuracy. On the other hand, animals plan complex motor actions not only fast but seemingly with little effort even on unseen tasks. This natural sense to infer temporal dynamics and coordination motivates us to approach robot control from a motor skill learning perspective to design fast and computationally light controllers that can be learned autonomously by the robot under mild modeling assumptions. This article introduces Phase Portrait Movement Primitives (PPMP), a primitive that predicts dynamics on a low dimensional phase space which in turn is used to govern the high dimensional kinematics of the task. The stark difference with other primitive formulations is a built-in mechanism for phase prediction in the form of coupled oscillators that replaces model-based state estimators such as Kalman filters. The policy is trained by optimizing the parameters of the oscillators whose output is connected to a kinematic distribution in the form of a phase portrait. The drastic reduction in dimensionality allows us to efficiently train and execute PPMPs on a real human-sized, dual-arm humanoid upper body on a task involving 20 degrees-of-freedom. We demonstrate PPMPs in interactions requiring fast reactions times while generating anticipative pose adaptation in both discrete and cyclic tasks.

Q-Learning of Straightforward Gait Pattern for Humanoid Robot Based on Automatic Training Platform

In this paper, an oscillator-based gait pattern with sinusoidal functions is designed and implemented on a field-programmable gate array (FPGA) chip to generate a trajectory plan and achieve bipedal locomotion for a small-sized humanoid robot. In order to let the robot can walk straight, the turning direction is viewed as a parameter of the gait pattern and Q-learning is used to obtain a straightforward gait pattern. Moreover, an automatic training platform is designed so that the learning process is automated. In this way, the turning direction can be adjusted flexibly and efficiently under the supervision of the automatic training platform. The experimental results show that the proposed learning framework allows the humanoid robot to gradually walk straight in the automated learning process.

Walking with perturbations: a guide for biped humans and robots

This paper provides an update on the neural control of bipedal walking in relation to bioinspired models and robots. It is argued that most current models or robots are based on the construct of a symmetrical central pattern generator (CPG). However, new evidence suggests that CPG functioning is basically asymmetrical with its flexor half linked more tightly to the rhythm generator. The stability of bipedal gait, which is an important problem for robots and biological systems, is also addressed. While it is not possible to determine how biological biped systems guarantee stability, robot solutions can be useful to propose new hypotheses for biology. In the second part of this review, the focus is on gait perturbations, which is an important topic in robotics in view of the frequent falls of robots when faced with perturbations. From the human physiology it is known that the initial reaction often consists of a brief interruption followed by an adequate response. For instance, the successful recovery from a trip is achieved using some basic reactions (termed elevating and lowering strategies), that depend on the phase of the step cycle of the trip occurrence. Reactions to stepping unexpectedly in a hole depend on comparing expected and real feedback. Implementation of these ideas in models and robotics starts to emerge, with the most advanced robots being able to learn how to fall safely and how to deal with complicated disturbances such as provided by walking on a split-belt.

A neural network with central pattern generators entrained by sensory feedback controls walking of a bipedal model

A neuromechanical simulation of a planar, bipedal walking robot has been developed. It is constructed as a simplified, planar musculoskeletal model of the biomechanics of the human lower body. The controller consists of a dynamic neural network with central pattern generators (CPGs) entrained by force and movement sensory feedback to generate appropriate muscle forces for walking. The CPG model is a two-level architecture, which consists of separate rhythm generator and pattern formation networks. The biped model walks stably in the sagittal plane without inertial sensors or a centralized posture controller or a 'baby walker' to help overcome gravity. Its gait is similar to humans' and it walks at speeds from 0.850 m s^-1 up to 1.289 m s^-1 with leg length of 0.84 m. The model walks over small unknown steps (6% of leg length) and up and down 5° slopes without any additional higher level control actions.

A central pattern generator approach to footstep transition for biped navigation

Existing sampling-based footstep planning method for biped navigation used an intermediate static posture for footstep transition. However, when adopting this approach, the robot is sensitive to modeling error and external environments, and also the transition between different gait patterns is unnatural. This article presents a central pattern generator approach to footstep transition for biped navigation. First, this approach decomposes the biped walking motion into five motion types and designs central pattern generator network for all joints of legs accordingly. Then, the central pattern generator parameters are simplified and the relationship between these parameters and footstep transition is formulated. By modifying the central pattern generator parameters, different walking gaits can be obtained. With sensing feedbacks, self-adaption walking on irregular terrains, such as walking on unknown sloped terrains and flat floor with tiny obstacles, is realized. Experiments were conducted both in simulator and on a physical biped robot. Results have shown that the proposed approach is able to generate gesture transition trajectory for biped robot navigation and realize a self-adaption walking for irregular terrains.

Spatiotemporal synchronization of biped walking patterns with multiple external inputs by style-phase adaptation

In this paper, we propose a framework for generating coordinated periodic movements of robotic systems with multiple external inputs. We developed an adaptive pattern generator model that is composed of a two-factor observation model with a style parameter and phase dynamics with a phase variable. The style parameter controls the spatial patterns of the generated trajectories, and the phase variable manages its temporal profiles. By exploiting the style-phase separation in the pattern generation, we can independently design adaptation schemes for the spatial and temporal profiles of the pattern generator to multiple external inputs. To validate the effectiveness of our proposed method, we applied it to a user-exoskeleton model to achieve user-adaptive walking assistance for which the exoskeleton robot's movements need to be coordinated with the user walking patterns and environment. As a result, the exoskeleton robot successfully performed stable biped walking behaviors for walking assistance even when the style of the observed walking pattern and the period were suddenly changed.

Creating the brain and interacting with the brain: an integrated approach to understanding the brain

In the past two decades, brain science and robotics have made gigantic advances in their own fields, and their interactions have generated several interdisciplinary research fields. First, in the 'understanding the brain by creating the brain' approach, computational neuroscience models have been applied to many robotics problems. Second, such brain-motivated fields as cognitive robotics and developmental robotics have emerged as interdisciplinary areas among robotics, neuroscience and cognitive science with special emphasis on humanoid robots. Third, in brain-machine interface research, a brain and a robot are mutually connected within a closed loop. In this paper, we review the theoretical backgrounds of these three interdisciplinary fields and their recent progress. Then, we introduce recent efforts to reintegrate these research fields into a coherent perspective and propose a new direction that integrates brain science and robotics where the decoding of information from the brain, robot control based on the decoded information and multimodal feedback to the brain from the robot are carried out in real time and in a closed loop.

Robust synchronization of coupled neural oscillators using the derivative-free nonlinear Kalman Filter

A synchronizing control scheme for coupled neural oscillators of the FitzHugh-Nagumo type is proposed. Using differential flatness theory the dynamical model of two coupled neural oscillators is transformed into an equivalent model in the linear canonical (Brunovsky) form. A similar linearized description is succeeded using differential geometry methods and the computation of Lie derivatives. For such a model it becomes possible to design a state feedback controller that assures the synchronization of the membrane's voltage variations for the two neurons. To compensate for disturbances that affect the neurons' model as well as for parametric uncertainties and variations a disturbance observer is designed based on Kalman Filtering. This consists of implementation of the standard Kalman Filter recursion on the linearized equivalent model of the coupled neurons and computation of state and disturbance estimates using the diffeomorphism (relations about state variables transformation) provided by differential flatness theory. After estimating the disturbance terms in the neurons' model their compensation becomes possible. The performance of the synchronization control loop is tested through simulation experiments.

Real-time Walking Pattern Generation for a Biped Robot with Hybrid CPG-ZMP Algorithm

Biped robots have better mobility than conventional wheeled robots. The bio-inspired method based on a central pattern generator (CPG) can be used to control biped robot walking in a manner like human beings. However, to achieve stable locomotion, it is difficult to modulate the parameters for the neural networks to coordinate every degree of freedom of the walking robot. The zero moment point (ZMP) method is very popular for the stability control of biped robot walking. However, the reference trajectories have low energy efficiency, lack naturalness and need significant offline calculation. This paper presents a new method for biped real-time walking generation using a hybrid CPG-ZMP control algorithm. The method can realize a stable walking pattern by combining the ZMP criterion with rhythmic motion control. The CPG component is designed to generate the desired motion for each robot joint, which is modulated by phase resetting according to foot contact information. By introducing the ZMP location, the activity of the CPG output signal is adjusted to coordinate the limbs’ motion and allow the robot to maintain balance during the process of locomotion. The numerical simulation results show that, compared with the CPG method, the new hybrid CPG-ZMP algorithm can enhance the robustness of the CPG parameters and improve the stability of the robot. In addition, the proposed algorithm is more energy efficient than the ZMP method. The results also demonstrate that the control system can generate an adaptive walking pattern through interactions between the robot, the CPG and the environment.

Visually guided gait modifications for stepping over an obstacle: a bio-inspired approach

There is an increasing interest in conceiving robotic systems that are able to move and act in an unstructured and not predefined environment, for which autonomy and adaptability are crucial features. In nature, animals are autonomous biological systems, which often serve as bio-inspiration models, not only for their physical and mechanical properties, but also their control structures that enable adaptability and autonomy-for which learning is (at least) partially responsible. This work proposes a system which seeks to enable a quadruped robot to online learn to detect and to avoid stumbling on an obstacle in its path. The detection relies in a forward internal model that estimates the robot's perceptive information by exploring the locomotion repetitive nature. The system adapts the locomotion in order to place the robot optimally before attempting to step over the obstacle, avoiding any stumbling. Locomotion adaptation is achieved by changing control parameters of a central pattern generator (CPG)-based locomotion controller. The mechanism learns the necessary alterations to the stride length in order to adapt the locomotion by changing the required CPG parameter. Both learning tasks occur online and together define a sensorimotor map, which enables the robot to learn to step over the obstacle in its path. Simulation results show the feasibility of the proposed approach.

Qualitative adaptive reward learning with success failure maps: applied to humanoid robot walking

In the human brain, rewards are encoded in a flexible and adaptive way after each novel stimulus. Neurons of the orbitofrontal cortex are the key reward structure of the brain. Neurobiological studies show that the anterior cingulate cortex of the brain is primarily responsible for avoiding repeated mistakes. According to vigilance threshold, which denotes the tolerance to risks, we can differentiate between a learning mechanism that takes risks and one that averts risks. The tolerance to risk plays an important role in such a learning mechanism. Results have shown the differences in learning capacity between risk-taking and risk-avert behaviors. These neurological properties provide promising inspirations for robot learning based on rewards. In this paper, we propose a learning mechanism that is able to learn from negative and positive feedback with reward coding adaptively. It is composed of two phases: evaluation and decision making. In the evaluation phase, we use a Kohonen self-organizing map technique to represent success and failure. Decision making is based on an early warning mechanism that enables avoiding repeating past mistakes. The behavior to risk is modulated in order to gain experiences for success and for failure. Success map is learned with adaptive reward that qualifies the learned task in order to optimize the efficiency. Our approach is presented with an implementation on the NAO humanoid robot, controlled by a bioinspired neural controller based on a central pattern generator. The learning system adapts the oscillation frequency and the motor neuron gain in pitch and roll in order to walk on flat and sloped terrain, and to switch between them.

On-line frequency adaptation and movement imitation for rhythmic robotic tasks

In this paper we present a novel method to obtain the basic frequency of an unknown periodic signal with an arbitrary waveform, which can work online with no additional signal processing or logical operations. The method originates from non-linear dynamical systems for frequency extraction, which are based on adaptive frequency oscillators in a feedback loop. In previous work, we had developed a method that could extract separate frequency components by using several adaptive frequency oscillators in a loop, but that method required a logical algorithm to identify the basic frequency. The novel method presented here uses a Fourier series representation in the feedback loop combined with a single oscillator. In this way it can extract the frequency and the phase of an unknown periodic signal in real time and without any additional signal processing or preprocessing. The method determines the Fourier series coefficients and can be used for dynamic Fourier series implementation. The proposed method can be used for the control of rhythmic robotic tasks, where only the extraction of the basic frequency is crucial. For demonstration several highly non-linear and dynamic periodic robotic tasks are shown, including also a task where an electromyography (EMG) signal is used in a feedback loop.

A simple walking strategy for biped walking based on an intermittent sinusoidal oscillator

This paper presents a control algorithm for biped walking by extension of previous work in the fields of central pattern generator (CPG) and passive walking. The algorithm takes advantage of the passive dynamics of walking, assisting only when necessary with an intermittent sinusoidal oscillator. The parameterized oscillator is used to drive the hip joint; the triggering and ceasing of the oscillator during a walking cycle can be modulated by the sensory feedback. The results from simulation indicate a stable, efficient gait, and robustness against model inaccuracy and environmental variation. We also examine the effects of oscillator parameters and link parameters on the gait, and design a controller to suppress the bifurcation phenomenon based on the error of prior step periods.

Biological inspiration used for robots motion synthesis

This work presents a biologically inspired method of gait generation. Bipedal gait pattern (for hip and knee joints) was taken into account giving the reference trajectories in a learning task. The four coupled oscillators were taught to generate the outputs similar to those in a human gait. After applying the correction functions the obtained generation method was validated using ZMP criterion. The formula suitable for real-time motion generation taking into account the positioning errors was also formulated. The small real robot prototype was tested to be able walk successfully following the elaborated motion pattern.