Self-organized adaptive legged locomotion in a compliant quadruped robot

Self-learning physical reservoir computer

A self-learning physical reservoir computer is demonstrated using an adaptive oscillator. Whereas physical reservoir computing repurposes the dynamics of a physical system for computation through machine learning, adaptive oscillators can innately learn and store information in plastic dynamic states. The adaptive state(s) can be used directly as physical node(s), but these plastic states can also be used to self-learn the optimal reservoir parameters for more complex tasks requiring virtual nodes from the base oscillator. Both this self-learning property for reconfigurable computing and the morphable logic gate property of the adaptive oscillator make this an ideal candidate for a multipurpose neuromorphic processor.

A change in behavioral state switches the pattern of motor output that underlies rhythmic head and orofacial movements

The breathing rhythm serves as a reference that paces orofacial motor actions and orchestrates active sensing. Past work has reported that pacing occurs solely at a fixed phase relative to sniffing. We re-evaluated this constraint as a function of exploratory behavior. Allocentric and egocentric rotations of the head and the electromyogenic activity of the motoneurons for head and orofacial movements were recorded in free-ranging rats as they searched for food. We found that a change in state from foraging to rearing is accompanied by a large phase shift in muscular activation relative to sniffing, and a concurrent change in the frequency of sniffing, so that pacing now occurs at one of the two phases. Further, head turning is biased such that an animal gathers a novel sample of its environment upon inhalation. In total, the coordination of active sensing has a previously unrealized computational complexity. This can emerge from hindbrain circuits with fixed architecture and credible synaptic time delays.

Robust and reusable self-organized locomotion of legged robots under adaptive physical and neural communications

Introduction

Animals such as cattle can achieve versatile and elegant behaviors through automatic sensorimotor coordination. Their self-organized movements convey an impression of adaptability, robustness, and motor memory. However, the adaptive mechanisms underlying such natural abilities of these animals have not been completely realized in artificial legged systems.

Methods

Hence, we propose adaptive neural control that can mimic these abilities through adaptive physical and neural communications. The control algorithm consists of distributed local central pattern generator (CPG)-based neural circuits for generating basic leg movements, an adaptive sensory feedback mechanism for generating self-organized phase relationships among the local CPG circuits, and an adaptive neural coupling mechanism for transferring and storing the formed phase relationships (a gait pattern) into the neural structure. The adaptive neural control was evaluated in experiments using a quadruped robot.

Results

The adaptive neural control enabled the robot to 1) rapidly and automatically form its gait (i.e., self-organized locomotion) within a few seconds, 2) memorize the gait for later recovery, and 3) robustly walk, even when a sensory feedback malfunction occurs. It also enabled maneuverability, with the robot being able to change its walking speed and direction. Moreover, implementing adaptive physical and neural communications provided an opportunity for understanding the mechanism of motor memory formation.

Discussion

Overall, this study demonstrates that the integration of the two forms of communications through adaptive neural control is a powerful way to achieve robust and reusable self-organized locomotion in legged robots.

Learning plastic matching of robot dynamics in closed-loop central pattern generators

Animals achieve agile locomotion performance with reduced control effort and energy efficiency by leveraging compliance in their muscles and tendons. However, it is not known how biological locomotion controllers learn to leverage the intelligence embodied in their leg mechanics. Here we present a framework to match control patterns and mechanics based on the concept of short-term elasticity and long-term plasticity. Inspired by animals, we design a robot, Morti, with passive elastic legs. The quadruped robot Morti is controlled by a bioinspired closed-loop central pattern generator that is designed to elastically mitigate short-term perturbations using sparse contact feedback. By minimizing the amount of corrective feedback on the long term, Morti learns to match the controller to its mechanics and learns to walk within 1 h. By leveraging the advantages of its mechanics, Morti improves its energy efficiency by 42% without explicit minimization in the cost function.

Crab-inspired compliant leg design method for adaptive locomotion of a multi-legged robot

Chinese mitten crabhas unique limb structures composed of a hard exoskeleton and flexible muscles. They enable the crab to locomote adaptively and safely on various terrains. In this work, we investigated the limb structures, motion principle, and gaits of the crab using a high-speed camera and a press machine. Then, a novel compliant robot leg design method is proposed, inspired by the crab limb. The leg comprises six hard scleromeres and a flexible thin-wall spring steel sheet (FSSS) mimicking the exoskeleton and muscle. The scleromeres connected one by one with rotational joints are designed with slots. The front end of the FSSS is fixed on the scleromere close to the ground. The rear end crosses the slots and is mounted at the shaft of a linear actuator installed at the rear scleromere. The leg bends and stretches when the actuator pushes and pulls the FSSS, respectively. The kinematic modeling, rigid-flexible coupling dynamic simulations, and leg prototype tests are conducted, which verify the leg design approach. Thirdly, we put forward a multi-legged robot with eight compliant legs and design its gait using the gaits of the crab. Finally, the robot's performance is evaluated, including the capabilities of walking on different terrains at adjustable speeds and body heights, traversing low channels, walking on slopes, and carrying loads. The results prove that the single-motor-actuated compliant legs and their dynamic coupling with the rigid robot body frame can enable them to have the ground clearance ability and realize the adaptive walking of the robot. The leg design methodology can be used to design multi-legged robots with the merits of compact, light, low mechanical complexity, high safety, and easy to control, for many applications, such as environmental monitoring, search and rescue.

A Quadruped Robot with Three-Dimensional Flexible Legs

As an important part of the quadruped robot, the leg determines its performance. Flexible legs or flexible joints aid in the buffering and adaptability of robots. At present, most flexible quadruped robots only have two-dimensional flexibility or use complex parallel structures to achieve three-dimensional flexibility. This research will propose a new type of three-dimensional flexible structure. This passive compliant three-dimensional flexibility reduces the weight and complex structure of the robot. The anti-impact performance of the robot is verified by a side impact experiment. The simulation and experiments show that the robot still has good stability even under a simple algorithm and that the flexible leg can reduce the impact on the quadruped robot and improve the environmental adaptability of the robot.

A four-state adaptive Hopf oscillator

Adaptive oscillators (AOs) are nonlinear oscillators with plastic states that encode information. Here, an analog implementation of a four-state adaptive oscillator, including design, fabrication, and verification through hardware measurement, is presented. The result is an oscillator that can learn the frequency and amplitude of an external stimulus over a large range. Notably, the adaptive oscillator learns parameters of external stimuli through its ability to completely synchronize without using any pre- or post-processing methods. Previously, Hopf oscillators have been built as two-state (a regular Hopf oscillator) and three-state (a Hopf oscillator with adaptive frequency) systems via VLSI and FPGA designs. Building on these important implementations, a continuous-time, analog circuit implementation of a Hopf oscillator with adaptive frequency and amplitude is achieved. The hardware measurements and SPICE simulation show good agreement. To demonstrate some of its functionality, the circuit’s response to several complex waveforms, including the response of a square wave, a sawtooth wave, strain gauge data of an impact of a nonlinear beam, and audio data of a noisy microphone recording, are reported. By learning both the frequency and amplitude, this circuit could be used to enhance applications of AOs for robotic gait, clock oscillators, analog frequency analyzers, and energy harvesting.

Structural Design and Crawling Pattern Generator of a Planar Quadruped Robot for High-Payload Locomotion

Load capacity is an important index to reflect the practicability of legged robots. Existing research into quadruped robots has not analyzed their load performance in terms of their structural design and control method from a systematic point of view. This paper proposes a structural design method and crawling pattern generator for a planar quadruped robot that can realize high-payload locomotion. First, the functions required to evaluate the leg's load capacity are established, and quantitative comparative analyses of the candidates are performed to select the leg structure with the best load capacity. We also propose a highly integrated design method for a driver module to improve the robot's load capacity. Second, in order to realize stable load locomotion, a novel crawling pattern generator based on trunk swaying is proposed which can realize lateral center of mass (CoM) movement by adjusting the leg lengths on both sides to change the CoM projection in the trunk width direction. Finally, loaded crawling simulations and experiments performed with our self-developed quadruped robot show that stable crawling with load ratios exceeding 66% can be realized, thus verifying the effectiveness and superiority of the proposed method.

Comparative study of forced oscillators for the adaptive generation of rhythmic movements in robot controllers

The interest of central pattern generators in robot motor coordination is universally recognized so much so that a lot of possibilities on different scales of modeling are nowadays available. While each method obviously has its advantages and drawbacks, some could be more suitable for human-robot interactions. In this paper, we compare three oscillator models: Matsuoka, Hopf and Rowat-Selverston models. These models are integrated to a control architecture for a robotic arm and evaluated in simulation during a simplified handshaking interaction which involves constrained rhythmic movements. Furthermore, Hebbian plasticity mechanisms are integrated to the Hopf and Rowat-Selverston models which can incorporate such mechanisms, contrary to the Matsuoka. Results show that the Matsuoka oscillator is subpar in all aspects and for the two others, that plasticity improves synchronization and leads to a significant decrease in the power consumption.

Model framework for emergence of synchronized oscillations

Autonomy is an important concept when investigating the mechanism whereby biological systems exhibit flexibility against unpredictable environmental changes. Herein we propose a parameter-tuning algorithm, based on a selection principle, that allows the emergence of synchronization between populations of oscillators through autonomous changes of the intrinsic parameters. With the algorithm, the populations exhibit self-recovery of the synchronized state after the existing synchronized state is broken suddenly; that is, the system chooses appropriate values of the intrinsic parameters to recover the synchronized state. We also propose a continuous model in which the selection is described by the replicator model and the parameter values are determined by the density profile of the oscillators in parameter space.

Development of an 8DOF quadruped robot and implementation of Inverse Kinematics using Denavit-Hartenberg convention

Quadruped robots can mimic animal walking gait and they have certain advantages like walking on terrain and extremely rough surfaces. Obstacles can impede the movement of wheeled vehicles, where a quadruped can adapt to avoid obstacles by adjusting its height. A quadruped robot is designed and developed for in this paper, which could be controlled by the Android operating system. The Inverse Kinematics Solutions are derived for the developed structure using Denavit-Hartenberg convention and using those solutions the movements are simulated using a custom-made 3D software. An Android application is developed, which is able to control the robot using Bluetooth. The robot currently has following six different movements: front, back, left, right walking, clockwise and anti-clockwise rotation. The robot uses the ultrasound sensor to detect any obstacle closer than 300 cm (maximum) and if an impediment appears, the robot will automatically move parallel to the obstacle until it is avoided. Currently, it can move at a speed of 15.5 cm/s (approximately). To complete a full rotation of 360°, it takes 6 seconds. It can be used to develop and implement any autonomous path-planning algorithm.

Adaptive Control Strategies for Interlimb Coordination in Legged Robots: A Review

Walking animals produce adaptive interlimb coordination during locomotion in accordance with their situation. Interlimb coordination is generated through the dynamic interactions of the neural system, the musculoskeletal system, and the environment, although the underlying mechanisms remain unclear. Recently, investigations of the adaptation mechanisms of living beings have attracted attention, and bio-inspired control systems based on neurophysiological findings regarding sensorimotor interactions are being developed for legged robots. In this review, we introduce adaptive interlimb coordination for legged robots induced by various factors (locomotion speed, environmental situation, body properties, and task). In addition, we show characteristic properties of adaptive interlimb coordination, such as gait hysteresis and different time-scale adaptations. We also discuss the underlying mechanisms and control strategies to achieve adaptive interlimb coordination and the design principle for the control system of legged robots.

Self-entrainment to optimal gaits of an underactuated biomimetic swimming robot using adaptive frequency oscillators

Underactuated compliant swimming robots are characterized by a simple mechanical structure, capable to mimic the body undulation of many fish species. One of the design issue for these robots is the generation and control of best performing swimming gaits. In this paper we propose a new controller, based on AFO oscillators, to address this issue. After analyzing the effects of the motion on the robot natural frequencies, we show that the closed loop system is able to generate self-sustained oscillations, at a characteristic frequency, while maximizing swimming velocity.

Neuromodulation and Synaptic Plasticity for the Control of Fast Periodic Movement: Energy Efficiency in Coupled Compliant Joints via PCA

There are multiple indications that the nervous system of animals tunes muscle output to exploit natural dynamics of the elastic locomotor system and the environment. This is an advantageous strategy especially in fast periodic movements, since the elastic elements store energy and increase energy efficiency and movement speed. Experimental evidence suggests that coordination among joints involves proprioceptive input and neuromodulatory influence originating in the brain stem. However, the neural strategies underlying the coordination of fast periodic movements remain poorly understood. Based on robotics control theory, we suggest that the nervous system implements a mechanism to accomplish coordination between joints by a linear coordinate transformation from the multi-dimensional space representing proprioceptive input at the joint level into a one-dimensional controller space. In this one-dimensional subspace, the movements of a whole limb can be driven by a single oscillating unit as simple as a reflex interneuron. The output of the oscillating unit is transformed back to joint space via the same transformation. The transformation weights correspond to the dominant principal component of the movement. In this study, we propose a biologically plausible neural network to exemplify that the central nervous system (CNS) may encode our controller design. Using theoretical considerations and computer simulations, we demonstrate that spike-timing-dependent plasticity (STDP) for the input mapping and serotonergic neuromodulation for the output mapping can extract the dominant principal component of sensory signals. Our simulations show that our network can reliably control mechanical systems of different complexity and increase the energy efficiency of ongoing cyclic movements. The proposed network is simple and consistent with previous biologic experiments. Thus, our controller could serve as a candidate to describe the neural control of fast, energy-efficient, periodic movements involving multiple coupled joints.

Self-Organizing Map With Time-Varying Structure to Plan and Control Artificial Locomotion

This paper presents an algorithm, self-organizing map-state trajectory generator (SOM-STG), to plan and control legged robot locomotion. The SOM-STG is based on an SOM with a time-varying structure characterized by constructing autonomously close-state trajectories from an arbitrary number of robot postures. Each trajectory represents a cyclical movement of the limbs of an animal. The SOM-STG was designed to possess important features of a central pattern generator, such as rhythmic pattern generation, synchronization between limbs, and swapping between gaits following a single command. The acquisition of data for SOM-STG is based on learning by demonstration in which the data are obtained from different demonstrator agents. The SOM-STG can construct one or more gaits for a simulated robot with six legs, can control the robot with any of the gaits learned, and can smoothly swap gaits. In addition, SOM-STG can learn to construct a state trajectory form observing an animal in locomotion. In this paper, a dog is the demonstrator agent.

A Novel Design of a Quadruped Robot for Research Purposes

This paper presents the design of a novel quadruped robot. The proposed design is characterized by a simple, modular design, and easy interfacing capabilities. The robot is built mostly from off-the-shelf components. The design includes four 3-DOF legs, the robot body and its electronics. The proposed robot is able to traverse rough terrain while carrying additional payloads. Such payloads can include both sensors and computational hardware. We present the robot design, the control system, and the forward and inverse kinematics of the robot, as well as experiments that are compared with simulation results.

A survey on CPG-inspired control models and system implementation

This paper surveys the developments of the last 20 years in the field of central pattern generator (CPG) inspired locomotion control, with particular emphasis on the fast emerging robotics-related applications. Functioning as a biological neural network, CPGs can be considered as a group of coupled neurons that generate rhythmic signals without sensory feedback; however, sensory feedback is needed to shape the CPG signals. The basic idea in engineering endeavors is to replicate this intrinsic, computationally efficient, distributed control mechanism for multiple articulated joints, or multi-DOF control cases. In terms of various abstraction levels, existing CPG control models and their extensions are reviewed with a focus on the relative advantages and disadvantages of the models, including ease of design and implementation. The main issues arising from design, optimization, and implementation of the CPG-based control as well as possible alternatives are further discussed, with an attempt to shed more light on locomotion control-oriented theories and applications. The design challenges and trends associated with the further advancement of this area are also summarized.

Visually guided gait modifications for stepping over an obstacle: a bio-inspired approach

There is an increasing interest in conceiving robotic systems that are able to move and act in an unstructured and not predefined environment, for which autonomy and adaptability are crucial features. In nature, animals are autonomous biological systems, which often serve as bio-inspiration models, not only for their physical and mechanical properties, but also their control structures that enable adaptability and autonomy-for which learning is (at least) partially responsible. This work proposes a system which seeks to enable a quadruped robot to online learn to detect and to avoid stumbling on an obstacle in its path. The detection relies in a forward internal model that estimates the robot's perceptive information by exploring the locomotion repetitive nature. The system adapts the locomotion in order to place the robot optimally before attempting to step over the obstacle, avoiding any stumbling. Locomotion adaptation is achieved by changing control parameters of a central pattern generator (CPG)-based locomotion controller. The mechanism learns the necessary alterations to the stride length in order to adapt the locomotion by changing the required CPG parameter. Both learning tasks occur online and together define a sensorimotor map, which enables the robot to learn to step over the obstacle in its path. Simulation results show the feasibility of the proposed approach.

Globally stable control of a dynamic bipedal walker using adaptive frequency oscillators

We present a control method for a simple limit-cycle bipedal walker that uses adaptive frequency oscillators (AFOs) to generate stable gaits. Existence of stable limit cycles is demonstrated with an inverted-pendulum model. This model predicts a proportional relationship between hip torque amplitude and stride frequency. The closed-loop walking control incorporates adaptive Fourier analysis to generate a uniform oscillator phase. Gait solutions (fixed points) are predicted via linearization of the walker model, and employed as initial conditions to generate exact solutions via simulation. Global stability is determined via a recursive algorithm that generates the approximate basin of attraction of a fixed point. We also present an initial study on the implementation of AFO-based control on a bipedal walker with realistic mass distribution and articulated knee joints.

Multigait soft robot

This manuscript describes a unique class of locomotive robot: A soft robot, composed exclusively of soft materials (elastomeric polymers), which is inspired by animals (e.g., squid, starfish, worms) that do not have hard internal skeletons. Soft lithography was used to fabricate a pneumatically actuated robot capable of sophisticated locomotion (e.g., fluid movement of limbs and multiple gaits). This robot is quadrupedal; it uses no sensors, only five actuators, and a simple pneumatic valving system that operates at low pressures (< 10 psi). A combination of crawling and undulation gaits allowed this robot to navigate a difficult obstacle. This demonstration illustrates an advantage of soft robotics: They are systems in which simple types of actuation produce complex motion.

The Stanford LittleDog: A learning and rapid replanning approach to quadruped locomotion

Legged robots have the potential to navigate a wide variety of terrain that is inaccessible to wheeled vehicles. In this paper we consider the planning and control tasks of navigating a quadruped robot over challenging terrain, including terrain that it has not seen until run-time. We present a software architecture that makes use of both static and dynamic gaits, as well as specialized dynamic maneuvers, to accomplish this task. Throughout the paper we highlight two themes that have been central to our approach: (1) the prevalent use of learning algorithms, and (2) a focus on rapid recovery and replanning techniques; we present several novel methods and algorithms that we developed for the quadruped and that illustrate these two themes. We evaluate the performance of these different methods, and also present and discuss the performance of our system on the official Learning Locomotion tests.