Error-Based Learning Mechanism for Fast Online Adaptation in Robot Motor Control

Memristive Circuit Implementation of Caenorhabditis Elegans Mechanism for Neuromorphic Computing

To overcome the energy efficiency bottleneck of the von Neumann architecture and scaling limit of silicon transistors, an emerging but promising solution is neuromorphic computing, a new computing paradigm inspired by how biological neural networks handle the massive amount of information in a parallel and efficient way. Recently, there is a surge of interest in the nematode worm Caenorhabditis elegans (C. elegans), an ideal model organism to probe the mechanisms of biological neural networks. In this article, we propose a neuron model for C. elegans with leaky integrate-and-fire (LIF) dynamics and adjustable integration time. We utilize these neurons to build the C. elegans neural network according to their neural physiology, which comprises: 1) sensory modules; 2) interneuron modules; and 3) motoneuron modules. Leveraging these block designs, we develop a serpentine robot system, which mimics the locomotion behavior of C. elegans upon external stimulus. Moreover, experimental results of C. elegans neurons presented in this article reveals the robustness (1% error w.r.t. 10% random noise) and flexibility of our design in term of parameter setting. The work paves the way for future intelligent systems by mimicking the C. elegans neural system.

FOESO-Net: A specific neural network for fast sensorless robot manipulator torque estimation

Contact torque sensing allows robot manipulators to cooperate with humans and detect accidental collisions in real time to ensure safety. Most sensorless torque estimation schemes, which are based on linear observer approaches, cannot compromise between non-negligible noise and high observation bandwidth. Therefore, fast time-varying nonlinear torque observation cannot be satisfied. To achieve this challenge, a customized network called FOESO-Net based on a novel fractional-order extended state observer is carefully designed in this paper. The network firstly chooses momentum as the benchmark state for torque estimation, which can avoid joint acceleration and model's inverse inertia matrix solution. Then, a fractional-order extended state observer (FOESO) is proposed from the perspective of momentum control to better adapt to the nonlinear fast time varying torque. In addition, a fractional-order neural network and a weight update neural network parallel architecture are constructed to enable fractional-order and dynamic weight-based adaptive learning of FOESO parameters. Formal analysis and proofs are made to show that the error of FOESO-Net is convergent. Finally, the effectiveness of the proposed method is verified by numerical simulations and a real collaborative robot platform. Moreover, compared with existing methods, the FOESO-Net based torque estimation method can reduce the estimation error and response time, which illustrates the superiority of the designed method.

Improvement of Reinforcement Learning With Supermodularity

Reinforcement learning (RL) is a promising approach to tackling learning and decision-making problems in a dynamic environment. Most studies on RL focus on the improvement of state evaluation or action evaluation. In this article, we investigate how to reduce action space by using supermodularity. We consider the decision tasks in the multistage decision process as a collection of parameterized optimization problems, where state parameters dynamically vary along with the time or stage. The optimal solutions of these parameterized optimization problems correspond to the optimal actions in RL. For a given Markov decision process (MDP) with supermodularity, the monotonicity of the optimal action set and the optimal selection with respect to state parameters can be obtained by using the monotone comparative statics. Accordingly, we propose a monotonicity cut to remove unpromising actions from the action space. Taking bin packing problem (BPP) as an example, we show how the supermodularity and monotonicity cut work in RL. Finally, we evaluate the monotonicity cut on the benchmark datasets reported in the literature and compare the proposed RL with some popular baseline algorithms. The results show that the monotonicity cut can effectively improve the performance of RL.

Frictional Anisotropic Locomotion and Adaptive Neural Control for a Soft Crawling Robot

Crawling animals with bendable soft bodies use the friction anisotropy of their asymmetric body structures to traverse various substrates efficiently. Although the effect of friction anisotropy has been investigated and applied to robot locomotion, the dynamic interactions between soft body bending at different frequencies (low and high), soft asymmetric surface structures at various aspect ratios (low, medium, and high), and different substrates (rough and smooth) have not been studied comprehensively. To address this lack, we developed a simple soft robot model with a bioinspired asymmetric structure (sawtooth) facing the ground. The robot uses only a single source of pressure for its pneumatic actuation. The frequency, teeth aspect ratio, and substrate parameters and the corresponding dynamic interactions were systematically investigated and analyzed. The study findings indicate that the anterior and posterior parts of the structure deform differently during the interaction, generating different frictional forces. In addition, these parts switched their roles dynamically from push to pull and vice versa in various states, resulting in the robot's emergent locomotion. Finally, autonomous adaptive crawling behavior of the robot was demonstrated using sensor-driven neural control with a miniature laser sensor installed in the anterior part of the robot. The robot successfully adapted its actuation frequency to reduce body bending and crawl through a narrow space, such as a tunnel. The study serves as a stepping stone for developing simple soft crawling robots capable of navigating cluttered and confined spaces autonomously.

Locomotion Control With Frequency and Motor Pattern Adaptations

Existing adaptive locomotion control mechanisms for legged robots are usually aimed at one specific type of adaptation and rarely combined with others. Adaptive mechanisms thus stay at a conceptual level without their coupling effect with other mechanisms being investigated. However, we hypothesize that the combination of adaptation mechanisms can be exploited for enhanced and more efficient locomotion control as in biological systems. Therefore, in this work, we present a central pattern generator (CPG) based locomotion controller integrating both a frequency and motor pattern adaptation mechanisms. We use the state-of-the-art Dual Integral Learner for frequency adaptation, which can automatically and quickly adapt the CPG frequency, enabling the entire motor pattern or output signal of the CPG to be followed at a proper high frequency with low tracking error. Consequently, the legged robot can move with high energy efficiency and perform the generated locomotion with high precision. The versatile state-of-the-art CPG-RBF network is used as a motor pattern adaptation mechanism. Using this network, the motor patterns or joint trajectories can be adapted to fit the robot's morphology and perform sensorimotor integration enabling online motor pattern adaptation based on sensory feedback. The results show that the two adaptation mechanisms can be combined for adaptive locomotion control of a hexapod robot in a complex environment. Using the CPG-RBF network for motor pattern adaptation, the hexapod learned basic straight forward walking, steering, and step climbing. In general, the frequency and motor pattern mechanisms complement each other well and their combination can be seen as an essential step toward further studies on adaptive locomotion control.

Insect-Inspired Robots: Bridging Biological and Artificial Systems

This review article aims to address common research questions in hexapod robotics. How can we build intelligent autonomous hexapod robots that can exploit their biomechanics, morphology, and computational systems, to achieve autonomy, adaptability, and energy efficiency comparable to small living creatures, such as insects? Are insects good models for building such intelligent hexapod robots because they are the only animals with six legs? This review article is divided into three main sections to address these questions, as well as to assist roboticists in identifying relevant and future directions in the field of hexapod robotics over the next decade. After an introduction in section (1), the sections will respectively cover the following three key areas: (2) biomechanics focused on the design of smart legs; (3) locomotion control; and (4) high-level cognition control. These interconnected and interdependent areas are all crucial to improving the level of performance of hexapod robotics in terms of energy efficiency, terrain adaptability, autonomy, and operational range. We will also discuss how the next generation of bioroboticists will be able to transfer knowledge from biology to robotics and vice versa.

Generic Neural Locomotion Control Framework for Legged Robots

In this article, we present a generic locomotion control framework for legged robots and a strategy for control policy optimization. The framework is based on neural control and black-box optimization. The neural control combines a central pattern generator (CPG) and a radial basis function (RBF) network to create a CPG-RBF network. The control network acts as a neural basis to produce arbitrary rhythmic trajectories for the joints of robots. The main features of the CPG-RBF network are: 1) it is generic since it can be applied to legged robots with different morphologies; 2) it has few control parameters, resulting in fast learning; 3) it is scalable, both in terms of policy/trajectory complexity and the number of legs that can be controlled using similar trajectories; 4) it does not rely heavily on sensory feedback to generate locomotion and is thus less prone to sensory faults; and 5) once trained, it is simple, minimal, and intuitive to use and analyze. These features will lead to an easy-to-use framework with fast convergence and the ability to encode complex locomotion control policies. In this work, we show that the framework can successfully be applied to three different simulated legged robots with varying morphologies and, even broken joints, to learn locomotion control policies. We also show that after learning, the control policies can also be successfully transferred to a real-world robot without any modifications. We, furthermore, show the scalability of the framework by implementing it as a central controller for all legs of a robot and as a decentralized controller for individual legs and leg pairs. By investigating the correlation between robot morphology and encoding type, we are able to present a strategy for control policy optimization. Finally, we show how sensory feedback can be integrated into the CPG-RBF network to enable online adaptation.

Distributed-force-feedback-based reflex with online learning for adaptive quadruped motor control

Biological motor control mechanisms (e.g., central pattern generators (CPGs), sensory feedback, reflexes, and motor learning) play a crucial role in the adaptive locomotion of animals. However, the interaction and integration of these mechanisms - necessary for generating the efficient, adaptive locomotion responses of legged robots to diverse terrains - have not yet been fully realized. One issue is that of achieving adaptive motor control for fast postural adaptation across various terrains. To address this issue, this study proposes a novel distributed-force-feedback-based reflex with online learning (DFRL). It integrates force-sensory feedback, reflexes, and learning to cooperate with CPGs in producing adaptive motor commands. The DFRL is based on a simple neural network that uses plastic synapses modulated online by a fast dual integral learner. Experimental results on different quadruped robots show that the DFRL can (1) automatically and rapidly adapt the CPG patterns (motor commands) of the robots, enabling them to realize appropriate body postures during locomotion and (2) enable the robots to effectively accommodate themselves to various slope terrains, including steep ones. Consequently, the DFRL-controlled robots can achieve efficient adaptive locomotion, to tackle complex terrains with diverse slopes.