Learning quadrupedal locomotion on deformable terrain

Multimodal locomotion ultra-thin soft robots for exploration of narrow spaces

From power plants on land to bridges over the sea, safety-critical built environments require periodic inspections for detecting issues to avoid functional discontinuities of these installations. However, navigation paths in these environments are usually challenging as they often contain difficult-to-access spaces (near-millimetre and submillimetre-high gaps) and multiple domains (solid, liquid and even aerial). In this paper, we address these challenges by developing a class of Thin Soft Robots (TS-Robot: thickness, 1.7 mm) that can access narrow spaces and perform cross-domain multimodal locomotion. We adopted a dual-actuation sandwich structure with a tuneable Poisson's ratio tensioning mechanism for developing the TS-Robots driven by dielectric elastomers, providing them with two types of gaits (linear and undulating), remarkable output force ( ~ 41 times their weight) and speed (1.16 times Body Length/s and 13.06 times Body Thickness/s). Here, we demonstrated that TS-Robots can crawl, climb, swim and collaborate for transitioning between domains in environments with narrow entries.

Adaptive Gait Acquisition through Learning Dynamic Stimulus Instinct of Bipedal Robot

Standard alternating leg motions serve as the foundation for simple bipedal gaits, and the effectiveness of the fixed stimulus signal has been proved in recent studies. However, in order to address perturbations and imbalances, robots require more dynamic gaits. In this paper, we introduce dynamic stimulus signals together with a bipedal locomotion policy into reinforcement learning (RL). Through the learned stimulus frequency policy, we induce the bipedal robot to obtain both three-dimensional (3D) locomotion and an adaptive gait under disturbance without relying on an explicit and model-based gait in both the training stage and deployment. In addition, a set of specialized reward functions focusing on reliable frequency reflections is used in our framework to ensure correspondence between locomotion features and the dynamic stimulus. Moreover, we demonstrate efficient sim-to-real transfer, making a bipedal robot called BITeno achieve robust locomotion and disturbance resistance, even in extreme situations of foot sliding in the real world. In detail, under a sudden change in torso velocity of -1.2 m/s in 0.65 s, the recovery time is within 1.5-2.0 s.

Learning agile soccer skills for a bipedal robot with deep reinforcement learning

We investigated whether deep reinforcement learning (deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies. We used deep RL to train a humanoid robot to play a simplified one-versus-one soccer game. The resulting agent exhibits robust and dynamic movement skills, such as rapid fall recovery, walking, turning, and kicking, and it transitions between them in a smooth and efficient manner. It also learned to anticipate ball movements and block opponent shots. The agent's tactical behavior adapts to specific game contexts in a way that would be impractical to manually design. Our agent was trained in simulation and transferred to real robots zero-shot. A combination of sufficiently high-frequency control, targeted dynamics randomization, and perturbations during training enabled good-quality transfer. In experiments, the agent walked 181% faster, turned 302% faster, took 63% less time to get up, and kicked a ball 34% faster than a scripted baseline.

Optimization method for human-robot command combinations of hexapod robot based on multi-objective constraints

Due to the heavy burden on human drivers when remotely controlling hexapod robots in complex terrain environments, there is a critical need for robot intelligence to assist in generating control commands. Therefore, this study proposes a mapping process framework that generates a combination of human-robot commands based on decision target values, focusing on the task of robot intelligence assisting drivers in generating human-robot command combinations. Furthermore, human-robot state constraints are quantified as geometric constraints on robot motion and driver fatigue constraints. By optimizing and filtering the feasible set of human-robot commands based on human-robot state constraints, instruction combinations are formed and recommended to the driver in real-time, thereby enhancing the efficiency and safety of human-machine coordination. To validate the effectiveness of the proposed method, a remote human-robot collaborative driving control system based on wearable devices is designed and implemented. Experimental results demonstrate that drivers utilizing the human-robot command recommendation system exhibit significantly improved robot walking stability and reduced collision rates compared to individual driving.

Learning Quadrupedal High-Speed Running on Uneven Terrain

Reinforcement learning (RL)-based controllers have been applied to the high-speed movement of quadruped robots on uneven terrains. The external disturbances increase as the robot moves faster on such terrains, affecting the stability of the robot. Many existing RL-based methods adopt higher control frequencies to respond quickly to the disturbance, which requires a significant computational cost. We propose a control framework that consists of an RL-based control policy updating at a low frequency and a model-based joint controller updating at a high frequency. Unlike previous methods, our policy outputs the control law for each joint, executed by the corresponding high-frequency joint controller to reduce the impact of external disturbances on the robot. We evaluated our method on various simulated terrains with height differences of up to 6 cm. We achieved a running motion of 1.8 m/s in the simulation using the Unitree A1 quadruped. The RL-based control policy updates at 50 Hz with a latency of 20 ms, while the model-based joint controller runs at 1000 Hz. The experimental results show that the proposed framework can overcome the latency caused by low-frequency updates, making it applicable for real-robot deployment.