SofaGym: An Open Platform for Reinforcement Learning Based on Soft Robot Simulations

Learning-Based Control for Soft Robot-Environment Interaction with Force/Position Tracking Capability

Soft robotics promises to achieve safe and efficient interactions with the environment by exploiting its inherent compliance and designing control strategies. However, effective control for the soft robot-environment interaction has been a challenging task. The challenges arise from the nonlinearity and complexity of soft robot dynamics, especially in situations where the environment is unknown and uncertainties exist, making it difficult to establish analytical models. In this study, we propose a learning-based optimal control approach as an attempt to address these challenges, which is an optimized combination of a feedforward controller based on probabilistic model predictive control and a feedback controller based on nonparametric learning methods. The approach is purely data-driven, without prior knowledge of soft robot dynamics and environment structures, and can be easily updated online to adapt to unknown environments. A theoretical analysis of the approach is provided to ensure its stability and convergence. The proposed approach enabled a soft robotic manipulator to track target positions and forces when interacting with a manikin in different cases. Moreover, comparisons with other data-driven control methods show a better performance of our approach. Overall, this work provides a viable learning-based control approach for soft robot-environment interactions with force/position tracking capability.

Exploring Embodied Intelligence in Soft Robotics: A Review

Soft robotics is closely related to embodied intelligence in the joint exploration of the means to achieve more natural and effective robotic behaviors via physical forms and intelligent interactions. Embodied intelligence emphasizes that intelligence is affected by the synergy of the brain, body, and environment, focusing on the interaction between agents and the environment. Under this framework, the design and control strategies of soft robotics depend on their physical forms and material properties, as well as algorithms and data processing, which enable them to interact with the environment in a natural and adaptable manner. At present, embodied intelligence has comprehensively integrated related research results on the evolution, learning, perception, decision making in the field of intelligent algorithms, as well as on the behaviors and controls in the field of robotics. From this perspective, the relevant branches of the embodied intelligence in the context of soft robotics were studied, covering the computation of embodied morphology; the evolution of embodied AI; and the perception, control, and decision making of soft robotics. Moreover, on this basis, important research progress was summarized, and related scientific problems were discussed. This study can provide a reference for the research of embodied intelligence in the context of soft robotics.

Soft DAgger: Sample-Efficient Imitation Learning for Control of Soft Robots

This paper presents Soft DAgger, an efficient imitation learning-based approach for training control solutions for soft robots. To demonstrate the effectiveness of the proposed algorithm, we implement it on a two-module soft robotic arm involved in the task of writing letters in 3D space. Soft DAgger uses a dynamic behavioral map of the soft robot, which maps the robot's task space to its actuation space. The map acts as a teacher and is responsible for predicting the optimal actions for the soft robot based on its previous state action history, expert demonstrations, and current position. This algorithm achieves generalization ability without depending on costly exploration techniques or reinforcement learning-based synthetic agents. We propose two variants of the control algorithm and demonstrate that good generalization capabilities and improved task reproducibility can be achieved, along with a consistent decrease in the optimization time and samples. Overall, Soft DAgger provides a practical control solution to perform complex tasks in fewer samples with soft robots. To the best of our knowledge, our study is an initial exploration of imitation learning with online optimization for soft robot control.