Learning of Sub-optimal Gait Controllers for Magnetic Walking Soft Millirobots

Learning Soft Millirobot Multimodal Locomotion with Sim-to-Real Transfer

With wireless multimodal locomotion capabilities, magnetic soft millirobots have emerged as potential minimally invasive medical robotic platforms. Due to their diverse shape programming capability, they can generate various locomotion modes, and their locomotion can be adapted to different environments by controlling the external magnetic field signal. Existing adaptation methods, however, are based on hand-tuned signals. Here, a learning-based adaptive magnetic soft millirobot multimodal locomotion framework empowered by sim-to-real transfer is presented. Developing a data-driven magnetic soft millirobot simulation environment, the periodic magnetic actuation signal is learned for a given soft millirobot in simulation. Then, the learned locomotion strategy is deployed to the real world using Bayesian optimization and Gaussian processes. Finally, automated domain recognition and locomotion adaptation for unknown environments using a Kullback-Leibler divergence-based probabilistic method are illustrated. This method can enable soft millirobot locomotion to quickly and continuously adapt to environmental changes and explore the actuation space for unanticipated solutions with minimum experimental cost.

Design and build of small-scale magnetic soft-bodied robots with multimodal locomotion

Small-scale magnetic soft-bodied robots can be designed to operate based on different locomotion modes to navigate and function inside unstructured, confined and varying environments. These soft millirobots may be useful for medical applications where the robots are tasked with moving inside the human body. Here we cover the entire process of developing small-scale magnetic soft-bodied millirobots with multimodal locomotion capability, including robot design, material preparation, robot fabrication, locomotion control and locomotion optimization. We describe in detail the design, fabrication and control of a sheet-shaped soft millirobot with 12 different locomotion modes for traversing different terrains, an ephyra jellyfish-inspired soft millirobot that can manipulate objects in liquids through various swimming modes, a larval zebrafish-inspired soft millirobot that can adjust its body stiffness for efficient propulsion in different swimming speeds and a dual stimuli-responsive sheet-shaped soft millirobot that can switch its locomotion modes automatically by responding to changes in the environmental temperature. The procedure is aimed at users with basic expertise in soft robot development. The procedure requires from a few days to several weeks to complete, depending on the degree of characterization required.

Learning-based intelligent trajectory planning for auto navigation of magnetic robots

Introduction: Electromagnetically controlled small-scale robots show great potential in precise diagnosis, targeted delivery, and minimally invasive surgery. The automatic navigation of such robots could reduce human intervention, as well as the risk and difficulty of surgery. However, it is challenging to build a precise kinematics model for automatic robotic control because the controlling process is affected by various delays and complex environments. Method: Here, we propose a learning-based intelligent trajectory planning strategy for automatic navigation of magnetic robots without kinematics modeling. The Long Short-Term Memory (LSTM) neural network is employed to establish a global mapping relationship between the current sequence in the electromagnetic actuation system and the trajectory coordinates. Result: We manually control the robot to move on a curved path 50 times to form the training database to train the LSTM network. The trained LSTM network is validated to output the current sequence for automatically controlling the magnetic robot to move on the same curved path and the tortuous and branched new paths in simulated vascular tracks. Discussion: The proposed trajectory planning strategy is expected to impact the clinical applications of robots.

Control of Magnetic Surgical Robots With Model-Based Simulators and Reinforcement Learning

Magnetically manipulated medical robots are a promising alternative to current robotic platforms, allowing for miniaturization and tetherless actuation. Controlling such systems autonomously may enable safe, accurate operation. However, classical control methods require rigorous models of magnetic fields, robot dynamics, and robot environments, which can be difficult to generate. Model-free reinforcement learning (RL) offers an alternative that can bypass these requirements. We apply RL to a robotic magnetic needle manipulation system. Reinforcement learning algorithms often require long runtimes, making them impractical for many surgical robotics applications, most of which require careful, constant monitoring. Our approach first constructs a model-based simulation (MBS) on guided real-world exploration, learning the dynamics of the environment. After intensive MBS environment training, we transfer the learned behavior from the MBS environment to the real-world. Our MBS method applies RL roughly 200 times faster than doing so in the real world, and achieves a 6 mm root-mean-square (RMS) error for a square reference trajectory. In comparison, pure simulation-based approaches fail to transfer, producing a 31 mm RMS error. These results demonstrate that MBS environments are a good solution for domains where running model-free RL is impractical, especially if an accurate simulation is not available.

3D-printed microrobots from design to translation

Microrobots have attracted the attention of scientists owing to their unique features to accomplish tasks in hard-to-reach sites in the human body. Microrobots can be precisely actuated and maneuvered individually or in a swarm for cargo delivery, sampling, surgery, and imaging applications. In addition, microrobots have found applications in the environmental sector (e.g., water treatment). Besides, recent advancements of three-dimensional (3D) printers have enabled the high-resolution fabrication of microrobots with a faster design-production turnaround time for users with limited micromanufacturing skills. Here, the latest end applications of 3D printed microrobots are reviewed (ranging from environmental to biomedical applications) along with a brief discussion over the feasible actuation methods (e.g., on- and off-board), and practical 3D printing technologies for microrobot fabrication. In addition, as a future perspective, we discussed the potential advantages of integration of microrobots with smart materials, and conceivable benefits of implementation of artificial intelligence (AI), as well as physical intelligence (PI). Moreover, in order to facilitate bench-to-bedside translation of microrobots, current challenges impeding clinical translation of microrobots are elaborated, including entry obstacles (e.g., immune system attacks) and cumbersome standard test procedures to ensure biocompatibility.

Microbots have attracted attention due to an ability to reach places and perform tasks which are not possible with conventional techniques in a wide range of applications. Here, the authors review the recent work in the field on the fabrication, application and actuation of 3D printed microbots offering a view of the direction of future microbot research.

Magnetic Soft Materials and Robots

In conventional classification, soft robots feature mechanical compliance as the main distinguishing factor from traditional robots made of rigid materials. Recent advances in functional soft materials have facilitated the emergence of a new class of soft robots capable of tether-free actuation in response to external stimuli such as heat, light, solvent, or electric or magnetic field. Among the various types of stimuli-responsive materials, magnetic soft materials have shown remarkable progress in their design and fabrication, leading to the development of magnetic soft robots with unique advantages and potential for many important applications. However, the field of magnetic soft robots is still in its infancy and requires further advancements in terms of design principles, fabrication methods, control mechanisms, and sensing modalities. Successful future development of magnetic soft robots would require a comprehensive understanding of the fundamental principle of magnetic actuation, as well as the physical properties and behavior of magnetic soft materials. In this review, we discuss recent progress in the design and fabrication, modeling and simulation, and actuation and control of magnetic soft materials and robots. We then give a set of design guidelines for optimal actuation performance of magnetic soft materials. Lastly, we summarize potential biomedical applications of magnetic soft robots and provide our perspectives on next-generation magnetic soft robots.

Task space adaptation via the learning of gait controllers of magnetic soft millirobots

Untethered small-scale soft robots have promising applications in minimally invasive surgery, targeted drug delivery, and bioengineering applications as they can directly and non-invasively access confined and hard-to-reach spaces in the human body. For such potential biomedical applications, the adaptivity of the robot control is essential to ensure the continuity of the operations, as task environment conditions show dynamic variations that can alter the robot's motion and task performance. The applicability of the conventional modeling and control methods is further limited for soft robots at the small-scale owing to their kinematics with virtually infinite degrees of freedom, inherent stochastic variability during fabrication, and changing dynamics during real-world interactions. To address the controller adaptation challenge to dynamically changing task environments, we propose using a probabilistic learning approach for a millimeter-scale magnetic walking soft robot using Bayesian optimization (BO) and Gaussian processes (GPs). Our approach provides a data-efficient learning scheme by finding the gait controller parameters while optimizing the stride length of the walking soft millirobot using a small number of physical experiments. To demonstrate the controller adaptation, we test the walking gait of the robot in task environments with different surface adhesion and roughness, and medium viscosity, which aims to represent the possible conditions for future robotic tasks inside the human body. We further utilize the transfer of the learned GP parameters among different task spaces and robots and compare their efficacy on the improvement of data-efficient controller learning.

Bayesian Machine Learning for Efficient Minimization of Defects in ALD Passivation Layers

Atomic layer deposition (ALD) is an enabling technology for encapsulating sensitive materials owing to its high-quality, conformal coating capability. Finding the optimum deposition parameters is vital to achieving defect-free layers; however, the high dimensionality of the parameter space makes a systematic study on the improvement of the protective properties of ALD films challenging. Machine-learning (ML) methods are gaining credibility in materials science applications by efficiently addressing these challenges and outperforming conventional techniques. Accordingly, this study reports the ML-based minimization of defects in an ALD-Al₂O₃ passivation layer for the corrosion protection of metallic copper using Bayesian optimization (BO). In all experiments, BO consistently minimizes the layer defect density by finding the optimum deposition parameters in less than three trials. Electrochemical tests show that the optimized layers have virtually zero film porosity and achieve five orders of magnitude reduction in corrosion current as compared to control samples. Optimized parameters of surface pretreatment using Ar/H₂ plasma, the deposition temperature above 200 °C, and 60 ms pulse time quadruple the corrosion resistance. The significant optimization of ALD layers presented in this study demonstrates the effectiveness of BO and its potential outreach to a broader audience, focusing on different materials and processes in materials science applications.