Learning Closed Loop Kinematic Controllers for Continuum Manipulators in Unstructured Environments

Utilising redundancy in musculoskeletal systems for adaptive stiffness and muscle failure compensation: a model-free inverse statics approach

Vertebrates possess a biomechanical structure with redundant muscles, enabling adaptability in uncertain and complex environments. Harnessing this inspiration, musculoskeletal systems offer advantages like variable stiffness and resilience to actuator failure and fatigue. Despite their potential, the complex structure presents modelling challenges that are difficult to explicitly formulate and control. This difficulty arises from the need for comprehensive knowledge of the musculoskeletal system, including details such as muscle arrangement, and fully accessible muscle and joint states. Whilst existing model-free methods do not need explicit formulations, they also underutilise the benefits of muscle redundancy. Consequently, they necessitate retraining in the event of muscle failure and require manual tuning of parameters to control joint stiffness limiting their applications under unknown payloads. Presented here is a model-free local inverse statics controller for musculoskeletal systems, employing a feedforward neural network trained on motor babbling data. Experiments with a musculoskeletal leg model showcase the controller's adaptability to complex structures, including mono and bi-articulate muscles. The controller can compensate for changes such as weight variations, muscle failures, and environmental interactions, retaining reasonable accuracy without the need for any additional retraining.

Visuo-dynamic self-modelling of soft robotic systems

Soft robots exhibit complex nonlinear dynamics with large degrees of freedom, making their modelling and control challenging. Typically, reduced-order models in time or space are used in addressing these challenges, but the resulting simplification limits soft robot control accuracy and restricts their range of motion. In this work, we introduce an end-to-end learning-based approach for fully dynamic modelling of any general robotic system that does not rely on predefined structures, learning dynamic models of the robot directly in the visual space. The generated models possess identical dimensionality to the observation space, resulting in models whose complexity is determined by the sensory system without explicitly decomposing the problem. To validate the effectiveness of our proposed method, we apply it to a fully soft robotic manipulator, and we demonstrate its applicability in controller development through an open-loop optimization-based controller. We achieve a wide range of dynamic control tasks including shape control, trajectory tracking and obstacle avoidance using a model derived from just 90 min of real-world data. Our work thus far provides the most comprehensive strategy for controlling a general soft robotic system, without constraints on the shape, properties, or dimensionality of the system.

Design and Kinematic Modeling of a Soft Continuum Telescopic Arm for the Self-Assembly Mechanism of a Modular Robot

In recent years, the development of mining robots has grown significantly, offering improved efficiency and safety in hazardous environments. However, there is still room for improvement in adaptability, scalability, and overall performance. The ROBOMINERS project, funded by the European Union's Horizon 2020 Research and Innovation Program, aims to facilitate Europe's access to mineral resources applying disruptive robotic concepts. One such concept is resilience, which can be achieved providing modular mining robots with the ability to reconfigure during operation. To address this challenge, this article presents the development and kinematic modeling of a soft, telescopic, continuum arm integrated into a modular robot. The arm serves as a mechanical interface for coupling different robotic modules or tools following the principle of the car crane. With a fully 3D-printed design, the arm features two sections of variable length that are driven by an innovative actuation method based on soft racks. It provides a 6 degrees of freedom (DoF) motion. The arm kinematic models are obtained by backbone parameterization assuming constant curvature and independent bending between sections for forward kinematics and applying a machine learning-based approach for inverse kinematics. The models are validated through the evaluation of two trajectories, measuring the deviation in each DoF and rack extension. Furthermore, a demonstration of the arm's coupling procedure between two robotic modules and one possible configuration of the robotic system showcases its functionality.

Adaptive Online Learning and Robust 3-D Shape Servoing of Continuum and Soft Robots in Unstructured Environments

In this article, we present a novel and generic data-driven method to servo-control the 3-D shape of continuum and soft robots based on proprioceptive sensing feedback. Developments of 3-D shape perception and control technologies are crucial for continuum and soft robots to perform tasks autonomously in surgical interventions. However, owing to the nonlinear properties of continuum robots, one main difficulty lies in the modeling of them, especially for soft robots with variable stiffness. To address this problem, we propose a versatile learning-based adaptive shape controller by leveraging proprioception of 3-D configuration from fiber Bragg grating (FBG) sensors, which can online estimate the unknown model of continuum robot against unexpected disturbances and exhibit an adaptive behavior to the unmodeled system without priori data exploration. Based on a new composite adaptation algorithm, the asymptotic convergences of the closed-loop system with learning parameters have been proven by Lyapunov theory. To validate the proposed method, we present a comprehensive experimental study using two continuum and soft robots both integrated with multicore FBGs, including a robotic-assisted colonoscope and multisection extensible soft manipulators. The results demonstrate the feasibility, adaptability, and superiority of our controller in various unstructured environments, as well as phantom experiments.

Performance evaluation of PSO-PID and PSO-FLC for continuum robot's developed modeling and control

Continuum robots are complex structures that require sophisticated modeling and control methods to achieve accurate position and motion tracking along desired trajectories. They are highly coupled, nonlinear systems with multiple degrees of freedom that pose a significant challenge for conventional approaches. In this paper, we propose a system dynamic model based on the Euler-Lagrange formulation with the assumption of piecewise constant curvature (PCC), where we accounts for the elasticity and gravity effects of the continuum robot. We also develop and apply a particle swarm optimization (PSO) algorithm to optimize the parameters of our developed controllers: an inverse dynamic proportional integral derivative (PID) controller and an inverse dynamic fuzzy logic controller (FLC), where we use the integral time of absolute error (ITAE) as the objective function for the PSO algorithm. We validate our proposed model and optimized controllers through different designed trajectories, simulated using our developed unique animated MATLAB simulation. The results show that the PSO-PID controller improves the rise time, overshoot percentage, and settling time by 16.3%, 31.1%, and 64.9%, respectively, compared to the PID controller without PSO. The PSO-FLC controller shows the best performance among all controllers, with a settling time of 0.7 s and a rise time of 0.4 s, leading to the highest level of precision in trajectory tracking. The ITAE error for the PSO-FLC controller is 11.4% and 29.9% lower than that of the PSO-PID and FLC controllers, respectively.

Soft DAgger: Sample-Efficient Imitation Learning for Control of Soft Robots

This paper presents Soft DAgger, an efficient imitation learning-based approach for training control solutions for soft robots. To demonstrate the effectiveness of the proposed algorithm, we implement it on a two-module soft robotic arm involved in the task of writing letters in 3D space. Soft DAgger uses a dynamic behavioral map of the soft robot, which maps the robot's task space to its actuation space. The map acts as a teacher and is responsible for predicting the optimal actions for the soft robot based on its previous state action history, expert demonstrations, and current position. This algorithm achieves generalization ability without depending on costly exploration techniques or reinforcement learning-based synthetic agents. We propose two variants of the control algorithm and demonstrate that good generalization capabilities and improved task reproducibility can be achieved, along with a consistent decrease in the optimization time and samples. Overall, Soft DAgger provides a practical control solution to perform complex tasks in fewer samples with soft robots. To the best of our knowledge, our study is an initial exploration of imitation learning with online optimization for soft robot control.

Comparison of Modern Control Methods for Soft Robots

With the rise in new soft robotic applications, the control requirements increase. Therefore, precise control methods for soft robots are required. However, the dynamic control of soft robots, which is required for fast movements, is still an open topic and will be discussed here. In this contribution, one kinematic and two dynamic control methods for soft robots are examined. Thereby, an LQI controller with gain scheduling, which is new to soft robotic applications, and an MPC controller are presented. The controllers are compared in a simulation regarding their accuracy and robustness. Additionally, the required implementation effort and computational effort is examined. For this purpose, the trajectory tracking control of a simple soft robot is studied for different trajectories. The soft robot is beam-shaped and tendon-actuated. It is modeled using the piecewise constant curvature model, which is one of the most popular modeling techniques in soft robotics. In this paper, it is shown that all three controllers are able to follow the examined trajectories. However, the dynamic controllers show much higher accuracy and robustness than the kinematic controller. Nevertheless, it should be noted that the implementation and computational effort for the dynamic controllers is significantly higher. Therefore, kinematic controllers should be used if movements are slow and small oscillations can be accepted, while dynamic controllers should be used for faster movements with higher accuracy or robustness requirements.

A Hydrogel-Based Self-Sensing Underwater Actuator

Soft robots made of hydrogels are suited for underwater exploration due to their biocompatibility and compliancy. Yet, reaching high dexterity and actuation force for hydrogel-based actuators is challenging. Meanwhile, real-time proprioception is critical for feedback control. Moreover, sensor integration to mimic living organisms remains problematic. To address these challenges, we introduce a hydrogel actuator driven by hydraulic force with a fast response (time constant 0.83 s). The highly stretchable and conductive hydrogel (1400% strain) is molded into the PneuNet shape, and two of them are further assembled symmetrically to actuate bi-directionally. Then, we demonstrate its bionic application for underwater swimming, showing 2 cm/s (0.19 BL/s) speed. Inspired by biological neuromuscular systems' sensory motion, which unifies the sensing and actuation in a single unit, we explore the hydrogel actuator's self-sensing capacity utilizing strain-induced resistance change. The results show that the soft actuator's proprioception can monitor the undulation in real-time with a sensitivity of 0.2%/degree. Furthermore, we take a finite-element method and first-order differential equations to model the actuator's bending in response to pressure. We show that such a model can precisely predict the robot's bending response over a range of pressures. With the self-sensing actuator and the proposed model, we expect the new approach can lead to future soft robots for underwater exploration with feedback control, and the underlying mechanism of the undulation control might offer significant insights for biomimetic research.

Using natural language processing to find research topics in Living Machines conferences and their intersections with Bioinspiration & Biomimetics publications

The number of published scientific articles is increasing dramatically and makes it difficult to keep track of research topics. This is particularly difficult in interdisciplinary research areas where different communities from different disciplines are working together. It would be useful to develop methods to automate the detection of research topics in a research domain. Here we propose a natural language processing (NLP) based method to automatically detect topics in defined corpora. We start by automatically generating a global state of the art of Living Machines conferences. Our NLP-based method classifies all published papers into different clusters corresponding to the research topic published in these conferences. We perform the same study on all papers published in the journals Bioinspiration & Biomimetics and Soft Robotics. In total this analysis concerns 2099 articles. Next, we analyze the intersection between the research themes published in the conferences and the corpora of these two journals. We also examine the evolution of the number of papers per research theme which determines the research trends. Together, these analyses provide a snapshot of the current state of the field, help to highlight open questions, and provide insights into the future.

Characterization of continuum robot arms under reinforcement learning and derived improvements

In robotics, soft continuum robot arms are a promising prospect owing to their redundancy and passivity; however, no comprehensive study exists that examines their characteristics compared to rigid manipulators. In this study, we examined the advantages of a continuum robot arm as compared to a typical and rigid seven-degree-of-freedom (7-DoF) robot manipulator in terms of performing various tasks through reinforcement learning. We conducted simulations for tasks with different characteristics that require control over position and force. Common tasks in robot manipulators, such as reaching, crank rotation, object throwing, and peg-in-hole were considered. The initial conditions of the robot and environment were randomized, aiming for evaluations including robustness. The results indicate that the continuum robot arm excels in the crank-rotation task, which is characterized by uncertainty in environmental conditions and cumulative rewards. However, the rigid robot arm learned better motions for the peg-in-hole task than the other tasks, which requires fine motion control of the end-effector. In the throwing task, the continuum robot arm scored well owing to its good handling of anisotropy. Moreover, we developed a reinforcement-learning method based on the comprehensive experimental results. The proposed method successfully improved the motion learning of a continuum robot arm by adding a technique to regulate the initial state of the robot. To the best of our knowledge, ours is the first reinforcement-learning experiment with multiple tasks on a single continuum robot arm and is the first report of a comparison between a single continuum robot arm and rigid manipulator on a wide range of tasks. This simulation study can make a significant contribution to the design of continuum arms and specification of their applications, and development of control and reinforcement learning methods.

Recent Progress in Modeling and Control of Bio-Inspired Fish Robots

Compared with traditional underwater vehicles, bio-inspired fish robots have the advantages of high efficiency, high maneuverability, low noise, and minor fluid disturbance. Therefore, they have gained an increasing research interest, which has led to a great deal of remarkable progress theoretically and practically in recent years. In this review, we first highlight our enhanced scientific understanding of bio-inspired propulsion and sensing underwater and then present the research progress and performance characteristics of different bio-inspired robot fish, classified by the propulsion method. Like the natural fish species they imitate, different types of bionic fish have different morphological structures and distinctive hydrodynamic properties. In addition, we select two pioneering directions about soft robotic control and multi-phase robotics. The hybrid dynamic control of soft robotic systems combines the accuracy of model-based control and the efficiency of model-free control, and is considered the proper way to optimize the classical control model with the intersection of multiple machine learning algorithms. Multi-phase robots provide a broader scope of application compared to ordinary bionic robot fish, with the ability of operating in air or on land outside the fluid. By introducing recent progress in related fields, we summarize the advantages and challenges of soft robotic control and multi-phase robotics, guiding the further development of bionic aquatic robots.

Review on generic methods for mechanical modeling, simulation and control of soft robots

In this review paper, we are interested in the models and algorithms that allow generic simulation and control of a soft robot. First, we start with a quick overview of modeling approaches for soft robots and available methods for calculating the mechanical compliance, and in particular numerical methods, like real-time Finite Element Method (FEM). We also show how these models can be updated based on sensor data. Then, we are interested in the problem of inverse kinematics, under constraints, with generic solutions without assumption on the robot shape, the type, the placement or the redundancy of the actuators, the material behavior… We are also interested by the use of these models and algorithms in case of contact with the environment. Moreover, we refer to dynamic control algorithms based on mechanical models, allowing for robust control of the positioning of the robot. For each of these aspects, this paper gives a quick overview of the existing methods and a focus on the use of FEM. Finally, we discuss the implementation and our contribution in the field for an open soft robotics research.

On the Mathematical Modeling of Slender Biomedical Continuum Robots

The passive, mechanical adaptation of slender, deformable robots to their environment, whether the robot be made of hard materials or soft ones, makes them desirable as tools for medical procedures. Their reduced physical compliance can provide a form of embodied intelligence that allows the natural dynamics of interaction between the robot and its environment to guide the evolution of the combined robot-environment system. To design these systems, the problems of analysis, design optimization, control, and motion planning remain of great importance because, in general, the advantages afforded by increased mechanical compliance must be balanced against penalties such as slower dynamics, increased difficulty in the design of control systems, and greater kinematic uncertainty. The models that form the basis of these problems should be reasonably accurate yet not prohibitively expensive to formulate and solve. In this article, the state-of-the-art modeling techniques for continuum robots are reviewed and cast in a common language. Classical theories of mechanics are used to outline formal guidelines for the selection of appropriate degrees of freedom in models of continuum robots, both in terms of number and of quality, for geometrically nonlinear models built from the general family of one-dimensional rod models of continuum mechanics. Consideration is also given to the variety of actuators found in existing designs, the types of interaction that occur between continuum robots and their biomedical environments, the imposition of constraints on degrees of freedom, and to the numerical solution of the family of models under study. Finally, some open problems of modeling are discussed and future challenges are identified.

A learning-based tip contact force estimation method for tendon-driven continuum manipulator

Although tendon-driven continuum manipulators have been extensively researched, how to realize tip contact force sensing in a more general and efficient way without increasing the diameter is still a challenge. Rather than use a complex modeling approach, this paper proposes a general tip contact force-sensing method based on a recurrent neural network that takes the tendons’ position and tension as the input of a recurrent neural network and the tip contact force of the continuum manipulator as the output and fits this static model by means of machine learning so that it may be used as a real-time contact force estimator. We also designed and built a corresponding three-degree-of-freedom contact force data acquisition platform based on the structure of a continuum manipulator designed in our previous studies. After obtaining training data, we built and compared the performances of a multi-layer perceptron-based contact force estimator as a baseline and three typical recurrent neural network-based contact force estimators through TensorFlow framework to verify the feasibility of this method. We also proposed a manually decoupled sub-estimators algorithm and evaluated the advantages and disadvantages of those two methods.

Dynamics and path tracking of continuum robotic arms using data-driven identification tools

In this paper, forward/inverse dynamics of a continuum robotic arm is developed using a data-driven approach, which could tackle uncertainties and extreme nonlinearities to obtain reliable solutions. By establishing a direct mapping between the actuator and task spaces, the unnecessary mappings of actuator-to-configuration then configuration-to-task are eliminated, to reduce extra computational cost. The proposed approach is validated through simulation (based on Cosserat rod theory) and experimental tests on RoboArm. Next, path tracking in the presence/absence of obstacles as well as load carrying maneuver are investigated. Finally, the obtained results concerning repeatability, scalability, and disturbance rejection performance of the approach are discussed.

Current Trends and Prospects in Compliant Continuum Robots: A Survey

Compliant continuum robots (CCRs) have slender and elastic bodies. Compared with a traditional serial robot, they have more degrees of freedom and can deform their flexible bodies to go through a constrained environment. In this paper, we classify CCRs according to basic transmission units. The merits, materials and potential drawbacks of each type of CCR are described. Drive systems depend on the basic transmission units significantly, and their advantages and disadvantages are reviewed and summarized. Variable stiffness and intrinsic sensing are desired characteristics of CCRs, and the methods of obtaining the two characteristics are discussed. Finally, we discuss the friction, buckling, singularity and twisting problems of CCRs, and emphasise the ways to reduce their effects, followed by several proposing perspectives, such as the collaborative CCRs.

First-Order Dynamic Modeling and Control of Soft Robots

Modeling of soft robots is typically performed at the static level or at a second-order fully dynamic level. Controllers developed upon these models have several advantages and disadvantages. Static controllers, based on the kinematic relations tend to be the easiest to develop, but by sacrificing accuracy, efficiency and the natural dynamics. Controllers developed using second-order dynamic models tend to be computationally expensive, but allow optimal control. Here we propose that the dynamic model of a soft robot can be reduced to first-order dynamical equation owing to their high damping and low inertial properties, as typically observed in nature, with minimal loss in accuracy. This paper investigates the validity of this assumption and the advantages it provides to the modeling and control of soft robots. Our results demonstrate that this model approximation is a powerful tool for developing closed-loop task-space dynamic controllers for soft robots by simplifying the planning and sensory feedback process with minimal effects on the controller accuracy.

Hierarchical control of soft manipulators towards unstructured interactions

Performing daily interaction tasks such as opening doors and pulling drawers in unstructured environments is a challenging problem for robots. The emergence of soft-bodied robots brings a new perspective to solving this problem. In this paper, inspired by humans performing interaction tasks through simple behaviors, we propose a hierarchical control system for soft arms, in which the low-level controller achieves motion control of the arm tip, the high-level controller controls the behaviors of the arm based on the low-level controller, and the top-level planner chooses what behaviors should be taken according to tasks. To realize the motion control of the soft arm in interacting with environments, we propose two control methods. The first is a feedback control method based on a simplified Jacobian model utilizing the motion laws of the soft arm that are not affected by environments during interaction. The second is a control method based on [Formula: see text]-learning, in which we present a novel method to increase training data by setting virtual goals. We implement the hierarchical control system on a platform with the Honeycomb Pneumatic Networks Arm (HPN Arm) and validate the effectiveness of this system on a series of typical daily interaction tasks, which demonstrates this proposed hierarchical control system could render the soft arms to perform interaction tasks as simply as humans, without force sensors or accurate models of the environments. This work provides a new direction for the application of soft-bodied arms and offers a new perspective for the physical interactions between robots and environments.

Model-free motion control of continuum robots based on a zeroing neurodynamic approach

As a result of inherent flexibility and structural compliance, continuum robots have great potential in practical applications and are attracting more and more attentions. However, these characteristics make it difficult to acquire the accurate kinematics of continuum robots due to uncertainties, deformation and external loads. This paper introduces a method based on a zeroing neurodynamic approach to solve the trajectory tracking problem of continuum robots. The proposed method can achieve the control of a bellows-driven continuum robot just relying on the actuator input and sensory output information, without knowing any information of the kinematic model. This approach reduces the computational load and can guarantee the real time control. The convergence, stability, and robustness of the proposed approach are proved by theoretical analyses. The effectiveness of the proposed method is verified by simulation studies including tracking performance, comparisons with other three methods, and robustness tests.

Energy-shaping control of soft continuum manipulators with in-plane disturbances

Soft continuum manipulators offer levels of compliance and inherent safety that can render them a superior alternative to conventional rigid robots for a variety of tasks, such as medical interventions or human–robot interaction. However, the ability of soft continuum manipulators to compensate for external disturbances needs to be further enhanced to meet the stringent requirements of many practical applications. In this paper, we investigate the control problem for soft continuum manipulators that consist of one inextensible segment of constant section, which bends under the effect of the internal pressure and is subject to unknown disturbances acting in the plane of bending. A rigid-link model of the manipulator with a single input pressure is employed for control purposes and an energy-shaping approach is proposed to derive the control law. A method for the adaptive estimation of disturbances is detailed and a disturbance compensation strategy is proposed. Finally, the effectiveness of the controller is demonstrated with simulations and with experiments on an inextensible soft continuum manipulator that employs pneumatic actuation.