Magnetic Soft Catheter Robot System for Minimally Invasive Treatments of Articular Cartilage Defects

Articular cartilage defects are among the most common orthopedic diseases, which seriously affect patients' health and daily activities, without prompt treatment. The repair biocarrier-based treatment has shown great promise. Total joint injection and open surgery are two main methods to deliver functional repair biocarriers into the knee joint. However, the exhibited drawbacks of these methods hinder their utility. The repair effect of total joint injection is unstable due to the low targeting rate of the repair biocarriers, whereas open surgery causes serious trauma to patients, thereby prolonging the postoperative healing time. In this study, we develop a magnetic soft catheter robot (MSCR) system to perform precise in situ repair of articular cartilage defects with minimal incision. The MSCR processes a size of millimeters, allowing it to enter the joint cavity through a tiny skin incision to reduce postoperative trauma. Meanwhile, a hybrid control strategy combining neural network and visual servo is applied to sequentially complete the coarse and fine positioning of the MSCR on the cartilage defect sites. After reaching the target, the photosensitive hydrogel is injected and anchored into the defect sites through the MSCR, ultimately completing the in situ cartilage repair. The in vitro and ex vivo experiments were conducted on a 3D printed human femur model and an isolated porcine femur, respectively, to demonstrate the potential of our system for the articular cartilage repair.

Adaptive Online Learning and Robust 3-D Shape Servoing of Continuum and Soft Robots in Unstructured Environments

In this article, we present a novel and generic data-driven method to servo-control the 3-D shape of continuum and soft robots based on proprioceptive sensing feedback. Developments of 3-D shape perception and control technologies are crucial for continuum and soft robots to perform tasks autonomously in surgical interventions. However, owing to the nonlinear properties of continuum robots, one main difficulty lies in the modeling of them, especially for soft robots with variable stiffness. To address this problem, we propose a versatile learning-based adaptive shape controller by leveraging proprioception of 3-D configuration from fiber Bragg grating (FBG) sensors, which can online estimate the unknown model of continuum robot against unexpected disturbances and exhibit an adaptive behavior to the unmodeled system without priori data exploration. Based on a new composite adaptation algorithm, the asymptotic convergences of the closed-loop system with learning parameters have been proven by Lyapunov theory. To validate the proposed method, we present a comprehensive experimental study using two continuum and soft robots both integrated with multicore FBGs, including a robotic-assisted colonoscope and multisection extensible soft manipulators. The results demonstrate the feasibility, adaptability, and superiority of our controller in various unstructured environments, as well as phantom experiments.

Learning-Based Control for Soft Robot-Environment Interaction with Force/Position Tracking Capability

Soft robotics promises to achieve safe and efficient interactions with the environment by exploiting its inherent compliance and designing control strategies. However, effective control for the soft robot-environment interaction has been a challenging task. The challenges arise from the nonlinearity and complexity of soft robot dynamics, especially in situations where the environment is unknown and uncertainties exist, making it difficult to establish analytical models. In this study, we propose a learning-based optimal control approach as an attempt to address these challenges, which is an optimized combination of a feedforward controller based on probabilistic model predictive control and a feedback controller based on nonparametric learning methods. The approach is purely data-driven, without prior knowledge of soft robot dynamics and environment structures, and can be easily updated online to adapt to unknown environments. A theoretical analysis of the approach is provided to ensure its stability and convergence. The proposed approach enabled a soft robotic manipulator to track target positions and forces when interacting with a manikin in different cases. Moreover, comparisons with other data-driven control methods show a better performance of our approach. Overall, this work provides a viable learning-based control approach for soft robot-environment interactions with force/position tracking capability.

Model-free kinematic control of redundant manipulators with simultaneous joint-physical-limit and joint-angular-drift handling

Model-free tracking control methods have been developed for redundant robotic manipulators with unknown kinematic or dynamic models. However, most existing works have not considered the problems of joint physical limits and joint angular drift. Joint angular drift refers to the phenomenon that the joint angles of a redundant manipulator are different from their initial state when the end-effector of the manipulator returns to its initial position. Therefore, this article proposes a model-free scheme for the tracking control of redundant manipulators as well as the avoidance of joint physical limits and joint angular drift. The proposed method aims to overcome these challenges by formulating the inverse kinematics problem as a constrained optimization problem, which incorporates the avoidance of joint angular drift into the optimization objective and takes the joint physical limits as a constraint. At the same time, an online Jacobian estimator is designed to observe the state of the manipulator. Specifically, it can estimate the Jacobian matrix of a redundant manipulator in real-time during the operation by fully exploiting the sensory data, without knowing the analytic robot model. Then, the optimization problem is integrated with the Jacobian estimator and solved by a discrete-time algorithm. Theoretical analysis is conducted to prove the stability and convergence of the proposed method. Moreover, the efficacy, practicability and superiority of the proposed method are supported by simulations and experiments based on different redundant manipulators.

Improved Jacobian matrix estimation applied to snake robots

Two manipulator Jacobian matrix estimators for constrained planar snake robots are developed and tested, which enables the implementation of Jacobian-based obstacle-aided locomotion (OAL) control schemes. These schemes use obstacles in the robot's vicinity to obtain propulsion. The devised estimators infer manipulator Jacobians for constrained planar snake robots in situations where the positions and number of surrounding obstacle constraints might change or are not precisely known. The first proposed estimator is an adaptation of contemporary research in soft robots and builds on convex optimization. The second estimator builds on the unscented Kalman filter. By simulations, we evaluate and compare the two devised algorithms in terms of their statistical performance, execution times, and robustness to measurement noise. We find that both algorithms lead to Jacobian matrix estimates that are similarly useful to predict end-effector movements. However, the unscented filter approach requires significantly lower computing resources and is not poised by convergence issues displayed by the convex optimization-based method. We foresee that the estimators may have use in other fields of research, such as soft robotics and visual servoing. The estimators may also be adapted for use in general non-planar snake robots.

Comparison of Modern Control Methods for Soft Robots

With the rise in new soft robotic applications, the control requirements increase. Therefore, precise control methods for soft robots are required. However, the dynamic control of soft robots, which is required for fast movements, is still an open topic and will be discussed here. In this contribution, one kinematic and two dynamic control methods for soft robots are examined. Thereby, an LQI controller with gain scheduling, which is new to soft robotic applications, and an MPC controller are presented. The controllers are compared in a simulation regarding their accuracy and robustness. Additionally, the required implementation effort and computational effort is examined. For this purpose, the trajectory tracking control of a simple soft robot is studied for different trajectories. The soft robot is beam-shaped and tendon-actuated. It is modeled using the piecewise constant curvature model, which is one of the most popular modeling techniques in soft robotics. In this paper, it is shown that all three controllers are able to follow the examined trajectories. However, the dynamic controllers show much higher accuracy and robustness than the kinematic controller. Nevertheless, it should be noted that the implementation and computational effort for the dynamic controllers is significantly higher. Therefore, kinematic controllers should be used if movements are slow and small oscillations can be accepted, while dynamic controllers should be used for faster movements with higher accuracy or robustness requirements.

Continuum Robots for Medical Interventions

Continuum robots are not constructed with discrete joints but, instead, change shape and position their tip by flexing along their entire length. Their narrow curvilinear shape makes them well suited to passing through body lumens, natural orifices, or small surgical incisions to perform minimally invasive procedures. Modeling and controlling these robots are, however, substantially more complex than traditional robots comprised of rigid links connected by discrete joints. Furthermore, there are many approaches to achieving robot flexure. Each presents its own design and modeling challenges, and to date, each has been pursued largely independently of the others. This article attempts to provide a unified summary of the state of the art of continuum robot architectures with respect to design for specific clinical applications. It also describes a unifying framework for modeling and controlling these systems while additionally explaining the elements unique to each architecture. The major research accomplishments are described for each topic and directions for the future progress needed to achieve widespread clinical use are identified.

Visual servoing of continuum robots: Methods, challenges, and prospects

BACKGROUND

Recent advancements in continuum robotics have accentuated developing efficient and stable controllers to handle shape deformation and compliance. The control of continuum robots (CRs) using physical sensors attached to the robot, particularly in confined spaces, is difficult due to their limited accuracy in three-dimensional deflections and challenging localisation. Therefore, using non-contact imaging sensors finds noticeable importance, particularly in medical scenarios. Accordingly, given the need for direct control of the robot tip and notable uncertainties in the kinematics and dynamics of CRs, many papers have focussed on the visual servoing (VS) of CRs in recent years.

METHODS

The significance of this research towards safe human-robot interaction has fuelled our survey on the previous methods, current challenges, and future opportunities.

RESULTS

Beginning with actuation modalities and modelling approaches, the paper investigates VS methods in medical and non-medical scenarios.

CONCLUSIONS

Finally, challenges and prospects of VS for CRs are discussed, followed by concluding remarks.

An Adaptive Time-Varying Impedance Controller for Manipulators

Aiming at the situation that the structural parameters of the general manipulators are uncertain, a time-varying impedance controller based on model reference adaptive control (MRAC) is proposed in this article. The proposed controller does not need to use acceleration-based feedback or to measure external loads and can tolerate considerable structure parameter errors. The global uniform asymptotic stability of the time-varying closed-loop system is analyzed, and a selection approach for control parameters is presented. It is demonstrated that, by using the proposed control parameter selection approach, the closed-loop system under the adaptive controller is equivalent to an existing result. The feasibility of the presented controller for the general manipulators is demonstrated by some numerical simulations.

Review on generic methods for mechanical modeling, simulation and control of soft robots

In this review paper, we are interested in the models and algorithms that allow generic simulation and control of a soft robot. First, we start with a quick overview of modeling approaches for soft robots and available methods for calculating the mechanical compliance, and in particular numerical methods, like real-time Finite Element Method (FEM). We also show how these models can be updated based on sensor data. Then, we are interested in the problem of inverse kinematics, under constraints, with generic solutions without assumption on the robot shape, the type, the placement or the redundancy of the actuators, the material behavior… We are also interested by the use of these models and algorithms in case of contact with the environment. Moreover, we refer to dynamic control algorithms based on mechanical models, allowing for robust control of the positioning of the robot. For each of these aspects, this paper gives a quick overview of the existing methods and a focus on the use of FEM. Finally, we discuss the implementation and our contribution in the field for an open soft robotics research.

A Survey for Machine Learning-Based Control of Continuum Robots

Soft continuum robots have been accepted as a promising category of biomedical robots, accredited to the robots’ inherent compliance that makes them safely interact with their surroundings. In its application of minimally invasive surgery, such a continuum concept shares the same view of robotization for conventional endoscopy/laparoscopy. Different from rigid-link robots with accurate analytical kinematics/dynamics, soft robots encounter modeling uncertainties due to intrinsic and extrinsic factors, which would deteriorate the model-based control performances. However, the trade-off between flexibility and controllability of soft manipulators may not be readily optimized but would be demanded for specific kinds of modeling approaches. To this end, data-driven modeling strategies making use of machine learning algorithms would be an encouraging way out for the control of soft continuum robots. In this article, we attempt to overview the current state of kinematic/dynamic model-free control schemes for continuum manipulators, particularly by learning-based means, and discuss their similarities and differences. Perspectives and trends in the development of new control methods are also investigated through the review of existing limitations and challenges.

On the Mathematical Modeling of Slender Biomedical Continuum Robots

The passive, mechanical adaptation of slender, deformable robots to their environment, whether the robot be made of hard materials or soft ones, makes them desirable as tools for medical procedures. Their reduced physical compliance can provide a form of embodied intelligence that allows the natural dynamics of interaction between the robot and its environment to guide the evolution of the combined robot-environment system. To design these systems, the problems of analysis, design optimization, control, and motion planning remain of great importance because, in general, the advantages afforded by increased mechanical compliance must be balanced against penalties such as slower dynamics, increased difficulty in the design of control systems, and greater kinematic uncertainty. The models that form the basis of these problems should be reasonably accurate yet not prohibitively expensive to formulate and solve. In this article, the state-of-the-art modeling techniques for continuum robots are reviewed and cast in a common language. Classical theories of mechanics are used to outline formal guidelines for the selection of appropriate degrees of freedom in models of continuum robots, both in terms of number and of quality, for geometrically nonlinear models built from the general family of one-dimensional rod models of continuum mechanics. Consideration is also given to the variety of actuators found in existing designs, the types of interaction that occur between continuum robots and their biomedical environments, the imposition of constraints on degrees of freedom, and to the numerical solution of the family of models under study. Finally, some open problems of modeling are discussed and future challenges are identified.

Visual Servoing Pushing Control of the Soft Robot with Active Pushing Force Regulation

Soft robots characterize operational safety due to inherent compliance from their soft mechanism, whereas their mechanism enhances the difficulty in accurate closed-loop control. To explore their applicability in manipulation tasks, in this article, we propose a visual servoing pushing controller considering the effect of contact. The controller is designed to simultaneously complete the positioning and manipulation tasks. To further improve the manipulation performance, an active pushing force regulation method is proposed and integrated into the controller. The proposed control algorithm is validated experimentally. The results show that the controller guarantees the convergence to the image error and meanwhile, it improves the pushing manipulation performance.

Current Trends and Prospects in Compliant Continuum Robots: A Survey

Compliant continuum robots (CCRs) have slender and elastic bodies. Compared with a traditional serial robot, they have more degrees of freedom and can deform their flexible bodies to go through a constrained environment. In this paper, we classify CCRs according to basic transmission units. The merits, materials and potential drawbacks of each type of CCR are described. Drive systems depend on the basic transmission units significantly, and their advantages and disadvantages are reviewed and summarized. Variable stiffness and intrinsic sensing are desired characteristics of CCRs, and the methods of obtaining the two characteristics are discussed. Finally, we discuss the friction, buckling, singularity and twisting problems of CCRs, and emphasise the ways to reduce their effects, followed by several proposing perspectives, such as the collaborative CCRs.

Hierarchical control of soft manipulators towards unstructured interactions

Performing daily interaction tasks such as opening doors and pulling drawers in unstructured environments is a challenging problem for robots. The emergence of soft-bodied robots brings a new perspective to solving this problem. In this paper, inspired by humans performing interaction tasks through simple behaviors, we propose a hierarchical control system for soft arms, in which the low-level controller achieves motion control of the arm tip, the high-level controller controls the behaviors of the arm based on the low-level controller, and the top-level planner chooses what behaviors should be taken according to tasks. To realize the motion control of the soft arm in interacting with environments, we propose two control methods. The first is a feedback control method based on a simplified Jacobian model utilizing the motion laws of the soft arm that are not affected by environments during interaction. The second is a control method based on [Formula: see text]-learning, in which we present a novel method to increase training data by setting virtual goals. We implement the hierarchical control system on a platform with the Honeycomb Pneumatic Networks Arm (HPN Arm) and validate the effectiveness of this system on a series of typical daily interaction tasks, which demonstrates this proposed hierarchical control system could render the soft arms to perform interaction tasks as simply as humans, without force sensors or accurate models of the environments. This work provides a new direction for the application of soft-bodied arms and offers a new perspective for the physical interactions between robots and environments.

Model-free motion control of continuum robots based on a zeroing neurodynamic approach

As a result of inherent flexibility and structural compliance, continuum robots have great potential in practical applications and are attracting more and more attentions. However, these characteristics make it difficult to acquire the accurate kinematics of continuum robots due to uncertainties, deformation and external loads. This paper introduces a method based on a zeroing neurodynamic approach to solve the trajectory tracking problem of continuum robots. The proposed method can achieve the control of a bellows-driven continuum robot just relying on the actuator input and sensory output information, without knowing any information of the kinematic model. This approach reduces the computational load and can guarantee the real time control. The convergence, stability, and robustness of the proposed approach are proved by theoretical analyses. The effectiveness of the proposed method is verified by simulation studies including tracking performance, comparisons with other three methods, and robustness tests.

Automatically steering cardiac catheters in vivo with respiratory motion compensation

A robotic system for automatically navigating ultrasound (US) imaging catheters can provide real-time intra-cardiac imaging for diagnosis and treatment while reducing the need for clinicians to perform manual catheter steering. Clinical deployment of such a system requires accurate navigation despite the presence of disturbances including cyclical physiological motions (e.g., respiration). In this work, we report results from in vivo trials of automatic target tracking using our system, which is the first to navigate cardiac catheters with respiratory motion compensation. The effects of respiratory disturbances on the US catheter are modeled and then applied to four-degree-of-freedom steering kinematics with predictive filtering. This enables the system to accurately steer the US catheter and aim the US imager at a target despite respiratory motion disturbance. In vivo animal respiratory motion compensation results demonstrate automatic US catheter steering to image a target ablation catheter with 1.05 mm and 1.33° mean absolute error. Robotic US catheter steering with motion compensation can improve cardiac catheterization techniques while reducing clinician effort and X-ray exposure.

Jacobian-Based Task-Space Motion Planning for MRI-Actuated Continuum Robots

Robot-assisted medical interventions, such as robotic catheter ablation, often require the robot to perform tasks on a tissue surface. This paper presents a task-space motion planning method that generates actuation trajectories which steer the end- effector of the MRI-actuated robot along desired trajectories on the surface. The continuum robot is modeled using the pseudo-rigid-body model, where the continuum body of the robot is approximated by rigid links joined by flexible joints. The quasistatic motion model of the robot is formulated as a potential energy minimization problem. The Jacobian of the quasistatic motion model is used in calculating the actuations that steer the tip in the desired directions. The proposed method is validated experimentally in a clinical 3-T MRI scanner.

Developments and Control of Biocompatible Conducting Polymer for Intracorporeal Continuum Robots

Dexterity of robots is highly required when it comes to integration for medical applications. Major efforts have been conducted to increase the dexterity at the distal parts of medical robots. This paper reports on developments toward integrating biocompatible conducting polymers (CP) into inherently dexterous concentric tube robot paradigm. In the form of tri-layer thin structures, CP micro-actuators produce high strains while requiring less than 1 V for actuation. Fabrication, characterization, and first integrations of such micro-actuators are presented. The integration is validated in a preliminary telescopic soft robot prototype with qualitative and quantitative performance assessment of accurate position control for trajectory tracking scenarios. Further, CP micro-actuators are integrated to a laser steering system in a closed-loop control scheme with displacements up to 5 mm. Our first developments aim toward intracorporeal medical robotics, with miniaturized actuators to be embedded into continuum robots.

Experimental validation of robot-assisted cardiovascular catheterization: model-based versus model-free control

PURPOSE

In cardiac electrophysiology, a long and flexible catheter is delivered to a cardiac chamber for the treatment of arrhythmias. Although several robot-assisted platforms have been commercialized, the disorientation in tele-operation is still not well solved. We propose a validation platform for robot-assisted cardiac EP catheterization, integrating a customized MR Safe robot, a standard clinically used EP catheter, and a human-robot interface. Both model-based and model-free control methods are implemented in the platform for quantitative evaluation and comparison.

METHODS

The model-based and model-free control methods were validated by subject test (ten participants), in which the subjects have to perform a simulated radiofrequency ablation task using both methods. A virtual endoscopic view of the catheter is also provided to enhance hand-to-eye coordination. Assessment indices for targeting accuracy and efficiency were acquired for the evaluation.

RESULTS

(1) Accuracy: The average distance measured from catheter tip to the closest lesion target during ablation of model-free method was 19.1% shorter than that of model-based control. (2) Efficiency: The model-free control reduced the total missed targets by 35.8% and the maximum continuously missed targets by 46.2%, both indices corresponded to a low p value ([Formula: see text]).

CONCLUSION

The model-free method performed better in terms of both accuracy and efficiency, indicating the model-free control could adapt to soft interaction with environment, as compared with the model-based control that does not consider contacts.

Iterative Jacobian-Based Inverse Kinematics and Open-Loop Control of an MRI-Guided Magnetically Actuated Steerable Catheter System

This paper presents an iterative Jacobian-based inverse kinematics method for an MRI-guided magnetically-actuated steerable intravascular catheter system. The catheter is directly actuated by magnetic torques generated on a set of current-carrying micro-coils embedded on the catheter tip, by the magnetic field of the magnetic resonance imaging (MRI) scanner. The Jacobian matrix relating changes of the currents through the coils to changes of the tip position is derived using a three dimensional kinematic model of the catheter deflection. The inverse kinematics is numerically computed by iteratively applying the inverse of the Jacobian matrix. The damped least square method is implemented to avoid numerical instability issues that exist during the computation of the inverse of the Jacobian matrix. The performance of the proposed inverse kinematics approach is validated using a prototype of the robotic catheter by comparing the actual trajectories of the catheter tip obtained via open-loop control with the desired trajectories. The results of reproducibility and accuracy evaluations demonstrate that the proposed Jacobian-based inverse kinematics method can be used to actuate the catheter in open-loop to successfully perform complex ablation trajectories required in atrial fibrillation ablation procedures. This study paves the way for effective and accurate closed-loop control of the robotic catheter with real-time feedback from MRI guidance in subsequent research.

Predictive Filtering in Motion Compensation with Steerable Cardiac Catheters

Robotic cardiac catheterization using ultrasound (US) imaging catheters provides real time imaging from within the heart while reducing the difficulty in manually steering a four degree-of-freedom (4-DOF) catheter. Accurate robotic catheter navigation in the heart is challenging due to a variety of disturbances including cyclical physiological motions, such as respiration. In this work we compensate for respiratory motion by using an Extended Kalman Filter (EKF) to predict target motion and by applying the predictions to steer the US imaging catheter. The system performance was measured in bench top experiments with phantom vasculature. The robotic system with predictive filtering tracked cyclically moving targets with 1.59 mm and 0.72° mean error. Accurately tracking moving structures can improve intra-procedural treatments and visualization.

Large-deflection statics analysis of active cardiac catheters through co-rotational modelling

This paper presents a co-rotational concept for large-deflection formulation of cardiac catheters. Using this approach, the catheter is first discretized with a number of equal length beam elements and nodes, and the rigid body motions of an individual beam element are separated from its deformations. Therefore, it is adequate for modelling arbitrarily large deflections of a catheter with linear elastic analysis at the local element level. A novel design of active cardiac catheter of 9 Fr in diameter at the beginning of the paper is proposed, which is based on the contra-rotating double helix patterns and is improved from the previous prototypes. The modelling section is followed by MATLAB simulations of various deflections when the catheter is exerted different types of loads. This proves the feasibility of the presented modelling approach. To the best knowledge of the authors, it is the first to utilize this methodology for large-deflection static analysis of the catheter, which will enable more accurate control of robot-assisted cardiac catheterization procedures. Future work would include further experimental validations.