A Fuzzy Reinforcement Learning Approach for Continuum Robot Control

Adaptive fuzzy control for tendon-sheath actuated bending-tip system with unknown friction for robotic flexible endoscope

Introduction

The tendon-sheath actuated bending-tip system (TAB) has been widely applied to long-distance transmission scenes due to its high maneuverability, safety, and compliance, such as in exoskeleton robots, rescue robots, and surgical robots design. Due to the suitability of operation in a narrow or tortuous environment, TAB has demonstrated great application potential in the area of minimally invasive surgery. However, TAB involves highly non-linear behavior due to hysteresis, creepage, and non-linear friction existing on the tendon routing, which is an enormous challenge for accurate control.

Methods

Considering the difficulties in the precise modeling of non-linearity friction, this paper proposes a novel fuzzy control scheme for the Euler-Lagrange dynamics model of TAB for achieving tracking performance and providing accurate friction compensation. Finally, the asymptotic stability of the closed-loop system is proved theoretically and the effectiveness of the controller is verified by numerical simulation carried out in MATLAB/Simulink.

Results

The desired angle can be reached quickly within 3 s by adopting the proposed controller without overshoot or oscillation in Tracking Experiment, demonstrating the regulation performance of the proposed control scheme. The proposed method still achieves the desired trajectory rapidly and accurately without steady-state errors in Varying-friction Experiment. The angle errors generated by external disturbances are < 1 deg under the proposed controller, which returns to zero in 2 s in Anti-disturbance Experiment. In contrast, comparative controllers take more time to be steady and are accompanied by oscillating and residual errors in all experiments.

Discussion

The proposed method is model-free control and has no strict requirement for the dynamics model and friction model. It is proved that advanced tracking performance and real-time response can be guaranteed under the presence of unknown bounded non-linear friction and time-varying non-linear dynamics.

Autonomous learning for fuzzy systems: a review

As one of the three pillars in computational intelligence, fuzzy systems are a powerful mathematical tool widely used for modelling nonlinear problems with uncertainties. Fuzzy systems take the form of linguistic IF-THEN fuzzy rules that are easy to understand for human. In this sense, fuzzy inference mechanisms have been developed to mimic human reasoning and decision-making. From a data analytic perspective, fuzzy systems provide an effective solution to build precise predictive models from imprecise data with great transparency and interpretability, thus facilitating a wide range of real-world applications. This paper presents a systematic review of modern methods for autonomously learning fuzzy systems from data, with an emphasis on the structure and parameter learning schemes of mainstream evolving, evolutionary, reinforcement learning-based fuzzy systems. The main purpose of this paper is to introduce the underlying concepts, underpinning methodologies, as well as outstanding performances of the state-of-the-art methods. It serves as a one-stop guide for readers learning the representative methodologies and foundations of fuzzy systems or who desire to apply fuzzy-based autonomous learning in other scientific disciplines and applied fields.

Characterization of continuum robot arms under reinforcement learning and derived improvements

In robotics, soft continuum robot arms are a promising prospect owing to their redundancy and passivity; however, no comprehensive study exists that examines their characteristics compared to rigid manipulators. In this study, we examined the advantages of a continuum robot arm as compared to a typical and rigid seven-degree-of-freedom (7-DoF) robot manipulator in terms of performing various tasks through reinforcement learning. We conducted simulations for tasks with different characteristics that require control over position and force. Common tasks in robot manipulators, such as reaching, crank rotation, object throwing, and peg-in-hole were considered. The initial conditions of the robot and environment were randomized, aiming for evaluations including robustness. The results indicate that the continuum robot arm excels in the crank-rotation task, which is characterized by uncertainty in environmental conditions and cumulative rewards. However, the rigid robot arm learned better motions for the peg-in-hole task than the other tasks, which requires fine motion control of the end-effector. In the throwing task, the continuum robot arm scored well owing to its good handling of anisotropy. Moreover, we developed a reinforcement-learning method based on the comprehensive experimental results. The proposed method successfully improved the motion learning of a continuum robot arm by adding a technique to regulate the initial state of the robot. To the best of our knowledge, ours is the first reinforcement-learning experiment with multiple tasks on a single continuum robot arm and is the first report of a comparison between a single continuum robot arm and rigid manipulator on a wide range of tasks. This simulation study can make a significant contribution to the design of continuum arms and specification of their applications, and development of control and reinforcement learning methods.

Design and experimental evaluation of an automated catheter operating system

Manual catheter-based interventions (CBIs) suffer from exposure of the interventionalists to X-ray, and dependence of their performance on the expertise and fatigue level of the interventionalists. Robot-assisted catheterization systems (RACS) have been introduced in recent years to improve the efficiency of CBIs; however, using them is still associated with some difficulties such as set-up dependency to a specific type of intervention instrument, not being portable, and offering limited options of operation modes. The objective of this research is to develop a new RACS to address these shortcomings. We propose Althea II as an improvement for our previously introduced RACS, Althea I. Althea II is designed for both research purposes and clinical applications including catheter-based cardiovascular interventions. Althea II benefits from a novel structural design leading to a significantly reduced weight and making the device inclusive for a broader range of intervention instruments. Also, a tip detection algorithm is developed and integrated into the graphical user interface (GUI) to enable image-based navigation, and accordingly, fully automatic navigation. Althea II has improved the outcome of catheter-based interventions by increased accuracy and precision of the intervention. The system can navigate the catheter tip to a designated target with an accuracy higher than 90% in both velocity and positioning mode. The device is associated with an upgraded GUI equipped with a strong tip detection algorithm with an accuracy of 0.05 mm. Moreover, Althea II gains from a quicker assembly time (20 minutes, which equals five times faster). The independency from specific catheters, several modes of function, an imaged-based feedback control, portability, and a remote function should allow operation even from beginners and reduce X-ray exposure. The preliminary research studies verified the accuracy and repeatability of Althea II, demonstrated the feasibility and applicability of using the set-up in multiple applications, and highlighted the improved set-up capabilities over the currently available RACS.