Continuous Shape Estimation of Continuum Robots Using X-ray Images

Design and Fabrication of a Fiber Bragg Grating Shape Sensor for Shape Reconstruction of a Continuum Manipulator

Continuum dexterous manipulators (CDMs) are suitable for performing tasks in a constrained environment due to their high dexterity and maneuverability. Despite the inherent advantages of CDMs in minimally invasive surgery, real-time control of CDMs' shape during nonconstant curvature bending is still challenging. This study presents a novel approach for the design and fabrication of a large deflection fiber Bragg grating (FBG) shape sensor embedded within the lumens inside the walls of a CDM with a large instrument channel. The shape sensor consisted of two fibers, each with three FBG nodes. A shape-sensing model was introduced to reconstruct the centerline of the CDM based on FBG wavelengths. Different experiments, including shape sensor tests and CDM shape reconstruction tests, were conducted to assess the overall accuracy of the shape-sensing. The FBG sensor evaluation results revealed the linear curvature-wavelength relationship with the large curvature detection of 0.045 mm and a high wavelength shift of up to 5.50 nm at a 90° bending angle in both the bending directions. The CDM's shape reconstruction experiments in a free environment demonstrated the shape-tracking accuracy of 0.216 ± 0.126 mm for positive/negative deflections. Also, the CDM shape reconstruction error for three cases of bending with obstacles was observed to be 0.436 ± 0.370 mm for the proximal case, 0.485 ± 0.418 mm for the middle case, and 0.312 ± 0.261 mm for the distal case. This study indicates the adequate performance of the FBG sensor and the effectiveness of the model for tracking the shape of the large-deflection CDM with nonconstant-curvature bending for minimally invasive orthopedic applications.

Continuum robot state estimation using Gaussian process regression on SE(3)

Continuum robots have the potential to enable new applications in medicine, inspection, and countless other areas due to their unique shape, compliance, and size. Excellent progress has been made in the mechanical design and dynamic modeling of continuum robots, to the point that there are some canonical designs, although new concepts continue to be explored. In this paper, we turn to the problem of state estimation for continuum robots that can been modeled with the common Cosserat rod model. Sensing for continuum robots might comprise external camera observations, embedded tracking coils, or strain gauges. We repurpose a Gaussian process (GP) regression approach to state estimation, initially developed for continuous-time trajectory estimation in SE(3). In our case, the continuous variable is not time but arclength and we show how to estimate the continuous shape (and strain) of the robot (along with associated uncertainties) given discrete, noisy measurements of both pose and strain along the length. We demonstrate our approach quantitatively through simulations as well as through experiments. Our evaluations show that accurate and continuous estimates of a continuum robot’s shape can be achieved, resulting in average end-effector errors between the estimated and ground truth shape as low as 3.5 mm and 0.016° in simulation or 3.3 mm and 0.035° for unloaded configurations and 6.2 mm and 0.041° for loaded ones during experiments, when using discrete pose measurements.

Shape Reconstruction Processes for Interventional Application Devices: State of the Art, Progress, and Future Directions

Soft and continuum robots are transforming medical interventions thanks to their flexibility, miniaturization, and multidirectional movement abilities. Although flexibility enables reaching targets in unstructured and dynamic environments, it also creates challenges for control, especially due to interactions with the anatomy. Thus, in recent years lots of efforts have been devoted for the development of shape reconstruction methods, with the advancement of different kinematic models, sensors, and imaging techniques. These methods can increase the performance of the control action as well as provide the tip position of robotic manipulators relative to the anatomy. Each method, however, has its advantages and disadvantages and can be worthwhile in different situations. For example, electromagnetic (EM) and Fiber Bragg Grating (FBG) sensor-based shape reconstruction methods can be used in small-scale robots due to their advantages thanks to miniaturization, fast response, and high sensitivity. Yet, the problem of electromagnetic interference in the case of EM sensors, and poor response to high strains in the case of FBG sensors need to be considered. To help the reader make a suitable choice, this paper presents a review of recent progress on shape reconstruction methods, based on a systematic literature search, excluding pure kinematic models. Methods are classified into two categories. First, sensor-based techniques are presented that discuss the use of various sensors such as FBG, EM, and passive stretchable sensors for reconstructing the shape of the robots. Second, imaging-based methods are discussed that utilize images from different imaging systems such as fluoroscopy, endoscopy cameras, and ultrasound for the shape reconstruction process. The applicability, benefits, and limitations of each method are discussed. Finally, the paper draws some future promising directions for the enhancement of the shape reconstruction methods by discussing open questions and alternative methods.

Real-Time Shape Estimation for Concentric Tube Continuum Robots with a Single Force/Torque Sensor

Shape-sensing in real-time is a key requirement for the development of advanced algorithms for concentric tube continuum robots when safe interaction with the environment is important e.g., for path planning, advanced control, and human-machine interaction. We propose a real-time shape-estimation algorithm for concentric tube continuum robots based on the force-torque information measured at the tubes' basis. It extends a shape estimation algorithm for elastic rods based on discrete Kirchhoff rod theory. For simplicity and efficiency of calculation, we combine it with a model under piece-wise constant curvature assumption, in which we model a concentric tube continuum robot as a combination of segments of planar constant curvatures lying on different equilibrium planes. We evaluate our approach for a single and two combined additively manufactured tubes and achieve an estimation frequency of 333 Hz for two combined tubes with a mean deviation along the backbone of the tubes of 1.91-5.22 mm.

Tip estimation approach for concentric tube robots using 2D ultrasound images and kinematic model

Concentric tube robot (CTR) is an efficient approach for minimally invasive surgery (MIS) and diagnosis due to its small size and high dexterity. To manipulate the robot accurately and safely inside the human body, tip position and shape information need to be well measured. In this paper, we propose a tip estimation method based on 2D ultrasound images with the help of the forward kinematic model of CTR. The forward kinematic model can help to provide a fast ultrasound scanning path and narrow the region of interest in ultrasound images. For each tube, only three scan positions are needed by combining the kinematic model prediction as prior knowledge. After that, the curve fitting method is used for its shape reconstruction, while its tip position can be estimated based on the constraints of its structure and length.7 This method provides the advantage that only three scan positions are needed for estimating the tip of each telescoping section. Moreover, no structure modification is needed on the robot, which makes it an appropriate approach for existing flexible surgical robots. Experimental results verified the feasibility of the proposed method and the tip estimation error is 0.59 mm. Graphical abstract In this paper, we propose a tip estimation method based on 2D Ultrasound images with the help of the forward kinematic model of CTR. The forward kinematic model can help to provide a fast Ultrasound scanning path and narrow the region of interest in Ultrasound images. For each tube, only three scan positions are needed by combining the kinematic model prediction as prior knowledge. After that, the curve fitting method is used for its shape reconstruction, while its tip position can be estimated based on the constraints of its structure and length.

Design of a multi-arm concentric-tube robot system for transnasal surgery

Concentric tube robot (CTR) has gradually attracted the attention of researchers on the basis of its small size and curved shape control ability. However, most of current experimental prototypes of CTR are single-arm structure, which can only carry out simple operation such as drug delivery or monitoring. In this paper, design and analysis of a three-arm CTR system is proposed. It has a four-DOF vision arm and two six-DOF manipulator arms, which equipped with special end effectors to achieve different surgical operations. Finally, a mean motion accuracy of 0.33 mm has been obtained quantitatively through teleoperation experiments. Moreover, tissue excision experiment in skull model is carried out to prove the effectiveness and feasibility of the proposed CTR system in nasopharyngeal carcinoma surgery. Graphical Abstract Platform of the proposed Multi-Arm Concentric Tube Robot system. (a) Configuration of the end-effectors with the CTR system. (b) The setup of the tissue removal experiment in a skull model.

Haptic Telerobotic Cardiovascular Intervention: A Review of Approaches, Methods, and Future Perspectives

Cardiac diseases are recognized as the leading cause of mortality, hospitalization, and medical prescription globally. The gold standard for the treatment of coronary artery stenosis is the percutaneous cardiac intervention that is performed under live X-ray imaging. Substantial clinical evidence shows that the surgeon and staff are prone to serious health problems due to X-ray exposure and occupational hazards. Telerobotic vascular intervention systems with a master-slave architecture reduced the X-ray exposure and enhanced the clinical outcomes; however, the loss of haptic feedback during surgery has been the main limitation of such systems. This paper is a review of the state of the art for haptic telerobotic cardiovascular interventions. A survey on the literature published between 2000 and 2019 was performed. Results of the survey were screened based on their relevance to this paper. Also, the leading research disciplines were identified based on the results of the survey. Furthermore, different approaches for sensor-based and model-based haptic telerobotic cardiovascular intervention, haptic rendering and actuation, and the pertinent methods were critically reviewed and compared. In the end, the current limitations of the state of the art, unexplored research areas as well as the future perspective of the research on this technology were laid out.

Real-Time Visual Tracking of Dynamic Surgical Suture Threads

In order to realize many of the potential benefits associated with robotically assisted minimally invasive surgery, the robot must be more than a remote controlled device. Currently, using a surgical robot can be challenging, fatiguing, and time consuming. Teaching the robot to actively assist surgical tasks, such as suturing, has the potential to vastly improve both patient outlook and the surgeon's efficiency. One obstacle to completing surgical sutures autonomously is the difficulty in tracking surgical suture threads. This paper presents novel stereo image processing algorithms for the detection, initialization, and tracking of a surgical suture thread. A Non Uniform Rational B-Spline (NURBS) curve is used to model a thin, deformable, and dynamic length thread. The NURBS model is initialized and grown from a single selected point located on the thread. The NURBS curve is optimized by minimizing the image matching energy between the projected stereo NURBS image and the segmented thread image. The algorithms are evaluated using suture threads, a calibrated test pattern, and a simulated thread image. Additionally, the accuracy of the algorithms presented are validated as they track a suture thread undergoing translation, deformation, and apparent length changes. All of the tracking is in real-time. Note to Practioners: Abstract-The problem of tracking a surgical suture thread was addressed in this work. Since the suture thread is highly deformable, any tracking algorithm must be robust to intersections, occlusions, knot tying, and length changes. The detection algorithm introduced in this paper is capable of distinguishing different threads when they intersect. The tracking algorithm presented here demonstrate that it is possible, using polynomial curves, to track a suture thread as it deforms, becomes occluded, changes length, and even ties a knot in real time. The detection algorithm can enhance directional thin features while the polynomial curve modeling can track any string like structure. Further integration of the polynomial curve with a feed-forward thread model could improve the stability and robustness of the thread tracking.

Real-time Adaptive Kinematic Model Estimation of Concentric Tube Robots

Kinematic models of concentric tube robots have matured from considering only tube bending to considering tube twisting as well as external loading. While these models have been demonstrated to approximate actual behavior, modeling error can be significant for medical applications that often call for positioning accuracy of 1-2mm. As an alternative to moving to more complex models, this paper proposes using sensing to adaptively update model parameters during robot operation. Advantages of this method are that the model is constantly tuning itself to provide high accuracy in the region of the workspace where it is currently operating. It also adapts automatically to changes in robot shape and compliance associated with the insertion and removal of tools through its lumen. As an initial exploration of this approach, a recursive on-line estimator is proposed and evaluated experimentally.

A Motion Planning Approach to Automatic Obstacle Avoidance during Concentric Tube Robot Teleoperation

Concentric tube robots are thin, tentacle-like devices that can move along curved paths and can potentially enable new, less invasive surgical procedures. Safe and effective operation of this type of robot requires that the robot's shaft avoid sensitive anatomical structures (e.g., critical vessels and organs) while the surgeon teleoperates the robot's tip. However, the robot's unintuitive kinematics makes it difficult for a human user to manually ensure obstacle avoidance along the entire tentacle-like shape of the robot's shaft. We present a motion planning approach for concentric tube robot teleoperation that enables the robot to interactively maneuver its tip to points selected by a user while automatically avoiding obstacles along its shaft. We achieve automatic collision avoidance by precomputing a roadmap of collision-free robot configurations based on a description of the anatomical obstacles, which are attainable via volumetric medical imaging. We also mitigate the effects of kinematic modeling error in reaching the goal positions by adjusting motions based on robot tip position sensing. We evaluate our motion planner on a teleoperated concentric tube robot and demonstrate its obstacle avoidance and accuracy in environments with tubular obstacles.

Conformational Modeling of Continuum Structures in Robotics and Structural Biology: A Review

Hyper-redundant (or snakelike) manipulators have many more degrees of freedom than are required to position and orient an object in space. They have been employed in a variety of applications ranging from search-and-rescue to minimally invasive surgical procedures, and recently they even have been proposed as solutions to problems in maintaining civil infrastructure and the repair of satellites. The kinematic and dynamic properties of snakelike robots are captured naturally using a continuum backbone curve equipped with a naturally evolving set of reference frames, stiffness properties, and mass density. When the snakelike robot has a continuum architecture, the backbone curve corresponds with the physical device itself. Interestingly, these same modeling ideas can be used to describe conformational shapes of DNA molecules and filamentous protein structures in solution and in cells. This paper reviews several classes of snakelike robots: (1) hyper-redundant manipulators guided by backbone curves; (2) flexible steerable needles; and (3) concentric tube continuum robots. It is then shown how the same mathematical modeling methods used in these robotics contexts can be used to model molecules such as DNA. All of these problems are treated in the context of a common mathematical framework based on the differential geometry of curves, continuum mechanics, and variational calculus. Both coordinate-dependent Euler-Lagrange formulations and coordinate-free Euler-Poincaré approaches are reviewed.