A data-driven soft sensor for needle deflection in heterogeneous tissue using just-in-time modelling

Hybrid Deep Learning and Model-Based Needle Shape Prediction

Needle insertion using flexible bevel tip needles are a common minimally-invasive surgical technique for prostate cancer interventions. Flexible, asymmetric bevel tip needles enable physicians for complex needle steering techniques to avoid sensitive anatomical structures during needle insertion. For accurate placement of the needle, predicting the trajectory of these needles intra-operatively would greatly reduce the need for frequently needle reinsertions thus improving patient comfort and positive outcomes. However, predicting the trajectory of the needle during insertion is a complex task that has yet to be solved due to random needle-tissue interactions. In this paper, we present and validate for the first time a hybrid deep learning and model-based approach to handle the intra-operative needle shape prediction problem through, leveraging a validated Lie-group theoretic model for needle shape representation. Furthermore, we present a novel self-supervised learning and method in conjunction with the Lie-group shape model for training these networks in the absence of data, enabling further refinement of these networks with transfer learning. Needle shape prediction was performed in single-layer and double-layer homogeneous phantom tissue for C- and S-shape needle insertions. Our method demonstrates an average root-mean-square prediction error of 1.03 mm over a dataset containing approximately 3,000 prediction samples with maximum prediction steps of 110 mm.

Model-free control for autonomous prevention of adverse events in robotics

Introduction: Preventive control is a critical feature in autonomous technology to ensure safe system operations. One application where safety is most important is robot-assisted needle interventions. During incisions into a tissue, adverse events such as mechanical buckling of the needle shaft and tissue displacements can occur on encounter with stiff membranes causing potential damage to the organ. Methods: To prevent these events before they occur, we propose a new control subroutine that autonomously chooses a) a reactive mechanism to stop the insertion procedure when a needle buckling or a severe tissue displacement event is predicted and b) an adaptive mechanism to continue the insertion procedure through needle steering control when a mild tissue displacement is detected. The subroutine is developed using a model-free control technique due to the nonlinearities of the unknown needle-tissue dynamics. First, an improved version of the model-free adaptive control (IMFAC) is developed by computing a fast time-varying partial pseudo derivative analytically from the dynamic linearization equation to enhance output convergence and robustness against external disturbances. Results and Discussion: Comparing IMFAC and MFAC algorithms on simulated nonlinear systems in MATLAB, IMFAC shows 20% faster output convergence against arbitrary disturbances. Next, IMFAC is integrated with event prediction algorithms from prior work to prevent adverse events during needle insertions in real time. Needle insertions in gelatin tissues with known environments show successful prevention of needle buckling and tissue displacement events. Needle insertions in biological tissues with unknown environments are performed using live fluoroscopic imaging as ground truth to verify timely prevention of adverse events. Finally, statistical ANOVA analysis on all insertion data shows the robustness of the prevention algorithm to various needles and tissue environments. Overall, the success rate of preventing adverse events in needle insertions through adaptive and reactive control was 95%, which is important toward achieving safety in robotic needle interventions.

Review of research on path planning and control methods of flexible steerable needle puncture robot

In the field of minimally invasive interventional therapy, the related research on the soft tissue puncture robot and its technology based on the flexible steerable needle as a research hot topic at present, and it has been developed rapidly in the past ten years. In order to better understand the development status of the flexible steerable needle puncture (FSNP) robot and provide reference for its design and improvement in subsequent research, it is necessary to introduce in two aspects of FSNP robot: the puncture path planning and the control methods. First, this article introduced the concept of the FSNP technology, and the necessity of the application of FSNP soft tissue robot in minimally invasive interventional surgery. Second, this article mainly introduced the principle of FSNP, the path planning of FSNP, the navigation and positioning control of the needle tip of the flexible steerable needle, the control method of FSNP system, and the controllable flexible needle. Finally, combined with the above analysis and introduction, it was pointed out that FSNP soft tissue robot and its related technology would be an important development direction in the field of minimally invasive interventional therapy in the future, and the current existing problems were pointed out. Meanwhile, the development trend of FSNP robot control technology was summarized and prospected.

Study of the surgical needle and biological soft tissue interaction phenomenon during insertion process for medical application: A Survey

The insertion of the surgical needle in soft tissue has involved significant interest in the current time because of its purpose in minimally invasive surgery (MIS) and percutaneous events like biopsies, PCNL, and brachytherapy. This study represents a review of the existing condition of investigation on insertion of a surgical needle in biological living soft tissue material. As observes the issue from numerous phases, like, analysis of the cutting forces modeling (insertion), tissue material deformation, analysis of the needle deflection for the period of the needle insertion, and the robot-controlled insertion procedures. All analysis confirms that the total needle insertion force is the total of dissimilar forces spread sideways the shaft of the insertion needle for example cutting force, stiffness force, and frictional force. Various investigations have analyzed all these kinds of forces during the needle insertion process. The force data in several measures are applied for recognizing the biological tissue materials as the needle is penetrated or for path planning. The deflection of the needle during insertion and tissue material deformation is the main trouble for defined needle placing and efforts have been prepared to model them. Applying existing models numerous insertion methods are established that are discussed in this review.

Image-guided robots for low dose rate prostate brachytherapy: Perspectives on safety in design and use

BACKGROUND

Image-guided brachytherapy (BT) robots can be used to assist urologists during seed implantation, thereby improving therapeutic effects. However, safety issues must be considered in the design of such robots, including their structure, mechanical movements, function, materials and actuators. Previous reviews focused on image-guided prostate BT robot technology (e.g., imaging and robot navigation technology and robot system introduction); however, this review is the first time that safety issues have been investigated as part of a study on low-dose-rate (LDR) prostate BT robots.

METHODS

Multiple electronic databases were searched for LDR prostate BT robot articles published during the last 24 years (1996-2020), with a particular focus on two aspects of robots: safety in design and use.

RESULTS

We retrieved a total of 26 LDR prostate BT robots. BT robots were divided into ultrasound, computed tomography, magnetic resonance imaging and fusion-guided systems. The conditions associated with each system were then analysed to develop a set of requirements for the safety of prostate BT robots. Recommendations are also provided for future BT robot development.

CONCLUSIONS

The transrectal approach for prostate seed implantation is safer than the traditional transperineal approach. Research into the control of a steerable needle by the urologists and robot, the needle deflection model, and robotic automated needle changing and seed injection equipment should be pursued in a future study.

Developing a novel force forecasting technique for early prediction of critical events in robotics

Safety critical events in robotic applications can often be characterized by forces between the robot end-effector and the environment. One application in which safe interaction between the robot and environment is critical is in the area of medical robots. In this paper, we propose a novel Compact Form Dynamic Linearization Model-Free Prediction (CFDL-MFP) technique to predict future values of any time-series sensor data, such as interaction forces. Existing time series forecasting methods have high computational times which motivates the development of a novel technique. Using Autoregressive Integrated Moving Average (ARIMA) forecasting as benchmark, the performance of the proposed model was evaluated in terms of accuracy, computation efficiency, and stability on various force profiles. The proposed algorithm was 11% more accurate than ARIMA and maximum computation time of CFDL-MFP was 4ms, compared to ARIMA (7390ms). Furthermore, we evaluate the model in the special case of predicting needle buckling events, before they occur, by using only axial force and needle-tip position data. The model was evaluated experimentally for robustness with steerable needle insertions into different tissues including gelatin and biological tissue. For a needle insertion velocity of 2.5mm/s, the proposed algorithm was able to predict needle buckling 2.03s sooner than human detections. In biological tissue, no false positive or false negative buckling detections occurred and the rates were low in artificial tissue. The proposed forecasting model can be used to ensure safe robot interactions with delicate environments by predicting adverse force-based events before they occur.

A MECHANICS-BASED MODEL FOR SIMULATING THE NEEDLE DEFLECTION IN TRANSVERSE ISOTROPIC TISSUE FOR A PERCUTANEOUS PUNCTURE

During the percutaneous puncture for robot-assisted brachytherapy, a medical needle is usually inserted into fiber-structured soft tissue which has transverse isotropic elasticity, such as muscle and skin, to deliver radioactive seeds that kill cancer cells. To place the radioactive seeds more accurately, it is necessary to assess the effect of the transverse isotropic elasticity on the needle deflection. A mechanics-based model for simulating the needle deflection in transverse isotropic tissue is developed in this paper. The anisotropic needle–tissue interaction forces are estimated and used as inputs to drive the model for simulating needle deflections for different insertion orientation angles. Automatic insertion experiments were performed on a single-layered porcine muscle at five different insertion orientation angles. The results show that the maximum difference in the tip deflection for the different insertion orientation angles is 2.99[Formula: see text]mm when the insertion depth is 50[Formula: see text]mm. The maximum simulated error of the needle axis deflection is 0.62[Formula: see text]mm for all insertion orientation angles. The developed model can successfully simulate the needle deflections inside transverse isotropic tissue for different insertion orientation angles. This work is useful for predicting and compensating for the deflection error for automatic needle insertion.