Robust path planning for flexible needle insertion using Markov decision processes

A survey on puncture models and path planning algorithms of bevel-tipped flexible needles

Percutaneous needle insertion is a minimally invasive surgery with broad medical application prospects, such as biopsy and brachytherapy. However, the currently adopted rigid needles have limitations, as they cannot bypass obstacles or correct puncture deviations and can only travel along a straight path. Bevel-tip flexible needles are increasingly being adopted to address these issues, owing to their needle body's ease of deformation and bending. Successful puncture of flexible needles relies on accurate models and path planning, ensuring the needle reaches the target while avoiding vital tissues. This review investigates puncture models and path-planning algorithms by reviewing recent literature, focusing on the path-planning part. According to the literature, puncture models can be divided into three types: mechanical, finite element method (FEM), and kinematic models, while path-planning algorithms are categorized and discussed following the division used for mobile robots, which differs from the conventional approach for flexible needles-an innovation in this review. This review systematically summarizes the following categories: graph theory search, sampling-based, intelligent search, local obstacle avoidance, and other algorithms, including their implementation, advantages, and disadvantages, to further explore the potential to overcome obstacles in path planning for minimally invasive puncture needles. Finally, this study proposes future development trends in path-planning algorithms, providing possible directions for subsequent research for bevel-tipped flexible needles. This research aims to provide a resource for researchers to quickly learn about common path-planning algorithms, their backgrounds, and puncture models.

Needle-tissue interaction model based needle path planning method

BACKGROUND AND OBJECTIVE

In needle insertion procedure, needle deflection and target movement will affect targeting accuracy. Existing planning algorithms rely on predetermined interaction force and parameters, which increase the targeting error for the patient-specific difference. In this paper, we proposed a needle-tissue interaction model based needle path planning method with patient-specific parameter identification algorithm, which is able to use iteration learning control and interaction model predicted information to improve targeting accuracy with the consideration of patient-specific differences.

METHODS

A 3D needle-tissue interaction deformation model has been constructed using local constraint method. The model, termed as the full computation model, predicts the needle-tissue interaction force using a Kriging-based model as well as the target movement and needle deflection simultaneously only requiring patient specific parameters. Needle paths without incorporating deformation, which is called static path, are generated by rapidly-exploring random trees algorithm first. Then, the needle-tissue interaction deformation model can calculate force and deformation of the static path and iterative learning control can correct the targeting error of moved target. In addition, the intraoperative parameter identification algorithm is proposed to identify patient-specific parameter. Simulations are carried out to verify the full computation model and needle path planning. A testbed is constructed and experiments are designed to validate the proposed method using phantom with common lesion-size obstacle markers and target markers. The deformation of tissue and needle are captured through charge coupled device camera.

RESULTS

Simulation results indicated the full computation model can simulate the needle-tissue interaction process and the proposed method can achieve needle path planning incorporating tissue deformation. Experiment results indicated the tissue deformation and needle deflection agree between model prediction and experiments. The proposed path planning method can reduce targeting error from maximum of 3.89 mm without incorporating deformation to less than 1 mm in 4 phantom experiments.

CONCLUSIONS

The full computation model based needle path planning is verified to be effective by experiments. The planning accuracy is improved based on the deformation predicted by full computation model and the desired accuracy is achieved.

Review of research on path planning and control methods of flexible steerable needle puncture robot

In the field of minimally invasive interventional therapy, the related research on the soft tissue puncture robot and its technology based on the flexible steerable needle as a research hot topic at present, and it has been developed rapidly in the past ten years. In order to better understand the development status of the flexible steerable needle puncture (FSNP) robot and provide reference for its design and improvement in subsequent research, it is necessary to introduce in two aspects of FSNP robot: the puncture path planning and the control methods. First, this article introduced the concept of the FSNP technology, and the necessity of the application of FSNP soft tissue robot in minimally invasive interventional surgery. Second, this article mainly introduced the principle of FSNP, the path planning of FSNP, the navigation and positioning control of the needle tip of the flexible steerable needle, the control method of FSNP system, and the controllable flexible needle. Finally, combined with the above analysis and introduction, it was pointed out that FSNP soft tissue robot and its related technology would be an important development direction in the field of minimally invasive interventional therapy in the future, and the current existing problems were pointed out. Meanwhile, the development trend of FSNP robot control technology was summarized and prospected.

Lie-Group Theoretic Approach to Shape-Sensing Using FBG-Sensorized Needles Including Double-Layer Tissue and S-Shape Insertions

Flexible bevel-tipped needles are often used for needle insertion in minimally-invasive surgical techniques due to their ability to be steered in cluttered environments. Shapesensing enables physicians to determine the location of needles intra-operatively without requiring radiation of the patient, enabling accurate needle placement. In this paper, we validate a theoretical method for flexible needle shape-sensing that allows for complex curvatures, extending upon a previous sensor-based model. This model combines curvature measurements from fiber Bragg grating (FBG) sensors and the mechanics of an inextensible elastic rod to determine and predict the 3D needle shape during insertion. We evaluate the model's shape sensing capabilities in C- and S-shape insertions in single-layer isotropic tissue, and C-shape insertions in two-layer isotropic tissue. Experiments on a four-active area, FBG-sensorized needle were performed in varying tissue stiffnesses and insertion scenarios under stereo vision to provide the 3D ground truth needle shape. The results validate a viable 3D needle shape-sensing model accounting for complex curvatures in flexible needles with mean needle shape sensing root-mean-square errors of 0.160 ± 0.055 mm over 650 needle insertions.

Robot-assisted flexible needle insertion using universal distributional deep reinforcement learning

PURPOSE

Flexible needle insertion is an important minimally invasive surgery approach for biopsy and radio-frequency ablation. This approach can minimize intraoperative trauma and improve postoperative recovery. We propose a new path planning framework using multi-goal deep reinforcement learning to overcome the difficulties in uncertain needle-tissue interactions and enhance the robustness of robot-assisted insertion process.

METHODS

This framework utilizes a new algorithm called universal distributional Q-learning (UDQL) to learn a stable steering policy and perform risk management by visualizing the learned Q-value distribution. To further improve the robustness, universal value function approximation is leveraged in the training process of UDQL to maximize generalization and connect to diagnosis by adapting fast re-planning and transfer learning.

RESULTS

Computer simulation and phantom experimental results show our proposed framework can securely steer flexible needles with high insertion accuracy and robustness. The framework also improves robustness by providing distribution information to clinicians for diagnosis and decision making during surgery.

CONCLUSIONS

Compared with previous methods, the proposed framework can perform multi-target needle insertion through single insertion point qunder continuous state space model with higher accuracy and robustness.