Integrating robot-assisted ultrasound tracking and 3D needle shape prediction for real-time tracking of the needle tip in needle steering procedures

Towards Design and Development of an MRI Conditional Robot to Enable Curvilinear Transperineal Prostate Biopsy

BACKGROUND

In-bore MRI prostate biopsy offers improved visualisation and detection of significant prostate cancer; however, it is not widely practiced in cancer diagnosis due to its associated costs.

METHODS

This work introduces the first prototype towards a 7-degrees-of-freedom (DOF) MRI-conditional piezoelectrically actuated robotic system for transperineal prostate biopsy. The robot enables needle insertions in the desired trajectories. Kinematic and static models of the active needle as well as automated control of the robot are presented.

RESULTS

It is shown that the controller can force the needle to realize the reference sine and triangular bending angles with an accuracy of 1.78 and 1.88°, respectively, in air. The trajectory tracking capability of the system in free space is shown with an RMS error of 0.86 mm and a standard deviation of 0.36 mm.

CONCLUSIONS

The robot's capability to steer the needle towards target inside a phantom and extract a sample was evaluated.

Curvilinear catheter implantation in HDR prostate brachytherapy: feasibility study

BACKGROUND

High-dose-rate (HDR) brachytherapy (BT) has been acknowledged as a widely utilized treatment for patients with intermediate- and high-risk prostate cancer, despite its side effects such as edema, incontinence, and impotence. Nevertheless, the treatment is consistently limited by the potential danger of excessive irradiation to organs-at-risk (OARs) like the urethra, bladder, and rectum.

PURPOSE

This study aims to introduce curvilinear catheter implantation in the prostate gland for HDR treatment. The objective is to improve the radiation dose distribution by offering access channels conformal to the prostate anatomy. This approach seeks to minimize toxicity to nearby OARs while utilizing a reduced number of needles, potentially leading to improved clinical outcomes.

METHODS

Curvilinear catheters were first pre-planned for an anonymized patient using Oncentra treatment planning system (TPS) and hybrid inverse planning optimization (HIPO) algorithm. The trajectories of the catheters were then analyzed using MATLAB to extract their radius of curvature. Tendon-driven active needles were then used to implant curvilinear catheters inside an anthropomorphic phantom.

RESULTS

Proposed curvilinear catheter implantation resulted in significant improvement in terms of dosimetric constraints to the OARs and coverage to the prostate. Tendon-driven active needles were shown to be capable of realizing the required pre-planned curvatures inside prostate. It was shown that the active needle can realize a desired radius of curvature and a desired trajectory with an average accuracy of 9.1 ± 8.6  and 1.27 ± 0.50 mm in air and inside a tissue-mimicking phantom, respectively.

CONCLUSION

This work demonstrates the feasibility of using tendon-driven active curvilinear catheter implantation in prostate to improve the outcomes of HDR-BT via improved radiation dose distribution to the prostate and reduced toxicity to the OARs.

Closed-Loop Control of a Tendon-Driven Active Needle for Tip Tracking at Desired Bending Angle for High-Dose-Rate Prostate Brachytherapy

Prostate cancer is the second most common malignancy in American men. High-dose-rate brachytherapy is a popular treatment technique in which a large, localized radiation dose is used to kill cancer. Utilization of curvilinear catheter implantation inside the prostate gland to provide access channels to host the radiation source has shown superiority in terms of improved dosimetric constraints compared to straight needles. To this aim, we have introduced an active needle to curve inside the prostate conformal to the patient's specific anatomical relationship for improved dose distribution to the prostate and reduced toxicity to the organs at risk (OARs). This work presents closed-loop control of our tendon-driven active needle in water medium and air using the position feedback of the tip obtained in real time from an ultrasound (US) or an electromagnetic (EM) tracking sensor, respectively. The active needle consists of a compliant flexure section to realize bending in two directions via actuation of two internal tendons. Tracking errors using US and EM tracker are estimated and compared. Results show that the bending angle of the active needle could be controlled using position feedback of the US or the EM tracking system with a bending angle error of less than 1.00 degree, when delay is disregarded. It is concluded that the actuation system and controller, presented in this work, are able to realize a desired bending angle at the active needle tip with reasonable accuracy paving the path for tip tracking and manipulation control evaluations in a prostate brachytherapy.

Hybrid Deep Learning and Model-Based Needle Shape Prediction

Needle insertion using flexible bevel tip needles are a common minimally-invasive surgical technique for prostate cancer interventions. Flexible, asymmetric bevel tip needles enable physicians for complex needle steering techniques to avoid sensitive anatomical structures during needle insertion. For accurate placement of the needle, predicting the trajectory of these needles intra-operatively would greatly reduce the need for frequently needle reinsertions thus improving patient comfort and positive outcomes. However, predicting the trajectory of the needle during insertion is a complex task that has yet to be solved due to random needle-tissue interactions. In this paper, we present and validate for the first time a hybrid deep learning and model-based approach to handle the intra-operative needle shape prediction problem through, leveraging a validated Lie-group theoretic model for needle shape representation. Furthermore, we present a novel self-supervised learning and method in conjunction with the Lie-group shape model for training these networks in the absence of data, enabling further refinement of these networks with transfer learning. Needle shape prediction was performed in single-layer and double-layer homogeneous phantom tissue for C- and S-shape needle insertions. Our method demonstrates an average root-mean-square prediction error of 1.03 mm over a dataset containing approximately 3,000 prediction samples with maximum prediction steps of 110 mm.

A Survey of Needle Steering Approaches in Minimally Invasive Surgery

In virtue of a curved insertion path inside tissues, needle steering techniques have revealed the potential with the assistance of medical robots and images. The superiority of this technique has been preliminarily verified with several maneuvers: target realignment, obstacle circumvention, and multi-target access. However, the momentum of needle steering approaches in the past decade leads to an open question-"How to choose an applicable needle steering approach for a specific clinical application?" This survey discusses this question in terms of design choices and clinical considerations, respectively. In view of design choices, this survey proposes a hierarchical taxonomy of current needle steering approaches. Needle steering approaches of different manipulations and designs are classified to systematically review the design choices and their influences on clinical treatments. In view of clinical consideration, this survey discusses the steerability and acceptability of the current needle steering approaches. On this basis, the pros and cons of the current needle steering approaches are weighed and their suitable applications are summarized. At last, this survey concluded with an outlook of the needle steering techniques, including the potential clinical applications and future developments in mechanical design.

A mechanics-based model for a tendon-driven active needle navigating inside a multiple-layer tissue

Percutaneous minimally invasive procedures such brachytherapy and biopsy require a flexible active needle for precise movement inside tissue and accurate placement at target positions for higher success rates for diagnosis and treatment, respectively. In a previous work, we presented a tendon-driven active needle to navigate inside tissue. This work presents a new model to predict the deflection of the tendon-driven needle while steering in a multiple-layer soft tissue. A multi-layer phantom tissue with different localized stiffness was developed for needle insertion tests followed by indentation tests to identify its mechanical properties. Using a robot that inserts and actively bends the tendon-driven needle inside the soft tissue while simultaneously tracking the needle through ultrasound imaging, various experiments were conducted for model validation. The proposed model was verified by comparing the simulation results to the empirical data. The results demonstrated the accuracy of the model in predicting the tendon-driven needle deflection in multiple-layer (different stiffness) soft tissue.

Modern Image-Guided Surgery: A Narrative Review of Medical Image Processing and Visualization

Medical image analysis forms the basis of image-guided surgery (IGS) and many of its fundamental tasks. Driven by the growing number of medical imaging modalities, the research community of medical imaging has developed methods and achieved functionality breakthroughs. However, with the overwhelming pool of information in the literature, it has become increasingly challenging for researchers to extract context-relevant information for specific applications, especially when many widely used methods exist in a variety of versions optimized for their respective application domains. By being further equipped with sophisticated three-dimensional (3D) medical image visualization and digital reality technology, medical experts could enhance their performance capabilities in IGS by multiple folds. The goal of this narrative review is to organize the key components of IGS in the aspects of medical image processing and visualization with a new perspective and insights. The literature search was conducted using mainstream academic search engines with a combination of keywords relevant to the field up until mid-2022. This survey systemically summarizes the basic, mainstream, and state-of-the-art medical image processing methods as well as how visualization technology like augmented/mixed/virtual reality (AR/MR/VR) are enhancing performance in IGS. Further, we hope that this survey will shed some light on the future of IGS in the face of challenges and opportunities for the research directions of medical image processing and visualization.

Learning-based needle tip tracking in 2D ultrasound by fusing visual tracking and motion prediction

Visual trackers are the most commonly adopted approach for needle tip tracking in ultrasound (US)-based procedures. However, they often perform unsatisfactorily in biological tissues due to the significant background noise and anatomical occlusion. This paper presents a learning-based needle tip tracking system, which consists of not only a visual tracking module, but also a motion prediction module. In the visual tracking module, two sets of masks are designed to improve the tracker's discriminability, and a template update submodule is used to keep up to date with the needle tip's current appearance. In the motion prediction module, a Transformer network-based prediction architecture estimates the target's current position according to its historical position data to tackle the problem of target's temporary disappearance. A data fusion module then integrates the results from the visual tracking and motion prediction modules to provide robust and accurate tracking results. Our proposed tracking system showed distinct improvement against other state-of-the-art trackers during the motorized needle insertion experiments in both gelatin phantom and biological tissue environments (e.g. 78% against <60% in terms of the tracking success rate in the most challenging scenario of "In-plane-static" during the tissue experiments). Its robustness was also verified in manual needle insertion experiments under varying needle velocities and directions, and occasional temporary needle tip disappearance, with its tracking success rate being >18% higher than the second best performing tracking system. The proposed tracking system, with its computational efficiency, tracking robustness, and tracking accuracy, will lead to safer targeting during existing clinical practice of US-guided needle operations and potentially be integrated in a tissue biopsy robotic system.

Teleoperated and Automated Control of a Robotic Tool for Targeted Prostate Biopsy

This work presents a robotic tool with bidirectional manipulation and control capabilities for targeted prostate biopsy interventions. Targeted prostate biopsy is an effective image-guided technique that results in detection of significant cancer with fewer cores and lower number of unnecessary biopsies compared to systematic biopsy. The robotic tool comprises of a compliant flexure section fabricated on a nitinol tube that enables bidirectional bending via actuation of two internal tendons, and a biopsy mechanism for extraction of tissue samples. The kinematic and static models of the compliant flexure section, as well as teleoperated and automated control of the robotic tool are presented and validated with experiments. It was shown that the controller can force the tip of the robotic tool to follow sinusoidal set-point positions with reasonable accuracy in air and inside a phantom tissue. Finally, the capability of the robotic tool to bend, reach targeted positions inside a phantom tissue, and extract a biopsy sample is evaluated.

Kinematics modelling and dynamics analysis of an SMA-actuated active flexible needle for feedback-controlled manipulation in phantom

Percutaneous needle-based procedures such as prostate brachytherapy demands for accurate placement of the needle tip at target locations. Recently, robotic needle insertion systems have been made available to help physicians in needle guidance and control inside tissue. It is often challenging to obtain an accurate and real-time position of the needle tip in clinical practice using medical imaging techniques. However, this information is vital for closed-loop control of the needles inside tissue. This work presents an SMA-actuated active flexible needle that is controlled inside a phantom without a need for a position sensor or a medical imaging device. The needle tip position feedback is found using shape sensing capabilities of the embedded SMA-wire actuators and a force sensor at the needle base. Three models were characterized and used to estimate needle tip position in real time. The control scheme was then tested on the active flexible needle to track a desired triangular trajectory in a phantom. It was shown that the control scheme presented in this work was able to manipulate the needle in this path with a reasonable accuracy.

HDR Brachytherapy Planning using Active Needles - Preliminary Investigation on Dose Planning

In this study we present a new approach to plan a high-dose-rate (HDR) prostate brachytherapy (BT) using active needles recently developed by our group. The active needles realize bi-directional bending inside the tissue, and thereby more compliant with the patient's anatomy compared with conventional straight needles. A computational method is presented to first generate a needle arrangement configuration based on the patient's prostate anatomy. The needle arrangement is generated to cover the prostate volume, providing accessible channels for the radiation source during a HDR BT. The needle arrangement configuration avoids healthy organs and prevents needle collision inside the body. Then a treatment plan is proposed to ensure sufficient prescribed dosage to the whole prostate gland. The method is applied to a prostate model reconstructed from an anonymized patient to show the feasibility of this method. Finally, the active needle's capability to generate the required bending is shown. We have shown that our method is able to automatically generate needle arrangement configuration using active needles, and plan for a treatment that meets the dose objectives while using fewer needles (about 20% of conventional straight needles) than the conventional HDR BT performed by straight needles.

Modeling and Operator Control of a Robotic Tool for Bidirectional Manipulation in Targeted Prostate Biopsy

This work introduces design, manipulation, and operator control of a bidirectional robotic tool for minimally invasive targeted prostate biopsy. The robotic tool is purposed to be used as a compliant flexure section of active biopsy needles. The design of the robotic tool comprises of a flexure section fabricated on a nitinol tube that enables bidirectional bending via actuation of two internal tendons. The statics of the flexure section is presented and validated with experimental data. Finally, the capability of the robotic tool to reach targeted positions inside prostate gland is evaluated.

Steering a Tendon-Driven Needle in High-Dose-Rate Prostate Brachytherapy for Patients with Pubic Arch Interference

High-dose-rate brachytherapy (HDR BT) is a radiation therapy that places radioactive sources at cancerous tissue using needles. HDR BT offers better dose conformality and sparing of clinical structures, lower operator dependency, and fewer acute irritative symptoms compared to the other form of BT (low-dose-rate (LDR)). However, use of HDR BT is limited for patients with pubic arch interference, where the transperineal path to the prostate is blocked. This study aims to introduce a tendon-driven needle that can bend inside tissue to reach desired positions inside prostate. Initial experiments in a phantom tissue showed the feasibility of the needle to get around the pubic arch for placement at hard-to-reach target positions.