Active Localization and Tracking of Needle and Target in Robotic Image-Guided Intervention Systems

Deformable cardiac surface tracking by adaptive estimation algorithms

This study presents a particle filter based framework to track cardiac surface from a time sequence of single magnetic resonance imaging (MRI) slices with the future goal of utilizing the presented framework for interventional cardiovascular magnetic resonance procedures, which rely on the accurate and online tracking of the cardiac surface from MRI data. The framework exploits a low-order parametric deformable model of the cardiac surface. A stochastic dynamic system represents the cardiac surface motion. Deformable models are employed to introduce shape prior to control the degree of the deformations. Adaptive filters are used to model complex cardiac motion in the dynamic model of the system. Particle filters are utilized to recursively estimate the current state of the system over time. The proposed method is applied to recover biventricular deformations and validated with a numerical phantom and multiple real cardiac MRI datasets. The algorithm is evaluated with multiple experiments using fixed and varying image slice planes at each time step. For the real cardiac MRI datasets, the average root-mean-square tracking errors of 2.61 mm and 3.42 mm are reported respectively for the fixed and varying image slice planes. This work serves as a proof-of-concept study for modeling and tracking the cardiac surface deformations via a low-order probabilistic model with the future goal of utilizing this method for the targeted interventional cardiac procedures under MR image guidance. For the real cardiac MRI datasets, the presented method was able to track the points-of-interests located on different sections of the cardiac surface within a precision of 3 pixels. The analyses show that the use of deformable cardiac surface tracking algorithm can pave the way for performing precise targeted intracardiac ablation procedures under MRI guidance. The main contributions of this work are twofold. First, it presents a framework for the tracking of whole cardiac surface from a time sequence of single image slices. Second, it employs adaptive filters to incorporate motion information in the tracking of nonrigid cardiac surface motion for temporal coherence.

MRI Distortion Correction and Robot-to-MRI Scanner Registration for an MRI-Guided Robotic System

Magnetic resonance imaging (MRI) guided robotic procedures require safe robotic instrument navigation and precise target localization. This depends on reliable tracking of the instrument from MR images, which requires accurate registration of the robot to the scanner. A novel differential image based robot-to-MRI scanner registration approach is proposed that utilizes a set of active fiducial coils, where background subtraction method is employed for coil detection. In order to use the presented preoperative registration approach jointly with the real-time high speed MRI image acquisition and reconstruction methods in real-time interventional procedures, the effects of the geometric MRI distortion in robot to scanner registration is analyzed using a custom distortion mapping algorithm. The proposed approach is validated by a set of target coils placed within the workspace, employing multi-planar capabilities of the scanner. Registration and validation errors are respectively 2.05 mm and 2.63 mm after the distortion correction showing an improvement of respectively 1.08 mm and 0.14 mm compared to the results without distortion correction.

Automatic needle tracking using Mask R-CNN for MRI-guided percutaneous interventions

PURPOSE

Accurate needle tracking provides essential information for MRI-guided percutaneous interventions. Passive needle tracking using MR images is challenged by variations of the needle-induced signal void feature in different situations. This work aimed to develop an automatic needle tracking algorithm for MRI-guided interventions based on the Mask Region Proposal-Based Convolutional Neural Network (R-CNN).

METHODS

Mask R-CNN was adapted and trained to segment the needle feature using 250 intra-procedural images from 85 MRI-guided prostate biopsy cases and 180 real-time images from MRI-guided needle insertion in ex vivo tissue. The segmentation masks were passed into the needle feature localization algorithm to extract the needle feature tip location and axis orientation. The proposed algorithm was tested using 208 intra-procedural images from 40 MRI-guided prostate biopsy cases, and 3 real-time MRI datasets in ex vivo tissue. The algorithm results were compared with human-annotated references.

RESULTS

In prostate datasets, the proposed algorithm achieved needle feature tip localization error with median Euclidean distance (dxy) of 0.71 mm and median difference in axis orientation angle (dθ) of 1.28°, respectively. In 3 real-time MRI datasets, the proposed algorithm achieved consistent dynamic needle feature tracking performance with processing time of 75 ms/image: (a) median dxy = 0.90 mm, median dθ = 1.53°; (b) median dxy = 1.31 mm, median dθ = 1.9°; (c) median dxy = 1.09 mm, median dθ = 0.91°.

CONCLUSIONS

The proposed algorithm using Mask R-CNN can accurately track the needle feature tip and axis on MR images from in vivo intra-procedural prostate biopsy cases and ex vivo real-time MRI experiments with a range of different conditions. The algorithm achieved pixel-level tracking accuracy in real time and has potential to assist MRI-guided percutaneous interventions.

Tip Estimation Method in Phantoms for Curved Needle Using 2D Transverse Ultrasound Images

Flexible needles have been widely used in minimally invasive surgeries, especially in percutaneous interventions. Among the interventions, tip position of the curved needle is very important, since it directly affects the success of the surgeries. In this paper, we present a method to estimate the tip position of a long-curved needle by using 2D transverse ultrasound images from a robotic ultrasound system. Ultrasound is first used to detect the cross section of long-flexible needle. A new imaging approach is proposed based on the selection of numbers of pixels with a higher gray level, which can directly remove the lower gray level to highlight the needle. After that, the needle shape tracking method is proposed by combining the image processing with the Kalman filter by using 3D needle positions, which develop a robust needle tracking procedure from 1 mm to 8 mm scan intervals. Shape reconstruction is then achieved using the curve fitting method. Finally, the needle tip position is estimated based on the curve fitting result. Experimental results showed that the estimation error of tip position is less than 1 mm within 4 mm scan intervals. The advantage of the proposed method is that the shape and tip position can be estimated through scanning the needle’s cross sections at intervals along the direction of needle insertion without detecting the tip.

Automatic Needle Segmentation and Localization in MRI With 3-D Convolutional Neural Networks: Application to MRI-Targeted Prostate Biopsy

Image guidance improves tissue sampling during biopsy by allowing the physician to visualize the tip and trajectory of the biopsy needle relative to the target in MRI, CT, ultrasound, or other relevant imagery. This paper reports a system for fast automatic needle tip and trajectory localization and visualization in MRI that has been developed and tested in the context of an active clinical research program in prostate biopsy. To the best of our knowledge, this is the first reported system for this clinical application and also the first reported system that leverages deep neural networks for segmentation and localization of needles in MRI across biomedical applications. Needle tip and trajectory were annotated on 583 T2-weighted intra-procedural MRI scans acquired after needle insertion for 71 patients who underwent transperineal MRI-targeted biopsy procedure at our institution. The images were divided into two independent training-validation and test sets at the patient level. A deep 3-D fully convolutional neural network model was developed, trained, and deployed on these samples. The accuracy of the proposed method, as tested on previously unseen data, was 2.80-mm average in needle tip detection and 0.98° in needle trajectory angle. An observer study was designed in which independent annotations by a second observer, blinded to the original observer, were compared with the output of the proposed method. The resultant error was comparable to the measured inter-observer concordance, reinforcing the clinical acceptability of the proposed method. The proposed system has the potential for deployment in clinical routine.