Towards the clinical integration of an image-guided navigation system for percutaneous liver tumor ablation using freehand 2D ultrasound images

Deep learning-based automatic pipeline for 3D needle localization on intra-procedural 3D MRI

PURPOSE

Accurate and rapid needle localization on 3D magnetic resonance imaging (MRI) is critical for MRI-guided percutaneous interventions. The current workflow requires manual needle localization on 3D MRI, which is time-consuming and cumbersome. Automatic methods using 2D deep learning networks for needle segmentation require manual image plane localization, while 3D networks are challenged by the need for sufficient training datasets. This work aimed to develop an automatic deep learning-based pipeline for accurate and rapid 3D needle localization on in vivo intra-procedural 3D MRI using a limited training dataset.

METHODS

The proposed automatic pipeline adopted Shifted Window (Swin) Transformers and employed a coarse-to-fine segmentation strategy: (1) initial 3D needle feature segmentation with 3D Swin UNEt TRansfomer (UNETR); (2) generation of a 2D reformatted image containing the needle feature; (3) fine 2D needle feature segmentation with 2D Swin Transformer and calculation of 3D needle tip position and axis orientation. Pre-training and data augmentation were performed to improve network training. The pipeline was evaluated via cross-validation with 49 in vivo intra-procedural 3D MR images from preclinical pig experiments. The needle tip and axis localization errors were compared with human intra-reader variation using the Wilcoxon signed rank test, with p < 0.05 considered significant.

RESULTS

The average end-to-end computational time for the pipeline was 6 s per 3D volume. The median Dice scores of the 3D Swin UNETR and 2D Swin Transformer in the pipeline were 0.80 and 0.93, respectively. The median 3D needle tip and axis localization errors were 1.48 mm (1.09 pixels) and 0.98°, respectively. Needle tip localization errors were significantly smaller than human intra-reader variation (median 1.70 mm; p < 0.01).

CONCLUSION

The proposed automatic pipeline achieved rapid pixel-level 3D needle localization on intra-procedural 3D MRI without requiring a large 3D training dataset and has the potential to assist MRI-guided percutaneous interventions.

Modeling of Respiratory Motion to Support the Minimally Invasive Destruction of Liver Tumors

OBJECTIVE

Respiratory movements are a significant factor that may hinder the use of image navigation systems during minimally invasive procedures used to destroy focal lesions in the liver. This article aims to present a method of estimating the displacement of the target point due to respiratory movements during the procedure, working in real time.

METHOD

The real-time method using skin markers and non-rigid registration algorithms has been implemented and tested for various classes of transformation. The method was validated using clinical data from 21 patients diagnosed with liver tumors. For each patient, each marker was treated as a target and the remaining markers as target position predictors, resulting in 162 configurations and 1095 respiratory cycles analyzed. In addition, the possibility of estimating the respiratory phase signal directly from intraoperative US images and the possibility of synchronization with the 4D CT respiratory sequence are also presented, based on ten patients.

RESULTS

The median value of the target registration error (TRE) was 3.47 for the non-rigid registration method using the combination of rigid transformation and elastic body spline curves, and an adaptation of the assessing quality using image registration circuits (AQUIRC) method. The average maximum distance was 3.4 (minimum: 1.6, maximum 6.8) mm.

CONCLUSIONS

The proposed method obtained promising real-time TRE values. It also allowed for the estimation of the TRE at a given geometric margin level to determine the estimated target position. Directions for further quantitative research and the practical possibility of combining both methods are also presented.

Assessing the value of volume navigation during ultrasound-guided radiofrequency- and microwave-ablations of liver lesions

Purpose

The goal of our study was to determine the influence of ultrasound (US)-coupled volume navigation on the use of computed tomography (CT) during minimally-invasive radiofrequency and microwave ablation procedures of liver lesions.

Method

Twenty-five patients with 40 liver lesions of different histological origin were retrospectively analysed. Lesions were ablated following standard protocol, using 1) conventional US-guidance, 2) manual registered volume navigation (mVNav), 3) automatic registered (aVNav) or 4) CT-guidance. In case of ultrasonographically inconspicuous lesions, conventional US-guidance was abandoned and mVNav was used. If mVNav was also unsuccessful, the procedure was either continued with aVNav or CT-guidance. The number, size and location of the lesions targeted using the different approaches were documented.

Results

Of the 40 lesions, sixteen (40.0 %) could be targeted with conventional US-guidance only, sixteen (40.0 %) with mVNav, three (7.5 %) with aVNav and five (12.5 %) only through the use of CT-guidance. Of the three alternatives (mVNav, aVNav and CT only) the mean size of the lesions targeted using mVNav (9.1 ± 4.6 mm) was significantly smaller from those targeted using US-guidance only (20.4 ± 9.4 mm; p < 0.001). The location of the lesions did not influence the selection of the modality used to guide the ablation.

Conclusions

In our cohort, mVNav allowed the ablation procedure to become less dependent on the use of CT. mVNav supported the ablation of lesions smaller than those that could be ablated with US only and doubled the application of minimally-invasive US-guided ablations.

A deep learning approach for 2D ultrasound and 3D CT/MR image registration in liver tumor ablation

BACKGROUND AND OBJECTIVE

Liver tumor ablation is often guided by ultrasound (US). Due to poor image quality, intraoperative US is fused with preoperative computed tomography or magnetic tomography (CT/MR) images to provide visual guidance. As of today, the underlying 2D US to 3D CT/MR registration problem remains a very challenging task.

METHODS

We propose a novel pipeline to address this registration problem. Contrary to previous work, we do not formulate the problem as a regression task, which - for the given registration problem - achieves a low performance regarding accuracy and robustness due to the limited US soft-tissue contrast and the inter-patient variability on liver vessels. Instead, we first estimate the US probe angle roughly by using a classification network. Given this coarse initialization, we then improve the registration by formulation of the problem as a segmentation task, estimating the US plane in the 3D CT/MR through segmentation.

RESULTS

We benchmark our approach on 1035 clinical images from 52 patients, yielding average registration errors of 11.6^° and 4.7 mm, which outperforms the state of the art SVR method[1].

CONCLUSION

Our results show the efficiency of the proposed registration pipeline, which has potential to improve the robustness and accuracy of intraoperative patient registration.

Target registration error reduction for percutaneous abdominal intervention

A real-time methodology that finds spatio-temporal correspondence between the positions of the target point in the pre-treatment 3DCT image and during the procedure was proposed. It based on minimizing the target registration error in III tier registration circuits. Particle Swarm Optimization and Differential Evaluation were used to find optimal values of Elastic Body Spline parameters in the generation of abdominal deformation field. Different transformation classes have been tested: rigid, affine, Thin Plate Spline, Elastic Body Spline. The lowest TRE was obtained for the swarm optimization algorithm - differential evolution for the rigid and affine version: 3.47 and 3.73 mm, respectively.

Two-dimensional ultrasound-computed tomography image registration for monitoring percutaneous hepatic intervention

PURPOSE

Deformable registration of ultrasound (US) and contrast enhanced computed tomography (CECT) images are essential for quantitative comparison of ablation boundaries and dimensions determined using these modalities. This comparison is essential as stiffness-based imaging using US has become popular and offers a nonionizing and cost-effective imaging modality for monitoring minimally invasive microwave ablation procedures. A sensible manual registration method is presented that performs the required CT-US image registration.

METHODS

The two-dimensional (2D) virtual CT image plane that corresponds to the clinical US B-mode was obtained by "virtually slicing" the 3D CT volume along the plane containing non-anatomical landmarks, namely points along the microwave ablation antenna. The initial slice plane was generated using the vector acquired by rotating the normal vector of the transverse (i.e., xz) plane along the angle subtended by the antenna. This plane was then further rotated along the ablation antenna and shifted along with the direction of normal vector to obtain similar anatomical structures, such as the liver surface and vasculature that is visualized on both the CT virtual slice and US B-mode images on 20 patients. Finally, an affine transformation was estimated using anatomic and non-anatomic landmarks to account for distortion between the colocated CT virtual slice and US B-mode image resulting in a final registered CT virtual slice. Registration accuracy was measured by estimating the Euclidean distance between corresponding registered points on CT and US B-mode images.

RESULTS

Mean and SD of the affine transformed registration error was 1.85 ± 2.14 (mm), computed from 20 coregistered data sets.

CONCLUSIONS

Our results demonstrate the ability to obtain 2D virtual CT slices that are registered to clinical US B-mode images. The use of both anatomical and non-anatomical landmarks result in accurate registration useful for validating ablative margins and comparison to electrode displacement elastography based images.

Accuracy of a CT-Ultrasound Fusion Imaging Guidance System Used for Hepatic Percutaneous Procedures

PURPOSE

To evaluate the accuracy of a fusion imaging guidance system using ultrasound (US) and computerized tomography (CT) as a real-time imaging modality for the positioning of a 22-gauge needle in the liver.

MATERIALS AND METHODS

The spatial coordinates of 23 spinal needles placed at the border of hepatic tumors before radiofrequency thermal ablation were determined in 23 patients. Needles were inserted up to the border of the tumor with the use of CT-US fusion imaging. A control CT scan was carried out to compare real (x, y, z) and virtual (x', y', z') coordinates of the tip of the needle (D for distal) and of a point on the needle located 3 cm proximally to the tip (P for proximal).

RESULTS

The mean Euclidian distances were 8.5 ± 4.7 mm and 10.5 ± 5.3 mm for D and P, respectively. The absolute value of mean differences of the 3 coordinates (|x' - x|, |y' - y|, and |z' - z|) were 4.06 ± 0.9, 4.21 ± 0.84, and 4.89 ± 0.89 mm for D and 3.96 ± 0.60, 4.41 ± 0.86, and 7.66 ± 1.27 mm for P. X = |x' - x| and Y = |y' - y| coordinates were <7 mm with a probability close to 1. Z = |z' - z| coordinate was not considered to be larger nor smaller than 7 mm (probability >7 mm close to 50%).

CONCLUSIONS

Positioning errors with the use of US-CT fusion imaging used in this study are not negligible for the insertion of a 22-gauge needle in the liver. Physicians must be aware of such possible errors to adapt the treatment when used for thermal ablation.

Selection of the respiratory phase in minimally invasive interventions for target registration error minimization

BACKGROUND

In minimally invasive surgery, the main challenge is precisely locating the target during the intervention. For abdominal intervention, one of most important factors causing target motion is breathing. The aim of the study is to efficiently predict target localization during the respiratory in breathing cycle.

METHOD

Analysis of target registration error (TRE) for the registration circuits method was used to find the breathing phase corresponding to the preoperative Computed Tomography spatial configuration. Then, Elastic Body Spline (EBS) for modeling deformation field and Particle Swarm Optimization method were used to find the desired values of EBS parameters: ∝ and stiffness were used.

RESULTS

The proposed methodology was verified during experiments conducted on 21 patients diagnosed with liver tumors. This ability of TRE reduction has been achieved for the respiratory phases founded in registration chain analysis.

CONCLUSIONS

The proposed method presents the usability of spatio-temporal analysis of collected real breathing data in order to estimate the position of a target during the respiratory cycle. This method has been developed to perform operations in real-time on a standard workstation.

Automatic liver segmentation in computed tomography using general-purpose shape modeling methods

BACKGROUND

Liver segmentation in computed tomography is required in many clinical applications. The segmentation methods used can be classified according to a number of criteria. One important criterion for method selection is the shape representation of the segmented organ. The aim of the work is automatic liver segmentation using general purpose shape modeling methods.

METHODS

As part of the research, methods based on shape information at various levels of advancement were used. The single atlas based segmentation method was used as the simplest shape-based method. This method is derived from a single atlas using the deformable free-form deformation of the control point curves. Subsequently, the classic and modified Active Shape Model (ASM) was used, using medium body shape models. As the most advanced and main method generalized statistical shape models, Gaussian Process Morphable Models was used, which are based on multi-dimensional Gaussian distributions of the shape deformation field.

RESULTS

Mutual information and sum os square distance were used as similarity measures. The poorest results were obtained for the single atlas method. For the ASM method in 10 analyzed cases for seven test images, the Dice coefficient was above 55[Formula: see text], of which for three of them the coefficient was over 70[Formula: see text], which placed the method in second place. The best results were obtained for the method of generalized statistical distribution of the deformation field. The DICE coefficient for this method was 88.5[Formula: see text] CONCLUSIONS: This value of 88.5 [Formula: see text] Dice coefficient can be explained by the use of general-purpose shape modeling methods with a large variance of the shape of the modeled object-the liver and limitations on the size of our training data set, which was limited to 10 cases. The obtained results in presented fully automatic method are comparable with dedicated methods for liver segmentation. In addition, the deforamtion features of the model can be modeled mathematically by using various kernel functions, which allows to segment the liver on a comparable level using a smaller learning set.

Target Registration Error minimization involving deformable organs using elastic body splines and Particle Swarm Optimization approach

In minimally invasive surgery one of the main challenges is the precise location of the target during the intervention. The aim of the study is to present usability of elastic body splines (EBS) to minimize TRE error. The method to find the desired EBS parameters values is presented with usage of Particle Swarm optimization approach. This ability of TRE minimization has been achieved for the respiratory phases corresponding to minimum FRE for abdominal (especially liver) surgery. The proposed methodology was verified during experiments conducted on 21 patients diagnosed with liver tumors. This method has been developed to perform operations in real-time on a standard workstation.