Tissue deformation and shape models in image-guided interventions: a discussion paper

Analysis of emerging trends and hot spots in respiratory biomechanics from 2003 to 2022 based on CiteSpace

Introduction: With the global prevalence of coronavirus disease 2019 (COVID-19), an increasing number of people are experiencing respiratory discomfort. Respiratory biomechanics can monitor breathing patterns and respiratory movements and it is easier to prevent, diagnose, treat or rehabilitate. However, there is still a lack of global knowledge structure in the field of respiratory biomechanics. With the help of CiteSpace software, we aim to help researchers identify potential collaborators and collaborating institutions, hotspots and research frontiers in respiratory biomechanics. Methods: Articles on respiratory biomechanics from 2003 to 2022 were retrieved from the Web of Science Core Collection by using a specific strategy, resulting a total of 2,850 publications. We used CiteSpace 6.1.R6 to analyze the year of publication, journal/journals cited, country, institution, author/authors cited, references, keywords and research trends. Co-citation maps were created to visually observe research hot spots and knowledge structures. Results and discussion: The number of annual publications gradually increased over the past 20 years. Medical Physics published the most articles and had the most citations in this study. The United States was the most influential country, with the highest number and centrality of publications. The most productive and influential institution was Harvard University in the United States. Keall PJ was the most productive author and MCCLELLAND JR was the most cited authors The article by Keall PJ (2006) article (cocitation counts: 55) and the article by McClelland JR (2013) were the most representative and symbolic references, with the highest cocitation number and centrality, respectively. The top keywords were "radiotherapy", "volume", and "ventilation". The top Frontier keywords were "organ motion," "deep inspiration," and "deep learning". The keywords were clustered to form seven labels. Currently, the main area of research in respiratory biomechanics is respiratory motion related to imaging techniques. Future research may focus on respiratory assistance techniques and respiratory detection techniques. At the same time, in the future, we will pay attention to personalized medicine and precision medicine, so that people can monitor their health status anytime and anywhere.

Complete region of interest reconstruction by fusing multiview deformable three-dimensional transesophageal echocardiography images

BACKGROUND

While three-dimensional transesophageal echocardiography (3D TEE) has been increasingly used for assessing cardiac anatomy and function, it still suffers from a limited field of view (FoV) of the ultrasound transducer. Therefore, it is difficult to examine a complete region of interest without moving the transducer. Existing methods extend the FoV of 3D TEE images by mosaicing multiview static images, which requires synchronization between 3D TEE images and electrocardiogram (ECG) signal to avoid deformations in the images and can only get the widened image at a specific phase.

PURPOSE

This work aims to develop a novel multiview nonrigid registration and fusion method to extend the FoV of 3D TEE images at different cardiac phases, avoiding the bias toward the specifically chosen phase.

METHODS

A multiview nonrigid registration and fusion method is proposed to enlarge the FoV of 3D TEE images by fusing dynamic images captured from different viewpoints sequentially. The deformation field for registering images is defined by a collection of affine transformations organized in a graph structure and is estimated by a direct (intensity-based) method. The accuracy of the proposed method is evaluated by comparing it with two B-spline-based methods, two Demons-based methods, and one learning-based method VoxelMorph. Twenty-nine sequences of in vivo 3D TEE images captured from four patients are used for the comparative experiments. Four performance metrics including checkerboard volumes, signed distance, mean absolute distance (MAD), and Dice similarity coefficient (DSC) are used jointly to evaluate the accuracy of the results. Additionally, paired t-tests are performed to examine the significance of the results.

RESULTS

The qualitative results show that the proposed method can align images more accurately and obtain the fused images with higher quality than the other five methods. Additionally, in the evaluation of the segmented left atrium (LA) walls for the pairwise registration and sequential fusion experiments, the proposed method achieves the MAD of (0.07 ± 0.03) mm for pairwise registration and (0.19 ± 0.02) mm for sequential fusion. Paired t-tests indicate that the results obtained from the proposed method are more accurate than those obtained by the state-of-the-art VoxelMorph and the diffeomorphic Demons methods at the significance level of 0.05. In the evaluation of left ventricle (LV) segmentations for the sequential fusion experiments, the proposed method achieves a DSC of (0.88 ± 0.08), which is also significantly better than diffeomorphic Demons at the 0.05 level. The FoVs of the final fused 3D TEE images obtained by our method are enlarged around two times compared with the original images.

CONCLUSIONS

Without selecting the static (ECG-gated) images from the same cardiac phase, this work addressed the problem of limited FoV of 3D TEE images in the deformable scenario, obtaining the fused images with high accuracy and good quality. The proposed method could provide an alternative to the conventional fusion methods that are biased toward the specifically chosen phase.

Patient-specific virtual three-dimensional surgical navigation for gastric cancer surgery: A prospective study for preoperative planning and intraoperative guidance

Introduction

Abdominal computed tomography (CT) can accurately demonstrate organs and vascular structures around the stomach, and its potential role for image guidance is becoming increasingly established. However, solely using two-dimensional CT images to identify critical anatomical structures is undeniably challenging and not surgeon-friendly. To validate the feasibility of a patient-specific 3-D surgical navigation system for preoperative planning and intraoperative guidance during robotic gastric cancer surgery.

Materials and methods

A prospective single-arm open-label observational study was conducted. Thirty participants underwent robotic distal gastrectomy for gastric cancer using a virtual surgical navigation system that provides patient-specific 3-D anatomical information with a pneumoperitoneum model using preoperative CT-angiography. Turnaround time and the accuracy of detecting vascular anatomy with its variations were measured, and perioperative outcomes were compared with a control group after propensity-score matching during the same study period.

Results

Among 36 registered patients, 6 were excluded from the study. Patient-specific 3-D anatomy reconstruction was successfully implemented without any problems in all 30 patients using preoperative CT. All vessels encountered during gastric cancer surgery were successfully reconstructed, and all vascular origins and variations were identical to operative findings. The operative data and short-term outcomes between the experimental and control group were comparable. The experimental group showed shorter anesthesia time (218.6 min vs. 230.3 min; P=0.299), operative time (177.1 min vs. 193.9 min; P=0.137), and console time (129.3 min vs. 147.4 min; P=0.101) than the control group, although the differences were not statistically significant.

Conclusions

Patient-specific 3-D surgical navigation system for robotic gastrectomy for gastric cancer is clinically feasible and applicable with an acceptable turnaround time. This system enables patient-specific preoperative planning and intraoperative navigation by visualizing all the anatomy required for gastrectomy in 3-D models without any error.

Clinical trial registration

Clinicaltrials.gov, identifier NCT05039333.

Adaptive Physics-Based Non-Rigid Registration for Immersive Image-Guided Neuronavigation Systems

Objective: In image-guided neurosurgery, co-registered preoperative anatomical, functional, and diffusion tensor imaging can be used to facilitate a safe resection of brain tumors in eloquent areas of the brain. However, the brain deforms during surgery, particularly in the presence of tumor resection. Non-Rigid Registration (NRR) of the preoperative image data can be used to create a registered image that captures the deformation in the intraoperative image while maintaining the quality of the preoperative image. Using clinical data, this paper reports the results of a comparison of the accuracy and performance among several non-rigid registration methods for handling brain deformation. A new adaptive method that automatically removes mesh elements in the area of the resected tumor, thereby handling deformation in the presence of resection is presented. To improve the user experience, we also present a new way of using mixed reality with ultrasound, MRI, and CT.

Materials and methods: This study focuses on 30 glioma surgeries performed at two different hospitals, many of which involved the resection of significant tumor volumes. An Adaptive Physics-Based Non-Rigid Registration method (A-PBNRR) registers preoperative and intraoperative MRI for each patient. The results are compared with three other readily available registration methods: a rigid registration implemented in 3D Slicer v4.4.0; a B-Spline non-rigid registration implemented in 3D Slicer v4.4.0; and PBNRR implemented in ITKv4.7.0, upon which A-PBNRR was based. Three measures were employed to facilitate a comprehensive evaluation of the registration accuracy: (i) visual assessment, (ii) a Hausdorff Distance-based metric, and (iii) a landmark-based approach using anatomical points identified by a neurosurgeon.

Results: The A-PBNRR using multi-tissue mesh adaptation improved the accuracy of deformable registration by more than five times compared to rigid and traditional physics based non-rigid registration, and four times compared to B-Spline interpolation methods which are part of ITK and 3D Slicer. Performance analysis showed that A-PBNRR could be applied, on average, in <2 min, achieving desirable speed for use in a clinical setting.

Conclusions: The A-PBNRR method performed significantly better than other readily available registration methods at modeling deformation in the presence of resection. Both the registration accuracy and performance proved sufficient to be of clinical value in the operating room. A-PBNRR, coupled with the mixed reality system, presents a powerful and affordable solution compared to current neuronavigation systems.

Population-based 3D respiratory motion modelling from convolutional autoencoders for 2D ultrasound-guided radiotherapy

Radiotherapy is a widely used treatment modality for various types of cancers. A challenge for precise delivery of radiation to the treatment site is the management of internal motion caused by the patient's breathing, especially around abdominal organs such as the liver. Current image-guided radiation therapy (IGRT) solutions rely on ionising imaging modalities such as X-ray or CBCT, which do not allow real-time target tracking. Ultrasound imaging (US) on the other hand is relatively inexpensive, portable and non-ionising. Although 2D US can be acquired at a sufficient temporal frequency, it doesn't allow for target tracking in multiple planes, while 3D US acquisitions are not adapted for real-time. In this work, a novel deep learning-based motion modelling framework is presented for ultrasound IGRT. Our solution includes an image similarity-based rigid alignment module combined with a deep deformable motion model. Leveraging the representational capabilities of convolutional autoencoders, our deformable motion model associates complex 3D deformations with 2D surrogate US images through a common learned low dimensional representation. The model is trained on a variety of deformations and anatomies which enables it to generate the 3D motion experienced by the liver of a previously unseen subject. During inference, our framework only requires two pre-treatment 3D volumes of the liver at extreme breathing phases and a live 2D surrogate image representing the current state of the organ. In this study, the presented model is evaluated on a 3D+t US data set of 20 volunteers based on image similarity as well as anatomical target tracking performance. We report results that surpass comparable methodologies in both metric categories with a mean tracking error of 3.5±2.4 mm, demonstrating the potential of this technique for IGRT.

PET- and SPECT-based navigation strategies to advance procedural accuracy in interventional radiology and image-guided surgery

INTRODUCTION

Nuclear medicine has a crucial role in interventional strategies where a combination between the increasing use of targeted radiotracers and intraprocedural detection modalities enable novel, but often complex, targeted procedures in both the fields of interventional radiology and surgery. 3D navigation approaches could assist the interventional radiologist or surgeon in such complex procedures.

EVIDENCE ACQUISITION

This review aimed to provide a comprehensive overview of the current application of computer-assisted navigation strategies based on nuclear imaging to assist in interventional radiology and image-guided surgery. This work starts with a brief overview of the typical navigation workflow from a technical perspective, which is followed by the different clinical applications organized based on their anatomical organ of interest.

EVIDENCE SYNTHESIS

Although many studies have proven the feasibility of PET- and SPECT-based navigation strategies for various clinical applications in both interventional radiology and surgery, the strategies are spread widely in both navigation workflows and clinical indications, evaluated in small patient groups. Hence, no golden standard has yet been established.

CONCLUSIONS

Despite that the clinical outcome is yet to be determined in large patient cohorts, navigation seems to be a promising technology to translate nuclear medicine findings, provided by PET- and SPECT-based molecular imaging, to the intervention and operating room. Interventional Nuclear Medicine (iNM) has an exciting future to come using both PET- and SPECT-based navigation.

Multiphysics modeling toward enhanced guidance in hepatic microwave ablation: a preliminary framework

We compare a surface-driven, model-based deformation correction method to a clinically relevant rigid registration approach within the application of image-guided microwave ablation for the purpose of demonstrating improved localization and antenna placement in a deformable hepatic phantom. Furthermore, we present preliminary computational modeling of microwave ablation integrated within the navigational environment to lay the groundwork for a more comprehensive procedural planning and guidance framework. To achieve this, we employ a simple, retrospective model of microwave ablation after registration, which allows a preliminary evaluation of the combined therapeutic and navigational framework. When driving registrations with full organ surface data (i.e., as could be available in a percutaneous procedure suite), the deformation correction method improved average ablation antenna registration error by 58.9% compared to rigid registration (i.e., 2.5 ± 1.1    mm , 5.6 ± 2.3    mm of average target error for corrected and rigid registration, respectively) and on average improved volumetric overlap between the modeled and ground-truth ablation zones from 67.0 ± 11.8 % to 85.6 ± 5.0 % for rigid and corrected, respectively. Furthermore, when using sparse-surface data (i.e., as is available in an open surgical procedure), the deformation correction improved registration error by 38.3% and volumetric overlap from 64.8 ± 12.4 % to 77.1 ± 8.0 % for rigid and corrected, respectively. We demonstrate, in an initial phantom experiment, enhanced navigation in image-guided hepatic ablation procedures and identify a clear multiphysics pathway toward a more comprehensive thermal dose planning and deformation-corrected guidance framework.

3D Registration of mpMRI for Assessment of Prostate Cancer Focal Therapy

RATIONALE AND OBJECTIVES

This study aimed to assess a novel method of three-dimensional (3D) co-registration of prostate magnetic resonance imaging (MRI) examinations performed before and after prostate cancer focal therapy.

MATERIALS AND METHODS

We developed a software platform for automatic 3D deformable co-registration of prostate MRI at different time points and applied this method to 10 patients who underwent focal ablative therapy. MRI examinations were performed preoperatively, as well as 1 week and 6 months post treatment. Rigid registration served as reference for assessing co-registration accuracy and precision.

RESULTS

Segmentation of preoperative and postoperative prostate revealed a significant postoperative volume decrease of the gland that averaged 6.49 cc (P = .017). Applying deformable transformation based on mutual information from 120 pairs of MRI slices, we refined by 2.9 mm (max. 6.25 mm) the alignment of the ablation zone, segmented from contrast-enhanced images on the 1-week postoperative examination, to the 6-month postoperative T2-weighted images. This represented a 500% improvement over the rigid approach (P = .001), corrected by volume. The dissimilarity by Dice index of the mapped ablation zone using deformable transformation vs rigid control was significantly (P = .04) higher at the ablation site than in the whole gland.

CONCLUSIONS

Our findings illustrate our method's ability to correct for deformation at the ablation site. The preliminary analysis suggests that deformable transformation computed from mutual information of preoperative and follow-up MRI is accurate in co-registration of MRI examinations performed before and after focal therapy. The ability to localize the previously ablated tissue in 3D space may improve targeting for image-guided follow-up biopsy within focal therapy protocols.

Vascular Structure Identification in Intraoperative 3D Contrast-Enhanced Ultrasound Data

In this paper, a method of vascular structure identification in intraoperative 3D Contrast-Enhanced Ultrasound (CEUS) data is presented. Ultrasound imaging is commonly used in brain tumor surgery to investigate in real time the current status of cerebral structures. The use of an ultrasound contrast agent enables to highlight tumor tissue, but also surrounding blood vessels. However, these structures can be used as landmarks to estimate and correct the brain shift. This work proposes an alternative method for extracting small vascular segments close to the tumor as landmark. The patient image dataset involved in brain tumor operations includes preoperative contrast T1MR (cT1MR) data and 3D intraoperative contrast enhanced ultrasound data acquired before (3D-iCEUS(start) and after (3D-iCEUS(end) tumor resection. Based on rigid registration techniques, a preselected vascular segment in cT1MR is searched in 3D-iCEUS(start) and 3D-iCEUS(end) data. The method was validated by using three similarity measures (Normalized Gradient Field, Normalized Mutual Information and Normalized Cross Correlation). Tests were performed on data obtained from ten patients overcoming a brain tumor operation and it succeeded in nine cases. Despite the small size of the vascular structures, the artifacts in the ultrasound images and the brain tissue deformations, blood vessels were successfully identified.

Population-based prediction of subject-specific prostate deformation for MR-to-ultrasound image registration

Statistical shape models of soft-tissue organ motion provide a useful means of imposing physical constraints on the displacements allowed during non-rigid image registration, and can be especially useful when registering sparse and/or noisy image data. In this paper, we describe a method for generating a subject-specific statistical shape model that captures prostate deformation for a new subject given independent population data on organ shape and deformation obtained from magnetic resonance (MR) images and biomechanical modelling of tissue deformation due to transrectal ultrasound (TRUS) probe pressure. The characteristics of the models generated using this method are compared with corresponding models based on training data generated directly from subject-specific biomechanical simulations using a leave-one-out cross validation. The accuracy of registering MR and TRUS images of the prostate using the new prostate models was then estimated and compared with published results obtained in our earlier research. No statistically significant difference was found between the specificity and generalisation ability of prostate shape models generated using the two approaches. Furthermore, no statistically significant difference was found between the landmark-based target registration errors (TREs) following registration using different models, with a median (95th percentile) TRE of 2.40 (6.19) mm versus 2.42 (7.15) mm using models generated with the new method versus a model built directly from patient-specific biomechanical simulation data, respectively (N = 800; 8 patient datasets; 100 registrations per patient). We conclude that the proposed method provides a computationally efficient and clinically practical alternative to existing complex methods for modelling and predicting subject-specific prostate deformation, such as biomechanical simulations, for new subjects. The method may also prove useful for generating shape models for other organs, for example, where only limited shape training data from dynamic imaging is available.

Effect of material property heterogeneity on biomechanical modeling of prostate under deformation

Biomechanical model based deformable image registration has been widely used to account for prostate deformation in various medical imaging procedures. Biomechanical material properties are important components of a biomechanical model. In this study, the effect of incorporating tumor-specific material properties in the prostate biomechanical model was investigated to provide insight into the potential impact of material heterogeneity on the prostate deformation calculations. First, a simple spherical prostate and tumor model was used to analytically describe the deformations and demonstrate the fundamental effect of changes in the tumor volume and stiffness in the modeled deformation. Next, using a clinical prostate model, a parametric approach was used to describe the variations in the heterogeneous prostate model by changing tumor volume, stiffness, and location, to show the differences in the modeled deformation between heterogeneous and homogeneous prostate models. Finally, five clinical prostatectomy examples were used in separately performed homogeneous and heterogeneous biomechanical model based registrations to describe the deformations between 3D reconstructed histopathology images and ex vivo magnetic resonance imaging, and examine the potential clinical impact of modeling biomechanical heterogeneity of the prostate. The analytical formulation showed that increasing the tumor volume and stiffness could significantly increase the impact of the heterogeneous prostate model in the calculated displacement differences compared to the homogeneous model. The parametric approach using a single prostate model indicated up to 4.8 mm of displacement difference at the tumor boundary compared to a homogeneous model. Such differences in the deformation of the prostate could be potentially clinically significant given the voxel size of the ex vivo MR images (0.3  ×  0.3  ×  0.3 mm). However, no significant changes in the registration accuracy were observed using heterogeneous models for the limited number of clinical prostatectomy patients modeled and evaluated in this study.

Personalising population-based respiratory motion models of the heart using neighbourhood approximation based on learnt anatomical features

Respiratory motion models have been proposed for the estimation and compensation of respiratory motion during image acquisition and image-guided interventions on organs in the chest and abdomen. However, such techniques are not commonly used in the clinic. Subject-specific motion models require a dynamic calibration scan that interrupts the clinical workflow and is often impractical to acquire, while population-based motion models are not as accurate as subject-specific motion models. To address this lack of accuracy, we propose a novel personalisation framework for population-based respiratory motion models and demonstrate its application to respiratory motion of the heart. The proposed method selects a subset of the population sample which is more likely to represent the cardiac respiratory motion of an unseen subject, thus providing a more accurate motion model. The selection is based only on anatomical features of the heart extracted from a static image. The features used are learnt using a neighbourhood approximation technique from a set of training datasets for which respiratory motion estimates are available. Results on a population sample of 28 adult healthy volunteers show average improvements in estimation accuracy of 20% compared to a standard population-based motion model, with an average value for the 50th and 95th quantiles of the estimation error of 1.6mm and 4.7 mm respectively. Furthermore, the anatomical features of the heart most strongly correlated to respiratory motion are investigated for the first time, showing the features on the apex in proximity to the diaphragm and the rib cage, on the left ventricle and interventricular septum to be good predictors of the similarity in cardiac respiratory motion.

Novel System for Real-Time Integration of 3-D Echocardiography and Fluoroscopy for Image-Guided Cardiac Interventions: Preclinical Validation and Clinical Feasibility Evaluation

Real-time imaging is required to guide minimally invasive catheter-based cardiac interventions. While transesophageal echocardiography allows for high-quality visualization of cardiac anatomy, X-ray fluoroscopy provides excellent visualization of devices. We have developed a novel image fusion system that allows real-time integration of 3-D echocardiography and the X-ray fluoroscopy. The system was validated in the following two stages: 1) preclinical to determine function and validate accuracy; and 2) in the clinical setting to assess clinical workflow feasibility and determine overall system accuracy. In the preclinical phase, the system was assessed using both phantom and porcine experimental studies. Median 2-D projection errors of 4.5 and 3.3 mm were found for the phantom and porcine studies, respectively. The clinical phase focused on extending the use of the system to interventions in patients undergoing either atrial fibrillation catheter ablation (CA) or transcatheter aortic valve implantation (TAVI). Eleven patients were studied with nine in the CA group and two in the TAVI group. Successful real-time view synchronization was achieved in all cases with a calculated median distance error of 2.2 mm in the CA group and 3.4 mm in the TAVI group. A standard clinical workflow was established using the image fusion system. These pilot data confirm the technical feasibility of accurate real-time echo-fluoroscopic image overlay in clinical practice, which may be a useful adjunct for real-time guidance during interventional cardiac procedures.

A Mechanics-Based Nonrigid Registration Method for Liver Surgery Using Sparse Intraoperative Data

In open abdominal image-guided liver surgery, sparse measurements of the organ surface can be taken intraoperatively via a laser-range scanning device or a tracked stylus with relatively little impact on surgical workflow. We propose a novel nonrigid registration method which uses sparse surface data to reconstruct a mapping between the preoperative CT volume and the intraoperative patient space. The mapping is generated using a tissue mechanics model subject to boundary conditions consistent with surgical supportive packing during liver resection therapy. Our approach iteratively chooses parameters which define these boundary conditions such that the deformed tissue model best fits the intraoperative surface data. Using two liver phantoms, we gathered a total of five deformation datasets with conditions comparable to open surgery. The proposed nonrigid method achieved a mean target registration error (TRE) of 3.3 mm for targets dispersed throughout the phantom volume, using a limited region of surface data to drive the nonrigid registration algorithm, while rigid registration resulted in a mean TRE of 9.5 mm. In addition, we studied the effect of surface data extent, the inclusion of subsurface data, the trade-offs of using a nonlinear tissue model, robustness to rigid misalignments, and the feasibility in five clinical datasets.

The virtual skeleton database: an open access repository for biomedical research and collaboration

BACKGROUND

Statistical shape models are widely used in biomedical research. They are routinely implemented for automatic image segmentation or object identification in medical images. In these fields, however, the acquisition of the large training datasets, required to develop these models, is usually a time-consuming process. Even after this effort, the collections of datasets are often lost or mishandled resulting in replication of work.

OBJECTIVE

To solve these problems, the Virtual Skeleton Database (VSD) is proposed as a centralized storage system where the data necessary to build statistical shape models can be stored and shared.

METHODS

The VSD provides an online repository system tailored to the needs of the medical research community. The processing of the most common image file types, a statistical shape model framework, and an ontology-based search provide the generic tools to store, exchange, and retrieve digital medical datasets. The hosted data are accessible to the community, and collaborative research catalyzes their productivity.

RESULTS

To illustrate the need for an online repository for medical research, three exemplary projects of the VSD are presented: (1) an international collaboration to achieve improvement in cochlear surgery and implant optimization, (2) a population-based analysis of femoral fracture risk between genders, and (3) an online application developed for the evaluation and comparison of the segmentation of brain tumors.

CONCLUSIONS

The VSD is a novel system for scientific collaboration for the medical image community with a data-centric concept and semantically driven search option for anatomical structures. The repository has been proven to be a useful tool for collaborative model building, as a resource for biomechanical population studies, or to enhance segmentation algorithms.

On mixed reality environments for minimally invasive therapy guidance: systems architecture, successes and challenges in their implementation from laboratory to clinic

Mixed reality environments for medical applications have been explored and developed over the past three decades in an effort to enhance the clinician's view of anatomy and facilitate the performance of minimally invasive procedures. These environments must faithfully represent the real surgical field and require seamless integration of pre- and intra-operative imaging, surgical instrument tracking, and display technology into a common framework centered around and registered to the patient. However, in spite of their reported benefits, few mixed reality environments have been successfully translated into clinical use. Several challenges that contribute to the difficulty in integrating such environments into clinical practice are presented here and discussed in terms of both technical and clinical limitations. This article should raise awareness among both developers and end-users toward facilitating a greater application of such environments in the surgical practice of the future.

Clinical application of image-enhanced minimally invasive robotic surgery for gastric cancer: a prospective observational study

BACKGROUND

This study was performed to validate the feasibility and role of image-guided robotic surgery using preoperative computed tomography (CT) images for the treatment of gastric cancer.

METHODS

Twelve patients scheduled to undergo robotic gastrectomy for gastric cancer were registered. Vessels encountered during gastrectomy were reconstructed using 3D software and their anatomical variation was evaluated using preoperatively performed CT-angiography. The vascular information was transferred to a robot console using a multi-input display mode. Radiologic findings acquired from preoperative CT by the radiologist were compared with intraoperative findings of the surgeon. This study is registered with www.clinicaltrials.gov as NCT01338948.

RESULTS

All 12 robotic gastrectomies were performed without any problems. All anatomical data acquired using 3D software were transferred successfully during surgery. Intraoperative vascular images depicted vasculatures around the stomach and could identify important vascular variations. During surgery, relevant vascular information led the surgeon to branch sites and facilitated lymphadenectomy around the vessels. Image-guidance during the operation provided a vascular map and enabled the surgeon to avoid accidental bleeding and damage to other organs by preventing vascular injuries.

CONCLUSION

Image-guided robotic surgery for gastric cancer using preoperative CT-angiography reconstructed during operation by a surgically trained radiologist who could adjust the images by anticipating the operative procedure was feasible and improved the efficiency of surgery by eliminating the possibility of vascular injuries.

Computational models in image guided interventions

In image-guided surgery and image-directed therapy a plan based on pre-procedure imaging is registered to the patient in the operating or treatment room using a 3D spatial localizer. The plan can be used as long as the transformation between plan and patient remains valid. Most systems use a rigid-body transformation restricting guidance to bony structures (e.g. orthopaedic surgery or maxillo-facial surgery) or structures that are rigidly related to bone (e.g. neurosurgery). Fully 3D intra-operative imaging such as interventional MR allows image guidance to be extended to structures that move or deform during an intervention. However, this technology is expensive, interferes significantly with standard surgical protocols and requires computationally expensive non-rigid registration of the plan to the current patient scan. This talk will describe four examples where computational models of motion and anatomy are combined with 2D intra-operative imaging to extend the scope of image directed methods. In the first, image guided neurosurgery, we show how intra-operative imaging may account for distortion caused by the intervention itself. In two further applications - percutaneous ablation of metastatic liver disease and external beam radiotherapy of the lung - we show how computational models of motion might be used in conjunction with a therapy plan to guide the intervention. In the final example, selected from orthopaedic surgery, we show recent advances that demonstrate how a statistical shape model generated from example 3D images, can be used to provide image guidance without any pre-operative 3D imaging.

Patient-specific treatment planning of electrochemotherapy: procedure design and possible pitfalls

Electrochemotherapy uses electroporation for enhancing chemotherapy. Electrochemotherapy can be performed using standard operating procedures with predefined electrode geometries, or using patient-specific treatment planning to predict electroporation. The latter relies on realistic computer models to provide optimal results (i.e. electric field distribution as well as electrodes' position and number) and is suitable for treatment of deep-seated tumors. Since treatment planning for deep-seated tumors has been used in radiotherapy, we expose parallelisms with radiotherapy in order to establish the procedure for electrochemotherapy of deep-seated tumors. We partitioned electrochemotherapy in the following phases: the mathematical model of electroporation, treatment planning, set-up verification, treatment delivery and monitoring, and response assessment. We developed a conceptual treatment planning software that incorporates mathematical models of electroporation. Preprocessing and segmentation of the patient's medical images are performed, and a 3D model is constructed which allows placement of electrodes and implementation of the mathematical model of electroporation. We demonstrated the feasibility of electrochemotherapy of deep-seated tumors treatment planning within a clinical study where treatment planning contributed to the effective electrochemotherapy treatment of deep-seated colorectal metastases in the liver. The described procedure can provide medical practitioners with information on using electrochemotherapy in the clinical setting. The main aims of this paper are: 1) to present the procedure for treating deep-seated tumors by electrochemotherapy based on patient-specific treatment planning, and 2) to identify gaps in knowledge and possible pitfalls of such procedure.

Navigated laparoscopic ultrasound in abdominal soft tissue surgery: technological overview and perspectives

PURPOSE

Two-dimensinal laparoscopic ultrasound (LUS) is commonly used for many laparoscopic procedures, but 3D LUS and navigation technology are not conventional tools in the clinic. Navigated LUS can help the user understand and interpret the ultrasound images in relation to the laparoscopic view and preoperative images. When combined with information from MRI or CT, navigated LUS has the potential to provide information about anatomic shifts during the procedure. In this paper, we present an overview of the ongoing technological research and development related to LUS combined with navigation technology, The purpose of this overview is threefold: (1) an introduction for those new to the field of navigated LUS; (2) an overview for those working in the field and; and (3) as a reference for those searching for literature on technological developments related to navigation in ultrasound-guided laparoscopic surgery.

METHODS

Databases were searched to identify relevant publications from the last 10 years.

RESULTS

We were able to identify 18 key papers in the area of navigated LUS for the abdomen, originating from about 10-11 groups. We present the literature overview, including descriptions of our own experience in the field, and a discussion of the important clinical and technological aspects related to navigated LUS.

CONCLUSIONS

LUS integrated with miniaturized tracking technology is likely to play an important role in guiding future laparoscopic surgery.

A reduced order explicit dynamic finite element algorithm for surgical simulation

Reduced order modelling, in which a full system response is projected onto a subspace of lower dimensionality, has been used previously to accelerate finite element solution schemes by reducing the size of the involved linear systems. In the present work we take advantage of a secondary effect of such reduction for explicit analyses, namely that the stable integration time step is increased far beyond that of the full system. This phenomenon alleviates one of the principal drawbacks of explicit methods, compared with implicit schemes. We present an explicit finite element scheme in which time integration is performed in a reduced basis. Futhermore, we present a simple procedure for imposing inhomogeneous essential boundary conditions, thus overcoming one of the principal deficiencies of such approaches. The computational benefits of the procedure within a GPU-based execution framework are examined, and an assessment of the errors introduced is given. It is shown that speedups approaching an order of magnitude are feasible, without introduction of prohibitive errors, and without hardware modifications. The procedure may have applications in interactive simulation and medical image-guidance problems, in which both speed and accuracy are vital.

Image-guided interventions: technology review and clinical applications

Image-guided interventions are medical procedures that use computer-based systems to provide virtual image overlays to help the physician precisely visualize and target the surgical site. This field has been greatly expanded by the advances in medical imaging and computing power over the past 20 years. This review begins with a historical overview and then describes the component technologies of tracking, registration, visualization, and software. Clinical applications in neurosurgery, orthopedics, and the cardiac and thoracoabdominal areas are discussed, together with a description of an evolving technology named Natural Orifice Transluminal Endoscopic Surgery (NOTES). As the trend toward minimally invasive procedures continues, image-guided interventions will play an important role in enabling new procedures, while improving the accuracy and success of existing approaches. Despite this promise, the role of image-guided systems must be validated by clinical trials facilitated by partnerships between scientists and physicians if this field is to reach its full potential.

A comparison of the accuracy of statistical models of prostate motion trained using data from biomechanical simulations

Statistical shape models (SSM) are widely used in medical image analysis to represent variability in organ shape. However, representing subject-specific soft-tissue motion using this technique is problematic for applications where imaging organ changes in an individual is not possible or impractical. One solution is to synthesise training data by using biomechanical modelling. However, for many clinical applications, generating a biomechanical model of the organ(s) of interest is a non-trivial task that requires a significant amount of user-interaction to segment an image and create a finite element mesh. In this study, we investigate the impact of reducing the effort required to generate SSMs and the accuracy with which such models can predict tissue displacements within the prostate gland due to transrectal ultrasound probe pressure. In this approach, the finite element mesh is based on a simplified geometric representation of the organs. For example, the pelvic bone is represented by planar surfaces, or the number of distinct tissue compartments is reduced. Such representations are much easier to generate from images than a geometrically accurate mesh. The difference in the median root-mean-square displacement error between different SSMs of prostate was <0.2 mm. We conclude that reducing the geometric complexity of the training model in this way made little difference to the absolute accuracy of SSMs to recover tissue displacements. The implication is that SSMs of organ motion based on simulated training data may be generated using simplified geometric representations, which are much more compatible with the time constraints of clinical workflows.

Real-time tracking of liver motion and deformation using a flexible needle

PURPOSE

A real-time 3D image guidance system is needed to facilitate treatment of liver masses using radiofrequency ablation, for example. This study investigates the feasibility and accuracy of using an electromagnetically tracked flexible needle inserted into the liver to track liver motion and deformation.

METHODS

This proof-of-principle study was conducted both ex vivo and in vivo with a CT scanner taking the place of an electromagnetic tracking system as the spatial tracker. Deformations of excised livers were artificially created by altering the shape of the stage on which the excised livers rested. Free breathing or controlled ventilation created deformations of live swine livers. The positions of the needle and test targets were determined through CT scans. The shape of the needle was reconstructed using data simulating multiple embedded electromagnetic sensors. Displacement of liver tissues in the vicinity of the needle was derived from the change in the reconstructed shape of the needle.

RESULTS

The needle shape was successfully reconstructed with tracking information of two on-needle points. Within 30 mm of the needle, the registration error of implanted test targets was 2.4 ± 1.0 mm ex vivo and 2.8 ± 1.5 mm in vivo.

CONCLUSION

A practical approach was developed to measure the motion and deformation of the liver in real time within a region of interest. The approach relies on redesigning the often-used seeker needle to include embedded electromagnetic tracking sensors. With the nonrigid motion and deformation information of the tracked needle, a single- or multimodality 3D image of the intraprocedural liver, now clinically obtained with some delay, can be updated continuously to monitor intraprocedural changes in hepatic anatomy. This capability may be useful in radiofrequency ablation and other percutaneous ablative procedures.

Multiplicative Jacobian Energy Decomposition method for fast porous visco-hyperelastic soft tissue model

Simulating soft tissues in real time is a significant challenge since a compromise between biomechanical accuracy and computational efficiency must be found. In this paper, we propose a new discretization method, the Multiplicative Jacobian Energy Decomposition (MJED) which is an alternative to the classical Galerkin FEM (Finite Element Method) formulation. This method for discretizing non-linear hyperelastic materials on linear tetrahedral meshes leads to faster stiffness matrix assembly for a large variety of isotropic and anisotropic materials. We show that our new approach, implemented within an implicit time integration scheme, can lead to fast and realistic liver deformations including hyperelasticity, porosity and viscosity.

Radiofrequency ablation of lung tumors in swine assisted by a navigation device with preprocedural volumetric planning

PURPOSE

To develop an image guidance system that incorporates volumetric planning of spherical ablations and electromagnetic tracking of radiofrequency (RF) electrodes during insertion.

MATERIALS AND METHODS

Simulated tumors were created in three live swine by percutaneously injecting agar nodules into the lung. A treatment plan was devised for each tumor with optimization software to solve the planning problem. The desired output was the minimum number of overlapping ablation spheres necessary to ablate each tumor and the margin. The insertion plan was executed with use of the electromagnetic tracking system that guided the insertion of the probe into precomputed locations. After a 72-hour survival period, animals were killed and histopathologic sections of the tissue were examined for cell viability and burn pattern analysis.

RESULTS

A planning algorithm to spherically cover the tumors and the margin was computed. Electromagnetic tracking allowed successful insertion of the instrument, and impedance roll-off was reached in all ablations. Depending on their size, the tumors and the tumor margins were successfully covered with two to four ablation spheres. The image registration error was 1.0 mm +/- 0.64. The overall error of probe insertion was 9.4 mm +/- 3.0 (N = 8). Analysis of histopathologic sections confirmed successful ablations of the tissue.

CONCLUSIONS

Computer-assisted RF ablation planning and electromagnetically tracked probe insertion were successful in three swine, validating the feasibility of electromagnetic tracking-assisted tumor targeting. Image misregistration caused by respiratory motion and tissue deformation contributed to the overall error of probe insertion.

Incorporation of preprocedural PET into CT-guided radiofrequency ablation of hepatic metastases: a nonrigid image registration validation study

This study evaluates the accuracy of augmenting initial intraprocedural computed tomography (CT) during radiofrequency ablation (RFA) of hepatic metastases with preprocedural positron emission tomography (PET) through a hardware-accelerated implementation of an automatic nonrigid PET-CT registration algorithm. The feasibility of augmenting intraprocedural CT with preprocedural PET to improve localization of CT-invisible but PET-positive tumors with images from actual RFA was explored. Preprocedural PET and intraprocedural CT images from 18 cases of hepatic RFA were included. All PET images in the study originated from a hybrid PET/CT scanner, and PET-CT registration was performed in two ways: (1) direct registration of preprocedural PET with intraprocedural CT and (2) indirect registration of preprocedural CT (i.e., the CT of hybrid PET/CT scan) with intraprocedural CT. A hardware-accelerated registration took approximately 2 min. Calculated registration errors were 7.0 and 8.4 mm for the direct and indirect methods, respectively. Overall, the direct registration was found to be statistically not distinct from that performed by a group of clinical experts. The accuracy, execution speed, and compactness of our implementation of nonrigid image registration suggest that existing PET can be overlaid on intraprocedural CT, promising a novel, technically feasible, and clinically viable approach for PET augmentation of CT guidance of RFA.

Training models of anatomic shape variability

Learning probability distributions of the shape of anatomic structures requires fitting shape representations to human expert segmentations from training sets of medical images. The quality of statistical segmentation and registration methods is directly related to the quality of this initial shape fitting, yet the subject is largely overlooked or described in an ad hoc way. This article presents a set of general principles to guide such training. Our novel method is to jointly estimate both the best geometric model for any given image and the shape distribution for the entire population of training images by iteratively relaxing purely geometric constraints in favor of the converging shape probabilities as the fitted objects converge to their target segmentations. The geometric constraints are carefully crafted both to obtain legal, nonself-interpenetrating shapes and to impose the model-to-model correspondences required for useful statistical analysis. The paper closes with example applications of the method to synthetic and real patient CT image sets, including same patient male pelvis and head and neck images, and cross patient kidney and brain images. Finally, we outline how this shape training serves as the basis for our approach to IGRT/ART.

Technology preview: X-ray fused with magnetic resonance during invasive cardiovascular procedures

BACKGROUND

We have developed and validated a system for real-time X-ray fused with magnetic resonance imaging, MRI (XFM), to guide catheter procedures with high spatial precision. Our implementation overlays roadmaps-MRI-derived soft-tissue features of interest-onto conventional X-ray fluoroscopy. We report our initial clinical experience applying XFM, using external fiducial markers, electrocardiogram (ECG)- gating, and automated real-time correction for gantry and table movement.

METHODS

This prospective case series for technical development was approved by the NHLBI Institutional Review Board and included 19 subjects. Multimodality external fiducial markers were affixed to patients' skin before MRI, which included contrast-enhanced, 3D T1-weighted, or breath-held and ECG-gated 2D steady state free precession imaging at 1.5T. MRI-derived roadmaps were manually segmented while patients were transferred to a calibrated X-ray fluoroscopy system. Image spaces were registered using the fiducial markers and thereafter permitted unrestricted gantry rotation, table panning, and magnification changes. Static and ECG-gated MRI data were transformed from 3D to 2D to correspond with gantry and table position and combined with live X-ray images.

RESULTS

Clinical procedures included graft coronary arteriography, right ventricular free-wall biopsy, and iliac and femoral artery recanalization and stenting. MRI roadmaps improved operator confidence, and in the biopsy cases, outperformed the best available alternative imaging modality. Registration errors were increased when external fiducial markers were affixed to more mobile skin positions, such as over the abdomen.

CONCLUSION

XFM using external fiducial markers is feasible during X-ray guided catheter treatments. Multimodality image fusion may prove a useful adjunct to invasive cardiovascular procedures.

FEM-based evaluation of deformable image registration for radiation therapy

This paper presents a new concept to automatically detect the neighborhood in an image where deformable registration is mis-performing. Specifically, the displacement vector field (DVF) from a deformable image registration is substituted into a finite-element-based elastic framework to calculate unbalanced energy in each element. The value of the derived energy indicates the quality of the DVF in its neighborhood. The new voxel-based evaluation approach is compared with three other validation criteria: landmark measurement, a finite element approach and visual comparison, for deformable registrations performed with the optical-flow-based 'demons' algorithm as well as thin-plate spline interpolation. This analysis was performed on three pairs of prostate CT images. The results of the analysis show that the four criteria give mutually comparable quantitative assessments on the six registration instances. As an objective concept, the unbalanced energy presents no requirement on boundary constraints in its calculation, different from traditional mechanical modeling. This method is automatic, and at voxel level suitable to evaluate deformable registration in a clinical setting.

FPGA-Accelerated Deformable Image Registration for Improved Target-Delineation During CT-Guided Interventions

Minimally invasive image-guided interventions (IGIs) are time and cost efficient, minimize unintended damage to healthy tissue, and lead to faster patient recovery. With the advent of multislice computed tomography (CT), many IGIs are now being performed under volumetric CT guidance. Registering pre-and intraprocedural images for improved intraprocedural target delineation is a fundamental need in the IGI workflow. Earlier approaches to meet this need primarily employed rigid body approximation, which may not be valid because of nonrigid tissue misalignment between these images. Intensity-based automatic deformable registration is a promising option to correct for this misalignment; however, the long execution times of these algorithms have prevented their use in clinical workflow. This article presents a field-programmable gate array-based architecture for accelerated implementation of mutual information (Ml)-based deformable registration. The reported implementation reduces the execution time of MI-based deformable registration from hours to a few minutes. This work also demonstrates successful registration of abdominal intraprocedural noncontrast CT (iCT) images with preprocedural contrast-enhanced CT (preCT) and positron emission tomography (PET) images using the reported solution. The registration accuracy for this application was evaluated using 5 iCT-preCT and 5 iCT-PET image pairs. The registration accuracy of the hardware implementation is comparable with that achieved using a software implementation and is on the order of a few millimeters. This registration accuracy, coupled with the execution speed and compact implementation of the reported solution, makes it suitable for integration in the IGI-workflow.

Estimation of the delivered patient dose in lung IMRT treatment based on deformable registration of 4D-CT data and Monte Carlo simulations

The purpose of this study is to accurately estimate the difference between the planned and the delivered dose due to respiratory motion and free breathing helical CT artefacts for lung IMRT treatments, and to estimate the impact of this difference on clinical outcome. Six patients with representative tumour motion, size and position were selected for this retrospective study. For each patient, we had acquired both a free breathing helical CT and a ten-phase 4D-CT scan. A commercial treatment planning system was used to create four IMRT plans for each patient. The first two plans were based on the GTV as contoured on the free breathing helical CT set, with a GTV to PTV expansion of 1.5 cm and 2.0 cm, respectively. The third plan was based on the ITV, a composite volume formed by the union of the CTV volumes contoured on free breathing helical CT, end-of-inhale (EOI) and end-of-exhale (EOE) 4D-CT. The fourth plan was based on GTV contoured on the EOE 4D-CT. The prescribed dose was 60 Gy for all four plans. Fluence maps and beam setup parameters of the IMRT plans were used by the Monte Carlo dose calculation engine MCSIM for absolute dose calculation on both the free breathing CT and 4D-CT data. CT deformable registration between the breathing phases was performed to estimate the motion trajectory for both the tumour and healthy tissue. Then, a composite dose distribution over the whole breathing cycle was calculated as a final estimate of the delivered dose. EUD values were computed on the basis of the composite dose for all four plans. For the patient with the largest motion effect, the difference in the EUD of CTV between the planed and the delivered doses was 33, 11, 1 and 0 Gy for the first, second, third and fourth plan, respectively. The number of breathing phases required for accurate dose prediction was also investigated. With the advent of 4D-CT, deformable registration and Monte Carlo simulations, it is feasible to perform an accurate calculation of the delivered dose, and compare our delivered dose with doses estimated using prior techniques.

Image-based guidance for minimally invasive surgical atrial fibrillation ablation

PURPOSE

Atrial fibrillation (AF) is the most common arrhythmia and results in an increased risk of ischaemic stroke. Recently, a European consortium has developed a new ablation device for minimally invasive surgical AF treatment. The device is controlled by a medical robot. Due to the minimal invasive usage, surgery using the new device needs appropriate navigation support. In this paper, we describe an image-based navigation application to guide the new device intraoperatively.

METHODS

The navigation procedure is based on intraoperative ultrasound. Variations in the position of the ablation device are transferred from the software controlling the robot to the navigation system. Due to the flexibility of the ablation device, a deformation model predicts the behaviour during repositioning. Ablation lines are interactively planned. Actually burned ablation lines are visualized during surgery. Several in vitro and ex vivo experimental set-ups were built up to test the feasibility.

RESULTS

The navigation workflow was implemented into navigation software using well-known open-source software toolkits. The navigation system has been integrated and tested successfully within the overall system. The ablation device could be localized on in vitro and ex vivo ultrasound images.

CONCLUSION

The performed trials proved the applicability of the navigation procedure. More in vivo tests are currently being performed to make the new device and the described navigation procedure ready for clinical use.

Non-rigid image registration: theory and practice

Image registration is an important enabling technology in medical image analysis. The current emphasis is on development and validation of application-specific non-rigid techniques, but there is already a plethora of techniques and terminology in use. In this paper we discuss the current state of the art of non-rigid registration to put on-going research in context and to highlight current and future clinical applications that might benefit from this technology. The philosophy and motivation underlying non-rigid registration is discussed and a guide to common terminology is presented. The core components of registration systems are described and outstanding issues of validity and validation are confronted.

A Complete Augmented Reality Guidance System for Liver Punctures: First Clinical Evaluation

We provided in an augmented reality guidance system for liver punctures, which has been validated on a static abdominal phantom. In this paper, we report the first in vivo experiments. We developed a strictly passive protocol to directly evaluate our system on patients. We show that the system algorithms work efficiently and we highlight the clinical constraints that we had to overcome (small operative field, weight and sterility of the tracked marker attached to the needle...). Finally, we investigate to what extent breathing motion can be neglected for free breathing patient. Results show that the guiding accuracy, close to 1 cm, is sufficient for large targets only (above 3 cm of diameter) when the breathing motion is neglected. In the near future, we aim at validating our system on smaller targets using a respiratory gating technique.