Virtual reality-enhanced ultrasound guidance: A novel technique for intracardiac interventions

Patient-Specific Inverse Modeling of In Vivo Cardiovascular Mechanics with Medical Image-Derived Kinematics as Input Data: Concepts, Methods, and Applications

Inverse modeling approaches in cardiovascular medicine are a collection of methodologies that can provide non-invasive patient-specific estimations of tissue properties, mechanical loads, and other mechanics-based risk factors using medical imaging as inputs. Its incorporation into clinical practice has the potential to improve diagnosis and treatment planning with low associated risks and costs. These methods have become available for medical applications mainly due to the continuing development of image-based kinematic techniques, the maturity of the associated theories describing cardiovascular function, and recent progress in computer science, modeling, and simulation engineering. Inverse method applications are multidisciplinary, requiring tailored solutions to the available clinical data, pathology of interest, and available computational resources. Herein, we review biomechanical modeling and simulation principles, methods of solving inverse problems, and techniques for image-based kinematic analysis. In the final section, the major advances in inverse modeling of human cardiovascular mechanics since its early development in the early 2000s are reviewed with emphasis on method-specific descriptions, results, and conclusions. We draw selected studies on healthy and diseased hearts, aortas, and pulmonary arteries achieved through the incorporation of tissue mechanics, hemodynamics, and fluid-structure interaction methods paired with patient-specific data acquired with medical imaging in inverse modeling approaches.

An Overview on Image Registration Techniques for Cardiac Diagnosis and Treatment

Image registration has been used for a wide variety of tasks within cardiovascular imaging. This study aims to provide an overview of the existing image registration methods to assist researchers and impart valuable resource for studying the existing methods or developing new methods and evaluation strategies for cardiac image registration. For the cardiac diagnosis and treatment strategy, image registration and fusion can provide complementary information to the physician by using the integrated image from these two modalities. This review also contains a description of various imaging techniques to provide an appreciation of the problems associated with implementing image registration, particularly for cardiac pathology intervention and treatments.

The state of the art of visualization in mixed reality image guided surgery

This paper presents a review of the state of the art of visualization in mixed reality image guided surgery (IGS). We used the DVV (data, visualization processing, view) taxonomy to classify a large unbiased selection of publications in the field. The goal of this work was not only to give an overview of current visualization methods and techniques in IGS but more importantly to analyze the current trends and solutions used in the domain. In surveying the current landscape of mixed reality IGS systems, we identified a strong need to assess which of the many possible data sets should be visualized at particular surgical steps, to focus on novel visualization processing techniques and interface solutions, and to evaluate new systems.

On mixed reality environments for minimally invasive therapy guidance: systems architecture, successes and challenges in their implementation from laboratory to clinic

Mixed reality environments for medical applications have been explored and developed over the past three decades in an effort to enhance the clinician's view of anatomy and facilitate the performance of minimally invasive procedures. These environments must faithfully represent the real surgical field and require seamless integration of pre- and intra-operative imaging, surgical instrument tracking, and display technology into a common framework centered around and registered to the patient. However, in spite of their reported benefits, few mixed reality environments have been successfully translated into clinical use. Several challenges that contribute to the difficulty in integrating such environments into clinical practice are presented here and discussed in terms of both technical and clinical limitations. This article should raise awareness among both developers and end-users toward facilitating a greater application of such environments in the surgical practice of the future.

Vessel bifurcation localization based on intraoperative three-dimensional ultrasound and catheter path for image-guided catheter intervention of oral cancers

We present a method to localize intraoperative target vessel bifurcations under bones for ultrasound (US) image-guided catheter interventions. A catheter path is recorded to acquire skeletons for the target vessel bifurcations that cannot be imaged by intraoperative US. The catheter path is combined with the centerlines of the three-dimensional (3D) US image to construct a preliminary skeleton. Based on the preliminary skeleton, the orientations of target vessels are determined by registration with the preoperative image and the bifurcations were localized by computing the vessel length. An accurate intraoperative vessel skeleton is obtained for correcting the preoperative image to compensate for vessel deformation. A reality check of the proposed method was performed in a phantom experiment. Reasonable results were obtained. The in vivo experiment verified the clinical workflow of the proposed method in an in vivo environment. The accuracy of the centerline length of the vessel for localizing the target artery bifurcation was 2.4mm. These results suggest that the proposed method can allow the catheter tip to stop at the target artery bifurcations and enter into the target arteries. This method can be applied for virtual reality-enhanced image-guided catheter intervention of oral cancers.

DVV: a taxonomy for mixed reality visualization in image guided surgery

Mixed reality visualizations are increasingly studied for use in image guided surgery (IGS) systems, yet few mixed reality systems have been introduced for daily use into the operating room (OR). This may be the result of several factors: the systems are developed from a technical perspective, are rarely evaluated in the field, and/or lack consideration of the end user and the constraints of the OR. We introduce the Data, Visualization processing, View (DVV) taxonomy which defines each of the major components required to implement a mixed reality IGS system. We propose that these components be considered and used as validation criteria for introducing a mixed reality IGS system into the OR. A taxonomy of IGS visualization systems is a step toward developing a common language that will help developers and end users discuss and understand the constituents of a mixed reality visualization system, facilitating a greater presence of future systems in the OR. We evaluate the DVV taxonomy based on its goodness of fit and completeness. We demonstrate the utility of the DVV taxonomy by classifying 17 state-of-the-art research papers in the domain of mixed reality visualization IGS systems. Our classification shows that few IGS visualization systems' components have been validated and even fewer are evaluated.

Accuracy considerations in image-guided cardiac interventions: experience and lessons learned

MOTIVATION

Medical imaging and its application in interventional guidance has revolutionized the development of minimally invasive surgical procedures leading to reduced patient trauma, fewer risks, and shorter recovery times. However, a frequently posed question with regard to an image guidance system is "how accurate is it?" On one hand, the accuracy challenge can be posed in terms of the tolerable clinical error associated with the procedure; on the other hand, accuracy is bound by the limitations of the system's components, including modeling, patient registration, and surgical instrument tracking, all of which ultimately impact the overall targeting capabilities of the system.

METHODS

While these processes are not unique to any interventional specialty, this paper discusses them in the context of two different cardiac image guidance platforms: a model-enhanced ultrasound platform for intracardiac interventions and a prototype system for advanced visualization in image-guided cardiac ablation therapy.

RESULTS

Pre-operative modeling techniques involving manual, semi-automatic and registration-based segmentation are discussed. The performance and limitations of clinically feasible approaches for patient registration evaluated both in the laboratory and in the operating room are presented. Our experience with two different magnetic tracking systems for instrument and ultrasound transducer localization is reported. Ultimately, the overall accuracy of the systems is discussed based on both in vitro and preliminary in vivo experience.

CONCLUSION

While clinical accuracy is specific to a particular patient and procedure and vastly dependent on the surgeon's experience, the system's engineering limitations are critical to determine whether the clinical requirements can be met.

A method for measuring the accuracy of multi-modal image fusion system for catheter-based cardiac interventions using a novel motion enabled targeting phantom

Targeted stem cell therapy offers great potential for the repair of infarcted cardiac tissue following heart attack. Safe delivery of stem-cells via catheter based interventions remains a challenge. A multi-modal image fusion approach has been considered for safe targeting of myocardial infarct border zones. In this paper we present an apparatus and method for measuring the accuracy of catheter-based injections using a multi-modal image fusion system. We also present results of the accuracy of our image fusion system under varying levels of cardio-respiratory motion.

Evaluation of model-enhanced ultrasound-assisted interventional guidance in a cardiac phantom

Minimizing invasiveness associated with cardiac procedures has led to limited visual access to the target tissues. To address these limitations, we have developed a visualization environment that integrates interventional ultrasound (US) imaging with preoperative anatomical models and virtual representations of the surgical instruments tracked in real time. In this paper, we present a comprehensive evaluation of our model-enhanced US-guidance environment by simulating clinically relevant interventions in vitro . We have demonstrated that model-enhanced US guidance provides a clinically desired targeting accuracy better than 3-mm rms and maintains this level of accuracy even in the case of image-to-patient misalignments that are often encountered in the clinic. These studies emphasize the benefits of integrating real-time imaging with preoperative data to enhance surgical navigation in the absence of direct vision during minimally invasive cardiac interventions.

Target tracking errors for 5D and 6D spatial measurement systems

In recent years, magnetic tracking systems, whose fundamental unit of measurement is a 5D transformation (three translational and two rotational degrees-of-freedom), have become much more popular. Two 5D sensors can be combined to obtain a 6D transformation similar to the ones provided by the point-based registration in optical tracking. However, estimates of the tool tip uncertainty, which we have called the target tracking error (TTE) since no registration is explicitly performed, are not available in the same manner as their optical counterpart. If the systematic bias error can be corrected and estimates of the 5D or 6D fiducial localizer error (FLE) are provided in the form of zero mean normally distributed random variables in [Formula: see text] and [Formula: see text], respectively, then the TTE can be modeled. In this paper, the required expressions that model the TTE as a function of the systematic bias, FLE and target location are derived and then validated using Monte Carlo simulations. We also show that the first order approximation is sufficient beyond the range of errors typically observed during an image-guided surgery (IGS) procedure. Applications of the models are described for a minimally invasive intracardiac surgical guidance system and needle-based therapy systems. Together with the target registration error (TRE) statistical models for point-based registration, the models presented in this article provide the basic framework for estimating the total system measurement uncertainty for an IGS system. Future work includes developing TRE models for commonly used registration methods that do not already have them.

How accurate is accurate enough? A brief overview on accuracy considerations in image-guided cardiac interventions

Image-guided interventions have revolutionized the development of minimally invasive surgical procedures, leading to reduced patient trauma, fewer risks and shorter recovery times. However, one of the most frequently posed question with regards to an image guidance system is how accurate it is. In this work we provide a brief overview on accuracy considerations from our perspective on cardiac image-guided procedures: what are the clinically-imposed accuracy constraints, how do these measure against the limitations of the image-guidance system, and how can surgeons directly benefit from real-time accuracy feedback to ensure optimal navigation at all times during the intervention?

Targeting accuracy under model-to-subject misalignments in model-guided cardiac surgery

In image-guided interventions, anatomical models of organs are often generated from pre-operative images and further employed in planning and guiding therapeutic procedures. However, the accuracy of these models, along with their registration to the subject are crucial for successful therapy delivery. These factors are amplified when manipulating soft tissue undergoing large deformations, such as the heart. When used in guiding beating-heart procedures, pre-operative models may not be sufficient for guidance and they are often complemented with real-time, intra-operative cardiac imaging. Here we demonstrate via in vitro endocardial "therapy" that ultrasound-enhanced model-guided navigation provides sufficient guidance to preserve a clinically-desired targeting accuracy of under 3 mm independently of the model-to-subject misregistrations. These results emphasize the direct benefit of integrating real-time imaging within intra-operative visualization environments considering that model-to-subject misalignments are often encountered clinically.

Real-time compounding of three-dimensional transesophageal echocardiographic volumes: the phantom study

3D ultrasound has attracted considerable interest in recent years as a low cost, mobile and real-time imaging modality for interventional cardiac applications. However, the low image quality and small field of view have been two major barriers preventing 3D ultrasound from being widely accepted as a solution to the guidance of cardiac interventions. By using the 3D transesophageal echographic (TEE) probe, it is possible to acquire images with better quality compared to the images acquired from traditional transthoracic probe (TTE). However, the 3D TEE volume has even smaller field of view and is insufficient to cover the whole geometry of the heart. Previously, we have developed a technique to compound 3D TTE volumes in real-time. In this study, we extend this technique to compound 3D TEE volumes by using an electromagnetic tracking system. In this pilot study, two different types of phantoms were used to evaluate our technique. The results suggest our method is accurate and efficient. The compounding error is approximately 2.5mm.

Virtual reality-enhanced ultrasound guidance for atrial ablation: in vitro epicardial study

In an effort to reduce morbidity of cardiac interventions, minimizing invasiveness inevitably leads to limited visual access to the surgical targets. To address these limitations, we provide the surgeons with a robust visualization environment that integrates interventional ultrasound imaging augmented with pre-operative anatomical models and virtual surgical instruments within a virtual reality environment. Here we present an in vitro study on a cardiac phantom that mimics an ablation therapy procedure, which allows us to assess the feasibility of our surgical system in comparison to traditional intra-operative ultrasound imaging. Following surgical target identification via an electro-anatomical model, the "ablation procedure" is performed blindly. A 2.8 mm RMS targeting error is achieved using our novel surgical system. This level of accuracy is adequate from both a clinical and engineering perspective, under the inherent procedure requirements and limitations of the system.

Inside the beating heart: an in vivo feasibility study on fusing pre- and intra-operative imaging for minimally invasive therapy

OBJECTIVE

An interventional system for minimally invasive cardiac surgery was developed for therapy delivery inside the beating heart, in absence of direct vision.

METHOD

A system was developed to provide a virtual reality (VR) environment that integrates pre-operative imaging, real-time intra-operative guidance using 2D trans-esophageal ultrasound, and models of the surgical tools tracked using a magnetic tracking system. Detailed 3D dynamic cardiac models were synthesized from high-resolution pre-operative MR data and registered within the intra-operative imaging environment. The feature-based registration technique was employed to fuse pre- and intra-operative data during in vivo intracardiac procedures on porcine subjects.

RESULTS

This method was found to be suitable for in vivo applications as it relies on easily identifiable landmarks, and hence, it ensures satisfactory alignment of pre- and intra-operative anatomy in the region of interest (4.8 mm RMS alignment accuracy) within the VR environment. Our initial experience in translating this work to guide intracardiac interventions, such as mitral valve implantation and atrial septal defect repair demonstrated feasibility of the methods.

CONCLUSION

Surgical guidance in the absence of direct vision and with no exposure to ionizing radiation was achieved, so our virtual environment constitutes a feasible candidate for performing various off-pump intracardiac interventions.

Surgical accuracy under virtual reality-enhanced ultrasound guidance: an in vitro epicardial dynamic study

In the context of our ongoing objective to reduce morbidity associated with cardiac interventions, minimizing invasiveness has inevitably led to more limited visual access to the target tissues. To ameliorate these challenges, we provide the surgeons with a complex visualization environment that integrates interventional ultrasound imaging augmented with pre-operative anatomical models and virtual surgical instruments within a virtual reality environment. In this paper we present an in vitro study on a cardiac phantom aimed at assessing the feasibility and targeting accuracy of our surgical system in comparison to traditional ultrasound imaging for intra-operative surgical guidance. The 'therapy delivery' was modeled in the context of a blinded procedure, mimicking a closed-chest intervention. Four users navigated a tracked pointer to a target, under guidance provide by either US imaging or virtual reality-enhanced ultrasound. A 2.8 mm RMS targeting error was achieved using our novel surgical system, which is adequate from both a clinical and engineering perspective, under the inherent procedure requirements and limitations of the system.