Image guidance for robotic minimally invasive coronary artery bypass

The Additional Diagnostic Value of the Three-dimensional Volume Rendering Imaging in Routine Radiology Practice

Three-dimensional volume rendering (3DVR) is useful in a wide variety of medical-imaging applications. The increasingly advanced capabilities of CT and MRI to acquire volumetric data sets with isotropic voxels have resulted in the increased use of the 3DVR techniques for clinical applications. The two most commonly used techniques are the maximum intensity projection (MIP) and, more recently, 3DVR. Several kinds of medical imaging data could be reconstructed for 3D display, including CT, MRI, and ultrasonography (US). In particular, the 3D CT imaging has been developed, improved, and widely used of late. Understanding the mechanisms of 3DVR is essential for the accurate evaluation of the resulting images. Although further research is required to detect the efficiency of 3DVR in radiological applications, with wider availability and improved diagnostic performance, 3DVR is likely to enjoy widespread acceptance in the radiology practice going forward.

An Overview on Image Registration Techniques for Cardiac Diagnosis and Treatment

Image registration has been used for a wide variety of tasks within cardiovascular imaging. This study aims to provide an overview of the existing image registration methods to assist researchers and impart valuable resource for studying the existing methods or developing new methods and evaluation strategies for cardiac image registration. For the cardiac diagnosis and treatment strategy, image registration and fusion can provide complementary information to the physician by using the integrated image from these two modalities. This review also contains a description of various imaging techniques to provide an appreciation of the problems associated with implementing image registration, particularly for cardiac pathology intervention and treatments.

Impact of Soft Tissue Heterogeneity on Augmented Reality for Liver Surgery

This paper presents a method for real-time augmented reality of internal liver structures during minimally invasive hepatic surgery. Vessels and tumors computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Compared to current methods, our method is able to locate the in-depth positions of the tumors based on partial three-dimensional liver tissue motion using a real-time biomechanical model. This model permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Experimentations conducted on phantom liver permits to measure the accuracy of the augmentation while real-time augmentation on in vivo human liver during real surgery shows the benefits of such an approach for minimally invasive surgery.

When change happens: computer assistance and image guidance for minimally invasive therapy

Computer-assisted interventions are medical procedures that rely on image guidance and computer-based systems to provide visualisation and navigation information to the clinician, when direct vision of the sites or targets to be treated is not available, during minimally invasive procedures. Recent advances in medical image acquisition and processing, accompanied by technological breakthroughs in image fusion, visualisation and display have accelerated the adoption of minimally invasive approaches for a variety of medical procedures. This Letter is intended to serve as a brief overview of available image guidance and computer-assisted technology in the context of popular minimally invasive applications, while outlining some of the limitations and challenges in the transition from laboratory to clinical care.

Predicting target vessel location on robot-assisted coronary artery bypass graft using CT to ultrasound registration

PURPOSE

Although robot-assisted coronary artery bypass grafting (RA-CABG) has gained more acceptance worldwide, its success still depends on the surgeon's experience and expertise, and the conversion rate to full sternotomy is in the order of 15%-25%. One of the reasons for conversion is poor pre-operative planning, which is based solely on pre-operative computed tomography (CT) images. In this paper, the authors propose a technique to estimate the global peri-operative displacement of the heart and to predict the intra-operative target vessel location, validated via both an in vitro and a clinical study.

METHODS

As the peri-operative heart migration during RA-CABG has never been reported in the literatures, a simple in vitro validation study was conducted using a heart phantom. To mimic the clinical workflow, a pre-operative CT as well as peri-operative ultrasound images at three different stages in the procedure (Stage(0)-following intubation; Stage(1)-following lung deflation; and Stage(2)-following thoracic insufflation) were acquired during the experiment. Following image acquisition, a rigid-body registration using iterative closest point algorithm with the robust estimator was employed to map the pre-operative stage to each of the peri-operative ones, to estimate the heart migration and predict the peri-operative target vessel location. Moreover, a clinical validation of this technique was conducted using offline patient data, where a Monte Carlo simulation was used to overcome the limitations arising due to the invisibility of the target vessel in the peri-operative ultrasound images.

RESULTS

For the in vitro study, the computed target registration error (TRE) at Stage(0), Stage(1), and Stage(2) was 2.1, 3.3, and 2.6 mm, respectively. According to the offline clinical validation study, the maximum TRE at the left anterior descending (LAD) coronary artery was 4.1 mm at Stage(0), 5.1 mm at Stage(1), and 3.4 mm at Stage(2).

CONCLUSIONS

The authors proposed a method to measure and validate peri-operative shifts of the heart during RA-CABG. In vitro and clinical validation studies were conducted and yielded a TRE in the order of 5 mm for all cases. As the desired clinical accuracy imposed by this procedure is on the order of one intercostal space (10-15 mm), our technique suits the clinical requirements. The authors therefore believe this technique has the potential to improve the pre-operative planning by updating peri-operative migration patterns of the heart and, consequently, will lead to reduced conversion to conventional open thoracic procedures.

Registration of 3D+t coronary CTA and monoplane 2D+t X-ray angiography

A method for registering preoperative 3D+t coronary CTA with intraoperative monoplane 2D+t X-ray angiography images is proposed to improve image guidance during minimally invasive coronary interventions. The method uses a patient-specific dynamic coronary model, which is derived from the CTA scan by centerline extraction and motion estimation. The dynamic coronary model is registered with the 2D+t X-ray sequence, considering multiple X-ray time points concurrently, while taking breathing induced motion into account. Evaluation was performed on 26 datasets of 17 patients by comparing projected model centerlines with manually annotated centerlines in the X-ray images. The proposed 3D+t/2D+t registration method performed better than a 3D/2D registration method with respect to the accuracy and especially the robustness of the registration. Registration with a median error of 1.47 mm was achieved.

Anatomical annotation on vascular structure in volume rendered images

The precise annotation of vascular structure is desired in computer-assisted systems to help surgeons identify each vessel branch. This paper proposes a method that annotates vessels on volume rendered images by rendering their names on them using a two-pass rendering process. In the first rendering pass, vessel surface models are generated using such properties as centerlines, radii, and running directions. Then the vessel names are drawn on the vessel surfaces. Finally, the vessel name images and the corresponding depth buffer are generated by a virtual camera at the viewpoint. In the second rendering pass, volume rendered images are generated by a ray casting volume rendering algorithm that considers the depth buffer generated in the first rendering pass. After the two-pass rendering is finished, an annotated image is generated by blending the volume rendered image with the surface rendered image. To confirm the effectiveness of our proposed method, we performed a computer-assisted system for the automated annotation of abdominal arteries. The experimental results show that vessel names can be drawn on the corresponding vessel surface in the volume rendered images at a computing cost that is nearly the same as that by volume rendering only. The proposed method has enormous potential to be adopted to annotate the vessels in the 3D medical images in clinical applications, such as image-guided surgery.

On mixed reality environments for minimally invasive therapy guidance: systems architecture, successes and challenges in their implementation from laboratory to clinic

Mixed reality environments for medical applications have been explored and developed over the past three decades in an effort to enhance the clinician's view of anatomy and facilitate the performance of minimally invasive procedures. These environments must faithfully represent the real surgical field and require seamless integration of pre- and intra-operative imaging, surgical instrument tracking, and display technology into a common framework centered around and registered to the patient. However, in spite of their reported benefits, few mixed reality environments have been successfully translated into clinical use. Several challenges that contribute to the difficulty in integrating such environments into clinical practice are presented here and discussed in terms of both technical and clinical limitations. This article should raise awareness among both developers and end-users toward facilitating a greater application of such environments in the surgical practice of the future.

Regression-based cardiac motion prediction from single-phase CTA

State of the art cardiac computed tomography (CT) enables the acquisition of imaging data of the heart over the entire cardiac cycle at concurrent high spatial and temporal resolution. However, in clinical practice, acquisition is increasingly limited to 3-D images. Estimating the shape of the cardiac structures throughout the entire cardiac cycle from a 3-D image is therefore useful in applications such as the alignment of preoperative computed tomography angiography (CTA) to intra-operative X-ray images for improved guidance in coronary interventions. We hypothesize that the motion of the heart is partially explained by its shape and therefore investigate the use of three regression methods for motion estimation from single-phase shape information. Quantitative evaluation on 150 4-D CTA images showed a small, but statistically significant, increase in the accuracy of the predicted shape sequences when using any of the regression methods, compared to shape-independent motion prediction by application of the mean motion. The best results were achieved using principal component regression resulting in point-to-point errors of 2.3±0.5 mm, compared to values of 2.7±0.6 mm for shape-independent motion estimation. Finally, we showed that this significant difference withstands small variations in important parameter settings of the landmarking procedure.

Current progress on augmented reality visualization in endoscopic surgery

PURPOSE OF REVIEW

Advancements in surgery are progressing at a rapid rate; however, there are still limitations, including the ability to accurately visualize the target organ, in particular during laparoscopic surgery. Augmented reality visualization is a novel technique that has been developed to allow the fusion of three-dimensional medical images, such as those from transrectal ultrasound or computed tomography/MRI, with live camera images in real-time. In this review, we describe the current advancements and future directions of augmented reality and its application to laparoscopic surgery.

RECENT FINDINGS

Geometrically-correct superimposed images can be generated by tracking of the laparoscope and registration of the target organ. The fused image between the live laparoscopic images and the reconstructed three-dimensional organ model aides the surgeon in his or her understanding of anatomical structures. Laparoscopic and robot-assisted surgeries in both general surgery and urology have been performed with technical success to date. The primary limitation of the current augmented reality systems is its infancy in dynamic tracking of organ motion or deformation. Recently, augmented reality systems with organ tracking based on real-time image analysis were developed. Further improvement and/or development of such new technologies would resolve these issues.

SUMMARY

Augmented reality visualization is a significant advancement, improving the precision of laparoscopic/endoscopic surgery. New technologies to improve the dynamic tracking of organ motion or deformation are currently under investigation.

Vision-based navigation in image-guided interventions

The trend toward minimally invasive surgical interventions has created new challenges for visualization during surgical procedures. However, at the same time, the introduction of high-definition digital endoscopy offers the opportunity to apply methods from computer vision to provide visualization enhancements such as anatomic reconstruction, surface registration, motion tracking, and augmented reality. This review provides a perspective on this rapidly evolving field. It first introduces the clinical and technical background necessary for developing vision-based algorithms for interventional applications. It then discusses several examples of clinical interventions where computer vision can be applied, including bronchoscopy, rhinoscopy, transnasal skull-base neurosurgery, upper airway interventions, laparoscopy, robotic-assisted surgery, and Natural Orifice Transluminal Endoscopic Surgery (NOTES). It concludes that the currently reported work is only the beginning. As the demand for minimally invasive procedures rises, computer vision in surgery will continue to advance through close interdisciplinary work between interventionists and engineers.

Augmented reality in laparoscopic surgical oncology

Minimally invasive surgery represents one of the main evolutions of surgical techniques aimed at providing a greater benefit to the patient. However, minimally invasive surgery increases the operative difficulty since the depth perception is usually dramatically reduced, the field of view is limited and the sense of touch is transmitted by an instrument. However, these drawbacks can currently be reduced by computer technology guiding the surgical gesture. Indeed, from a patient's medical image (US, CT or MRI), Augmented Reality (AR) can increase the surgeon's intra-operative vision by providing a virtual transparency of the patient. AR is based on two main processes: the 3D visualization of the anatomical or pathological structures appearing in the medical image, and the registration of this visualization on the real patient. 3D visualization can be performed directly from the medical image without the need for a pre-processing step thanks to volume rendering. But better results are obtained with surface rendering after organ and pathology delineations and 3D modelling. Registration can be performed interactively or automatically. Several interactive systems have been developed and applied to humans, demonstrating the benefit of AR in surgical oncology. It also shows the current limited interactivity due to soft organ movements and interaction between surgeon instruments and organs. If the current automatic AR systems show the feasibility of such system, it is still relying on specific and expensive equipment which is not available in clinical routine. Moreover, they are not robust enough due to the high complexity of developing a real-time registration taking organ deformation and human movement into account. However, the latest results of automatic AR systems are extremely encouraging and show that it will become a standard requirement for future computer-assisted surgical oncology. In this article, we will explain the concept of AR and its principles. Then, we will review the existing interactive and automatic AR systems in digestive surgical oncology, highlighting their benefits and limitations. Finally, we will discuss the future evolutions and the issues that still have to be tackled so that this technology can be seamlessly integrated in the operating room.