Multimodal registration and data fusion for cardiac resynchronization therapy optimization

Multi-modality cardiac image computing: A survey

Multi-modality cardiac imaging plays a key role in the management of patients with cardiovascular diseases. It allows a combination of complementary anatomical, morphological and functional information, increases diagnosis accuracy, and improves the efficacy of cardiovascular interventions and clinical outcomes. Fully-automated processing and quantitative analysis of multi-modality cardiac images could have a direct impact on clinical research and evidence-based patient management. However, these require overcoming significant challenges including inter-modality misalignment and finding optimal methods to integrate information from different modalities. This paper aims to provide a comprehensive review of multi-modality imaging in cardiology, the computing methods, the validation strategies, the related clinical workflows and future perspectives. For the computing methodologies, we have a favored focus on the three tasks, i.e., registration, fusion and segmentation, which generally involve multi-modality imaging data, either combining information from different modalities or transferring information across modalities. The review highlights that multi-modality cardiac imaging data has the potential of wide applicability in the clinic, such as trans-aortic valve implantation guidance, myocardial viability assessment, and catheter ablation therapy and its patient selection. Nevertheless, many challenges remain unsolved, such as missing modality, modality selection, combination of imaging and non-imaging data, and uniform analysis and representation of different modalities. There is also work to do in defining how the well-developed techniques fit in clinical workflows and how much additional and relevant information they introduce. These problems are likely to continue to be an active field of research and the questions to be answered in the future.

Spatiotemporal registration and fusion of transthoracic echocardiography and volumetric coronary artery tree

PURPOSE

Cardiac multimodal image fusion can offer an image with various types of information in a single image. Many coronary stenosis, which are anatomically clear, are not functionally significant. The treatment of such kind of stenosis can cause irreversible effects on the patient. Thus, choosing the best treatment planning depend on anatomical and functional information is very beneficial.

METHODS

An algorithm for the fusion of coronary computed tomography angiography (CCTA) as an anatomical and transthoracic echocardiography (TTE) as a functional modality is presented. CCTA and TTE are temporally registered using manifold learning. A pattern search optimization algorithm, using normalized mutual information, is used to find the best match slice to TTE frame from CCTA volume. By employing a free-form deformation, the heart's non-rigid deformations are modeled. The spatiotemporal registered TTE frame is embedded to achieve the fusion result.

RESULTS

The accuracy is evaluated on CCTA and TTE data obtained from 10 patients. In temporal registration, mean absolute error of 1.97 [Formula: see text] 1.23 is resulted from comparing the output frame numbers from the algorithm and from manual assignment by an expert. In spatial registration, the accuracy of the similarity between the best match slice from CCTA volume and TTE frame is resulted in 1.82 [Formula: see text] 0.024 mm, 6.74 [Formula: see text] 0.013 mm, and 0.901 [Formula: see text] 0.0548 due to mean absolute distance, Hausdorff distance, and Dice similarity coefficient, respectively.

CONCLUSION

Without the use of ECG and Optical tracking systems, a semiautomatic framework of spatiotemporal registration and fusion of CCTA volume and TTE frame is presented. The experimental results showed the effectiveness of our proposed method to create complementary information from TTE and CCTA, which may help in the early diagnosis and effective treatment of cardiovascular diseases (CVDs).

Multi-modality medical image fusion technique using multi-objective differential evolution based deep neural networks

The advancements in automated diagnostic tools allow researchers to obtain more and more information from medical images. Recently, to obtain more informative medical images, multi-modality images have been used. These images have significantly more information as compared to traditional medical images. However, the construction of multi-modality images is not an easy task. The proposed approach, initially, decomposes the image into sub-bands using a non-subsampled contourlet transform (NSCT) domain. Thereafter, an extreme version of the Inception (Xception) is used for feature extraction of the source images. The multi-objective differential evolution is used to select the optimal features. Thereafter, the coefficient of determination and the energy loss based fusion functions are used to obtain the fused coefficients. Finally, the fused image is computed by applying the inverse NSCT. Extensive experimental results show that the proposed approach outperforms the competitive multi-modality image fusion approaches.

A review of current trends in three-dimensional analysis of left ventricular myocardial strain

Three-dimensional (3D) left ventricular (LV) myocardial strain measurements using transthoracic 3D echocardiography speckle tracking analysis have several advantages over two-dimensional (2D) LV strain measurements, because 3D strain values are derived from the entire LV myocardium, yielding more accurate estimates of global and regional LV function. In this review article, we summarize the current status of 3D LV myocardial strain. Specifically, we describe how 3D LV strain analysis is performed. Next, we compare characteristics of 2D and 3D strain, and we explain validation of 3D strain measurements, feasibility and measurement differences between 2D and 3D strain, reference values of 3D strain, and its applications in several clinical scenarios. In some parts of this review, we used a meta-analysis to draw reliable conclusions. We also describe the added value of 3D over 2D strain in several specific pathologies and prognoses. Finally, we discuss novel techniques using 3D strain and suggest its future directions.

Deformable multimodal registration for navigation in beating-heart cardiac surgery

PURPOSE

Minimally invasive beating-heart surgery is currently performed using endoscopes and without navigation. Registration of intraoperative ultrasound to a preoperative cardiac CT scan is a valuable step toward image-guided navigation.

METHODS

The registration was achieved by first extracting a representative point set from each ultrasound image in the sequence using a deformable registration. A template shape representing the cardiac chambers was deformed through a hierarchy of affine transformations to match each ultrasound image using a generalized expectation maximization algorithm. These extracted point sets were matched to the CT by exhaustively searching over a large number of precomputed slices of 3D geometry. The result is a similarity transformation mapping the intraoperative ultrasound to preoperative CT.

RESULTS

Complete data sets were acquired for four patients. Transesophageal echocardiography ultrasound sequences were deformably registered to a model of oriented points with a mean error of 2.3 mm. Ultrasound and CT scans were registered to a mean of 3 mm, which is comparable to the error of 2.8 mm expected by merging ultrasound registration with uncertainty of cardiac CT.

CONCLUSION

The proposed algorithm registered 3D CT with dynamic 2D intraoperative imaging. The algorithm aligned the images in both space and time, needing neither dynamic CT imaging nor intraoperative electrocardiograms. The accuracy was sufficient for navigation in thoracoscopically guided beating-heart surgery.

An Overview on Image Registration Techniques for Cardiac Diagnosis and Treatment

Image registration has been used for a wide variety of tasks within cardiovascular imaging. This study aims to provide an overview of the existing image registration methods to assist researchers and impart valuable resource for studying the existing methods or developing new methods and evaluation strategies for cardiac image registration. For the cardiac diagnosis and treatment strategy, image registration and fusion can provide complementary information to the physician by using the integrated image from these two modalities. This review also contains a description of various imaging techniques to provide an appreciation of the problems associated with implementing image registration, particularly for cardiac pathology intervention and treatments.

A Planning and Guidance Platform for Cardiac Resynchronization Therapy

Patients with drug-refractory heart failure can greatly benefit from cardiac resynchronization therapy (CRT). A CRT device can resynchronize the contractions of the left ventricle (LV) leading to reduced mortality. Unfortunately, 30%-50% of patients do not respond to treatment when assessed by objective criteria such as cardiac remodeling. A significant contributing factor is the suboptimal placement of the LV lead. It has been shown that placing this lead away from scar and at the point of latest mechanical activation can improve response rates. This paper presents a comprehensive and highly automated system that uses scar and mechanical activation to plan and guide CRT procedures. Standard clinical preoperative magnetic resonance imaging is used to extract scar and mechanical activation information. The data are registered to a single 3-D coordinate system and visualized in novel 2-D and 3-D American Heart Association plots enabling the clinician to select target segments. During the procedure, the planning information is overlaid onto live fluoroscopic images to guide lead deployment. The proposed platform has been used during 14 CRT procedures and validated on synthetic, phantom, volunteer, and patient data.

Multimodality registration of two-dimensional echocardiography and cardiac CT for mitral valve diagnosis and surgical planning

A registration method to fuse two-dimensional (2-D) echocardiography images with cardiac computed tomography (CT) volume is presented. The method consists of two major procedures: temporal and spatial registrations. In temporal registration, the echocardiography frames at similar cardiac phases as the CT volume were interpolated based on electrocardiogram signal information, and the noise of the echocardiography image was reduced using the speckle reducing anisotropic diffusion technique. For spatial registration, an intensity-based normalized mutual information method was applied with a pattern search optimization algorithm to produce an interpolated cardiac CT image. The proposed registration framework does not require optical tracking information. Dice coefficient and Hausdorff distance for the left atrium assessments were [Formula: see text] and [Formula: see text], respectively; for left ventricle, they were [Formula: see text] and [Formula: see text], respectively. There was no significant difference in the mitral valve annulus diameter measurement between the manually and automatically registered CT images. The transformation parameters showed small deviations ([Formula: see text] deviation in translation and [Formula: see text] for rotation) between manual and automatic registrations. The proposed method aids the physician in diagnosing mitral valve disease as well as provides surgical guidance during the treatment procedure.

Automatic segmentation and quantification of the cardiac structures from non-contrast-enhanced cardiac CT scans

Early structural changes to the heart, including the chambers and the coronary arteries, provide important information on pre-clinical heart disease like cardiac failure. Currently, contrast-enhanced cardiac computed tomography angiography (CCTA) is the preferred modality for the visualization of the cardiac chambers and the coronaries. In clinical practice not every patient undergoes a CCTA scan; many patients receive only a non-contrast-enhanced calcium scoring CT scan (CTCS), which has less radiation dose and does not require the administration of contrast agent. Quantifying cardiac structures in such images is challenging, as they lack the contrast present in CCTA scans. Such quantification would however be relevant, as it enables population based studies with only a CTCS scan. The purpose of this work is therefore to investigate the feasibility of automatic segmentation and quantification of cardiac structures viz whole heart, left atrium, left ventricle, right atrium, right ventricle and aortic root from CTCS scans. A fully automatic multi-atlas-based segmentation approach is used to segment the cardiac structures. Results show that the segmentation overlap between the automatic method and that of the reference standard have a Dice similarity coefficient of 0.91 on average for the cardiac chambers. The mean surface-to-surface distance error over all the cardiac structures is [Formula: see text] mm. The automatically obtained cardiac chamber volumes using the CTCS scans have an excellent correlation when compared to the volumes in corresponding CCTA scans, a Pearson correlation coefficient (R) of 0.95 is obtained. Our fully automatic method enables large-scale assessment of cardiac structures on non-contrast-enhanced CT scans.

2D to 3D fusion of echocardiography and cardiac CT for TAVR and TAVI image guidance

This study proposed a registration framework to fuse 2D echocardiography images of the aortic valve with preoperative cardiac CT volume. The registration facilitates the fusion of CT and echocardiography to aid the diagnosis of aortic valve diseases and provide surgical guidance during transcatheter aortic valve replacement and implantation. The image registration framework consists of two major steps: temporal synchronization and spatial registration. Temporal synchronization allows time stamping of echocardiography time series data to identify frames that are at similar cardiac phase as the CT volume. Spatial registration is an intensity-based normalized mutual information method applied with pattern search optimization algorithm to produce an interpolated cardiac CT image that matches the echocardiography image. Our proposed registration method has been applied on the short-axis "Mercedes Benz" sign view of the aortic valve and long-axis parasternal view of echocardiography images from ten patients. The accuracy of our fully automated registration method was 0.81 ± 0.08 and 1.30 ± 0.13 mm in terms of Dice coefficient and Hausdorff distance for short-axis aortic valve view registration, whereas for long-axis parasternal view registration it was 0.79 ± 0.02 and 1.19 ± 0.11 mm, respectively. This accuracy is comparable to gold standard manual registration by expert. There was no significant difference in aortic annulus diameter measurement between the automatically and manually registered CT images. Without the use of optical tracking, we have shown the applicability of this technique for effective fusion of echocardiography with preoperative CT volume to potentially facilitate catheter-based surgery.

Multi-modal data fusion for Cardiac Resynchronization Therapy planning and assistance

Cardiac Resynchronization Therapy (CRT) has been validated as an efficient treatment for selected patients suffering from heart failure with cardiac dyssynchrony. In case of bi-ventricular stimulation, the response to the therapy may be improved by an optimal choice of the left ventricle (LV) pacing sites. The characterization of LV properties to select the best candidate sites and to precise their access modes would be useful for the clinician in pre- and per-operative stages. For that purpose, we propose a new pre-operative analysis solution integrating previously developed multi-modal data registration methods and a new segmentation process of their coronary venous access. Moreover, a novel visualization interface is proposed to help the clinician to visualize the most relevant pacing sites and their access during the implantation in the operating room. This work is illustrated on real CRT data patients.

Synchronization and Registration of Cine Magnetic Resonance and Dynamic Computed Tomography Images of the Heart

The synchronization and registration of dynamic computed tomography (CT) and magnetic resonance images (MRI) of the heart is required to perform a combined analysis of their complementary information. We propose a novel method that synchronizes and registers intrapatient dynamic CT and cine-MRI short axis view (SAX). For the synchronization step, a normalized cross-correlation curve is computed from each image sequence to describe the global cardiac dynamics. The time axes of these curves are then warped using an adapted dynamic time warping (DTW) procedure. The adaptation constrains the time deformation to obtain a coherent warping function. The registration step then computes the rigid transformation that maximizes the multiimage normalized mutual information of DTW-synchronized images. The DTW synchronization and the multiimage registration were evaluated using dynamic CT and cine-SAX acquisitions from nine patients undergoing cardiac resynchronization therapy. The distance between the end-systolic phases after DTW was used to evaluate the synchronization. Mean errors, expressed as a percentage of the RR-intervals, were 3.9% and 3.7% after adapted DTW synchronization against 10.8% and 11.3% after linear synchronization, for dynamic CT and cine-SAX, respectively. This suggests that the adapted DTW synchronization leads to a coherent warping of cardiac dynamics. The multiimage registration was evaluated using fiducial points. Compared to a monoimage and a two-image registration, the multiimage registration of DTW-synchronized images obtained the lowest mean fiducial error showing that the use of dynamic voxel intensity information improves the registration.