Temporal diffeomorphic free-form deformation: Application to motion and strain estimation from 3D echocardiography

Modeling 3D Cardiac Contraction and Relaxation With Point Cloud Deformation Networks

Global single-valued biomarkers, such as ejection fraction, are widely used in clinical practice to assess cardiac function. However, they only approximate the heart's true 3D deformation process, thus limiting diagnostic accuracy and the understanding of cardiac mechanics. Metrics based on 3D shape have been proposed to alleviate these shortcomings. In this work, we present the Point Cloud Deformation Network (PCD-Net) as a novel geometric deep learning approach for direct modeling of 3D cardiac mechanics of the biventricular anatomy between the extreme ends of the cardiac cycle. Its encoder-decoder architecture combines a low-dimensional latent space with recent advances in point cloud deep learning for effective multi-scale feature learning directly on flexible and memory-efficient point cloud representations of the cardiac anatomy. We first evaluate the PCD-Net's predictive capability for both cardiac contraction and relaxation on a large UK Biobank dataset of over 10,000 subjects and find average Chamfer distances between the predicted and ground truth anatomies below the pixel resolution of the underlying image acquisition. We then show the PCD-Net's ability to capture subpopulation-specific differences in 3D cardiac mechanics between normal and myocardial infarction (MI) subjects and visualize abnormal phenotypes between predicted normal 3D shapes and corresponding observed ones. Finally, we demonstrate that the PCD-Net's learned 3D deformation encodings outperform multiple clinical and machine learning benchmarks by 11% in terms of area under the receiver operating characteristic curve for the tasks of prevalent MI detection and incident MI prediction and by 7% in terms of Harrell's concordance index for MI survival analysis.

Multi-Task Learning for Motion Analysis and Segmentation in 3D Echocardiography

Characterizing left ventricular deformation and strain using 3D+time echocardiography provides useful insights into cardiac function and can be used to detect and localize myocardial injury. To achieve this, it is imperative to obtain accurate motion estimates of the left ventricle. In many strain analysis pipelines, this step is often accompanied by a separate segmentation step; however, recent works have shown both tasks to be highly related and can be complementary when optimized jointly. In this work, we present a multi-task learning network that can simultaneously segment the left ventricle and track its motion between multiple time frames. Two task-specific networks are trained using a composite loss function. Cross-stitch units combine the activations of these networks by learning shared representations between the tasks at different levels. We also propose a novel shape-consistency unit that encourages motion propagated segmentations to match directly predicted segmentations. Using a combined synthetic and in-vivo 3D echocardiography dataset, we demonstrate that our proposed model can achieve excellent estimates of left ventricular motion displacement and myocardial segmentation. Additionally, we observe strong correlation of our image-based strain measurements with crystal-based strain measurements as well as good correspondence with SPECT perfusion mappings. Finally, we demonstrate the clinical utility of the segmentation masks in estimating ejection fraction and sphericity indices that correspond well with benchmark measurements.

DeepMesh: Mesh-Based Cardiac Motion Tracking Using Deep Learning

3D motion estimation from cine cardiac magnetic resonance (CMR) images is important for the assessment of cardiac function and the diagnosis of cardiovascular diseases. Current state-of-the art methods focus on estimating dense pixel-/voxel-wise motion fields in image space, which ignores the fact that motion estimation is only relevant and useful within the anatomical objects of interest, e.g., the heart. In this work, we model the heart as a 3D mesh consisting of epi- and endocardial surfaces. We propose a novel learning framework, DeepMesh, which propagates a template heart mesh to a subject space and estimates the 3D motion of the heart mesh from CMR images for individual subjects. In DeepMesh, the heart mesh of the end-diastolic frame of an individual subject is first reconstructed from the template mesh. Mesh-based 3D motion fields with respect to the end-diastolic frame are then estimated from 2D short- and long-axis CMR images. By developing a differentiable mesh-to-image rasterizer, DeepMesh is able to leverage 2D shape information from multiple anatomical views for 3D mesh reconstruction and mesh motion estimation. The proposed method estimates vertex-wise displacement and thus maintains vertex correspondences between time frames, which is important for the quantitative assessment of cardiac function across different subjects and populations. We evaluate DeepMesh on CMR images acquired from the UK Biobank. We focus on 3D motion estimation of the left ventricle in this work. Experimental results show that the proposed method quantitatively and qualitatively outperforms other image-based and mesh-based cardiac motion tracking methods.

Co-attention spatial transformer network for unsupervised motion tracking and cardiac strain analysis in 3D echocardiography

Myocardial ischemia/infarction causes wall-motion abnormalities in the left ventricle. Therefore, reliable motion estimation and strain analysis using 3D+time echocardiography for localization and characterization of myocardial injury is valuable for early detection and targeted interventions. Previous unsupervised cardiac motion tracking methods rely on heavily-weighted regularization functions to smooth out the noisy displacement fields in echocardiography. In this work, we present a Co-Attention Spatial Transformer Network (STN) for improved motion tracking and strain analysis in 3D echocardiography. Co-Attention STN aims to extract inter-frame dependent features between frames to improve the motion tracking in otherwise noisy 3D echocardiography images. We also propose a novel temporal constraint to further regularize the motion field to produce smooth and realistic cardiac displacement paths over time without prior assumptions on cardiac motion. Our experimental results on both synthetic and in vivo 3D echocardiography datasets demonstrate that our Co-Attention STN provides superior performance compared to existing methods. Strain analysis from Co-Attention STNs also correspond well with the matched SPECT perfusion maps, demonstrating the clinical utility for using 3D echocardiography for infarct localization.

Generative myocardial motion tracking via latent space exploration with biomechanics-informed prior

Myocardial motion and deformation are rich descriptors that characterize cardiac function. Image registration, as the most commonly used technique for myocardial motion tracking, is an ill-posed inverse problem which often requires prior assumptions on the solution space. In contrast to most existing approaches which impose explicit generic regularization such as smoothness, in this work we propose a novel method that can implicitly learn an application-specific biomechanics-informed prior and embed it into a neural network-parameterized transformation model. Particularly, the proposed method leverages a variational autoencoder-based generative model to learn a manifold for biomechanically plausible deformations. The motion tracking then can be performed via traversing the learnt manifold to search for the optimal transformations while considering the sequence information. The proposed method is validated on three public cardiac cine MRI datasets with comprehensive evaluations. The results demonstrate that the proposed method can outperform other approaches, yielding higher motion tracking accuracy with reasonable volume preservation and better generalizability to varying data distributions. It also enables better estimates of myocardial strains, which indicates the potential of the method in characterizing spatiotemporal signatures for understanding cardiovascular diseases.

DragNet: Learning-based deformable registration for realistic cardiac MR sequence generation from a single frame

Deformable image registration (DIR) can be used to track cardiac motion. Conventional DIR algorithms aim to establish a dense and non-linear correspondence between independent pairs of images. They are, nevertheless, computationally intensive and do not consider temporal dependencies to regulate the estimated motion in a cardiac cycle. In this paper, leveraging deep learning methods, we formulate a novel hierarchical probabilistic model, termed DragNet, for fast and reliable spatio-temporal registration in cine cardiac magnetic resonance (CMR) images and for generating synthetic heart motion sequences. DragNet is a variational inference framework, which takes an image from the sequence in combination with the hidden states of a recurrent neural network (RNN) as inputs to an inference network per time step. As part of this framework, we condition the prior probability of the latent variables on the hidden states of the RNN utilised to capture temporal dependencies. We further condition the posterior of the motion field on a latent variable from hierarchy and features from the moving image. Subsequently, the RNN updates the hidden state variables based on the feature maps of the fixed image and the latent variables. Different from traditional methods, DragNet performs registration on unseen sequences in a forward pass, which significantly expedites the registration process. Besides, DragNet enables generating a large number of realistic synthetic image sequences given only one frame, where the corresponding deformations are also retrieved. The probabilistic framework allows for computing spatio-temporal uncertainties in the estimated motion fields. Our results show that DragNet performance is comparable with state-of-the-art methods in terms of registration accuracy, with the advantage of offering analytical pixel-wise motion uncertainty estimation across a cardiac cycle and being a motion generator. We will make our code publicly available.

MulViMotion: Shape-Aware 3D Myocardial Motion Tracking From Multi-View Cardiac MRI

Recovering the 3D motion of the heart from cine cardiac magnetic resonance (CMR) imaging enables the assessment of regional myocardial function and is important for understanding and analyzing cardiovascular disease. However, 3D cardiac motion estimation is challenging because the acquired cine CMR images are usually 2D slices which limit the accurate estimation of through-plane motion. To address this problem, we propose a novel multi-view motion estimation network (MulViMotion), which integrates 2D cine CMR images acquired in short-axis and long-axis planes to learn a consistent 3D motion field of the heart. In the proposed method, a hybrid 2D/3D network is built to generate dense 3D motion fields by learning fused representations from multi-view images. To ensure that the motion estimation is consistent in 3D, a shape regularization module is introduced during training, where shape information from multi-view images is exploited to provide weak supervision to 3D motion estimation. We extensively evaluate the proposed method on 2D cine CMR images from 580 subjects of the UK Biobank study for 3D motion tracking of the left ventricular myocardium. Experimental results show that the proposed method quantitatively and qualitatively outperforms competing methods.

DeepStrain: A Deep Learning Workflow for the Automated Characterization of Cardiac Mechanics

Myocardial strain analysis from cinematic magnetic resonance imaging (cine-MRI) data provides a more thorough characterization of cardiac mechanics than volumetric parameters such as left-ventricular ejection fraction, but sources of variation including segmentation and motion estimation have limited its wider clinical use. We designed and validated a fast, fully-automatic deep learning (DL) workflow to generate both volumetric parameters and strain measures from cine-MRI data consisting of segmentation and motion estimation convolutional neural networks. The final motion network design, loss function, and associated hyperparameters are the result of a thorough ad hoc implementation that we carefully planned specific for strain quantification, tested, and compared to other potential alternatives. The optimal configuration was trained using healthy and cardiovascular disease (CVD) subjects (n = 150). DL-based volumetric parameters were correlated (>0.98) and without significant bias relative to parameters derived from manual segmentations in 50 healthy and CVD test subjects. Compared to landmarks manually-tracked on tagging-MRI images from 15 healthy subjects, landmark deformation using DL-based motion estimates from paired cine-MRI data resulted in an end-point-error of 2.9 ± 1.5 mm. Measures of end-systolic global strain from these cine-MRI data showed no significant biases relative to a tagging-MRI reference method. On 10 healthy subjects, intraclass correlation coefficient for intra-scanner repeatability was good to excellent (>0.75) for all global measures and most polar map segments. In conclusion, we developed and evaluated the first end-to-end learning-based workflow for automated strain analysis from cine-MRI data to quantitatively characterize cardiac mechanics of healthy and CVD subjects.

Learning-Based Regularization for Cardiac Strain Analysis via Domain Adaptation

Reliable motion estimation and strain analysis using 3D+ time echocardiography (4DE) for localization and characterization of myocardial injury is valuable for early detection and targeted interventions. However, motion estimation is difficult due to the low-SNR that stems from the inherent image properties of 4DE, and intelligent regularization is critical for producing reliable motion estimates. In this work, we incorporated the notion of domain adaptation into a supervised neural network regularization framework. We first propose a semi-supervised Multi-Layered Perceptron (MLP) network with biomechanical constraints for learning a latent representation that is shown to have more physiologically plausible displacements. We extended this framework to include a supervised loss term on synthetic data and showed the effects of biomechanical constraints on the network's ability for domain adaptation. We validated the semi-supervised regularization method on in vivo data with implanted sonomicrometers. Finally, we showed the ability of our semi-supervised learning regularization approach to identify infarct regions using estimated regional strain maps with good agreement to manually traced infarct regions from postmortem excised hearts.

Learning a Generative Motion Model From Image Sequences Based on a Latent Motion Matrix

We propose to learn a probabilistic motion model from a sequence of images for spatio-temporal registration. Our model encodes motion in a low-dimensional probabilistic space - the motion matrix - which enables various motion analysis tasks such as simulation and interpolation of realistic motion patterns allowing for faster data acquisition and data augmentation. More precisely, the motion matrix allows to transport the recovered motion from one subject to another simulating for example a pathological motion in a healthy subject without the need for inter-subject registration. The method is based on a conditional latent variable model that is trained using amortized variational inference. This unsupervised generative model follows a novel multivariate Gaussian process prior and is applied within a temporal convolutional network which leads to a diffeomorphic motion model. Temporal consistency and generalizability is further improved by applying a temporal dropout training scheme. Applied to cardiac cine-MRI sequences, we show improved registration accuracy and spatio-temporally smoother deformations compared to three state-of-the-art registration algorithms. Besides, we demonstrate the model's applicability for motion analysis, simulation and super-resolution by an improved motion reconstruction from sequences with missing frames compared to linear and cubic interpolation.

A 4D continuous representation of myocardial velocity fields from tissue phase mapping magnetic resonance imaging

Myocardial velocities carry important diagnostic information in a range of cardiac diseases, and play an important role in diagnosing and grading left ventricular diastolic dysfunction. Tissue Phase Mapping (TPM) Magnetic Resonance Imaging (MRI) enables discrete sampling of the myocardium’s underlying smooth and continuous velocity field. This paper presents a post-processing framework for constructing a spatially and temporally smooth and continuous representation of the myocardium’s velocity field from TPM data. In the proposed scheme, the velocity field is represented through either linear or cubic B-spline basis functions. The framework facilitates both interpolation and noise reducing approximation. As a proof-of-concept, the framework was evaluated using artificially noisy (i.e., synthetic) velocity fields created by adding different levels of noise to an original TPM data. The framework’s ability to restore the original velocity field was investigated using Bland-Altman statistics. Moreover, we calculated myocardial material point trajectories through temporal integration of the original and synthetic fields. The effect of noise reduction on the calculated trajectories was investigated by assessing the distance between the start and end position of material points after one complete cardiac cycle (end point error). We found that the Bland-Altman limits of agreement between the original and the synthetic velocity fields were reduced after application of the framework. Furthermore, the integrated trajectories exhibited consistently lower end point error. These results suggest that the proposed method generates a realistic continuous representation of myocardial velocity fields from noisy and discrete TPM data. Linear B-splines resulted in narrower limits of agreement between the original and synthetic fields, compared to Cubic B-splines. The end point errors were also consistently lower for Linear B-splines than for cubic. Linear B-splines therefore appear to be more suitable for TPM data.

Spatiotemporal Free-Form Registration Method Assisted by a Minimum Spanning Tree During Discontinuous Transformations

The sliding motion along the boundaries of discontinuous regions has been actively studied in B-spline free-form deformation framework. This study focusses on the sliding motion for a velocity field-based 3D+t registration. The discontinuity of the tangent direction guides the deformation of the object region, and a separate control of two regions provides a better registration accuracy. The sliding motion under the velocity field-based transformation is conducted under the [Formula: see text]-Rényi entropy estimator using a minimum spanning tree (MST) topology. Moreover, a new topology changing method of the MST is proposed. The topology change is performed as follows: inserting random noise, constructing the MST, and removing random noise while preserving a local connection consistency of the MST. This random noise process (RNP) prevents the [Formula: see text]-Rényi entropy-based registration from degrading in sliding motion, because the RNP creates a small disturbance around special locations. Experiments were performed using two publicly available datasets: the DIR-Lab dataset, which consists of 4D pulmonary computed tomography (CT) images, and a benchmarking framework dataset for cardiac 3D ultrasound. For the 4D pulmonary CT images, RNP produced a significantly improved result for the original MST with sliding motion (p<0.05). For the cardiac 3D ultrasound dataset, only a discontinuity-based registration indicated activity of the RNP. In contrast, the single MST without sliding motion did not show any improvement. These experiments proved the effectiveness of the RNP for sliding motion.

CardIAc: an open-source application for myocardial strain analysis

PURPOSE

This paper presents CardIAc, an open-source application designed as an alternative to commercial software for left ventricle myocardial strain quantification in short-axis cardiac magnetic resonance images. The aim is to provide a useful extension for myocardial strain analysis that can be easily adapted to incorporate different strategies of motion tracking to improve the strain accuracy. In this way, users with programming skills can easily modify the code and adjust the program's performance according to their own scientific or clinical requirements. The software is intended for research and clinical use is not advised.

METHODS

CardIAc was developed as a 3D Slicer extension for an easy installation and usability. The main contribution of this article is to provide a general workflow, going from data and segmentation loading, 3D heart modeling, analysis and several options for visualization of the myocardial strain.

RESULTS

CardIAc strain feature was evaluated on a public dataset (Cardiac Motion Analysis Challenge-STACOM 2011) of 15 volunteers, and a synthetic one generated from this real dataset. Results on the real dataset show that cardIAc achieves suitable accuracy for myocardial motion estimation with a median error of 3.66 mm. In particular, global strain curves show strong correlation with the bibliography for healthy patients and similar approaches. On the other hand, results on the synthetic dataset show a mean global error of 4.07%, 7.76% and 8.18% for circumferential, radial and longitudinal strain.

CONCLUSION

This paper introduces a new open-source application for strain analysis distributed under a BSD-style open-source license. Results demonstrate the capability and merits of the proposed application for strain analysis.

Cardiac motion estimation from medical images: a regularisation framework applied on pairwise image registration displacement fields

Accurate cardiac motion estimation from medical images such as ultrasound is important for clinical evaluation. We present a novel regularisation layer for cardiac motion estimation that will be applied after image registration and demonstrate its effectiveness. The regularisation utilises a spatio-temporal model of motion, b-splines of Fourier, to fit to displacement fields from pairwise image registration. In the process, it enforces spatial and temporal smoothness and consistency, cyclic nature of cardiac motion, and better adherence to the stroke volume of the heart. Flexibility is further given for inclusion of any set of registration displacement fields. The approach gave high accuracy. When applied to human adult Ultrasound data from a Cardiac Motion Analysis Challenge (CMAC), the proposed method is found to have 10% lower tracking error over CMAC participants. Satisfactory cardiac motion estimation is also demonstrated on other data sets, including human fetal echocardiography, chick embryonic heart ultrasound images, and zebrafish embryonic microscope images, with the average Dice coefficient between estimation motion and manual segmentation at 0.82-0.87. The approach of performing regularisation as an add-on layer after the completion of image registration is thus a viable option for cardiac motion estimation that can still have good accuracy. Since motion estimation algorithms are complex, dividing up regularisation and registration can simplify the process and provide flexibility. Further, owing to a large variety of existing registration algorithms, such an approach that is usable on any algorithm may be useful.

Machine Learning Approaches for Myocardial Motion and Deformation Analysis

Information about myocardial motion and deformation is key to differentiate normal and abnormal conditions. With the advent of approaches relying on data rather than pre-conceived models, machine learning could either improve the robustness of motion quantification or reveal patterns of motion and deformation (rather than single parameters) that differentiate pathologies. We review machine learning strategies for extracting motion-related descriptors and analyzing such features among populations, keeping in mind constraints specific to the cardiac application.

Hierarchical Template Matching for 3D Myocardial Tracking and Cardiac Strain Estimation

Myocardial tracking and strain estimation can non-invasively assess cardiac functioning using subject-specific MRI. As the left-ventricle does not have a uniform shape and functioning from base to apex, the development of 3D MRI has provided opportunities for simultaneous 3D tracking, and 3D strain estimation. We have extended a Local Weighted Mean (LWM) transformation function for 3D, and incorporated in a Hierarchical Template Matching model to solve 3D myocardial tracking and strain estimation problem. The LWM does not need to solve a large system of equations, provides smooth displacement of myocardial points, and adapt local geometric differences in images. Hence, 3D myocardial tracking can be performed with 1.49 mm median error, and without large error outliers. The maximum error of tracking is up to 24% reduced compared to benchmark methods. Moreover, the estimated strain can be insightful to improve 3D imaging protocols, and the computer code of LWM could also be useful for geo-spatial and manufacturing image analysis researchers.

Flow network tracking for spatiotemporal and periodic point matching: Applied to cardiac motion analysis

The accurate quantification of left ventricular (LV) deformation/strain shows significant promise for quantitatively assessing cardiac function for use in diagnosis and therapy planning. However, accurate estimation of the displacement of myocardial tissue and hence LV strain has been challenging due to a variety of issues, including those related to deriving tracking tokens from images and following tissue locations over the entire cardiac cycle. In this work, we propose a point matching scheme where correspondences are modeled as flow through a graphical network. Myocardial surface points are set up as nodes in the network and edges define neighborhood relationships temporally. The novelty lies in the constraints that are imposed on the matching scheme, which render the correspondences one-to-one through the entire cardiac cycle, and not just two consecutive frames. The constraints also encourage motion to be cyclic, which an important characteristic of LV motion. We validate our method by applying it to the estimation of quantitative LV displacement and strain estimation using 8 synthetic and 8 open-chested canine 4D echocardiographic image sequences, the latter with sonomicrometric crystals implanted on the LV wall. We were able to achieve excellent tracking accuracy on the synthetic dataset and observed a good correlation with crystal-based strains on the in-vivo data.

Robust Optical Flow Estimation in Cardiac Ultrasound Images Using a Sparse Representation

This paper introduces a robust 2-D cardiac motion estimation method. The problem is formulated as an energy minimization with an optical flow-based data fidelity term and two regularization terms imposing spatial smoothness and the sparsity of the motion field in an appropriate cardiac motion dictionary. Robustness to outliers, such as imaging artefacts and anatomical motion boundaries, is introduced using robust weighting functions for the data fidelity term as well as for the spatial and sparse regularizations. The motion fields and the weights are computed jointly using an iteratively re-weighted minimization strategy. The proposed robust approach is evaluated on synthetic data and realistic simulation sequences with available ground-truth by comparing the performance with state-of-the-art algorithms. Finally, the proposed method is validated using two sequences of in vivo images. The obtained results show the interest of the proposed approach for 2-D cardiac ultrasound imaging.

Cardiac motion and spillover correction for quantitative PET imaging using dynamic MRI

PURPOSE

Cardiac positron emission tomography/magnetic resonance imaging (PET/MRI) acquisition presents novel clinical applications thanks to the combination of viability and metabolic imaging (PET) and functional and structural imaging (MRI). However, the resolution of PET, as well as cardiac and respiratory motion in nongated cardiac imaging acquisition protocols, leads to a reduction in image quality and severe quantitative bias. Respiratory or cardiac motion is customarily addressed with gated reconstruction which results in higher noise.

METHODS

Inspired by a method that has been used in brain PET, a practical correction approach, designed to overcome these existing limitations for quantitative PET imaging, was developed and applied in the context of cardiac PET/MRI. The correction approach for PET data consists of computing the mean density map of each underlying moving region, as obtained with MRI, and translating them to the PET space taking into account the PET spatial and temporal resolution. Using these tissue density maps, the method then constructs a system of linear equations that models the activity recovery and cross-contamination coefficients, which can be solved for the true activity values. Physical and numerical cardiac phantoms were employed in order to quantify the proposed correction. The full correction pipeline was then used to assess differences in metabolic function between scar and healthy myocardium in eight patients with recent acute myocardial infarction using [¹¹ C]-acetate. Data from ten additional patients, injected with [¹⁸ F]-FDG, were used to compare the method to the standard electrocardiography (ECG)-gated approach.

RESULTS

The proposed method resulted in better recovery (from 32% to 95% on the simulated phantom model) and less residual activity than the standard approach. Higher signal-to-noise and contrast-to-noise ratios than ECG-gating were also witnessed (Signal-to-noise ratio (SNR) increased from 2.92 to 5.24, contrast-to-noise ratio (CNR) increased from 62.9 to 145.9 when compared to a four-gate reconstruction). Finally, the relevance of this correction using [¹¹ C]-acetate PET patient data, for which erroneous physiological conclusions could have been made based on the uncorrected data, was established as the correction led to the expected clinical results.

CONCLUSIONS

An efficient and simple method to correct for the quantitative biases in PET measurements caused by cardiac motion has been developed. Validation experiments using phantom and patient data showed improved accuracy and reliability with this approach when compared to simpler strategies such as gated acquisition or optimal regions of interest (ROI).

Regularization-Free Strain Mapping in Three Dimensions, With Application to Cardiac Ultrasound

Quantifying dynamic strain fields from time-resolved volumetric medical imaging and microscopy stacks is a pressing need for radiology and mechanobiology. A critical limitation of all existing techniques is regularization: because these volumetric images are inherently noisy, the current strain mapping techniques must impose either displacement regularization and smoothing that sacrifices spatial resolution, or material property assumptions that presuppose a material model, as in hyperelastic warping. Here, we present, validate, and apply the first three-dimensional (3D) method for estimating mechanical strain directly from raw 3D image stacks without either regularization or assumptions about material behavior. We apply the method to high-frequency ultrasound images of mouse hearts to diagnose myocardial infarction. We also apply the method to present the first ever in vivo quantification of elevated strain fields in the heart wall associated with the insertion of the chordae tendinae. The method shows promise for broad application to dynamic medical imaging modalities, including high-frequency ultrasound, tagged magnetic resonance imaging, and confocal fluorescence microscopy.

A noninvasive method for the determination of in vivo mitral valve leaflet strains

Assessment of mitral valve (MV) function is important in many diagnostic, prognostic, and surgical planning applications for treatment of MV disease. Yet, to date, there are no accepted noninvasive methods for determination of MV leaflet deformation, which is a critical metric of MV function. In this study, we present a novel, completely noninvasive computational method to estimate MV leaflet in-plane strains from clinical-quality real-time three-dimensional echocardiography (rt-3DE) images. The images were first segmented to produce meshed medial-surface leaflet geometries of the open and closed states. To establish material point correspondence between the two states, an image-based morphing pipeline was implemented within a finite element (FE) modeling framework in which MV closure was simulated by pressurizing the open-state geometry, and local corrective loads were applied to enforce the actual MV closed shape. This resulted in a complete map of local systolic leaflet membrane strains, obtained from the final FE mesh configuration. To validate the method, we utilized an extant in vitro database of fiducially labeled MVs, imaged in conditions mimicking both the healthy and diseased states. Our method estimated local anisotropic in vivo strains with less than 10% error and proved to be robust to changes in boundary conditions similar to those observed in ischemic MV disease. Next, we applied our methodology to ovine MVs imaged in vivo with rt-3DE and compared our results to previously published findings of in vivo MV strains in the same type of animal as measured using surgically sutured fiducial marker arrays. In regions encompassed by fiducial markers, we found no significant differences in circumferential(P = 0.240) or radial (P = 0.808) strain estimates between the marker-based measurements and our novel noninvasive method. This method can thus be used for model validation as well as for studies of MV disease and repair.

A momentum-based diffeomorphic demons framework for deformable MR-CT image registration

Neuro-navigated procedures require a high degree of geometric accuracy but are subject to geometric error from complex deformation in the deep brain-e.g. regions about the ventricles due to egress of cerebrospinal fluid (CSF) upon neuroendoscopic approach or placement of a ventricular shunt. We report a multi-modality, diffeomorphic, deformable registration method using momentum-based acceleration of the Demons algorithm to solve the transformation relating preoperative MRI and intraoperative CT as a basis for high-precision guidance. The registration method (pMI-Demons) extends the mono-modality, diffeomorphic form of the Demons algorithm to multi-modality registration using pointwise mutual information (pMI) as a similarity metric. The method incorporates a preprocessing step to nonlinearly stretch CT image values and incorporates a momentum-based approach to accelerate convergence. Registration performance was evaluated in phantom and patient images: first, the sensitivity of performance to algorithm parameter selection (including update and displacement field smoothing, histogram stretch, and the momentum term) was analyzed in a phantom study over a range of simulated deformations; and second, the algorithm was applied to registration of MR and CT images for four patients undergoing minimally invasive neurosurgery. Performance was compared to two previously reported methods (free-form deformation using mutual information (MI-FFD) and symmetric normalization using mutual information (MI-SyN)) in terms of target registration error (TRE), Jacobian determinant (J), and runtime. The phantom study identified optimal or nominal settings of algorithm parameters for translation to clinical studies. In the phantom study, the pMI-Demons method achieved comparable registration accuracy to the reference methods and strongly reduced outliers in TRE (p [Formula: see text] 0.001 in Kolmogorov-Smirnov test). Similarly, in the clinical study: median TRE  =  1.54 mm (0.83-1.66 mm interquartile range, IQR) for pMI-Demons compared to 1.40 mm (1.02-1.67 mm IQR) for MI-FFD and 1.64 mm (0.90-1.92 mm IQR) for MI-SyN. The pMI-Demons and MI-SyN methods yielded diffeomorphic transformations (J  >  0) that preserved topology, whereas MI-FFD yielded unrealistic (J  <  0) deformations subject to tissue folding and tearing. Momentum-based acceleration gave a ~35% speedup of the pMI-Demons method, providing registration runtime of 10.5 min (reduced to 2.2 min on GPU), compared to 15.5 min for MI-FFD and 34.7 min for MI-SyN. The pMI-Demons method achieved registration accuracy comparable to MI-FFD and MI-SyN, maintained diffeomorphic transformation similar to MI-SyN, and accelerated runtime in a manner that facilitates translation to image-guided neurosurgery.

Quantitative Assessment of the Effect of the Out-of-Plane Movement of the Homogenous Ellipsoidal Model of the Left Ventricle on the Deformation Measures Estimated Using 2-D Speckle Tracking-An In-Silico Study

Effect of the out-of-plane (OOP) movement amplitude on estimates of global displacements (radial, circumferential) and strains (radial , circumferential ) was studied in an ellipsoidal model of the left ventricle using finite-element modeling (FEM), synthetic ultrasonic data, and short-axis view. This effect was assessed using median of the absolute relative error (RE) of the global parameters. FEM provided node displacements for synthetic ultrasonic data and reference data generation. Displacements were estimated using block-matching (BM) and B-spline (BS) methods. FEM-derived data analysis, free from errors resulting from speckle tracking, indicated that the tissue motion introduced REs of global strain estimates below 4.5%. The effect of the OOP motion amplitude on strain estimates was strain specific and depended on the displacement estimation method. In the case of , the increase of the OOP amplitude resulted in quasi-linear increase of the RE from approximately 10% to 15%. The modulus of the end-systolic (ES) errors of the estimates almost linearly increased with increasing OOP amplitude approximately from 10% to 16%. REs of the estimate were close to 80% and 40%, respectively, in the case of the BM and BS methods, and increased with increasing OOP amplitude. The modulus of the ES errors of the estimates in the case of the BS method was about -40% and showed low sensitivity to the OOP amplitude; in the BM case, these errors varied approximately from -70% to -58% for OOP amplitude from 0 to 15 mm.

An Iterative Diffeomorphic Algorithm for Registration of Subdivision Surfaces: Application to Congenital Heart Disease

In this paper, we present a new diffeomorphic registration algorithm for the registration of 3D models to 3D points. A biventricular template is iteratively fitted to the data by a series of implicitly constrained diffeomorphic linear least squares fits with decreasing regularization weights before performing an explicitly constrained diffeomorphic fit. The algorithm has been tested on a set of manual contours from 20 patients with a variety of congenital heart disease. Registration accuracy was assessed by calculating the mean point-to-point distance and the Dice overlap metric. Results showed that the method was able to accurately fit the biventricular model to 3D points and that the deformable model was able to fit all the pathologies while being diffeomorphic. The algorithm took approximately 5 minutes to fit each case, with an average of 52,580 points per case.

Three-dimensional speckle-tracking echocardiography: benefits and limitations of integrating myocardial mechanics with three-dimensional imaging

Three-dimensional (3D) speckle-tracking echocardiography (3DSTE) is an advanced imaging technique designed for left ventricular (LV) myocardial deformation analysis based on 3D data sets. 3DSTE has the potential to overcome some of the intrinsic limitations of two-dimensional STE (2DSTE) in the assessment of complex LV myocardial mechanics, offering additional deformation parameters (such as area strain) and a comprehensive quantitation of LV geometry and function from a single 3D acquisition. Albeit being a relatively young technique still undergoing technological developments, several experimental studies and clinical investigations have already demonstrated the reliability and feasibility of 3DSTE, as well as several advantages of 3DSTE over 2DSTE. This technique has provided new insights into LV mechanics in several clinical fields, such as the objective assessment of global and regional LV function in ischemic and non-ischemic heart diseases, the evaluation of LV mechanical dyssynchrony, as well as the detection of subclinical cardiac dysfunction in cardiovascular conditions at risk of progression to overt heart failure. However, 3DSTE generally requires patient's breathhold and regular rhythm for enabling an ECG-gated multi-beat 3D acquisition. In addition, the measurements, normal limits and cut-off values pertaining to 3D strain parameters are currently vendor-specific and highly dependent on the 3D ultrasound equipment used. Technological advances with improvement in spatial and temporal resolution and a standardized methodology for obtaining vendor-independent 3D strain measurements are expected in the future for a widespread application of 3DSTE in both clinical and research arenas. The purpose of this review is to summarize currently available data on 3DSTE methodology (feasibility, accuracy and reproducibility), strengths and weaknesses with respect to 2DSTE, as well as the main clinical applications and future research priorities of this emerging technology.

Fast left ventricle tracking using localized anatomical affine optical flow

In daily clinical cardiology practice, left ventricle (LV) global and regional function assessment is crucial for disease diagnosis, therapy selection, and patient follow-up. Currently, this is still a time-consuming task, spending valuable human resources. In this work, a novel fast methodology for automatic LV tracking is proposed based on localized anatomically constrained affine optical flow. This novel method can be combined to previously proposed segmentation frameworks or manually delineated surfaces at an initial frame to obtain fully delineated datasets and, thus, assess both global and regional myocardial function. Its feasibility and accuracy were investigated in 3 distinct public databases, namely in realistically simulated 3D ultrasound, clinical 3D echocardiography, and clinical cine cardiac magnetic resonance images. The method showed accurate tracking results in all databases, proving its applicability and accuracy for myocardial function assessment. Moreover, when combined to previous state-of-the-art segmentation frameworks, it outperformed previous tracking strategies in both 3D ultrasound and cardiac magnetic resonance data, automatically computing relevant cardiac indices with smaller biases and narrower limits of agreement compared to reference indices. Simultaneously, the proposed localized tracking method showed to be suitable for online processing, even for 3D motion assessment. Importantly, although here evaluated for LV tracking only, this novel methodology is applicable for tracking of other target structures with minimal adaptations.

Phase Vector Incompressible Registration Algorithm for Motion Estimation From Tagged Magnetic Resonance Images

Tagged magnetic resonance imaging has been used for decades to observe and quantify motion and strain of deforming tissue. It is challenging to obtain 3-D motion estimates due to a tradeoff between image slice density and acquisition time. Typically, interpolation methods are used either to combine 2-D motion extracted from sparse slice acquisitions into 3-D motion or to construct a dense volume from sparse acquisitions before image registration methods are applied. This paper proposes a new phase-based 3-D motion estimation technique that first computes harmonic phase volumes from interpolated tagged slices and then matches them using an image registration framework. The approach uses several concepts from diffeomorphic image registration with a key novelty that defines a symmetric similarity metric on harmonic phase volumes from multiple orientations. The material property of harmonic phase solves the aperture problem of optical flow and intensity-based methods and is robust to tag fading. A harmonic magnitude volume is used in enforcing incompressibility in the tissue regions. The estimated motion fields are dense, incompressible, diffeomorphic, and inverse-consistent at a 3-D voxel level. The method was evaluated using simulated phantoms, human brain data in mild head accelerations, human tongue data during speech, and an open cardiac data set. The method shows comparable accuracy to three existing methods while demonstrating low computation time and robustness to tag fading and noise.

Fast Simulation of Dynamic Ultrasound Images Using the GPU

Simulated ultrasound data is a valuable tool for development and validation of quantitative image analysis methods in echocardiography. Unfortunately, simulation time can become prohibitive for phantoms consisting of a large number of point scatterers. The COLE algorithm by Gao et al. is a fast convolution-based simulator that trades simulation accuracy for improved speed. We present highly efficient parallelized CPU and GPU implementations of the COLE algorithm with an emphasis on dynamic simulations involving moving point scatterers. We argue that it is crucial to minimize the amount of data transfers from the CPU to achieve good performance on the GPU. We achieve this by storing the complete trajectories of the dynamic point scatterers as spline curves in the GPU memory. This leads to good efficiency when simulating sequences consisting of a large number of frames, such as B-mode and tissue Doppler data for a full cardiac cycle. In addition, we propose a phase-based subsample delay technique that efficiently eliminates flickering artifacts seen in B-mode sequences when COLE is used without enough temporal oversampling. To assess the performance, we used a laptop computer and a desktop computer, each equipped with a multicore Intel CPU and an NVIDIA GPU. Running the simulator on a high-end TITAN X GPU, we observed two orders of magnitude speedup compared to the parallel CPU version, three orders of magnitude speedup compared to simulation times reported by Gao et al. in their paper on COLE, and a speedup of 27000 times compared to the multithreaded version of Field II, using numbers reported in a paper by Jensen. We hope that by releasing the simulator as an open-source project we will encourage its use and further development.

Ultrasound 2D strain measurement for arm lymphedema using deformable registration: A feasibility study

Purpose

Lymphedema, a swelling of the extremity, is a debilitating morbidity of cancer treatment. Current clinical evaluation of lymphedema is often based on medical history and physical examinations, which is subjective. In this paper, the authors report an objective, quantitative 2D strain imaging approach using a hybrid deformable registration to measure soft-tissue stiffness and assess the severity of lymphedema.

Methods

The authors have developed a new 2D strain imaging method using registration of pre- and post-compression ultrasound B-mode images, which combines the statistical intensity- and structure-based similarity measures using normalized mutual information (NMI) metric and normalized sum-of-squared-differences (NSSD), with an affine-based global and B-spline-based local transformation model. This 2D strain method was tested through a series of experiments using elastography phantom under various pressures. Clinical feasibility was tested with four participants: two patients with arm lymphedema following breast-cancer radiotherapy and two healthy volunteers.

Results

The phantom experiments have shown that the proposed registration-based strain method significantly increased the signal-to-noise and contrast-to-noise ratio under various pressures as compared with the commonly used cross-correlation-based elastography method. In the pilot study, the strain images were successfully generated for all participants. The averaged strain values of the lymphedema affected arms were much higher than those of the normal arms.

Conclusions

The authors have developed a deformable registration-based 2D strain method for the evaluation of arm lymphedema. The initial findings are encouraging and a large clinical study is warranted to further evaluate this 2D ultrasound strain imaging technology.

Longitudinal Analysis of Pre-Term Neonatal Cerebral Ventricles From 3D Ultrasound Images Using Spatial-Temporal Deformable Registration

Preterm neonates with a very low birth weight of less than 1,500 grams are at increased risk for developing intraventricular hemorrhage (IVH), which is a major cause of brain injury in preterm neonates. Quantitative measurements of ventricular dilatation or shrinkage play an important role in monitoring patients and evaluating treatment options. 3D ultrasound (US) has been developed to monitor ventricle volume as a biomarker for ventricular changes. However, ventricle volume as a global indicator does not allow for precise analysis of local ventricular changes, which could be linked to specific neurological problems often seen in the patient population later in life. In this work, a 3D+t spatial-temporal deformable registration approachis proposed, which is applied to the analysis of the detailed local changes of preterm IVH neonatal ventricles from 3D US images. In particular, a novel sequential convex/dual optimization algorithm is introduced to extract the optimal 3D+t spatial-temporal deformable field, which simultaneously optimizes the sequence of 3D deformation fieldswhile enjoying both efficiencyand simplicity in numerics. The developed registration technique was evaluated by comparing two manually extracted ventricle surfaces from the baseline and the registered follow-up images using the metrics of Dice similarity coefficient (DSC), mean absolute surface distance (MAD), and maximum absolute surface distance (MAXD). The performed experiments using 14 patients with 5 time-point images per patient show that the proposed 3D+t registration approach accurately recovered the longitudinal deformation of ventricle surfaces from 3D US images. The proposed approach may be potentially used to analyse the change pattern of cerebral ventricles of IVH patients, their response to different treatment options, and to elucidate the deficiencies that a patient could have later in life. To the best of our knowledge, this paper reports the first study on the longitudinalanalysis of neonatal ventricular system from 3D US images.

Diffeomorphic Multi-Frame Non-Rigid Registration of Cell Nuclei in 2D and 3D Live Cell Images

To gain a better understanding of cellular and molecular processes, it is important to quantitatively analyze the motion of subcellular particles in live cell microscopy image sequences. Since, generally, the subcellular particles move and cell nuclei move as well as deform, it is important to decouple the movement of particles from that of the cell nuclei using non-rigid registration methods. We have developed a diffeomorphic multi-frame approach for non-rigid registration of cell nuclei in 2D and 3D live cell fluorescence microscopy images. Our non-rigid registration approach is based on local optic flow estimation, exploits information from multiple consecutive image frames, and determines diffeomorphic transformations in the log-domain, which allows efficient computation of the inverse transformations. To register single images of an image sequence to a reference image, we use a temporally weighted mean image, which is constructed based on inverse transformations and multiple consecutive frames. Using multiple consecutive frames improves the registration accuracy compared to pairwise registration, and using a temporally weighted mean image significantly reduces the computation time compared with previous work. In addition, we use a flow boundary preserving method for regularization of computed deformation vector fields, which prevents from over-smoothing compared to standard Gaussian filtering. Our approach has been successfully applied to 2D and 3D synthetic as well as real live cell microscopy image sequences, and an experimental comparison with non-rigid pairwise, multi-frame, and temporal groupwise registration has been carried out.

Reflections on ultrasound image analysis

Ultrasound (US) image analysis has advanced considerably in twenty years. Progress in ultrasound image analysis has always been fundamental to the advancement of image-guided interventions research due to the real-time acquisition capability of ultrasound and this has remained true over the two decades. But in quantitative ultrasound image analysis - which takes US images and turns them into more meaningful clinical information - thinking has perhaps more fundamentally changed. From roots as a poor cousin to Computed Tomography (CT) and Magnetic Resonance (MR) image analysis, both of which have richer anatomical definition and thus were better suited to the earlier eras of medical image analysis which were dominated by model-based methods, ultrasound image analysis has now entered an exciting new era, assisted by advances in machine learning and the growing clinical and commercial interest in employing low-cost portable ultrasound devices outside traditional hospital-based clinical settings. This short article provides a perspective on this change, and highlights some challenges ahead and potential opportunities in ultrasound image analysis which may both have high impact on healthcare delivery worldwide in the future but may also, perhaps, take the subject further away from CT and MR image analysis research with time.

Kernel Bundle Diffeomorphic Image Registration Using Stationary Velocity Fields and Wendland Basis Functions

In this paper, we propose a multi-scale, multi-kernel shape, compactly supported kernel bundle framework for stationary velocity field-based image registration (Wendland kernel bundle stationary velocity field, wKB-SVF). We exploit the possibility of directly choosing kernels to construct a reproducing kernel Hilbert space (RKHS) instead of imposing it from a differential operator. The proposed framework allows us to minimize computational cost without sacrificing the theoretical foundations of SVF-based diffeomorphic registration. In order to recover deformations occurring at different scales, we use compactly supported Wendland kernels at multiple scales and orders to parameterize the velocity fields, and the framework allows simultaneous optimization over all scales. The performance of wKB-SVF is extensively compared to the 14 non-rigid registration algorithms presented in a recent comparison paper. On both MGH10 and CUMC12 datasets, the accuracy of wKB-SVF is improved when compared to other registration algorithms. In a disease-specific application for intra-subject registration, atrophy scores estimated using the proposed registration scheme separates the diagnostic groups of Alzheimer's and normal controls better than the state-of-the-art segmentation technique. Experimental results show that wKB-SVF is a robust, flexible registration framework that allows theoretically well-founded and computationally efficient multi-scale representation of deformations and is equally well-suited for both inter- and intra-subject image registration.

A Spatio-Temporal Atlas and Statistical Model of the Tongue During Speech from Cine-MRI

Statistical modeling of tongue motion during speech using cine magnetic resonance imaging (MRI) provides key information about the relationship between structure and motion of the tongue. In order to study the variability of tongue shape and motion in populations, a consistent integration and characterization of inter-subject variability is needed. In this paper, a method to construct a spatio-temporal atlas comprising a mean motion model and statistical modes of variation during speech is presented. The model is based on the cine-MRI from twenty two normal speakers and consists of several steps involving both spatial and temporal alignment problems independently. First, all images are registered into a common reference space, which is taken to be a neutral resting position of the tongue. Second, the tongue shapes of each individual relative to this reference space are produced. Third, a time warping approach (several are evaluated) is used to align the time frames of each subject to a common time series of initial mean images. Finally, the spatio-temporal atlas is created by time-warping each subject, generating new mean images at each time, and producing shape statistics around these mean images using principal component analysis at each reference time frame. Experimental results consist of comparison of various parameters and methods in creation of the atlas and a demonstration of the final modes of variations at various key time frames in a sample phrase.

Influence of ultrasound speckle tracking strategies for motion and strain estimation

Speckle Tracking is one of the most prominent techniques used to estimate the regional movement of the heart based on ultrasound acquisitions. Many different approaches have been proposed, proving their suitability to obtain quantitative and qualitative information regarding myocardial deformation, motion and function assessment. New proposals to improve the basic algorithm usually focus on one of these three steps: (1) the similarity measure between images and the speckle model; (2) the transformation model, i.e. the type of motion considered between images; (3) the optimization strategies, such as the use of different optimization techniques in the transformation step or the inclusion of structural information. While many contributions have shown their good performance independently, it is not always clear how they perform when integrated in a whole pipeline. Every step will have a degree of influence over the following and hence over the final result. Thus, a Speckle Tracking pipeline must be analyzed as a whole when developing novel methods, since improvements in a particular step might be undermined by the choices taken in further steps. This work presents two main contributions: (1) We provide a complete analysis of the influence of the different steps in a Speckle Tracking pipeline over the motion and strain estimation accuracy. (2) The study proposes a methodology for the analysis of Speckle Tracking systems specifically designed to provide an easy and systematic way to include other strategies. We close the analysis with some conclusions and recommendations that can be used as an orientation of the degree of influence of the models for speckle, the transformation models, interpolation schemes and optimization strategies over the estimation of motion features. They can be further use to evaluate and design new strategy into a Speckle Tracking system.

Robust Spatio-Temporal Registration of 4D Cardiac Ultrasound Sequences

Registration of multiple 3D ultrasound sectors in order to provide an extended field of view is important for the appreciation of larger anatomical structures at high spatial and temporal resolution. In this paper, we present a method for fully automatic spatio-temporal registration between two partially overlapping 3D ultrasound sequences. The temporal alignment is solved by aligning the normalized cross correlation-over-time curves of the sequences. For the spatial alignment, corresponding 3D Scale Invariant Feature Transform (SIFT) features are extracted from all frames of both sequences independently of the temporal alignment. A rigid transform is then calculated by least squares minimization in combination with random sample consensus. The method is applied to 16 echocardiographic sequences of the left and right ventricles and evaluated against manually annotated temporal events and spatial anatomical landmarks. The mean distances between manually identified landmarks in the left and right ventricles after automatic registration were (mean ± SD) 4.3 ± 1.2 mm compared to a reference error of 2.8 ± 0.6 mm with manual registration. For the temporal alignment, the absolute errors in valvular event times were 14.4 ± 11.6 ms for Aortic Valve (AV) opening, 18.6 ± 16.0 ms for AV closing, and 34.6 ± 26.4 ms for mitral valve opening, compared to a mean inter-frame time of 29 ms.

Anatomical Image Registration Using Volume Conservation to Assess Cardiac Deformation From 3D Ultrasound Recordings

Myocardial deformation imaging can provide valuable insights in myocardial mechanics and help in the diagnosis, prognosis and follow-up of cardiac diseases. However, extracting these indices in 3D is challenging due to the limitations in spatial and temporal resolution of the current volumetric ultrasound systems. For this purpose, we developed an anatomical free-form deformation image registration framework which is locally adapted to the anatomy of the heart. In this work we explored whether incorporating a myocardial volume conservation regularizer would improve strain estimates. We evaluated our technique on in silico echo sequences featuring realistic speckle textures and showed the volume conservation regularizer to be beneficial in reducing strain errors further when used in combination with a smoothness penalty. This combination led to more physiological boundary conditions. It also made distinguishing ischemic from normal segments easier in clinical images.

Estimation of Myocardial Strain and Contraction Phase From Cine MRI Using Variational Data Assimilation

This paper presents a new method to estimate left ventricle deformations using variational data assimilation that combines image observations from cine MRI and a dynamic evolution model of the heart. The main contribution of the model is that it embeds parameters modeling the contraction / relaxation process. It estimates myocardial motion and contraction parameters simultaneously, providing accurate complementary information for diagnosis. The method was applied to synthetic datasets with known ground truth motion and to 47 patients MRI datasets acquired at three slice locations (base, mid-ventricle and apex). Radial and circumferential strain components were compared to those obtained with a reference tag tracking software, exhibiting good agreement with intraclass correlation coefficients (ICC) above 0.8. Results were also evaluated against wall motion score indices used to assess cardiac kinetics in clinical practice. The assimilation process overcame issues caused by temporal artifacts as a result of the dynamic model, compared to using the observation term alone. Moreover we found that the new dynamic model, consisting of a piecewise transport model acting independently on systole and diastole performed better than the standard continuous transport model, which oversmooths temporal variations. Estimated strain and contraction parameters significantly correlated to clinical scores, making them promising features for diagnosing not only hypokinesia but also dyskinesia.

Integration of Multi-Plane Tissue Doppler and B-Mode Echocardiographic Images for Left Ventricular Motion Estimation

Although modern ultrasound acquisition systems allow recording of 3D echocardiographic images, tracking anatomical structures from them is still challenging. In addition, since these images are typically created from information obtained across several cardiac cycles, it is not yet possible to acquire high-quality 3D images from patients presenting varying heart rhythms. In this paper, we propose a method to estimate the motion field from multi-plane echocardiographic images of the left ventricle, which are acquired simultaneously during a single cardiac cycle. The method integrates tri-plane B-mode and tissue Doppler images acquired at different rotation angles around the long axis of the left ventricle. It uses a diffeomorphic continuous spatio-temporal transformation model with a spherical data representation for a better interpolation in the circumferential direction. This framework allows exploiting the spatial relation among the acquired planes. In addition, higher temporal resolution of the transformation in the beam direction is achieved by uncoupling the estimation of the different components of the velocity field. The method was validated using a realistic synthetic dataset including healthy and ischemic cases, obtaining errors of 0.14 ± 0.09 mm for displacements, 0.96 ± 1.03% for longitudinal strain and 3.94 ± 4.38% for radial strain estimation. In addition, the method was also demonstrated on a healthy volunteer and two patients with ischemia.

Temporal Super Resolution Enhancement of Echocardiographic Images Based on Sparse Representation

A challenging issue for echocardiographic image interpretation is the accurate analysis of small transient motions of myocardium and valves during real-time visualization. A higher frame rate video may reduce this difficulty, and temporal super resolution (TSR) is useful for illustrating the fast-moving structures. In this paper, we introduce a novel framework that optimizes TSR enhancement of echocardiographic images by utilizing temporal information and sparse representation. The goal of this method is to increase the frame rate of echocardiographic videos, and therefore enable more accurate analyses of moving structures. For the proposed method, we first derived temporal information by extracting intensity variation time curves (IVTCs) assessed for each pixel. We then designed both low-resolution and high-resolution overcomplete dictionaries based on prior knowledge of the temporal signals and a set of prespecified known functions. The IVTCs can then be described as linear combinations of a few prototype atoms in the low-resolution dictionary. We used the Bayesian compressive sensing (BCS) sparse recovery algorithm to find the sparse coefficients of the signals. We extracted the sparse coefficients and the corresponding active atoms in the low-resolution dictionary to construct new sparse coefficients corresponding to the high-resolution dictionary. Using the estimated atoms and the high-resolution dictionary, a new IVTC with more samples was constructed. Finally, by placing the new IVTC signals in the original IVTC positions, we were able to reconstruct the original echocardiography video with more frames. The proposed method does not require training of low-resolution and high-resolution dictionaries, nor does it require motion estimation; it does not blur fast-moving objects, and does not have blocking artifacts.

Estimation of Large-Scale Organ Motion in B-Mode Ultrasound Image Sequences: A Survey

Reviewed here are methods developed for following (i.e., tracking) structures in medical B-mode ultrasound time sequences during large-scale motion. The resulting motion estimation problem and its key components are defined. The main tracking approaches are described, and their strengths and weaknesses are discussed. Existing motion estimation methods, tested on multiple in vivo sequences, are categorized with respect to their clinical applications, namely, cardiac, respiratory and muscular motion. A large number of works in this field had to be discarded as thorough validation of the results was missing. The remaining relevant works identified indicate the possibility of reaching an average tracking accuracy up to 1-2 mm. Real-time performance can be achieved using several methods. Yet only very few of these have progressed to clinical practice. The latest trends include incorporation of complementary and prior information. Advances are expected from common evaluation databases and 4-D ultrasound scanning technologies.

Physical Constraint Finite Element Model for Medical Image Registration

Due to being derived from linear assumption, most elastic body based non-rigid image registration algorithms are facing challenges for soft tissues with complex nonlinear behavior and with large deformations. To take into account the geometric nonlinearity of soft tissues, we propose a registration algorithm on the basis of Newtonian differential equation. The material behavior of soft tissues is modeled as St. Venant-Kirchhoff elasticity, and the nonlinearity of the continuum represents the quadratic term of the deformation gradient under the Green- St.Venant strain. In our algorithm, the elastic force is formulated as the derivative of the deformation energy with respect to the nodal displacement vectors of the finite element; the external force is determined by the registration similarity gradient flow which drives the floating image deforming to the equilibrium condition. We compared our approach to three other models: 1) the conventional linear elastic finite element model (FEM); 2) the dynamic elastic FEM; 3) the robust block matching (RBM) method. The registration accuracy was measured using three similarities: MSD (Mean Square Difference), NC (Normalized Correlation) and NMI (Normalized Mutual Information), and was also measured using the mean and max distance between the ground seeds and corresponding ones after registration. We validated our method on 60 image pairs including 30 medical image pairs with artificial deformation and 30 clinical image pairs for both the chest chemotherapy treatment in different periods and brain MRI normalization. Our method achieved a distance error of 0.320±0.138 mm in x direction and 0.326±0.111 mm in y direction, MSD of 41.96±13.74, NC of 0.9958±0.0019, NMI of 1.2962±0.0114 for images with large artificial deformations; and average NC of 0.9622±0.008 and NMI of 1.2764±0.0089 for the real clinical cases. Student’s t-test demonstrated that our model statistically outperformed the other methods in comparison (p-values <0.05).

Construction of An Unbiased Spatio-Temporal Atlas of the Tongue During Speech

Quantitative characterization and comparison of tongue motion during speech and swallowing present fundamental challenges because of striking variations in tongue structure and motion across subjects. A reliable and objective description of the dynamics tongue motion requires the consistent integration of inter-subject variability to detect the subtle changes in populations. To this end, in this work, we present an approach to constructing an unbiased spatio-temporal atlas of the tongue during speech for the first time, based on cine-MRI from twenty two normal subjects. First, we create a common spatial space using images from the reference time frame, a neutral position, in which the unbiased spatio-temporal atlas can be created. Second, we transport images from all time frames of all subjects into this common space via the single transformation. Third, we construct atlases for each time frame via groupwise diffeomorphic registration, which serves as the initial spatio-temporal atlas. Fourth, we update the spatio-temporal atlas by realigning each time sequence based on the Lipschitz norm on diffeomorphisms between each subject and the initial atlas. We evaluate and compare different configurations such as similarity measures to build the atlas. Our proposed method permits to accurately and objectively explain the main pattern of tongue surface motion.

Analysis of passive cardiac constitutive laws for parameter estimation using 3D tagged MRI

An unresolved issue in patient-specific models of cardiac mechanics is the choice of an appropriate constitutive law, able to accurately capture the passive behavior of the myocardium, while still having uniquely identifiable parameters tunable from available clinical data. In this paper, we aim to facilitate this choice by examining the practical identifiability and model fidelity of constitutive laws often used in cardiac mechanics. Our analysis focuses on the use of novel 3D tagged MRI, providing detailed displacement information in three dimensions. The practical identifiability of each law is examined by generating synthetic 3D tags from in silico simulations, allowing mapping of the objective function landscape over parameter space and comparison of minimizing parameter values with original ground truth values. Model fidelity was tested by comparing these laws with the more complex transversely isotropic Guccione law, by characterizing their passive end-diastolic pressure–volume relation behavior, as well as by considering the in vivo case of a healthy volunteer. These results show that a reduced form of the Holzapfel–Ogden law provides the best balance between identifiability and model fidelity across the tests considered.

Right ventricular strain analysis from three-dimensional echocardiography by using temporally diffeomorphic motion estimation

PURPOSE

Quantitative analysis of right ventricle (RV) motion is important for study of the mechanism of congenital and acquired diseases. Unlike left ventricle (LV), motion estimation of RV is more difficult because of its complex shape and thin myocardium. Although attempts of finite element models on MR images and speckle tracking on echocardiography have shown promising results on RV strain analysis, these methods can be improved since the temporal smoothness of the motion is not considered.

METHODS

The authors have proposed a temporally diffeomorphic motion estimation method in which a spatiotemporal transformation is estimated by optimization of a registration energy functional of the velocity field in their earlier work. The proposed motion estimation method is a fully automatic process for general image sequences. The authors apply the method by combining with a semiautomatic myocardium segmentation method to the RV strain analysis of three-dimensional (3D) echocardiographic sequences of five open-chest pigs under different steady states.

RESULTS

The authors compare the peak two-point strains derived by their method with those estimated from the sonomicrometry, the results show that they have high correlation. The motion of the right ventricular free wall is studied by using segmental strains. The baseline sequence results show that the segmental strains in their methods are consistent with results obtained by other image modalities such as MRI. The image sequences of pacing steady states show that segments with the largest strain variation coincide with the pacing sites.

CONCLUSIONS

The high correlation of the peak two-point strains of their method and sonomicrometry under different steady states demonstrates that their RV motion estimation has high accuracy. The closeness of the segmental strain of their method to those from MRI shows the feasibility of their method in the study of RV function by using 3D echocardiography. The strain analysis of the pacing steady states shows the potential utility of their method in study on RV diseases.