Constrained manifold learning for the characterization of pathological deviations from normality

Characterizing interactions between cardiac shape and deformation by non-linear manifold learning

In clinical routine, high-dimensional descriptors of the cardiac function such as shape and deformation are reduced to scalars (e.g. volumes or ejection fraction), which limit the characterization of complex diseases. Besides, these descriptors undergo interactions depending on disease, which may bias their computational analysis. In this paper, we aim at characterizing such interactions by unsupervised manifold learning. We propose to use a sparsified version of Multiple Manifold Learning to align the latent spaces encoding each descriptor and weighting the strength of the alignment depending on each pair of samples. While this framework was up to now only applied to link different datasets from the same manifold, we demonstrate its relevance to characterize the interactions between different but partially related descriptors of the cardiac function (shape and deformation). We benchmark our approach against linear and non-linear embedding strategies, among which the fusion of manifolds by Multiple Kernel Learning, the independent embedding of each descriptor by Diffusion Maps, and a strict alignment based on pairwise correspondences. We first evaluated the methods on a synthetic dataset from a 0D cardiac model where the interactions between descriptors are fully controlled. Then, we transfered them to a population of right ventricular meshes from 310 subjects (100 healthy and 210 patients with right ventricular disease) obtained from 3D echocardiography, where the link between shape and deformation is key for disease understanding. Our experiments underline the relevance of jointly considering shape and deformation descriptors, and that manifold alignment is preferable over fusion for our application. They also confirm at a finer scale the characteristic traits of the right ventricular diseases in our population.

A probabilistic deep motion model for unsupervised cardiac shape anomaly assessment

Automatic shape anomaly detection in large-scale imaging data can be useful for screening suboptimal segmentations and pathologies altering the cardiac morphology without intensive manual labour. We propose a deep probabilistic model for local anomaly detection in sequences of heart shapes, modelled as point sets, in a cardiac cycle. A deep recurrent encoder-decoder network captures the spatio-temporal dependencies to predict the next shape in the cycle and thus derive the outlier points that are attributed to excessive deviations from the network prediction. A predictive mixture distribution models the inlier and outlier classes via Gaussian and uniform distributions, respectively. A Gibbs sampling Expectation-Maximisation (EM) algorithm computes soft anomaly scores of the points via the posterior probabilities of each class in the E-step and estimates the parameters of the network and the predictive distribution in the M-step. We demonstrate the versatility of the method using two shape datasets derived from: (i) one million biventricular CMR images from 20,000 participants in the UK Biobank (UKB), and (ii) routine diagnostic imaging from Multi-Centre, Multi-Vendor, and Multi-Disease Cardiac Image (M&Ms). Experiments show that the detected shape anomalies in the UKB dataset are mostly associated with poor segmentation quality, and the predicted shape sequences show significant improvement over the input sequences. Furthermore, evaluations on U-Net based shapes from the M&Ms dataset reveals that the anomalies are attributable to the underlying pathologies that affect the ventricles. The proposed model can therefore be used as an effective mechanism to sift shape anomalies in large-scale cardiac imaging pipelines for further analysis.

Applications of artificial intelligence in cardiovascular imaging

Research into artificial intelligence (AI) has made tremendous progress over the past decade. In particular, the AI-powered analysis of images and signals has reached human-level performance in many applications owing to the efficiency of modern machine learning methods, in particular deep learning using convolutional neural networks. Research into the application of AI to medical imaging is now very active, especially in the field of cardiovascular imaging because of the challenges associated with acquiring and analysing images of this dynamic organ. In this Review, we discuss the clinical questions in cardiovascular imaging that AI can be used to address and the principal methodological AI approaches that have been developed to solve the related image analysis problems. Some approaches are purely data-driven and rely mainly on statistical associations, whereas others integrate anatomical and physiological information through additional statistical, geometric and biophysical models of the human heart. In a structured manner, we provide representative examples of each of these approaches, with particular attention to the underlying computational imaging challenges. Finally, we discuss the remaining limitations of AI approaches in cardiovascular imaging (such as generalizability and explainability) and how they can be overcome.

Machine Learning Approaches for Myocardial Motion and Deformation Analysis

Information about myocardial motion and deformation is key to differentiate normal and abnormal conditions. With the advent of approaches relying on data rather than pre-conceived models, machine learning could either improve the robustness of motion quantification or reveal patterns of motion and deformation (rather than single parameters) that differentiate pathologies. We review machine learning strategies for extracting motion-related descriptors and analyzing such features among populations, keeping in mind constraints specific to the cardiac application.

Regional Multi-View Learning for Cardiac Motion Analysis: Application to Identification of Dilated Cardiomyopathy Patients

OBJECTIVE

The aim of this paper is to describe an automated diagnostic pipeline that uses as input only ultrasound (US) data, but is at the same time informed by a training database of multimodal magnetic resonance (MR) and US image data.

METHODS

We create a multimodal cardiac motion atlas from three-dimensional (3-D) MR and 3-D US data followed by multi-view machine learning algorithms to combine and extract the most meaningful cardiac descriptors for classification of dilated cardiomyopathy (DCM) patients using US data only. More specifically, we propose two algorithms based on multi-view linear discriminant analysis and multi-view Laplacian support vector machines (MvLapSVMs). Furthermore, a novel regional multi-view approach is proposed to exploit the regional relationships between the two modalities.

RESULTS

We evaluate our pipeline on the classification task of discriminating between normals and DCM patients. Results show that the use of multi-view classifiers together with a cardiac motion atlas results in a statistically significant improvement in accuracy compared to classification without the multimodal atlas. MvLapSVM was able to achieve the highest accuracy for both the global approach (92.71%) and the regional approach (94.32%).

CONCLUSION

Our work represents an important contribution to the understanding of cardiac motion, which is an important aid in the quantification of the contractility and function of the left ventricular myocardium.

SIGNIFICANCE

The intended workflow of the developed pipeline is to make use of the prior knowledge from the multimodal atlas to enable robust extraction of indicators from 3-D US images for detecting DCM patients.

Myocardial strain computed at multiple spatial scales from tagged magnetic resonance imaging: Estimating cardiac biomarkers for CRT patients

Abnormal cardiac motion can indicate different forms of disease, which can manifest at different spatial scales in the myocardium. Many studies have sought to characterise particular motion abnormalities associated with specific diseases, and to utilise motion information to improve diagnoses. However, the importance of spatial scale in the analysis of cardiac deformation has not been extensively investigated. We build on recent work on the analysis of myocardial strains at different spatial scales using a cardiac motion atlas to find the optimal scales for estimating different cardiac biomarkers. We apply a multi-scale strain analysis to a 43 patient cohort of cardiac resynchronisation therapy (CRT) patients using tagged magnetic resonance imaging data for (1) predicting response to CRT, (2) identifying septal flash, (3) estimating QRS duration, and (4) identifying the presence of ischaemia. A repeated, stratified cross-validation is used to demonstrate the importance of spatial scale in our analysis, revealing different optimal spatial scales for the estimation of different biomarkers.

A multimodal spatiotemporal cardiac motion atlas from MR and ultrasound data

Cardiac motion atlases provide a space of reference in which the motions of a cohort of subjects can be directly compared. Motion atlases can be used to learn descriptors that are linked to different pathologies and which can subsequently be used for diagnosis. To date, all such atlases have been formed and applied using data from the same modality. In this work we propose a framework to build a multimodal cardiac motion atlas from 3D magnetic resonance (MR) and 3D ultrasound (US) data. Such an atlas will benefit from the complementary motion features derived from the two modalities, and furthermore, it could be applied in clinics to detect cardiovascular disease using US data alone. The processing pipeline for the formation of the multimodal motion atlas initially involves spatial and temporal normalisation of subjects' cardiac geometry and motion. This step was accomplished following a similar pipeline to that proposed for single modality atlas formation. The main novelty of this paper lies in the use of a multi-view algorithm to simultaneously reduce the dimensionality of both the MR and US derived motion data in order to find a common space between both modalities to model their variability. Three different dimensionality reduction algorithms were investigated: principal component analysis, canonical correlation analysis and partial least squares regression (PLS). A leave-one-out cross validation on a multimodal data set of 50 volunteers was employed to quantify the accuracy of the three algorithms. Results show that PLS resulted in the lowest errors, with a reconstruction error of less than 2.3 mm for MR-derived motion data, and less than 2.5  mm for US-derived motion data. In addition, 1000 subjects from the UK Biobank database were used to build a large scale monomodal data set for a systematic validation of the proposed algorithms. Our results demonstrate the feasibility of using US data alone to analyse cardiac function based on a multimodal motion atlas.

Quantitative Analysis of Electro-Anatomical Maps: Application to an Experimental Model of Left Bundle Branch Block/Cardiac Resynchronization Therapy

Electro-anatomical maps (EAMs) are commonly acquired in clinical routine for guiding ablation therapies. They provide voltage and activation time information on a 3-D anatomical mesh representation, making them useful for analyzing the electrical activation patterns in specific pathologies. However, the variability between the different acquisitions and anatomies hampers the comparison between different maps. This paper presents two contributions for the analysis of electrical patterns in EAM data from biventricular surfaces of cardiac chambers. The first contribution is an integrated automatic 2-D disk representation (2-D bull’s eye plot) of the left ventricle (LV) and right ventricle (RV) obtained with a quasi-conformal mapping from the 3-D EAM meshes, that allows an analysis of cardiac resynchronization therapy (CRT) lead positioning, interpretation of global (total activation time), and local indices (local activation time (LAT), surrogates of conduction velocity, inter-ventricular, and transmural delays) that characterize changes in the electrical activation pattern. The second contribution is a set of indices derived from the electrical activation: speed maps, computed from LAT values, to study the electrical wave propagation, and histograms of isochrones to analyze regional electrical heterogeneities in the ventricles. We have applied the proposed methods to look for the underlying physiological mechanisms of left bundle branch block (LBBB) and CRT, with the goal of optimizing the therapy by improving CRT response. To better illustrate the benefits of the proposed tools, we created a set of synthetically generated and fully controlled activation patterns, where the proposed representation and indices were validated. Then, the proposed analysis tools are used to analyze EAM data from an experimental swine model of induced LBBB with an implanted CRT device. We have analyzed and compared the electrical activation patterns at baseline, LBBB, and CRT stages in four animals: two without any structural disease and two with an induced infarction. By relating the CRT lead location with electrical dyssynchrony, we evaluated current hypotheses about lead placement in CRT and showed that optimal pacing sites should target the RV lead close to the apex and the LV one distant from it.

A framework for combining a motion atlas with non-motion information to learn clinically useful biomarkers: Application to cardiac resynchronisation therapy response prediction

We present a framework for combining a cardiac motion atlas with non-motion data. The atlas represents cardiac cycle motion across a number of subjects in a common space based on rich motion descriptors capturing 3D displacement, velocity, strain and strain rate. The non-motion data are derived from a variety of sources such as imaging, electrocardiogram (ECG) and clinical reports. Once in the atlas space, we apply a novel supervised learning approach based on random projections and ensemble learning to learn the relationship between the atlas data and some desired clinical output. We apply our framework to the problem of predicting response to Cardiac Resynchronisation Therapy (CRT). Using a cohort of 34 patients selected for CRT using conventional criteria, results show that the combination of motion and non-motion data enables CRT response to be predicted with 91.2% accuracy (100% sensitivity and 62.5% specificity), which compares favourably with the current state-of-the-art in CRT response prediction.

Infarct Localization From Myocardial Deformation: Prediction and Uncertainty Quantification by Regression From a Low-Dimensional Space

Diagnosing and localizing myocardial infarct is crucial for early patient management and therapy planning. We propose a new method for predicting the location of myocardial infarct from local wall deformation, which has value for risk stratification from routine examinations such as (3D) echocardiography. The pipeline combines non-linear dimensionality reduction of deformation patterns and two multi-scale kernel regressions. Confidence in the diagnosis is assessed by a map of local uncertainties, which integrates plausible infarct locations generated from the space of reduced dimensionality. These concepts were tested on 500 synthetic cases generated from a realistic cardiac electromechanical model, and 108 pairs of 3D echocardiographic sequences and delayed-enhancement magnetic resonance images from real cases. Infarct prediction is made at a spatial resolution around 4 mm, more than 10 times smaller than the current diagnosis, made regionally. Our method is accurate, and significantly outperforms the clinically-used thresholding of the deformation patterns (on real data: sensitivity/specificity of 0.828/0.804, area under the curve: 0.909 versus 0.742 for the most predictive strain component). Uncertainty adds value to refine the diagnosis and eventually re-examine suspicious cases.

Principles of cardiovascular magnetic resonance feature tracking and echocardiographic speckle tracking for informed clinical use

Tissue tracking technology of routinely acquired cardiovascular magnetic resonance (CMR) cine acquisitions has increased the apparent ease and availability of non-invasive assessments of myocardial deformation in clinical research and practice. Its widespread availability thanks to the fact that this technology can in principle be applied on images that are part of every CMR or echocardiographic protocol. However, the two modalities are based on very different methods of image acquisition and reconstruction, each with their respective strengths and limitations. The image tracking methods applied are not necessarily directly comparable between the modalities, or with those based on dedicated CMR acquisitions for strain measurement such as tagging or displacement encoding. Here we describe the principles underlying the image tracking methods for CMR and echocardiography, and the translation of the resulting tracking estimates into parameters suited to describe myocardial mechanics. Technical limitations are presented with the objective of suggesting potential solutions that may allow informed and appropriate use in clinical applications.

Precision Imaging: more descriptive, predictive and integrative imaging

Medical image analysis has grown into a matured field challenged by progress made across all medical imaging technologies and more recent breakthroughs in biological imaging. The cross-fertilisation between medical image analysis, biomedical imaging physics and technology, and domain knowledge from medicine and biology has spurred a truly interdisciplinary effort that stretched outside the original boundaries of the disciplines that gave birth to this field and created stimulating and enriching synergies. Consideration on how the field has evolved and the experience of the work carried out over the last 15 years in our centre, has led us to envision a future emphasis of medical imaging in Precision Imaging. Precision Imaging is not a new discipline but rather a distinct emphasis in medical imaging borne at the cross-roads between, and unifying the efforts behind mechanistic and phenomenological model-based imaging. It captures three main directions in the effort to deal with the information deluge in imaging sciences, and thus achieve wisdom from data, information, and knowledge. Precision Imaging is finally characterised by being descriptive, predictive and integrative about the imaged object. This paper provides a brief and personal perspective on how the field has evolved, summarises and formalises our vision of Precision Imaging for Precision Medicine, and highlights some connections with past research and current trends in the field.

Regional manifold learning for disease classification

While manifold learning from images itself has become widely used in medical image analysis, the accuracy of existing implementations suffers from viewing each image as a single data point. To address this issue, we parcellate images into regions and then separately learn the manifold for each region. We use the regional manifolds as low-dimensional descriptors of high-dimensional morphological image features, which are then fed into a classifier to identify regions affected by disease. We produce a single ensemble decision for each scan by the weighted combination of these regional classification results. Each weight is determined by the regional accuracy of detecting the disease. When applied to cardiac magnetic resonance imaging of 50 normal controls and 50 patients with reconstructive surgery of Tetralogy of Fallot, our method achieves significantly better classification accuracy than approaches learning a single manifold across the entire image domain.

Hierarchical manifold learning for regional image analysis

We present a novel method of hierarchical manifold learning which aims to automatically discover regional properties of image datasets. While traditional manifold learning methods have become widely used for dimensionality reduction in medical imaging, they suffer from only being able to consider whole images as single data points. We extend conventional techniques by additionally examining local variations, in order to produce spatially-varying manifold embeddings that characterize a given dataset. This involves constructing manifolds in a hierarchy of image patches of increasing granularity, while ensuring consistency between hierarchy levels. We demonstrate the utility of our method in two very different settings: 1) to learn the regional correlations in motion within a sequence of time-resolved MR images of the thoracic cavity; 2) to find discriminative regions of 3-D brain MR images associated with neurodegenerative disease.

Understanding the mechanisms amenable to CRT response: from pre-operative multimodal image data to patient-specific computational models

This manuscript describes our recent developments towards better understanding of the mechanisms amenable to cardiac resynchronization therapy response. We report the results from a full multimodal dataset corresponding to eight patients from the euHeart project. The datasets include echocardiography, MRI and electrophysiological studies. We investigate two aspects. The first one focuses on pre-operative multimodal image data. From 2D echocardiography and 3D tagged MRI images, we compute atlas based dyssynchrony indices. We complement these indices with presence and extent of scar tissue and correlate them with CRT response. The second one focuses on computational models. We use pre-operative imaging to generate a patient-specific computational model. We show results of a fully automatic personalized electromechanical simulation. By case-per-case discussion of the results, we highlight the potential and key issues of this multimodal pipeline for the understanding of the mechanisms of CRT response and a better patient selection.

Atlas-based quantification of myocardial motion abnormalities: added-value for understanding the effect of cardiac resynchronization therapy

Statistical atlases may help improving the analysis of cardiac wall-motion abnormalities. This study aims at demonstrating the clinical value of such a method to better understand the effect of cardiac resynchronization therapy (CRT). We compared an atlas of normal septal motion built using apical four-chamber two-dimensional echocardiographic sequences from healthy volunteers with 88 patients undergoing CRT at baseline and at 12 months follow-up. Abnormal motion was quantified locally using a p value based on a statistical distance to normality. Reduction ≥15% in left ventricle end-systolic volume defined CRT response. Responders showed significantly higher reduction of abnormalities (p ≤ 0.001). Non-responders conserved abnormal septal motion at the end of the isovolumic contraction (IVC). A specific inward-outward motion of the septum during IVC predominated in responders and was corrected at follow-up. The method is of interest to characterize patterns of mechanical dyssynchrony and to study the link between their evolution and CRT response.