A combined manifold learning analysis of shape and appearance to characterize neonatal brain development

Independent components of human brain morphology

Quantification of brain morphology has become an important cornerstone in understanding brain structure. Measures of cortical morphology such as thickness and surface area are frequently used to compare groups of subjects or characterise longitudinal changes. However, such measures are often treated as independent from each other. A recently described scaling law, derived from a statistical physics model of cortical folding, demonstrates that there is a tight covariance between three commonly used cortical morphology measures: cortical thickness, total surface area, and exposed surface area. We show that assuming the independence of cortical morphology measures can hide features and potentially lead to misinterpretations. Using the scaling law, we account for the covariance between cortical morphology measures and derive novel independent measures of cortical morphology. By applying these new measures, we show that new information can be gained; in our example we show that distinct morphological alterations underlie healthy ageing compared to temporal lobe epilepsy, even on the coarse level of a whole hemisphere. We thus provide a conceptual framework for characterising cortical morphology in a statistically valid and interpretable manner, based on theoretical reasoning about the shape of the cortex.

Visualizing Alzheimer's disease progression in low dimensional manifolds

While tomographic neuroimaging data is information rich, objective, and with high sensitivity in the study of brain diseases such as Alzheimer's disease (AD), its direct use in clinical practice and in regulated clinical trial (CT) still has many challenges. Taking CT as an example, unless the relevant policy and the perception of the primary outcome measures change, the need to construct univariate indices (out of the 3-D imaging data) to serve as CT's primary outcome measures will remain the focus of active research. More relevant to this current study, an overall global index that summarizes multiple complicated features from neuroimages should be developed in order to provide high diagnostic accuracy and sensitivity in tracking AD progression over time in clinical setting. Such index should also be practically intuitive and logically explainable to patients and their families. In this research, we propose a new visualization tool, derived from the manifold-based nonlinear dimension reduction of brain MRI features, to track AD progression over time. In specific, we investigate the locally linear embedding (LLE) method using a dataset from Alzheimer's Disease Neuroimaging Initiative (ADNI), which includes the longitudinal MRIs from 562 subjects. About 20% of them progressed to the next stage of dementia. Using only the baseline data of cognitively unimpaired (CU) and AD subjects, LLE reduces the feature dimension to two and a subject's AD progression path can be plotted in this low dimensional LLE feature space. In addition, the likelihood of being categorized to AD is indicated by color. This LLE map is a new data visualization tool that can assist in tracking AD progression over time.

Thought Chart: tracking the thought with manifold learning during emotion regulation

The Nash embedding theorem demonstrates that any compact manifold can be isometrically embedded in a Euclidean space. Assuming the complex brain states form a high-dimensional manifold in a topological space, we propose a manifold learning framework, termed Thought Chart, to reconstruct and visualize the manifold in a low-dimensional space. Furthermore, it serves as a data-driven approach to discover the underlying dynamics when the brain is engaged in a series of emotion and cognitive regulation tasks. EEG-based temporal dynamic functional connectomes are created based on 20 psychiatrically healthy participants’ EEG recordings during resting state and an emotion regulation task. Graph dissimilarity space embedding was applied to all the dynamic EEG connectomes. In order to visualize the learned manifold in a lower dimensional space, local neighborhood information is reconstructed via k-nearest neighbor-based nonlinear dimensionality reduction (NDR) and epsilon distance-based NDR. We showed that two neighborhood constructing approaches of NDR embed the manifold in a two-dimensional space, which we named Thought Chart. In Thought Chart, different task conditions represent distinct trajectories. Properties such as the distribution or average length in the 2-D space may serve as useful parameters to explore the underlying cognitive load and emotion processing during the complex task. In sum, this framework is a novel data-driven approach to the learning and visualization of underlying neurophysiological dynamics of complex functional brain data.

Modelling neuroanatomical variation during childhood and adolescence with neighbourhood-preserving embedding

Brain development is a dynamic process with tissue-specific alterations that reflect complex and ongoing biological processes taking place during childhood and adolescence. Accurate identification and modelling of these anatomical processes in vivo with MRI may provide clinically useful imaging markers of individual variability in development. In this study, we use manifold learning to build a model of age- and sex-related anatomical variation using multiple magnetic resonance imaging metrics. Using publicly available data from a large paediatric cohort (n = 768), we apply a multi-metric machine learning approach combining measures of tissue volume, cortical area and cortical thickness into a low-dimensional data representation. We find that neuroanatomical variation due to age and sex can be captured by two orthogonal patterns of brain development and we use this model to simultaneously predict age with a mean error of 1.5-1.6 years and sex with an accuracy of 81%. We validate this model in an independent developmental cohort. We present a framework for modelling anatomical development during childhood using manifold embedding. This model accurately predicts age and sex based on image-derived markers of cerebral morphology and generalises well to independent populations.

Learning and combining image neighborhoods using random forests for neonatal brain disease classification

It is challenging to characterize and classify normal and abnormal brain development during early childhood. To reduce the complexity of heterogeneous data population, manifold learning techniques are increasingly applied, which find a low-dimensional representation of the data, while preserving all relevant information. The neighborhood definition used for constructing manifold representations of the population is crucial for preserving the similarity structure and it is highly application dependent. The recently proposed neighborhood approximation forests learn a neighborhood structure in a dataset based on a user-defined distance. We propose a framework to learn multiple pairwise distances in a population of brain images and to combine them in an unsupervised manner optimally in a manifold learning step. Unlike other methods that only use a univariate distance measure, our method allows for a natural combination of multiple distances from heterogeneous sources. As a result, it yields a representation of the population that preserves the multiple distances. Furthermore, our method also selects the most predictive features associated with the distances. We evaluate our method in neonatal magnetic resonance images of three groups (term controls, patients affected by intrauterine growth restriction and mild isolated ventriculomegaly). We show that combining multiple distances related to the condition improves the overall characterization and classification of the three clinical groups compared to the use of single distances and classical unsupervised manifold learning.

Computing group cardinality constraint solutions for logistic regression problems

We derive an algorithm to directly solve logistic regression based on cardinality constraint, group sparsity and use it to classify intra-subject MRI sequences (e.g. cine MRIs) of healthy from diseased subjects. Group cardinality constraint models are often applied to medical images in order to avoid overfitting of the classifier to the training data. Solutions within these models are generally determined by relaxing the cardinality constraint to a weighted feature selection scheme. However, these solutions relate to the original sparse problem only under specific assumptions, which generally do not hold for medical image applications. In addition, inferring clinical meaning from features weighted by a classifier is an ongoing topic of discussion. Avoiding weighing features, we propose to directly solve the group cardinality constraint logistic regression problem by generalizing the Penalty Decomposition method. To do so, we assume that an intra-subject series of images represents repeated samples of the same disease patterns. We model this assumption by combining series of measurements created by a feature across time into a single group. Our algorithm then derives a solution within that model by decoupling the minimization of the logistic regression function from enforcing the group sparsity constraint. The minimum to the smooth and convex logistic regression problem is determined via gradient descent while we derive a closed form solution for finding a sparse approximation of that minimum. We apply our method to cine MRI of 38 healthy controls and 44 adult patients that received reconstructive surgery of Tetralogy of Fallot (TOF) during infancy. Our method correctly identifies regions impacted by TOF and generally obtains statistically significant higher classification accuracy than alternative solutions to this model, i.e., ones relaxing group cardinality constraints.

Multi-template analysis of human perirhinal cortex in brain MRI: Explicitly accounting for anatomical variability

RATIONAL

The human perirhinal cortex (PRC) plays critical roles in episodic and semantic memory and visual perception. The PRC consists of Brodmann areas 35 and 36 (BA35, BA36). In Alzheimer's disease (AD), BA35 is the first cortical site affected by neurofibrillary tangle pathology, which is closely linked to neural injury in AD. Large anatomical variability, manifested in the form of different cortical folding and branching patterns, makes it difficult to segment the PRC in MRI scans. Pathology studies have found that in ~97% of specimens, the PRC falls into one of three discrete anatomical variants. However, current methods for PRC segmentation and morphometry in MRI are based on single-template approaches, which may not be able to accurately model these discrete variants METHODS: A multi-template analysis pipeline that explicitly accounts for anatomical variability is used to automatically label the PRC and measure its thickness in T2-weighted MRI scans. The pipeline uses multi-atlas segmentation to automatically label medial temporal lobe cortices including entorhinal cortex, PRC and the parahippocampal cortex. Pairwise registration between label maps and clustering based on residual dissimilarity after registration are used to construct separate templates for the anatomical variants of the PRC. An optimal path of deformations linking these templates is used to establish correspondences between all the subjects. Experimental evaluation focuses on the ability of single-template and multi-template analyses to detect differences in the thickness of medial temporal lobe cortices between patients with amnestic mild cognitive impairment (aMCI, n=41) and age-matched controls (n=44).

RESULTS

The proposed technique is able to generate templates that recover the three dominant discrete variants of PRC and establish more meaningful correspondences between subjects than a single-template approach. The largest reduction in thickness associated with aMCI, in absolute terms, was found in left BA35 using both regional and summary thickness measures. Further, statistical maps of regional thickness difference between aMCI and controls revealed different patterns for the three anatomical variants.

A discriminative feature selection approach for shape analysis: Application to fetal brain cortical folding

The development of post-processing reconstruction techniques has opened new possibilities for the study of in-utero fetal brain MRI data. Recent cortical surface analysis have led to the computation of quantitative maps characterizing brain folding of the developing brain. In this paper, we describe a novel feature selection-based approach that is used to extract the most discriminative and sparse set of features of a given dataset. The proposed method is used to sparsely characterize cortical folding patterns of an in-utero fetal MR dataset, labeled with heterogeneous gestational age ranging from 26 weeks to 34 weeks. The proposed algorithm is validated on a synthetic dataset with both linear and non-linear dynamics, supporting its ability to capture deformation patterns across the dataset within only a few features. Results on the fetal brain dataset show that the temporal process of cortical folding related to brain maturation can be characterized by a very small set of points, located in anatomical regions changing across time. Quantitative measurements of growth against time are extracted from the set selected features to compare multiple brain regions (e.g. lobes and hemispheres) during the considered period of gestation.

Learning clinically useful information from images: Past, present and future

Over the last decade, research in medical imaging has made significant progress in addressing challenging tasks such as image registration and image segmentation. In particular, the use of model-based approaches has been key in numerous, successful advances in methodology. The advantage of model-based approaches is that they allow the incorporation of prior knowledge acting as a regularisation that favours plausible solutions over implausible ones. More recently, medical imaging has moved away from hand-crafted, and often explicitly designed models towards data-driven, implicit models that are constructed using machine learning techniques. This has led to major improvements in all stages of the medical imaging pipeline, from acquisition and reconstruction to analysis and interpretation. As more and more imaging data is becoming available, e.g., from large population studies, this trend is likely to continue and accelerate. At the same time new developments in machine learning, e.g., deep learning, as well as significant improvements in computing power, e.g., parallelisation on graphics hardware, offer new potential for data-driven, semantic and intelligent medical imaging. This article outlines the work of the BioMedIA group in this area and highlights some of the challenges and opportunities for future work.

Multivariate Analyses Applied to Healthy Neurodevelopment in Fetal, Neonatal, and Pediatric MRI

Multivariate analysis (MVA) is a class of statistical and pattern recognition techniques that involve the processing of data that contains multiple measurements per sample. MVA can be used to address a wide variety of neurological medical imaging related challenges including the evaluation of healthy brain development, the automated analysis of brain tissues and structures through image segmentation, evaluating the effects of genetic and environmental factors on brain development, evaluating sensory stimulation's relationship with functional brain activity and much more. Compared to adult imaging, pediatric, neonatal and fetal imaging have attracted less attention from MVA researchers, however, recent years have seen remarkable MVA research growth in pre-adult populations. This paper presents the results of a systematic review of the literature focusing on MVA applied to healthy subjects in fetal, neonatal and pediatric magnetic resonance imaging (MRI) of the brain. While the results of this review demonstrate considerable interest from the scientific community in applications of MVA technologies in brain MRI, the field is still young and significant research growth will continue into the future.

Learning-based prediction of gestational age from ultrasound images of the fetal brain

We propose an automated framework for predicting gestational age (GA) and neurodevelopmental maturation of a fetus based on 3D ultrasound (US) brain image appearance. Our method capitalizes on age-related sonographic image patterns in conjunction with clinical measurements to develop, for the first time, a predictive age model which improves on the GA-prediction potential of US images. The framework benefits from a manifold surface representation of the fetal head which delineates the inner skull boundary and serves as a common coordinate system based on cranial position. This allows for fast and efficient sampling of anatomically-corresponding brain regions to achieve like-for-like structural comparison of different developmental stages. We develop bespoke features which capture neurosonographic patterns in 3D images, and using a regression forest classifier, we characterize structural brain development both spatially and temporally to capture the natural variation existing in a healthy population (N=447) over an age range of active brain maturation (18-34weeks). On a routine clinical dataset (N=187) our age prediction results strongly correlate with true GA (r=0.98,accurate within±6.10days), confirming the link between maturational progression and neurosonographic activity observable across gestation. Our model also outperforms current clinical methods by ±4.57 days in the third trimester-a period complicated by biological variations in the fetal population. Through feature selection, the model successfully identified the most age-discriminating anatomies over this age range as being the Sylvian fissure, cingulate, and callosal sulci.

A Feature-Based Approach to Big Data Analysis of Medical Images

This paper proposes an inference method well-suited to large sets of medical images. The method is based upon a framework where distinctive 3D scale-invariant features are indexed efficiently to identify approximate nearest-neighbor (NN) feature matches-in O (log N) computational complexity in the number of images N. It thus scales well to large data sets, in contrast to methods based on pair-wise image registration or feature matching requiring O(N) complexity. Our theoretical contribution is a density estimator based on a generative model that generalizes kernel density estimation and K-nearest neighbor (KNN) methods.. The estimator can be used for on-the-fly queries, without requiring explicit parametric models or an off-line training phase. The method is validated on a large multi-site data set of 95,000,000 features extracted from 19,000 lung CT scans. Subject-level classification identifies all images of the same subjects across the entire data set despite deformation due to breathing state, including unintentional duplicate scans. State-of-the-art performance is achieved in predicting chronic pulmonary obstructive disorder (COPD) severity across the 5-category GOLD clinical rating, with an accuracy of 89% if both exact and one-off predictions are considered correct.

Visualization and unsupervised predictive clustering of high-dimensional multimodal neuroimaging data

BACKGROUND

Neuroimaging machine learning studies have largely utilized supervised algorithms - meaning they require both neuroimaging scan data and corresponding target variables (e.g. healthy vs. diseased) to be successfully 'trained' for a prediction task. Noticeably, this approach may not be optimal or possible when the global structure of the data is not well known and the researcher does not have an a priori model to fit the data.

NEW METHOD

We set out to investigate the utility of an unsupervised machine learning technique; t-distributed stochastic neighbour embedding (t-SNE) in identifying 'unseen' sample population patterns that may exist in high-dimensional neuroimaging data. Multimodal neuroimaging scans from 92 healthy subjects were pre-processed using atlas-based methods, integrated and input into the t-SNE algorithm. Patterns and clusters discovered by the algorithm were visualized using a 2D scatter plot and further analyzed using the K-means clustering algorithm.

COMPARISON WITH EXISTING METHODS

t-SNE was evaluated against classical principal component analysis.

CONCLUSION

Remarkably, based on unlabelled multimodal scan data, t-SNE separated study subjects into two very distinct clusters which corresponded to subjects' gender labels (cluster silhouette index value=0.79). The resulting clusters were used to develop an unsupervised minimum distance clustering model which identified 93.5% of subjects' gender. Notably, from a neuropsychiatric perspective this method may allow discovery of data-driven disease phenotypes or sub-types of treatment responders.

Regional manifold learning for disease classification

While manifold learning from images itself has become widely used in medical image analysis, the accuracy of existing implementations suffers from viewing each image as a single data point. To address this issue, we parcellate images into regions and then separately learn the manifold for each region. We use the regional manifolds as low-dimensional descriptors of high-dimensional morphological image features, which are then fed into a classifier to identify regions affected by disease. We produce a single ensemble decision for each scan by the weighted combination of these regional classification results. Each weight is determined by the regional accuracy of detecting the disease. When applied to cardiac magnetic resonance imaging of 50 normal controls and 50 patients with reconstructive surgery of Tetralogy of Fallot, our method achieves significantly better classification accuracy than approaches learning a single manifold across the entire image domain.

Hierarchical manifold learning for regional image analysis

We present a novel method of hierarchical manifold learning which aims to automatically discover regional properties of image datasets. While traditional manifold learning methods have become widely used for dimensionality reduction in medical imaging, they suffer from only being able to consider whole images as single data points. We extend conventional techniques by additionally examining local variations, in order to produce spatially-varying manifold embeddings that characterize a given dataset. This involves constructing manifolds in a hierarchy of image patches of increasing granularity, while ensuring consistency between hierarchy levels. We demonstrate the utility of our method in two very different settings: 1) to learn the regional correlations in motion within a sequence of time-resolved MR images of the thoracic cavity; 2) to find discriminative regions of 3-D brain MR images associated with neurodegenerative disease.

Modeling the variability in brain morphology and lesion distribution in multiple sclerosis by deep learning

Changes in brain morphology and white matter lesions are two hallmarks of multiple sclerosis (MS) pathology, but their variability beyond volumetrics is poorly characterized. To further our understanding of complex MS pathology, we aim to build a statistical model of brain images that can automatically discover spatial patterns of variability in brain morphology and lesion distribution. We propose building such a model using a deep belief network (DBN), a layered network whose parameters can be learned from training images. In contrast to other manifold learning algorithms, the DBN approach does not require a prebuilt proximity graph, which is particularly advantageous for modeling lesions, because their sparse and random nature makes defining a suitable distance measure between lesion images challenging. Our model consists of a morphology DBN, a lesion DBN, and a joint DBN that models concurring morphological and lesion patterns. Our results show that this model can automatically discover the classic patterns of MS pathology, as well as more subtle ones, and that the parameters computed have strong relationships to MS clinical scores.

A general framework for context-specific image segmentation using reinforcement learning

This paper presents an online reinforcement learning framework for medical image segmentation. The concept of context-specific segmentation is introduced such that the model is adaptive not only to a defined objective function but also to the user's intention and prior knowledge. Based on this concept, a general segmentation framework using reinforcement learning is proposed, which can assimilate specific user intention and behavior seamlessly in the background. The method is able to establish an implicit model for a large state-action space and generalizable to different image contents or segmentation requirements based on learning in situ. In order to demonstrate the practical value of the method, example applications of the technique to four different segmentation problems are presented. Detailed validation results have shown that the proposed framework is able to significantly reduce user interaction, while maintaining both segmentation accuracy and consistency.

COMBINING REGIONAL METRICS FOR DISEASE-RELATED BRAIN POPULATION ANALYSIS

In this paper, we present a new metric combining regional measurements to improve image based population studies that use manifold learning techniques. These studies currently rely on a single score over the whole brain image domain. Thus, they require large amount of training data to uncover spatially complex variation in the whole brain impacted by diseases. We reduce the impact of this issue by first computing pairwise measurements in local regions separately and then combining regional measurements into a single pairwise metric. We apply the new metric to learn the manifold of ADNI data and evaluate the resulting morphological representation by fitting multiple linear regression models to the mini-mental state examination (MMSE) score. The regression models show that the morphological representations from the proposed metric achieves higher estimation accuracy of MMSE score compared to those from the conventional global scores.

QUANTIFYING REGIONAL GROWTH PATTERNS THROUGH LONGITUDINAL ANALYSIS OF DISTANCES BETWEEN MULTIMODAL MR INTENSITY DISTRIBUTIONS

Quantitative analysis of early brain development through imaging is critical for identifying pathological development, which may in turn affect treatment procedures. We propose a framework for analyzing spatiotemporal patterns of brain maturation by quantifying intensity changes in longitudinal MR images. We use a measure of divergence between a pair of intensity distributions to study the changes that occur within specific regions, as well as between a pair of anatomical regions, over time. The change within a specific region is measured as the contrast between white matter and gray matter tissue belonging to that region. The change between a pair of regions is measured as the divergence between regional image appearances, summed over all tissue classes. We use kernel regression to integrate the temporal information across different subjects in a consistent manner. We applied our method on multimodal MRI data with T1-weighted (T1W) and T2-weighted (T2W) scans of each subject at the approximate ages of 6 months, 12 months, and 24 months. The results demonstrate that brain maturation begins at posterior regions and that frontal regions develop later, which matches previously published histological, qualitative and morphometric studies. Our multimodal analysis also confirms that T1W and T2W modalities capture different properties of the maturation process, a phenomena referred to as T2 time lag compared to T1. The proposed method has potential for analyzing regional growth patterns across different populations and for isolating specific critical maturation phases in different MR modalities.

STATISTICAL GROWTH MODELING OF LONGITUDINAL DT-MRI FOR REGIONAL CHARACTERIZATION OF EARLY BRAIN DEVELOPMENT

A population growth model that represents the growth trajectories of individual subjects is critical to study and understand neurodevelopment. This paper presents a framework for jointly estimating and modeling individual and population growth trajectories, and determining significant regional differences in growth pattern characteristics applied to longitudinal neuroimaging data. We use non-linear mixed effect modeling where temporal change is modeled by the Gompertz function. The Gompertz function uses intuitive parameters related to delay, rate of change, and expected asymptotic value; all descriptive measures which can answer clinical questions related to growth. Our proposed framework combines nonlinear modeling of individual trajectories, population analysis, and testing for regional differences. We apply this framework to the study of early maturation in white matter regions as measured with diffusion tensor imaging (DTI). Regional differences between anatomical regions of interest that are known to mature differently are analyzed and quantified. Experiments with image data from a large ongoing clinical study show that our framework provides descriptive, quantitative information on growth trajectories that can be directly interpreted by clinicians. To our knowledge, this is the first longitudinal analysis of growth functions to explain the trajectory of early brain maturation as it is represented in DTI.