Multi-task prediction of infant cognitive scores from longitudinal incomplete neuroimaging data

Hypergraph-regularized multimodal learning by graph diffusion for imaging genetics based Alzheimer's Disease diagnosis

Recent studies show that multi-modal data fusion techniques combining information from diverse sources are helpful to diagnose and predict complex brain disorders. However, most existing diagnosis methods have only simply employed a feature combination strategy for multiple imaging and genetic data, ignoring the imaging phenotypes associated with the risk gene information. To this end, we present a hypergraph-regularized multimodal learning by graph diffusion (HMGD) for joint association learning and outcome prediction. Specifically, we first present a graph diffusion method for enhancing similarity measures among subjects given from multi-modality phenotypes, which fully uses multiple input similarity graphs and integrates them into a unified graph with valuable geometric structures among different imaging phenotypes. Then, we employ the unified graph to represent the high-order similarity relationships among subjects, and enforce a hypergraph-regularized term to incorporate both inter- and cross-modality information for selecting the imaging phenotypes associated with the risk single nucleotide polymorphism (SNP). Finally, a multi-kernel support vector machine (MK-SVM) is adopted to fuse such phenotypic features selected from different modalities for the final diagnosis and prediction. The proposed approach is experimentally explored on brain imaging genetic data of the Alzheimer's Disease Neuroimaging Initiative (ADNI) datasets. Relevant results present that the proposed approach is superior to several competing algorithms, and realizes strong associations and discovers significant consistent and robust ROIs across different imaging phenotypes associated with the genetic risk biomarkers to guide disease interpretation and prediction.

Machine Learning and Prediction in Fetal, Infant, and Toddler Neuroimaging: A Review and Primer

Predictive models in neuroimaging are increasingly designed with the intent to improve risk stratification and support interventional efforts in psychiatry. Many of these models have been developed in samples of children school-aged or older. Nevertheless, despite growing evidence that altered brain maturation during the fetal, infant, and toddler (FIT) period modulates risk for poor mental health outcomes in childhood, these models are rarely implemented in FIT samples. Applications of predictive modeling in children of these ages provide an opportunity to develop powerful tools for improved characterization of the neural mechanisms underlying development. To facilitate the broader use of predictive models in FIT neuroimaging, we present a brief primer and systematic review on the methods used in current predictive modeling FIT studies. Reflecting on current practices in more than 100 studies conducted over the past decade, we provide an overview of topics, modalities, and methods commonly used in the field and under-researched areas. We then outline ethical and future considerations for neuroimaging researchers interested in predicting health outcomes in early life, including researchers who may be relatively new to either advanced machine learning methods or using FIT data. Altogether, the last decade of FIT research in machine learning has provided a foundation for accelerating the prediction of early-life trajectories across the full spectrum of illness and health.

Multi-Modal Imaging Genetics Data Fusion via a Hypergraph-Based Manifold Regularization: Application to Schizophrenia Study

Recent studies show that multi-modal data fusion techniques combine information from diverse sources for comprehensive diagnosis and prognosis of complex brain disorder, often resulting in improved accuracy compared to single-modality approaches. However, many existing data fusion methods extract features from homogeneous networs, ignoring heterogeneous structural information among multiple modalities. To this end, we propose a Hypergraph-based Multi-modal data Fusion algorithm, namely HMF. Specifically, we first generate a hypergraph similarity matrix to represent the high-order relationships among subjects, and then enforce the regularization term based upon both the inter- and intra-modality relationships of the subjects. Finally, we apply HMF to integrate imaging and genetics datasets. Validation of the proposed method is performed on both synthetic data and real samples from schizophrenia study. Results show that our algorithm outperforms several competing methods, and reveals significant interactions among risk genes, environmental factors and abnormal brain regions.

Functional brain activity is highly associated with cortical myelination in neonates

Functional organization of the human cerebral cortex is highly constrained by underlying brain structures, but how functional activity is associated with different brain structures during development is not clear, especially at the neonatal stage. Since long-range functional connectivity is far from mature in the dynamically developing neonatal brain, it is of great scientific significance to investigate the relationship between different structural and functional features at the local level. To this end, for the first time, correlation and regression analyses were performed to examine the relationship between cortical morphology, cortical myelination, age, and local brain functional activity, as well as functional connectivity strength using high-resolution structural and resting-state functional MRI data of 177 neonates (29-44 postmenopausal weeks, 98 male and 79 female) from both static and dynamic perspectives. We found that cortical myelination was most strongly associated with local brain functional activity across the cerebral cortex than other cortical structural features while controlling the age effect. These findings suggest the crucial role of cortical myelination in local brain functional development at birth, providing valuable insights into the fundamental biological basis of functional activity at this early developmental stage.

Path Signature Neural Network of Cortical Features for Prediction of Infant Cognitive Scores

Studies have shown that there is a tight connection between cognition skills and brain morphology during infancy. Nonetheless, it is still a great challenge to predict individual cognitive scores using their brain morphological features, considering issues like the excessive feature dimension, small sample size and missing data. Due to the limited data, a compact but expressive feature set is desirable as it can reduce the dimension and avoid the potential overfitting issue. Therefore, we pioneer the path signature method to further explore the essential hidden dynamic patterns of longitudinal cortical features. To form a hierarchical and more informative temporal representation, in this work, a novel cortical feature based path signature neural network (CF-PSNet) is proposed with stacked differentiable temporal path signature layers for prediction of individual cognitive scores. By introducing the existence embedding in path generation, we can improve the robustness against the missing data. Benefiting from the global temporal receptive field of CF-PSNet, characteristics consisted in the existing data can be fully leveraged. Further, as there is no need for the whole brain to work for a certain cognitive ability, a top K selection module is used to select the most influential brain regions, decreasing the model size and the risk of overfitting. Extensive experiments are conducted on an in-house longitudinal infant dataset within 9 time points. By comparing with several recent algorithms, we illustrate the state-of-the-art performance of our CF-PSNet (i.e., root mean square error of 0.027 with the time latency of 518 milliseconds for each sample).

Multiview Diffusion Map Improves Prediction of Fluid Intelligence With Two Paradigms of fMRI Analysis

OBJECTIVE

To understand the association between brain networks and behaviors of an individual, most studies build predictive models based on functional connectivity (FC) from a single dataset with linear analysis techniques. Such approaches may fail to capture the nonlinear structure of brain networks and neglect the complementary information contained in FC networks (FCNs) from multiple datasets. To address this challenging issue, we use multiview dimensionality reduction to extract a coherent low-dimensional representation of the FCNs from resting-state and emotion identification task-based functional magnetic resonance imaging (fMRI) datasets.

METHODS

We propose a scheme based on multiview diffusion map to extract intrinsic features while preserving the underlying geometric structure of high dimensional datasets. This method is robust to noise and small variations in the data.

RESULTS

After validation on the Philadelphia Neurodevelopmental Cohort data, the predictive model built from both resting-state and emotion identification task-based fMRI datasets outperforms the one using each individual fMRI dataset. In addition, the proposed model achieves better prediction performance than principal component analysis (PCA) and three other competing data fusion methods.

CONCLUSION

Our framework for combing multiple FCNs in one predictive model exhibits improved prediction performance.

SIGNIFICANCE

To our knowledge, we demonstrate a first application of multiview diffusion map to successfully fuse different types of fMRI data for predicting fluid intelligence (gF).

Precision Medicine in Neonates: A Tailored Approach to Neonatal Brain Injury

Despite advances in neonatal care to prevent neonatal brain injury and neurodevelopmental impairment, predicting long-term outcome in neonates at risk for brain injury remains difficult. Early prognosis is currently based on cranial ultrasound (CUS), MRI, EEG, NIRS, and/or general movements assessed at specific ages, and predicting outcome in an individual (precision medicine) is not yet possible. New algorithms based on large databases and machine learning applied to clinical, neuromonitoring, and neuroimaging data and genetic analysis and assays measuring multiple biomarkers (omics) can fulfill the needs of modern neonatology. A synergy of all these techniques and the use of automatic quantitative analysis might give clinicians the possibility to provide patient-targeted decision-making for individualized diagnosis, therapy, and outcome prediction. This review will first focus on common neonatal neurological diseases, associated risk factors, and most common treatments. After that, we will discuss how precision medicine and machine learning (ML) approaches could change the future of prediction and prognosis in this field.

Ensemble Manifold Regularized Multi-Modal Graph Convolutional Network for Cognitive Ability Prediction

OBJECTIVE

Multi-modal functional magnetic resonance imaging (fMRI) can be used to make predictions about individual behavioral and cognitive traits based on brain connectivity networks.

METHODS

To take advantage of complementary information from multi-modal fMRI, we propose an interpretable multi-modal graph convolutional network (MGCN) model, incorporating both fMRI time series and functional connectivity (FC) between each pair of brain regions. Specifically, our model learns a graph embedding from individual brain networks derived from multi-modal data. A manifold-based regularization term is enforced to consider the relationships of subjects both within and between modalities. Furthermore, we propose the gradient-weighted regression activation mapping (Grad-RAM) and the edge mask learning to interpret the model, which is then used to identify significant cognition-related biomarkers.

RESULTS

We validate our MGCN model on the Philadelphia Neurodevelopmental Cohort to predict individual wide range achievement test (WRAT) score. Our model obtains superior predictive performance over GCN with a single modality and other competing approaches. The identified biomarkers are cross-validated from different approaches.

CONCLUSION AND SIGNIFICANCE

This paper develops a new interpretable graph deep learning framework for cognition prediction, with the potential to overcome the limitations of several current data-fusion models. The results demonstrate the power of MGCN in analyzing multi-modal fMRI and discovering significant biomarkers for human brain studies.

Infant Cognitive Scores Prediction with Multi-stream Attention-Based Temporal Path Signature Features

There is stunning rapid development of human brains in the first year of life. Some studies have revealed the tight connection between cognition skills and cortical morphology in this period. Nonetheless, it is still a great challenge to predict cognitive scores using brain morphological features, given issues like small sample size and missing data in longitudinal studies. In this work, for the first time, we introduce the path signature method to explore hidden analytical and geometric properties of longitudinal cortical morphology features. A novel BrainPSNet is proposed with a differentiable temporal path signature layer to produce informative representations of different time points and various temporal granules. Further, a two-stream neural network is included to combine groups of raw features and path signature features for predicting the cognitive score. More importantly, considering different influences of each brain region on the cognitive function, we design a learning-based attention mask generator to automatically weight regions correspondingly. Experiments are conducted on an in-house longitudinal dataset. By comparing with several recent algorithms, the proposed method achieves the state-of-the-art performance. The relationship between morphological features and cognitive abilities is also analyzed.

Joint Multi-Modal Longitudinal Regression and Classification for Alzheimer's Disease Prediction

Alzheimer's disease (AD) is a serious neurodegenerative condition that affects millions of individuals across the world. As the average age of individuals in the United States and the world increases, the prevalence of AD will continue to grow. To address this public health problem, the research community has developed computational approaches to sift through various aspects of clinical data and uncover their insights, among which one of the most challenging problem is to determine the biological mechanisms that cause AD to develop. To study this problem, in this paper we present a novel Joint Multi-Modal Longitudinal Regression and Classification method and show how it can be used to identify the cognitive status of the participants in the Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort and the underlying biological mechanisms. By intelligently combining clinical data of various modalities (i.e., genetic information and brain scans) using a variety of regularizations that can identify AD-relevant biomarkers, we perform the regression and classification tasks simultaneously. Because the proposed objective is a non-smooth optimization problem that is difficult to solve in general, we derive an efficient iterative algorithm and rigorously prove its convergence. To validate our new method in predicting the cognitive scores of patients and their clinical diagnosis, we conduct comprehensive experiments on the ADNI cohort. Our promising results demonstrate the benefits and flexibility of the proposed method. We anticipate that our new method is of interest to clinical communities beyond AD research and have open-sourced the code of our method online.11 The code package for the proposed Joint Multi-Modal Longitudinal Regression and Classification model have been made publicly available online at https://github.com/minds-mines/jmmlrc.

Partial Least Squares Regression Performs Well in MRI-Based Individualized Estimations

Estimation of individuals' cognitive, behavioral and demographic (CBD) variables based on MRI has attracted much research interest in the past decade, and effective machine learning techniques are of great importance for these estimations. Partial least squares regression (PLSR) is an attractive machine learning technique that can accommodate both single- and multi-label learning in a simple framework, while its potential for MRI-based estimations of CBD variables remains to be explored. In this study, we systemically investigated the performance of PLSR in MRI-based estimations of individuals' CBD variables, especially its performance in simultaneous estimation of multiple CBD variables (multi-label learning). We performed the study on the dataset included in the HCP S1200 release. Resting state functional connections (RSFCs) were used as features, and a total of 10 CBD variables (e.g., age, gender, grip strength, and picture vocabulary) were estimated. The results showed that PLSR performed well in both single- and multi-label learning. In fact, the present estimations were better than those reported in literatures, as indicated by stronger correlations between the estimated and actual CBD variables, as well as high gender classification accuracy (97.8% in this study). Moreover, the RSFCs that contributed to the estimations exhibited strong correlations with the CBD variable estimated, that is, PLSR algorithm automatically selected the RSFCs closely related to one CBD variable to establish predictive models for the variable. Besides, the estimation accuracies based on RSFCs among 100, 200, and 300 regions of interest (ROIs) were higher than those based on RSFCs among 15, 25, and 50 ROIs; the estimation accuracies based on RSFCs evaluated using partial correlation were higher than those based on RSFCs evaluated using full correlation. In addition to the aforementioned virtues, PLSR is efficient in model training and testing, and it is simple and easy to use. Therefore, PLSR can be a favorable choice for future MRI-based estimations of CBD variables.

A Manifold Regularized Multi-Task Learning Model for IQ Prediction From Two fMRI Paradigms

OBJECTIVE

Multi-modal brain functional connectivity (FC) data have shown great potential for providing insights into individual variations in behavioral and cognitive traits. The joint learning of multi-modal data can utilize intrinsic association, and thus can boost learning performance. Although several multi-task based learning models have already been proposed by viewing feature learning on each modality as one task, most of them ignore the structural information inherent across the modalities, which may play an important role in extracting discriminative features.

METHODS

In this paper, we propose a new manifold regularized multi-task learning model by simultaneously considering between-subject and between-modality relationships. Specifically, the l_2,1-norm (i.e., group-sparsity) regularizer is enforced to jointly select a few common features across different modalities. A novelly designed manifold regularizer is further imposed as a crucial underpinning to preserve the structural information both within and between modalities. Such designed regularizers will make our model more adaptive to realistic neuroimaging data, which are usually of small sample size but high dimensional features.

RESULTS

Our model is validated on the Philadelphia Neurodevelopmental Cohort dataset, where our modalities are regarded as two types of functional MRI (fMRI) data collected under two paradigms. We conduct experimental studies on fMRI-based FC network data in two task conditions for intelligence quotient (IQ) prediction. The results show that our proposed model can not only achieve improved prediction performance, but also yield a set of IQ-relevant biomarkers.

CONCLUSION AND SIGNIFICANCE

This paper develops a new multi-task learning model, enabling the discovery of significant biomarkers that may account for a proportion of the variance in human intelligence.

Infant Brain Development Prediction With Latent Partial Multi-View Representation Learning

The early postnatal period witnesses rapid and dynamic brain development. However, the relationship between brain anatomical structure and cognitive ability is still unknown. Currently, there is no explicit model to characterize this relationship in the literature. In this paper, we explore this relationship by investigating the mapping between morphological features of the cerebral cortex and cognitive scores. To this end, we introduce a multi-view multi-task learning approach to intuitively explore complementary information from different time-points and handle the missing data issue in longitudinal studies simultaneously. Accordingly, we establish a novel model, latent partial multi-view representation learning. Our approach regards data from different time-points as different views and constructs a latent representation to capture the complementary information from incomplete time-points. The latent representation explores the complementarity across different time-points and improves the accuracy of prediction. The minimization problem is solved by the alternating direction method of multipliers. Experimental results on both synthetic and real data validate the effectiveness of our proposed algorithm.