Editorial: Predicting Chronological Age From Structural Neuroimaging: The Predictive Analytics Competition 2019

Exploring the potential of representation and transfer learning for anatomical neuroimaging: Application to psychiatry

The perspective of personalized medicine for brain disorders requires efficient learning models for anatomical neuroimaging-based prediction of clinical conditions. There is now a consensus on the benefit of deep learning (DL) in addressing many medical imaging tasks, such as image segmentation. However, for single-subject prediction problems, recent studies yielded contradictory results when comparing DL with Standard Machine Learning (SML) on top of classical feature extraction. Most existing comparative studies were limited in predicting phenotypes of little clinical interest, such as sex and age, and using a single dataset. Moreover, they conducted a limited analysis of the employed image pre-processing and feature selection strategies. This paper extensively compares DL and SML prediction capacity on five multi-site problems, including three increasingly complex clinical applications in psychiatry namely schizophrenia, bipolar disorder, and Autism Spectrum Disorder (ASD) diagnosis. To compensate for the relative scarcity of neuroimaging data on these clinical datasets, we also evaluate three pre-training strategies for transfer learning from brain imaging of the general healthy population: self-supervised learning, generative modeling and supervised learning with age. Overall, we find similar performance between randomly initialized DL and SML for the three clinical tasks and a similar scaling trend for sex prediction. This was replicated on an external dataset. We also show highly correlated discriminative brain regions between DL and linear ML models in all problems. Nonetheless, we demonstrate that self-supervised pre-training on large-scale healthy population imaging datasets (N≈10k), along with Deep Ensemble, allows DL to learn robust and transferable representations to smaller-scale clinical datasets (N≤1k). It largely outperforms SML on 2 out of 3 clinical tasks both in internal and external test sets. These findings suggest that the improvement of DL over SML in anatomical neuroimaging mainly comes from its capacity to learn meaningful and useful abstract representations of the brain anatomy, and it sheds light on the potential of transfer learning for personalized medicine in psychiatry.

Predicting 'Brainage' in late childhood to adolescence (6-17yrs) using structural MRI, morphometric similarity, and machine learning

Brain development is regularly studied using structural MRI. Recently, studies have used a combination of statistical learning and large-scale imaging databases of healthy children to predict an individual's age from structural MRI. This data-driven, predicted 'Brainage' typically differs from the subjects chronological age, with this difference a potential measure of individual difference. Few studies have leveraged higher-order or connectomic representations of structural MRI data for this Brainage approach. We leveraged morphometric similarity as a network-level approach to structural MRI to generate predictive models of age. We benchmarked these novel Brainage approaches using morphometric similarity against more typical, single feature (i.e., cortical thickness) approaches. We showed that these novel methods did not outperform cortical thickness or cortical volume measures. All models were significantly biased by age, but robust to motion confounds. The main results show that, whilst morphometric similarity mapping may be a novel way to leverage additional information from a T1-weighted structural MRI beyond individual features, in the context of a Brainage framework, morphometric similarity does not provide more accurate predictions of age. Morphometric similarity as a network-level approach to structural MRI may be poorly positioned to study individual differences in brain development in healthy participants in this way.

Artificial Intelligence in Neuroradiology: A Review of Current Topics and Competition Challenges

There is an expanding body of literature that describes the application of deep learning and other machine learning and artificial intelligence methods with potential relevance to neuroradiology practice. In this article, we performed a literature review to identify recent developments on the topics of artificial intelligence in neuroradiology, with particular emphasis on large datasets and large-scale algorithm assessments, such as those used in imaging AI competition challenges. Numerous applications relevant to ischemic stroke, intracranial hemorrhage, brain tumors, demyelinating disease, and neurodegenerative/neurocognitive disorders were discussed. The potential applications of these methods to spinal fractures, scoliosis grading, head and neck oncology, and vascular imaging were also reviewed. The AI applications examined perform a variety of tasks, including localization, segmentation, longitudinal monitoring, diagnostic classification, and prognostication. While research on this topic is ongoing, several applications have been cleared for clinical use and have the potential to augment the accuracy or efficiency of neuroradiologists.

Dynamic network properties of the superior temporal gyrus mediate the impact of brain age gap on chronic aphasia severity

Brain structure deteriorates with aging and predisposes an individual to more severe language impairments (aphasia) after a stroke. However, the underlying mechanisms of this relation are not well understood. Here we use an approach to model brain network properties outside the stroke lesion, network controllability, to investigate relations among individualized structural brain connections, brain age, and aphasia severity in 93 participants with chronic post-stroke aphasia. Controlling for the stroke lesion size, we observe that lower average controllability of the posterior superior temporal gyrus (STG) mediates the relation between advanced brain aging and aphasia severity. Lower controllability of the left posterior STG signifies that activity in the left posterior STG is less likely to yield a response in other brain regions due to the topological properties of the structural brain networks. These results indicate that advanced brain aging among individuals with post-stroke aphasia is associated with disruption of dynamic properties of a critical language-related area, the STG, which contributes to worse aphasic symptoms. Because brain aging is variable among individuals with aphasia, our results provide further insight into the mechanisms underlying the variance in clinical trajectories in post-stroke aphasia.

Predicting 'Brainage' in the Developmental Period using Structural MRI, Morphometric Similarity, and Machine Learning

Brain development is regularly studied using structural MRI. Recently, studies have used a combination of statistical learning and large-scale imaging databases of healthy-children to predict an individual's age from structural MRI. This data-driven, 'brainage' typically differs from the subjects chronological age, with this difference a potential measure of individual difference. Few studies have leveraged higher-order or connectomic representations of structural MRI data for this brainage approach. We leveraged morphometric similarity as a network-level approach to structural MRI to generate predictive models of age. We benchmarked these novel brain-age approaches using morphometric similarity against more typical, single feature (i.e. cortical thickness) approaches. We showed that these novel methods did not outperform cortical thickness or cortical volume measures. All models were significantly biased by age, but robust to motion confounds. The main results show that, whilst morphometric similarity mapping may be a novel way to leverage additional information from a T1-weighted structural MRI beyond individual features, in the context of a brain-age framework, morphometric similarity does not explain more variance than individual structural features. Morphometric similarity as a network-level approach to structural MRI may be poorly positioned to study individual differences in brain development in healthy individuals.

Anatomically interpretable deep learning of brain age captures domain-specific cognitive impairment

The gap between chronological age (CA) and biological brain age, as estimated from magnetic resonance images (MRIs), reflects how individual patterns of neuroanatomic aging deviate from their typical trajectories. MRI-derived brain age (BA) estimates are often obtained using deep learning models that may perform relatively poorly on new data or that lack neuroanatomic interpretability. This study introduces a convolutional neural network (CNN) to estimate BA after training on the MRIs of 4,681 cognitively normal (CN) participants and testing on 1,170 CN participants from an independent sample. BA estimation errors are notably lower than those of previous studies. At both individual and cohort levels, the CNN provides detailed anatomic maps of brain aging patterns that reveal sex dimorphisms and neurocognitive trajectories in adults with mild cognitive impairment (MCI, N = 351) and Alzheimer's disease (AD, N = 359). In individuals with MCI (54% of whom were diagnosed with dementia within 10.9 y from MRI acquisition), BA is significantly better than CA in capturing dementia symptom severity, functional disability, and executive function. Profiles of sex dimorphism and lateralization in brain aging also map onto patterns of neuroanatomic change that reflect cognitive decline. Significant associations between BA and neurocognitive measures suggest that the proposed framework can map, systematically, the relationship between aging-related neuroanatomy changes in CN individuals and in participants with MCI or AD. Early identification of such neuroanatomy changes can help to screen individuals according to their AD risk.

Structural and Functional MRI Data Differentially Predict Chronological Age and Behavioral Memory Performance

Human cognitive abilities decline with increasing chronological age, with decreased explicit memory performance being most strongly affected. However, some older adults show "successful aging," that is, relatively preserved cognitive ability in old age. One explanation for this could be higher brain-structural integrity in these individuals. Alternatively, the brain might recruit existing resources more efficiently or employ compensatory cognitive strategies. Here, we approached this question by testing multiple candidate variables from structural and functional neuroimaging for their ability to predict chronological age and memory performance, respectively. Prediction was performed using support vector machine (SVM) classification and regression across and within two samples of young (N = 106) and older (N = 153) adults. The candidate variables were (1) behavioral response frequencies in an episodic memory test; (2) recently described functional magnetic resonance imaging (fMRI) scores reflecting preservation of functional memory networks; (3) whole-brain fMRI contrasts for novelty processing and subsequent memory; (4) resting-state fMRI maps quantifying voxel-wise signal fluctuation; and (5) gray matter volume estimated from structural MRIs. While age group could be reliably decoded from all variables, chronological age within young and older subjects was best predicted from gray matter volume. In contrast, memory performance was best predicted from task-based fMRI contrasts and particularly single-value fMRI scores, whereas gray matter volume has no predictive power with respect to memory performance in healthy adults. Our results suggest that superior memory performance in healthy older adults is better explained by efficient recruitment of memory networks rather than by preserved brain structure.

OpenBHB: a Large-Scale Multi-Site Brain MRI Data-set for Age Prediction and Debiasing

Prediction of chronological age from neuroimaging in the healthy population is an important issue because the deviations from normal brain age may highlight abnormal trajectories towards brain disorders. As a first step, ML models have emerged to predict chronological age from brain MRI, as a proxy measure of biological age. However, there is currently no consensus w.r.t which Machine Learning (ML) model is best suited for this task, largely because of a lack of public benchmark. Furthermore, new large emerging population neuroimaging datasets are often biased by the acquisition center images are coming from. This bias heavily deteriorates models generalization capacities, especially for Deep Learning (DL) algorithms that are known to overfit rapidly on the simplest features (known as simplicity bias). Here we propose a new public benchmarking resource, namely Open Big Healthy Brains (OpenBHB), along with a challenge for both brain age prediction and site-effect removal through a representation learning framework. OpenBHB is large-scale, gathering >5K 3D T1 brain MRI from Healthy Controls (HC) and highly multi-sites, aggregating >60 centers worldwide and 10 studies. OpenBHB is expected to grow both in terms of available modalities and number of subjects. All OpenBHB datasets are uniformly preprocessed, including quality check, with container technologies that consist in: 3D Voxel-Based Morphometry maps (VBM from CAT12), quasi-raw (simple linear alignment of images), and Surface-Based Morphometry indices (SBM, from FreeSurfer). The OpenBHB challenge is permanent and we provide all tools, materials and tutorials for participants to easily submit and benchmark their model against each other on a public leaderboard.

Predicting brain age from functional connectivity in symptomatic and preclinical Alzheimer disease

"Brain-predicted age" quantifies apparent brain age compared to normative neuroimaging trajectories. Advanced brain-predicted age has been well established in symptomatic Alzheimer disease (AD), but is underexplored in preclinical AD. Prior brain-predicted age studies have typically used structural MRI, but resting-state functional connectivity (FC) remains underexplored. Our model predicted age from FC in 391 cognitively normal, amyloid-negative controls (ages 18-89). We applied the trained model to 145 amyloid-negative, 151 preclinical AD, and 156 symptomatic AD participants to test group differences. The model accurately predicted age in the training set. FC-predicted brain age gaps (FC-BAG) were significantly older in symptomatic AD and significantly younger in preclinical AD compared to controls. There was minimal correspondence between networks predictive of age and AD. Elevated FC-BAG may reflect network disruption during symptomatic AD. Reduced FC-BAG in preclinical AD was opposite to the expected direction, and may reflect a biphasic response to preclinical AD pathology or may be driven by inconsistency between age-related vs. AD-related networks. Overall, FC-predicted brain age may be a sensitive AD biomarker.

Recommendations for machine learning benchmarks in neuroimaging

The field of neuroimaging has embraced methods from machine learning in a variety of ways. Although an increasing number of initiatives have published open-access neuroimaging datasets, specifically designed benchmarks are rare in the field. In this article, we first describe how benchmarks in computer science and biomedical imaging have fostered methodological progress in machine learning. Second, we identify the special characteristics of neuroimaging data and outline what researchers have to ensure when establishing a neuroimaging benchmark, how datasets should be composed and how adequate evaluation criteria can be chosen. Based on lessons learned from machine learning benchmarks, we argue for an extended evaluation procedure that, next to applying suitable performance metrics, focuses on scientifically relevant aspects such as explainability, robustness, uncertainty, computational efficiency and code quality. Lastly, we envision a collaborative neuroimaging benchmarking platform that combines the discussed aspects in a collaborative and agile framework, allowing researchers across disciplines to work together on the key predictive problems of the field of neuroimaging and psychiatry.

Ten years of image analysis and machine learning competitions in dementia

Machine learning methods exploiting multi-parametric biomarkers, especially based on neuroimaging, have huge potential to improve early diagnosis of dementia and to predict which individuals are at-risk of developing dementia. To benchmark algorithms in the field of machine learning and neuroimaging in dementia and assess their potential for use in clinical practice and clinical trials, seven grand challenges have been organized in the last decade: MIRIAD (2012), Alzheimer's Disease Big Data DREAM (2014), CADDementia (2014), Machine Learning Challenge (2014), MCI Neuroimaging (2017), TADPOLE (2017), and the Predictive Analytics Competition (2019). Based on two challenge evaluation frameworks, we analyzed how these grand challenges are complementing each other regarding research questions, datasets, validation approaches, results and impact. The seven grand challenges addressed questions related to screening, clinical status estimation, prediction and monitoring in (pre-clinical) dementia. There was little overlap in clinical questions, tasks and performance metrics. Whereas this aids providing insight on a broad range of questions, it also limits the validation of results across challenges. The validation process itself was mostly comparable between challenges, using similar methods for ensuring objective comparison, uncertainty estimation and statistical testing. In general, winning algorithms performed rigorous data pre-processing and combined a wide range of input features. Despite high state-of-the-art performances, most of the methods evaluated by the challenges are not clinically used. To increase impact, future challenges could pay more attention to statistical analysis of which factors (i.e., features, models) relate to higher performance, to clinical questions beyond Alzheimer's disease, and to using testing data beyond the Alzheimer's Disease Neuroimaging Initiative. Grand challenges would be an ideal venue for assessing the generalizability of algorithm performance to unseen data of other cohorts. Key for increasing impact in this way are larger testing data sizes, which could be reached by sharing algorithms rather than data to exploit data that cannot be shared. Given the potential and lessons learned in the past ten years, we are excited by the prospects of grand challenges in machine learning and neuroimaging for the next ten years and beyond.