Efficacy of MRI data harmonization in the age of machine learning: a multicenter study across 36 datasets

A data integration method for new advances in development cognitive neuroscience

Combining existing datasets to investigate key questions in developmental cognitive neuroscience brings exciting opportunities and unique challenges. However, many data pooling methods require identical or harmonized methodologies that are often not feasible. We propose Integrative Data Analysis (IDA) as a promising framework to advance developmental cognitive neuroscience with secondary data analysis. IDA serves to test hypotheses by combining data of the same construct from commensurate (but not identical) measures. To overcome idiosyncrasies of neuroimaging data, IDA explicitly evaluates if measures across studies assess the same construct. Moreover, IDA allows investigators to examine meaningful individual variability by de-confounding source-specific differences. To demonstrate IDA's potential, we explain foundational concepts, outline necessary steps, and apply IDA to volumetric measures of hippocampal subfields from 443 4- to 17-year-olds across three independent studies. We identified commensurate measures of Cornu Ammonis (CA) 1, dentate gyrus (DG)/CA3, and Subiculum (Sub). Model testing supported use of IDA to create IDA factor scores. We found age-related differences in DG/CA3, not but CA1 and Sub volume in the integrated dataset. By successfully demonstrating IDA, our hope is that future innovations come from the combination of existing neuroimaging data to create representative integrated samples when testing critical developmental questions.

Harmonization for Parkinson's Disease Multi-Dataset T1 MRI Morphometry Classification

Classification of disease and healthy volunteer cohorts provides a useful clinical alternative to traditional group statistics due to individualized, personalized predictions. Classifiers for neurodegenerative disease can be trained on structural MRI morphometry, but require large multi-scanner datasets, introducing confounding batch effects. We test ComBat, a common harmonization model, in an example application to classify subjects with Parkinson's disease from healthy volunteers and identify common pitfalls, including data leakage. We used a multi-dataset cohort of 372 subjects (216 with Parkinson's disease, 156 healthy volunteers) from 11 identified scanners. We extracted both FreeSurfer and the determinant of Jacobian morphometry to compare single-scanner and multi-scanner classification pipelines. We confirm the presence of batch effects by running single scanner classifiers which could achieve wildly divergent AUCs on scanner-specific datasets (mean:0.651 ± 0.144). Multi-scanner classifiers that considered neurobiological batch effects between sites could easily achieve a test AUC of 0.902, though pipelines that prevented data leakage could only achieve a test AUC of 0.550. We conclude that batch effects remain a major issue for classification problems, such that even impressive single-scanner classifiers are unlikely to generalize to multiple scanners, and that solving for batch effects in a classifier problem must avoid circularity and reporting overly optimistic results.

Quality over quantity: powering neuroimaging samples in psychiatry

Neuroimaging has been widely adopted in psychiatric research, with hopes that these non-invasive methods will provide important clues to the underpinnings and prediction of various mental health symptoms and outcomes. However, the translational impact of neuroimaging has not yet reached its promise, despite the plethora of computational methods, tools, and datasets at our disposal. Some have lamented that too many psychiatric neuroimaging studies have been underpowered with respect to sample size. In this review, we encourage this discourse to shift from a focus on sheer increases in sample size to more thoughtful choices surrounding experimental study designs. We propose considerations at multiple decision points throughout the study design, data modeling and analysis process that may help researchers working in psychiatric neuroimaging boost power for their research questions of interest without necessarily increasing sample size. We also provide suggestions for leveraging multiple datasets to inform each other and strengthen our confidence in the generalization of findings to both population-level and clinical samples. Through a greater emphasis on improving the quality of brain-based and clinical measures rather than merely quantity, meaningful and potentially translational clinical associations with neuroimaging measures can be achieved with more modest sample sizes in psychiatry.

ComBat models for harmonization of resting-state EEG features in multisite studies

OBJECTIVE

Pooling multisite resting-state electroencephalography (rsEEG) datasets may introduce bias due to batch effects (i.e., cross-site differences in the rsEEG related to scanner/sample characteristics). The Combining Batches (ComBat) models, introduced for microarray expression and adapted for neuroimaging, can control for batch effects while preserving the variability of biological covariates. We aim to evaluate four ComBat harmonization methods in a pooled sample from five independent rsEEG datasets of young and old adults.

METHODS

RsEEG signals (n = 374) were automatically preprocessed. Oscillatory and aperiodic rsEEG features were extracted in sensor space. Features were harmonized using neuroCombat (standard ComBat used in neuroimaging), neuroHarmonize (variant with nonlinear adjustment of covariates), OPNested-GMM (variant based on Gaussian Mixture Models to fit bimodal feature distributions), and HarmonizR (variant based on resampling to handle missing feature values). Relationships between rsEEG features and age were explored before and after harmonizing batch effects.

RESULTS

Batch effects were identified in rsEEG features. All ComBat methods reduced batch effects and features' dispersion; HarmonizR and OPNested-GMM ComBat achieved the greatest performance. Harmonized Beta power, individual Alpha peak frequency, Aperiodic exponent, and offset in posterior electrodes showed significant relations with age. All ComBat models maintained the direction of observed relationships while increasing the effect size.

CONCLUSIONS

ComBat models, particularly HarmonizeR and OPNested-GMM ComBat, effectively control for batch effects in rsEEG spectral features.

SIGNIFICANCE

This workflow can be used in multisite studies to harmonize batch effects in sensor-space rsEEG spectral features while preserving biological associations.

HarmonyTM: multi-center data harmonization applied to distributed learning for Parkinson's disease classification

Purpose

Distributed learning is widely used to comply with data-sharing regulations and access diverse datasets for training machine learning (ML) models. The traveling model (TM) is a distributed learning approach that sequentially trains with data from one center at a time, which is especially advantageous when dealing with limited local datasets. However, a critical concern emerges when centers utilize different scanners for data acquisition, which could potentially lead models to exploit these differences as shortcuts. Although data harmonization can mitigate this issue, current methods typically rely on large or paired datasets, which can be impractical to obtain in distributed setups.

Approach

We introduced HarmonyTM, a data harmonization method tailored for the TM. HarmonyTM effectively mitigates bias in the model's feature representation while retaining crucial disease-related information, all without requiring extensive datasets. Specifically, we employed adversarial training to "unlearn" bias from the features used in the model for classifying Parkinson's disease (PD). We evaluated HarmonyTM using multi-center three-dimensional (3D) neuroimaging datasets from 83 centers using 23 different scanners.

Results

Our results show that HarmonyTM improved PD classification accuracy from 72% to 76% and reduced (unwanted) scanner classification accuracy from 53% to 30% in the TM setup.

Conclusion

HarmonyTM is a method tailored for harmonizing 3D neuroimaging data within the TM approach, aiming to minimize shortcut learning in distributed setups. This prevents the disease classifier from leveraging scanner-specific details to classify patients with or without PD-a key aspect for deploying ML models for clinical applications.

Deep Learning for MRI Segmentation and Molecular Subtyping in Glioblastoma: Critical Aspects from an Emerging Field

Deep learning (DL) has been applied to glioblastoma (GBM) magnetic resonance imaging (MRI) assessment for tumor segmentation and inference of molecular, diagnostic, and prognostic information. We comprehensively overviewed the currently available DL applications, critically examining the limitations that hinder their broader adoption in clinical practice and molecular research. Technical limitations to the routine application of DL include the qualitative heterogeneity of MRI, related to different machinery and protocols, and the absence of informative sequences, possibly compensated by artificial image synthesis. Moreover, taking advantage from the available benchmarks of MRI, algorithms should be trained on large amounts of data. Additionally, the segmentation of postoperative imaging should be further addressed to limit the inaccuracies previously observed for this task. Indeed, molecular information has been promisingly integrated in the most recent DL tools, providing useful prognostic and therapeutic information. Finally, ethical concerns should be carefully addressed and standardized to allow for data protection. DL has provided reliable results for GBM assessment concerning MRI analysis and segmentation, but the routine clinical application is still limited. The current limitations could be prospectively addressed, giving particular attention to data collection, introducing new technical advancements, and carefully regulating ethical issues.

Lifespan reference curves for harmonizing multi-site regional brain white matter metrics from diffusion MRI

Age-related white matter (WM) microstructure maturation and decline occur throughout the human lifespan, complementing the process of gray matter development and degeneration. Here, we create normative lifespan reference curves for global and regional WM microstructure by harmonizing diffusion MRI (dMRI)-derived data from ten public datasets (N = 40,898 subjects; age: 3-95 years; 47.6% male). We tested three harmonization methods on regional diffusion tensor imaging (DTI) based fractional anisotropy (FA), a metric of WM microstructure, extracted using the ENIGMA-DTI pipeline. ComBat-GAM harmonization provided multi-study trajectories most consistent with known WM maturation peaks. Lifespan FA reference curves were validated with test-retest data and used to assess the effect of the ApoE4 risk factor for dementia in WM across the lifespan. We found significant associations between ApoE4 and FA in WM regions associated with neurodegenerative disease even in healthy individuals across the lifespan, with regional age-by-genotype interactions. Our lifespan reference curves and tools to harmonize new dMRI data to the curves are publicly available as eHarmonize (https://github.com/ahzhu/eharmonize).