Sample size requirement for achieving multisite harmonization using structural brain MRI features

Autism-associated brain differences can be observed in utero using MRI

Developmental changes that occur before birth are thought to be associated with the development of autism spectrum disorders. Identifying anatomical predictors of early brain development may contribute to our understanding of the neurobiology of autism spectrum disorders and allow for earlier and more effective identification and treatment of autism spectrum disorders. In this study, we used retrospective clinical brain magnetic resonance imaging data from fetuses who were diagnosed with autism spectrum disorders later in life (prospective autism spectrum disorders) in order to identify the earliest magnetic resonance imaging-based regional volumetric biomarkers. Our results showed that magnetic resonance imaging-based autism spectrum disorder biomarkers can be found as early as in the fetal period and suggested that the increased volume of the insular cortex may be the most promising magnetic resonance imaging-based fetal biomarker for the future emergence of autism spectrum disorders, along with some additional, potentially useful changes in regional volumes and hemispheric asymmetries.

Harmonizing T1-Weighted Images to Improve Consistency of Brain Morphology Among Different Scanner Manufacturers in Alzheimer's disease

BACKGROUND

Brain MRI scanner variability can introduce bias in measurements. Harmonizing scanner variability is crucial.

PURPOSE

To develop a harmonization method aimed at removing scanner variability, and to evaluate the consistency of results in multicenter studies.

STUDY TYPE

Retrospective.

POPULATION

Multicenter data from 170 healthy participants (males/females = 98/72; age = 73.8 ± 7.3) and 170 Alzheimer's disease patients (males/females = 98/72; age = 76.2 ± 8.5) were compared with reference data from another 340 participants.

FIELD STRENGTH/SEQUENCE

3-T, magnetization prepared rapid gradient echo and turbo field echo; 1.5-T, inversion recovery prepared fast spoiled gradient echo T1-weighted sequences.

ASSESSMENT

Gray matter (GM) brain images, obtained through segmentation of T1-weighted images, were utilized to evaluate the performance of the harmonization method using common orthogonal basis extraction (HCOBE) and four other methods (removal of artificial voxel effect by linear regression, RAVEL; Z_score; general linear model, GLM; ComBat). Linear discriminant analysis (LDA) was used to access the effectiveness of different methods in reducing scanner variability. The performance of harmonization methods in preserving GM volumes heterogeneity was evaluated by the similarity of the relationship between GM proportion and age in the reference and multicenter data. Furthermore, the consistency of the harmonized multicenter data with the reference data were evaluated based on classification results (train/test = 7/3) and brain atrophy.

STATISTICAL TESTS

Two-sample t-tests, area under the curve (AUC), and Dice coefficients were used to analyze the consistency of results from the reference and harmonized multicenter data. A P-value <0.01 was considered statistically significant.

RESULTS

HCOBE reduced the scanner variability from 0.09 before harmonization to 0.003 (ideal: 0, RAVEL/Z_score/GLM/ComBat = 0.087/0.003/0.006/0.013). GM volumes showed no significant difference (P = 0.52) between the reference and HCOBE-harmonized multicenter data. Consistency evaluation showed that AUC values of 0.95 for both reference and HCOBE-harmonized multicenter data (RAVEL/Z_score/GLM/ComBat = 0.86/0.86/0.84/0.89), and the Dice coefficient increased from 0.73 before harmonization to 0.82 (ideal: 1, RAVEL/Z_score/GLM/ComBat = 0.39/0.64/0.59/0.74).

DATA CONCLUSION

HCOBE may help to remove scanner variability and could improve the consistency of results in multicenter studies.

LEVEL OF EVIDENCE

2 TECHNICAL EFFICACY STAGE: 1.

Empirical assessment of the assumptions of ComBat with diffusion tensor imaging

Purpose

Diffusion tensor imaging (DTI) is a magnetic resonance imaging technique that provides unique information about white matter microstructure in the brain but is susceptible to confounding effects introduced by scanner or acquisition differences. ComBat is a leading approach for addressing these site biases. However, despite its frequent use for harmonization, ComBat's robustness toward site dissimilarities and overall cohort size have not yet been evaluated in terms of DTI.

Approach

As a baseline, we match N = 358 participants from two sites to create a "silver standard" that simulates a cohort for multi-site harmonization. Across sites, we harmonize mean fractional anisotropy and mean diffusivity, calculated using participant DTI data, for the regions of interest defined by the JHU EVE-Type III atlas. We bootstrap 10 iterations at 19 levels of total sample size, 10 levels of sample size imbalance between sites, and 6 levels of mean age difference between sites to quantify (i) β AGE , the linear regression coefficient of the relationship between FA and age; (ii) γ ^ s f * , the ComBat-estimated site-shift; and (iii) δ ^ s f * , the ComBat-estimated site-scaling. We characterize the reliability of ComBat by evaluating the root mean squared error in these three metrics and examine if there is a correlation between the reliability of ComBat and a violation of assumptions.

Results

ComBat remains well behaved for β AGE when N > 162 and when the mean age difference is less than 4 years. The assumptions of the ComBat model regarding the normality of residual distributions are not violated as the model becomes unstable.

Conclusion

Prior to harmonization of DTI data with ComBat, the input cohort should be examined for size and covariate distributions of each site. Direct assessment of residual distributions is less informative on stability than bootstrap analysis. We caution use ComBat of in situations that do not conform to the above thresholds.

Efficacy of MRI data harmonization in the age of machine learning: a multicenter study across 36 datasets

Pooling publicly-available MRI data from multiple sites allows to assemble extensive groups of subjects, increase statistical power, and promote data reuse with machine learning techniques. The harmonization of multicenter data is necessary to reduce the confounding effect associated with non-biological sources of variability in the data. However, when applied to the entire dataset before machine learning, the harmonization leads to data leakage, because information outside the training set may affect model building, and potentially falsely overestimate performance. We propose a 1) measurement of the efficacy of data harmonization; 2) harmonizer transformer, i.e., an implementation of the ComBat harmonization allowing its encapsulation among the preprocessing steps of a machine learning pipeline, avoiding data leakage by design. We tested these tools using brain T1-weighted MRI data from 1740 healthy subjects acquired at 36 sites. After harmonization, the site effect was removed or reduced, and we showed the data leakage effect in predicting individual age from MRI data, highlighting that introducing the harmonizer transformer into a machine learning pipeline allows for avoiding data leakage by design.

Image harmonization: A review of statistical and deep learning methods for removing batch effects and evaluation metrics for effective harmonization

Magnetic resonance imaging and computed tomography from multiple batches (e.g. sites, scanners, datasets, etc.) are increasingly used alongside complex downstream analyses to obtain new insights into the human brain. However, significant confounding due to batch-related technical variation, called batch effects, is present in this data; direct application of downstream analyses to the data may lead to biased results. Image harmonization methods seek to remove these batch effects and enable increased generalizability and reproducibility of downstream results. In this review, we describe and categorize current approaches in statistical and deep learning harmonization methods. We also describe current evaluation metrics used to assess harmonization methods and provide a standardized framework to evaluate newly-proposed methods for effective harmonization and preservation of biological information. Finally, we provide recommendations to end-users to advocate for more effective use of current methods and to methodologists to direct future efforts and accelerate development of the field.