DeepHarmony: A deep learning approach to contrast harmonization across scanner changes

DeepComBat: A statistically motivated, hyperparameter-robust, deep learning approach to harmonization of neuroimaging data

Neuroimaging data acquired using multiple scanners or protocols are increasingly available. However, such data exhibit technical artifacts across batches which introduce confounding and decrease reproducibility. This is especially true when multi-batch data are analyzed using complex downstream models which are more likely to pick up on and implicitly incorporate batch-related information. Previously proposed image harmonization methods have sought to remove these batch effects; however, batch effects remain detectable in the data after applying these methods. We present DeepComBat, a deep learning harmonization method based on a conditional variational autoencoder and the ComBat method. DeepComBat combines the strengths of statistical and deep learning methods in order to account for the multivariate relationships between features while simultaneously relaxing strong assumptions made by previous deep learning harmonization methods. As a result, DeepComBat can perform multivariate harmonization while preserving data structure and avoiding the introduction of synthetic artifacts. We apply this method to cortical thickness measurements from a cognitive-aging cohort and show DeepComBat qualitatively and quantitatively outperforms existing methods in removing batch effects while preserving biological heterogeneity. Additionally, DeepComBat provides a new perspective for statistically motivated deep learning harmonization methods.

Exploring approaches to tackle cross-domain challenges in brain medical image segmentation: a systematic review

Introduction

Brain medical image segmentation is a critical task in medical image processing, playing a significant role in the prediction and diagnosis of diseases such as stroke, Alzheimer's disease, and brain tumors. However, substantial distribution discrepancies among datasets from different sources arise due to the large inter-site discrepancy among different scanners, imaging protocols, and populations. This leads to cross-domain problems in practical applications. In recent years, numerous studies have been conducted to address the cross-domain problem in brain image segmentation.

Methods

This review adheres to the standards of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for data processing and analysis. We retrieved relevant papers from PubMed, Web of Science, and IEEE databases from January 2018 to December 2023, extracting information about the medical domain, imaging modalities, methods for addressing cross-domain issues, experimental designs, and datasets from the selected papers. Moreover, we compared the performance of methods in stroke lesion segmentation, white matter segmentation and brain tumor segmentation.

Results

A total of 71 studies were included and analyzed in this review. The methods for tackling the cross-domain problem include Transfer Learning, Normalization, Unsupervised Learning, Transformer models, and Convolutional Neural Networks (CNNs). On the ATLAS dataset, domain-adaptive methods showed an overall improvement of ~3 percent in stroke lesion segmentation tasks compared to non-adaptive methods. However, given the diversity of datasets and experimental methodologies in current studies based on the methods for white matter segmentation tasks in MICCAI 2017 and those for brain tumor segmentation tasks in BraTS, it is challenging to intuitively compare the strengths and weaknesses of these methods.

Conclusion

Although various techniques have been applied to address the cross-domain problem in brain image segmentation, there is currently a lack of unified dataset collections and experimental standards. For instance, many studies are still based on n-fold cross-validation, while methods directly based on cross-validation across sites or datasets are relatively scarce. Furthermore, due to the diverse types of medical images in the field of brain segmentation, it is not straightforward to make simple and intuitive comparisons of performance. These challenges need to be addressed in future research.

Clinical applications of deep learning in neuroinflammatory diseases: A scoping review

BACKGROUND

Deep learning (DL) is an artificial intelligence technology that has aroused much excitement for predictive medicine due to its ability to process raw data modalities such as images, text, and time series of signals.

OBJECTIVES

Here, we intend to give the clinical reader elements to understand this technology, taking neuroinflammatory diseases as an illustrative use case of clinical translation efforts. We reviewed the scope of this rapidly evolving field to get quantitative insights about which clinical applications concentrate the efforts and which data modalities are most commonly used.

METHODS

We queried the PubMed database for articles reporting DL algorithms for clinical applications in neuroinflammatory diseases and the radiology.healthairegister.com website for commercial algorithms.

RESULTS

The review included 148 articles published between 2018 and 2024 and five commercial algorithms. The clinical applications could be grouped as computer-aided diagnosis, individual prognosis, functional assessment, the segmentation of radiological structures, and the optimization of data acquisition. Our review highlighted important discrepancies in efforts. The segmentation of radiological structures and computer-aided diagnosis currently concentrate most efforts with an overrepresentation of imaging. Various model architectures have addressed different applications, relatively low volume of data, and diverse data modalities. We report the high-level technical characteristics of the algorithms and synthesize narratively the clinical applications. Predictive performances and some common a priori on this topic are finally discussed.

CONCLUSION

The currently reported efforts position DL as an information processing technology, enhancing existing modalities of paraclinical investigations and bringing perspectives to make innovative ones actionable for healthcare.

Streamlining neuroradiology workflow with AI for improved cerebrovascular structure monitoring

Radiological imaging to examine intracranial blood vessels is critical for preoperative planning and postoperative follow-up. Automated segmentation of cerebrovascular anatomy from Time-Of-Flight Magnetic Resonance Angiography (TOF-MRA) can provide radiologists with a more detailed and precise view of these vessels. This paper introduces a domain generalized artificial intelligence (AI) solution for volumetric monitoring of cerebrovascular structures from multi-center MRAs. Our approach utilizes a multi-task deep convolutional neural network (CNN) with a topology-aware loss function to learn voxel-wise segmentation of the cerebrovascular tree. We use Decorrelation Loss to achieve domain regularization for the encoder network and auxiliary tasks to provide additional regularization and enable the encoder to learn higher-level intermediate representations for improved performance. We compare our method to six state-of-the-art 3D vessel segmentation methods using retrospective TOF-MRA datasets from multiple private and public data sources scanned at six hospitals, with and without vascular pathologies. The proposed model achieved the best scores in all the qualitative performance measures. Furthermore, we have developed an AI-assisted Graphical User Interface (GUI) based on our research to assist radiologists in their daily work and establish a more efficient work process that saves time.

Fighting the scanner effect in brain MRI segmentation with a progressive level-of-detail network trained on multi-site data

Many clinical and research studies of the human brain require accurate structural MRI segmentation. While traditional atlas-based methods can be applied to volumes from any acquisition site, recent deep learning algorithms ensure high accuracy only when tested on data from the same sites exploited in training (i.e., internal data). Performance degradation experienced on external data (i.e., unseen volumes from unseen sites) is due to the inter-site variability in intensity distributions, and to unique artefacts caused by different MR scanner models and acquisition parameters. To mitigate this site-dependency, often referred to as the scanner effect, we propose LOD-Brain, a 3D convolutional neural network with progressive levels-of-detail (LOD), able to segment brain data from any site. Coarser network levels are responsible for learning a robust anatomical prior helpful in identifying brain structures and their locations, while finer levels refine the model to handle site-specific intensity distributions and anatomical variations. We ensure robustness across sites by training the model on an unprecedentedly rich dataset aggregating data from open repositories: almost 27,000 T1w volumes from around 160 acquisition sites, at 1.5 - 3T, from a population spanning from 8 to 90 years old. Extensive tests demonstrate that LOD-Brain produces state-of-the-art results, with no significant difference in performance between internal and external sites, and robust to challenging anatomical variations. Its portability paves the way for large-scale applications across different healthcare institutions, patient populations, and imaging technology manufacturers. Code, model, and demo are available on the project website.

Contrast-enhanced to non-contrast-enhanced image translation to exploit a clinical data warehouse of T1-weighted brain MRI

BACKGROUND

Clinical data warehouses provide access to massive amounts of medical images, but these images are often heterogeneous. They can for instance include images acquired both with or without the injection of a gadolinium-based contrast agent. Harmonizing such data sets is thus fundamental to guarantee unbiased results, for example when performing differential diagnosis. Furthermore, classical neuroimaging software tools for feature extraction are typically applied only to images without gadolinium. The objective of this work is to evaluate how image translation can be useful to exploit a highly heterogeneous data set containing both contrast-enhanced and non-contrast-enhanced images from a clinical data warehouse.

METHODS

We propose and compare different 3D U-Net and conditional GAN models to convert contrast-enhanced T1-weighted (T1ce) into non-contrast-enhanced (T1nce) brain MRI. These models were trained using 230 image pairs and tested on 77 image pairs from the clinical data warehouse of the Greater Paris area.

RESULTS

Validation using standard image similarity measures demonstrated that the similarity between real and synthetic T1nce images was higher than between real T1nce and T1ce images for all the models compared. The best performing models were further validated on a segmentation task. We showed that tissue volumes extracted from synthetic T1nce images were closer to those of real T1nce images than volumes extracted from T1ce images.

CONCLUSION

We showed that deep learning models initially developed with research quality data could synthesize T1nce from T1ce images of clinical quality and that reliable features could be extracted from the synthetic images, thus demonstrating the ability of such methods to help exploit a data set coming from a clinical data warehouse.

Image harmonization improves consistency of intra-rater delineations of MS lesions in heterogeneous MRI

Clinical magnetic resonance images (MRIs) lack a standard intensity scale due to differences in scanner hardware and the pulse sequences used to acquire the images. When MRIs are used for quantification, as in the evaluation of white matter lesions (WMLs) in multiple sclerosis, this lack of intensity standardization becomes a critical problem affecting both the staging and tracking of the disease and its treatment. This paper presents a study of harmonization on WML segmentation consistency, which is evaluated using an object detection classification scheme that incorporates manual delineations from both the original and harmonized MRIs. A cohort of ten people scanned on two different imaging platforms was studied. An expert rater, blinded to the image source, manually delineated WMLs on images from both scanners before and after harmonization. It was found that there is closer agreement in both global and per-lesion WML volume and spatial distribution after harmonization, demonstrating the importance of image harmonization prior to the creation of manual delineations. These results could lead to better truth models in both the development and evaluation of automated lesion segmentation algorithms.

Growing importance of brain morphometry analysis in the clinical routine: The hidden impact of MR sequence parameters

Volumetric assessment based on structural MRI is increasingly recognized as an auxiliary tool to visual reading, also in examinations acquired in the clinical routine. However, MRI acquisition parameters can significantly influence these measures, which must be considered when interpreting the results on an individual patient level. This Technical Note shall demonstrate the problem. Using data from a dedicated experiment, we show the influence of two crucial sequence parameters on the GM/WM contrast and their impact on the measured volumes. A simulated contrast derived from acquisition parameters TI/TR may serve as surrogate and is highly correlated (r=0.96) with the measured contrast.

Bile acid metabolites predict multiple sclerosis progression and supplementation is safe in progressive disease

Background

Bile acid metabolism is altered in multiple sclerosis (MS) and tauroursodeoxycholic acid (TUDCA) supplementation ameliorated disease in mouse models of MS.

Methods

Global metabolomics was performed in an observational cohort of people with MS followed by pathway analysis to examine relationships between baseline metabolite levels and subsequent brain and retinal atrophy. A double-blind, placebo-controlled trial, was completed in people with progressive MS (PMS), randomized to receive either TUDCA (2g daily) or placebo for 16 weeks. Participants were followed with serial clinical and laboratory assessments. Primary outcomes were safety and tolerability of TUDCA, and exploratory outcomes included changes in clinical, laboratory and gut microbiome parameters.

Results

In the observational cohort, higher primary bile acid levels at baseline predicted slower whole brain, brain substructure and specific retinal layer atrophy. In the clinical trial, 47 participants were included in our analyses (21 in placebo arm, 26 in TUDCA arm). Adverse events did not significantly differ between arms (p=0.77). The TUDCA arm demonstrated increased serum levels of multiple bile acids. No significant differences were noted in clinical or fluid biomarker outcomes. Central memory CD4+ and Th1/17 cells decreased, while CD4+ naïve cells increased in the TUDCA arm compared to placebo. Changes in the composition and function of gut microbiota were also noted in the TUDCA arm compared to placebo.

Conclusion

Bile acid metabolism in MS is linked with brain and retinal atrophy. TUDCA supplementation in PMS is safe, tolerable and has measurable biological effects that warrant further evaluation in larger trials with a longer treatment duration.

DeepResBat: deep residual batch harmonization accounting for covariate distribution differences

Pooling MRI data from multiple datasets requires harmonization to reduce undesired inter-site variabilities, while preserving effects of biological variables (or covariates). The popular harmonization approach ComBat uses a mixed effect regression framework that explicitly accounts for covariate distribution differences across datasets. There is also significant interest in developing harmonization approaches based on deep neural networks (DNNs), such as conditional variational autoencoder (cVAE). However, current DNN approaches do not explicitly account for covariate distribution differences across datasets. Here, we provide mathematical results, suggesting that not accounting for covariates can lead to suboptimal harmonization outcomes. We propose two DNN-based harmonization approaches that explicitly account for covariate distribution differences across datasets: covariate VAE (coVAE) and DeepResBat. The coVAE approach is a natural extension of cVAE by concatenating covariates and site information with site- and covariate-invariant latent representations. DeepResBat adopts a residual framework inspired by ComBat. DeepResBat first removes the effects of covariates with nonlinear regression trees, followed by eliminating site differences with cVAE. Finally, covariate effects are added back to the harmonized residuals. Using three datasets from three different continents with a total of 2787 participants and 10085 anatomical T1 scans, we find that DeepResBat and coVAE outperformed ComBat, CovBat and cVAE in terms of removing dataset differences, while enhancing biological effects of interest. However, coVAE hallucinates spurious associations between anatomical MRI and covariates even when no association exists. Therefore, future studies proposing DNN-based harmonization approaches should be aware of this false positive pitfall. Overall, our results suggest that DeepResBat is an effective deep learning alternative to ComBat.

Learning multi-site harmonization of magnetic resonance images without traveling human phantoms

Harmonization improves Magn. Reson. Imaging (MRI) data consistency and is central to effective integration of diverse imaging data acquired across multiple sites. Recent deep learning techniques for harmonization are predominantly supervised in nature and hence require imaging data of the same human subjects to be acquired at multiple sites. Data collection as such requires the human subjects to travel across sites and is hence challenging, costly, and impractical, more so when sufficient sample size is needed for reliable network training. Here we show how harmonization can be achieved with a deep neural network that does not rely on traveling human phantom data. Our method disentangles site-specific appearance information and site-invariant anatomical information from images acquired at multiple sites and then employs the disentangled information to generate the image of each subject for any target site. We demonstrate with more than 6,000 multi-site T1- and T2-weighted images that our method is remarkably effective in generating images with realistic site-specific appearances without altering anatomical details. Our method allows retrospective harmonization of data in a wide range of existing modern large-scale imaging studies, conducted via different scanners and protocols, without additional data collection.

Robust fiber orientation distribution function estimation using deep constrained spherical deconvolution for diffusion-weighted magnetic resonance imaging

Purpose

Diffusion-weighted magnetic resonance imaging (DW-MRI) is a critical imaging method for capturing and modeling tissue microarchitecture at a millimeter scale. A common practice to model the measured DW-MRI signal is via fiber orientation distribution function (fODF). This function is the essential first step for the downstream tractography and connectivity analyses. With recent advantages in data sharing, large-scale multisite DW-MRI datasets are being made available for multisite studies. However, measurement variabilities (e.g., inter- and intrasite variability, hardware performance, and sequence design) are inevitable during the acquisition of DW-MRI. Most existing model-based methods [e.g., constrained spherical deconvolution (CSD)] and learning-based methods (e.g., deep learning) do not explicitly consider such variabilities in fODF modeling, which consequently leads to inferior performance on multisite and/or longitudinal diffusion studies.

Approach

In this paper, we propose a data-driven deep CSD method to explicitly constrain the scan-rescan variabilities for a more reproducible and robust estimation of brain microstructure from repeated DW-MRI scans. Specifically, the proposed method introduces a three-dimensional volumetric scanner-invariant regularization scheme during the fODF estimation. We study the Human Connectome Project (HCP) young adults test-retest group as well as the MASiVar dataset (with inter- and intrasite scan/rescan data). The Baltimore Longitudinal Study of Aging dataset is employed for external validation.

Results

From the experimental results, the proposed data-driven framework outperforms the existing benchmarks in repeated fODF estimation. By introducing the contrastive loss with scan/rescan data, the proposed method achieved a higher consistency while maintaining higher angular correlation coefficients with the CSD modeling. The proposed method is assessing the downstream connectivity analysis and shows increased performance in distinguishing subjects with different biomarkers.

Conclusion

We propose a deep CSD method to explicitly reduce the scan-rescan variabilities, so as to model a more reproducible and robust brain microstructure from repeated DW-MRI scans. The plug-and-play design of the proposed approach is potentially applicable to a wider range of data harmonization problems in neuroimaging.

USE-Evaluator: Performance metrics for medical image segmentation models supervised by uncertain, small or empty reference annotations in neuroimaging

Performance metrics for medical image segmentation models are used to measure the agreement between the reference annotation and the predicted segmentation. Usually, overlap metrics, such as the Dice, are used as a metric to evaluate the performance of these models in order for results to be comparable. However, there is a mismatch between the distributions of cases and the difficulty level of segmentation tasks in public data sets compared to clinical practice. Common metrics used to assess performance fail to capture the impact of this mismatch, particularly when dealing with datasets in clinical settings that involve challenging segmentation tasks, pathologies with low signal, and reference annotations that are uncertain, small, or empty. Limitations of common metrics may result in ineffective machine learning research in designing and optimizing models. To effectively evaluate the clinical value of such models, it is essential to consider factors such as the uncertainty associated with reference annotations, the ability to accurately measure performance regardless of the size of the reference annotation volume, and the classification of cases where reference annotations are empty. We study how uncertain, small, and empty reference annotations influence the value of metrics on a stroke in-house data set regardless of the model. We examine metrics behavior on the predictions of a standard deep learning framework in order to identify suitable metrics in such a setting. We compare our results to the BRATS 2019 and Spinal Cord public data sets. We show how uncertain, small, or empty reference annotations require a rethinking of the evaluation. The evaluation code was released to encourage further analysis of this topic https://github.com/SophieOstmeier/UncertainSmallEmpty.git.

A three-dimensional deep learning model for inter-site harmonization of structural MR images of the brain: Extensive validation with a multicenter dataset

In multicenter MRI studies, pooling the imaging data can introduce site-related variabilities and can therefore bias the subsequent analyses. To harmonize the intensity distributions of brain MR images in a multicenter dataset, unsupervised deep learning methods can be employed. Here, we developed a model based on cycle-consistent adversarial networks for the harmonization of T1-weighted brain MR images. In contrast to previous works, it was designed to process three-dimensional whole-brain images in a stable manner while optimizing computation resources. Using six different MRI datasets for healthy adults (n=1525 in total) with different acquisition parameters, we tested the model in (i) three pairwise harmonizations with site effects of various sizes, (ii) an overall harmonization of the six datasets with different age distributions, and (iii) a traveling-subject dataset. Our results for intensity distributions, brain volumes, image quality metrics and radiomic features indicated that the MRI characteristics at the various sites had been effectively homogenized. Next, brain age prediction experiments and the observed correlation between the gray-matter volume and age showed that thanks to an appropriate training strategy and despite biological differences between the dataset populations, the model reinforced biological patterns. Furthermore, radiologic analyses of the harmonized images attested to the conservation of the radiologic information in the original images. The robustness of the harmonization model (as judged with various datasets and metrics) demonstrates its potential for application in retrospective multicenter studies.

Image-encoded biological and non-biological variables may be used as shortcuts in deep learning models trained on multisite neuroimaging data

OBJECTIVE

This work investigates if deep learning (DL) models can classify originating site locations directly from magnetic resonance imaging (MRI) scans with and without correction for intensity differences.

MATERIAL AND METHODS

A large database of 1880 T1-weighted MRI scans collected across 41 sites originally for Parkinson's disease (PD) classification was used to classify sites in this study. Forty-six percent of the datasets are from PD patients, while 54% are from healthy participants. After preprocessing the T1-weighted scans, 2 additional data types were generated: intensity-harmonized T1-weighted scans and log-Jacobian deformation maps resulting from nonlinear atlas registration. Corresponding DL models were trained to classify sites for each data type. Additionally, logistic regression models were used to investigate the contribution of biological (age, sex, disease status) and non-biological (scanner type) variables to the models' decision.

RESULTS

A comparison of the 3 different types of data revealed that DL models trained using T1-weighted and intensity-harmonized T1-weighted scans can classify sites with an accuracy of 85%, while the model using log-Jacobian deformation maps achieved a site classification accuracy of 54%. Disease status and scanner type were found to be significant confounders.

DISCUSSION

Our results demonstrate that MRI scans encode relevant site-specific information that models could use as shortcuts that cannot be removed using simple intensity harmonization methods.

CONCLUSION

The ability of DL models to exploit site-specific biases as shortcuts raises concerns about their reliability, generalization, and deployability in clinical settings.

HACA3: A unified approach for multi-site MR image harmonization

The lack of standardization and consistency of acquisition is a prominent issue in magnetic resonance (MR) imaging. This often causes undesired contrast variations in the acquired images due to differences in hardware and acquisition parameters. In recent years, image synthesis-based MR harmonization with disentanglement has been proposed to compensate for the undesired contrast variations. The general idea is to disentangle anatomy and contrast information from MR images to achieve cross-site harmonization. Despite the success of existing methods, we argue that major improvements can be made from three aspects. First, most existing methods are built upon the assumption that multi-contrast MR images of the same subject share the same anatomy. This assumption is questionable, since different MR contrasts are specialized to highlight different anatomical features. Second, these methods often require a fixed set of MR contrasts for training (e.g., both T1-weighted and T2-weighted images), limiting their applicability. Lastly, existing methods are generally sensitive to imaging artifacts. In this paper, we present Harmonization with Attention-based Contrast, Anatomy, and Artifact Awareness (HACA3), a novel approach to address these three issues. HACA3 incorporates an anatomy fusion module that accounts for the inherent anatomical differences between MR contrasts. Furthermore, HACA3 can be trained and applied to any combination of MR contrasts and is robust to imaging artifacts. HACA3 is developed and evaluated on diverse MR datasets acquired from 21 sites with varying field strengths, scanner platforms, and acquisition protocols. Experiments show that HACA3 achieves state-of-the-art harmonization performance under multiple image quality metrics. We also demonstrate the versatility and potential clinical impact of HACA3 on downstream tasks including white matter lesion segmentation for people with multiple sclerosis and longitudinal volumetric analyses for normal aging subjects. Code is available at https://github.com/lianruizuo/haca3.

Style transfer generative adversarial networks to harmonize multisite MRI to a single reference image to avoid overcorrection

Recent work within neuroimaging consortia have aimed to identify reproducible, and often subtle, brain signatures of psychiatric or neurological conditions. To allow for high-powered brain imaging analyses, it is often necessary to pool MR images that were acquired with different protocols across multiple scanners. Current retrospective harmonization techniques have shown promise in removing site-related image variation. However, most statistical approaches may over-correct for technical, scanning-related, variation as they cannot distinguish between confounded image-acquisition based variability and site-related population variability. Such statistical methods often require that datasets contain subjects or patient groups with similar clinical or demographic information to isolate the acquisition-based variability. To overcome this limitation, we consider site-related magnetic resonance (MR) imaging harmonization as a style transfer problem rather than a domain transfer problem. Using a fully unsupervised deep-learning framework based on a generative adversarial network (GAN), we show that MR images can be harmonized by inserting the style information encoded from a single reference image, without knowing their site/scanner labels a priori. We trained our model using data from five large-scale multisite datasets with varied demographics. Results demonstrated that our style-encoding model can harmonize MR images, and match intensity profiles, without relying on traveling subjects. This model also avoids the need to control for clinical, diagnostic, or demographic information. We highlight the effectiveness of our method for clinical research by comparing extracted cortical and subcortical features, brain-age estimates, and case-control effect sizes before and after the harmonization. We showed that our harmonization removed the site-related variances, while preserving the anatomical information and clinical meaningful patterns. We further demonstrated that with a diverse training set, our method successfully harmonized MR images collected from unseen scanners and protocols, suggesting a promising tool for ongoing collaborative studies. Source code is released in USC-IGC/style_transfer_harmonization (github.com).

MISPEL: A supervised deep learning harmonization method for multi-scanner neuroimaging data

Large-scale data obtained from aggregation of already collected multi-site neuroimaging datasets has brought benefits such as higher statistical power, reliability, and robustness to the studies. Despite these promises from growth in sample size, substantial technical variability stemming from differences in scanner specifications exists in the aggregated data and could inadvertently bias any downstream analyses on it. Such a challenge calls for data normalization and/or harmonization frameworks, in addition to comprehensive criteria to estimate the scanner-related variability and evaluate the harmonization frameworks. In this study, we propose MISPEL (Multi-scanner Image harmonization via Structure Preserving Embedding Learning), a supervised multi-scanner harmonization method that is naturally extendable to more than two scanners. We also designed a set of criteria to investigate the scanner-related technical variability and evaluate the harmonization techniques. As an essential requirement of our criteria, we introduced a multi-scanner matched dataset of 3T T1 images across four scanners, which, to the best of our knowledge is one of the few datasets of this kind. We also investigated our evaluations using two popular segmentation frameworks: FSL and segmentation in statistical parametric mapping (SPM). Lastly, we compared MISPEL to popular methods of normalization and harmonization, namely White Stripe, RAVEL, and CALAMITI. MISPEL outperformed these methods and is promising for many other neuroimaging modalities.

MICDIR: Multi-scale inverse-consistent deformable image registration using UNetMSS with self-constructing graph latent

Image registration is the process of bringing different images into a common coordinate system - a technique widely used in various applications of computer vision, such as remote sensing, image retrieval, and, most commonly, medical imaging. Deep learning based techniques have been applied successfully to tackle various complex medical image processing problems, including medical image registration. Over the years, several image registration techniques have been proposed using deep learning. Deformable image registration techniques such as Voxelmorph have been successful in capturing finer changes and providing smoother deformations. However, Voxelmorph, as well as ICNet and FIRE, do not explicitly encode global dependencies (i.e. the overall anatomical view of the supplied image) and, therefore, cannot track large deformations. In order to tackle the aforementioned problems, this paper extends the Voxelmorph approach in three different ways. To improve the performance in case of small as well as large deformations, supervision of the model at different resolutions has been integrated using a multi-scale UNet. To support the network to learn and encode the minute structural co-relations of the given image-pairs, a self-constructing graph network (SCGNet) has been used as the latent of the multi-scale UNet - which can improve the learning process of the model and help the model to generalise better. And finally, to make the deformations inverse-consistent, cycle consistency loss has been employed. On the task of registration of brain MRIs, the proposed method achieved significant improvements over ANTs and VoxelMorph, obtaining a Dice score of 0.8013 ± 0.0243 for intramodal and 0.6211 ± 0.0309 for intermodal, while VoxelMorph achieved 0.7747 ± 0.0260 and 0.6071 ± 0.0510, respectively.

ImUnity: A generalizable VAE-GAN solution for multicenter MR image harmonization

ImUnity is an original 2.5D deep-learning model designed for efficient and flexible MR image harmonization. A VAE-GAN network, coupled with a confusion module and an optional biological preservation module, uses multiple 2D slices taken from different anatomical locations in each subject of the training database, as well as image contrast transformations for its training. It eventually generates 'corrected' MR images that can be used for various multi-center population studies. Using 3 open source databases (ABIDE, OASIS and SRPBS), which contain MR images from multiple acquisition scanner types or vendors and a large range of subjects ages, we show that ImUnity: (1) outperforms state-of-the-art methods in terms of quality of images generated using traveling subjects; (2) removes sites or scanner biases while improving patients classification; (3) harmonizes data coming from new sites or scanners without the need for an additional fine-tuning and (4) allows the selection of multiple MR reconstructed images according to the desired applications. Tested here on T1-weighted images, ImUnity could be used to harmonize other types of medical images.

Learning from pseudo-labels: deep networks improve consistency in longitudinal brain volume estimation

Introduction

Brain atrophy is a critical biomarker of disease progression and treatment response in neurodegenerative diseases such as multiple sclerosis (MS). Confounding factors such as inconsistent imaging acquisitions hamper the accurate measurement of brain atrophy in the clinic. This study aims to develop and validate a robust deep learning model to overcome these challenges; and to evaluate its impact on the measurement of disease progression.

Methods

Voxel-wise pseudo-atrophy labels were generated using SIENA, a widely adopted tool for the measurement of brain atrophy in MS. Deformation maps were produced for 195 pairs of longitudinal 3D T1 scans from patients with MS. A 3D U-Net, namely DeepBVC, was specifically developed overcome common variances in resolution, signal-to-noise ratio and contrast ratio between baseline and follow up scans. The performance of DeepBVC was compared against SIENA using McLaren test-retest dataset and 233 in-house MS subjects with MRI from multiple time points. Clinical evaluation included disability assessment with the Expanded Disability Status Scale (EDSS) and traditional imaging metrics such as lesion burden.

Results

For 3 subjects in test-retest experiments, the median percent brain volume change (PBVC) for DeepBVC and SIENA was 0.105 vs. 0.198% (subject 1), 0.061 vs. 0.084% (subject 2), 0.104 vs. 0.408% (subject 3). For testing consistency across multiple time points in individual MS subjects, the mean (± standard deviation) PBVC difference of DeepBVC and SIENA were 0.028% (± 0.145%) and 0.031% (±0.154%), respectively. The linear correlation with baseline T2 lesion volume were r = -0.288 (p < 0.05) and r = -0.249 (p < 0.05) for DeepBVC and SIENA, respectively. There was no significant correlation of disability progression with PBVC as estimated by either method (p = 0.86, p = 0.84).

Discussion

DeepBVC is a deep learning powered brain volume change estimation method for assessing brain atrophy used T1-weighted images. Compared to SIENA, DeepBVC demonstrates superior performance in reproducibility and in the context of common clinical scan variances such as imaging contrast, voxel resolution, random bias field, and signal-to-noise ratio. Enhanced measurement robustness, automation, and processing speed of DeepBVC indicate its potential for utilisation in both research and clinical environments for monitoring disease progression and, potentially, evaluating treatment effectiveness.

Image harmonization: A review of statistical and deep learning methods for removing batch effects and evaluation metrics for effective harmonization

Magnetic resonance imaging and computed tomography from multiple batches (e.g. sites, scanners, datasets, etc.) are increasingly used alongside complex downstream analyses to obtain new insights into the human brain. However, significant confounding due to batch-related technical variation, called batch effects, is present in this data; direct application of downstream analyses to the data may lead to biased results. Image harmonization methods seek to remove these batch effects and enable increased generalizability and reproducibility of downstream results. In this review, we describe and categorize current approaches in statistical and deep learning harmonization methods. We also describe current evaluation metrics used to assess harmonization methods and provide a standardized framework to evaluate newly-proposed methods for effective harmonization and preservation of biological information. Finally, we provide recommendations to end-users to advocate for more effective use of current methods and to methodologists to direct future efforts and accelerate development of the field.

Multiple sclerosis lesion segmentation: revisiting weighting mechanisms for federated learning

Background and introduction

Federated learning (FL) has been widely employed for medical image analysis to facilitate multi-client collaborative learning without sharing raw data. Despite great success, FL's applications remain suboptimal in neuroimage analysis tasks such as lesion segmentation in multiple sclerosis (MS), due to variance in lesion characteristics imparted by different scanners and acquisition parameters.

Methods

In this work, we propose the first FL MS lesion segmentation framework via two effective re-weighting mechanisms. Specifically, a learnable weight is assigned to each local node during the aggregation process, based on its segmentation performance. In addition, the segmentation loss function in each client is also re-weighted according to the lesion volume for the data during training.

Results

The proposed method has been validated on two FL MS segmentation scenarios using public and clinical datasets. Specifically, the case-wise and voxel-wise Dice score of the proposed method under the first public dataset is 65.20 and 74.30, respectively. On the second in-house dataset, the case-wise and voxel-wise Dice score is 53.66, and 62.31, respectively.

Discussions and conclusions

The Comparison experiments on two FL MS segmentation scenarios using public and clinical datasets have demonstrated the effectiveness of the proposed method by significantly outperforming other FL methods. Furthermore, the segmentation performance of FL incorporating our proposed aggregation mechanism can achieve comparable performance to that from centralized training with all the raw data.

DeepComBat: A Statistically Motivated, Hyperparameter-Robust, Deep Learning Approach to Harmonization of Neuroimaging Data

Neuroimaging data from multiple batches (i.e. acquisition sites, scanner manufacturer, datasets, etc.) are increasingly necessary to gain new insights into the human brain. However, multi-batch data, as well as extracted radiomic features, exhibit pronounced technical artifacts across batches. These batch effects introduce confounding into the data and can obscure biological effects of interest, decreasing the generalizability and reproducibility of findings. This is especially true when multi-batch data is used alongside complex downstream analysis models, such as machine learning methods. Image harmonization methods seeking to remove these batch effects are important for mitigating these issues; however, significant multivariate batch effects remain in the data following harmonization by current state-of-the-art statistical and deep learning methods. We present DeepCombat, a deep learning harmonization method based on a conditional variational autoencoder architecture and the ComBat harmonization model. DeepCombat learns and removes subject-level batch effects by accounting for the multivariate relationships between features. Additionally, DeepComBat relaxes a number of strong assumptions commonly made by previous deep learning harmonization methods and is empirically robust across a wide range of hyperparameter choices. We apply this method to neuroimaging data from a large cognitive-aging cohort and find that DeepCombat outperforms existing methods, as assessed by a battery of machine learning methods, in removing scanner effects from cortical thickness measurements while preserving biological heterogeneity. Additionally, DeepComBat provides a new perspective for statistically-motivated deep learning harmonization methods.

Virtual high-resolution MR angiography from non-angiographic multi-contrast MRIs: synthetic vascular model populations for in-silico trials

Despite success on multi-contrast MR image synthesis, generating specific modalities remains challenging. Those include Magnetic Resonance Angiography (MRA) that highlights details of vascular anatomy using specialised imaging sequences for emphasising inflow effect. This work proposes an end-to-end generative adversarial network that can synthesise anatomically plausible, high-resolution 3D MRA images using commonly acquired multi-contrast MR images (e.g. T1/T2/PD-weighted MR images) for the same subject whilst preserving the continuity of vascular anatomy. A reliable technique for MRA synthesis would unleash the research potential of very few population databases with imaging modalities (such as MRA) that enable quantitative characterisation of whole-brain vasculature. Our work is motivated by the need to generate digital twins and virtual patients of cerebrovascular anatomy for in-silico studies and/or in-silico trials. We propose a dedicated generator and discriminator that leverage the shared and complementary features of multi-source images. We design a composite loss function for emphasising vascular properties by minimising the statistical difference between the feature representations of the target images and the synthesised outputs in both 3D volumetric and 2D projection domains. Experimental results show that the proposed method can synthesise high-quality MRA images and outperform the state-of-the-art generative models both qualitatively and quantitatively. The importance assessment reveals that T2 and PD-weighted images are better predictors of MRA images than T1; and PD-weighted images contribute to better visibility of small vessel branches towards the peripheral regions. In addition, the proposed approach can generalise to unseen data acquired at different imaging centres with different scanners, whilst synthesising MRAs and vascular geometries that maintain vessel continuity. The results show the potential for use of the proposed approach to generating digital twin cohorts of cerebrovascular anatomy at scale from structural MR images typically acquired in population imaging initiatives.

Deep learning-based harmonization of CT reconstruction kernels towards improved clinical task performance

OBJECTIVES

To develop a deep learning-based harmonization framework, assessing whether it can improve performance of radiomics models given different kernels in different clinical tasks and additionally generalize to mitigate the effects of new/unobserved kernels on radiomics features.

METHODS

Patient data with 2 reconstruction kernels and phantom data with 22 reconstruction kernels were included. Eighty-five patients were studied for lymph node metastasis (LNM) prediction, and 164 patients for differential diagnosis between lung cancer (LC) and pulmonary tuberculosis (TB). Two convolutional neural network (CNN) models were developed to convert images (i) from B70f to B30f (CNNa) and (ii) from B30f to B70f (CNNb). Model performance between the two kernels was evaluated using AUC and compared with other well-known harmonization methods. Patient-normalized feature difference (PNFD) was used to identify the incompatible kernels (i.e., kernel with median PNFD > 1) with baseline (B30f/B70f), and measure the ability of the CNN models to convert the non-comparable kernels.

RESULTS

For LC versus pulmonary TB diagnosis, AUCs of CNNa vs. others were 0.85 vs. 0.54-0.74 (p = 0.0001-0.0003), and for CNNb vs. others: 0.87 vs. 0.54-0.86 (p = 0.0001-0.55). For LNM prediction, AUCs of CNNa vs. others were 0.68 vs. 0.56-0.61 (p = 0.10-0.39), and for CNNb vs. others: 0.78 vs. 0.70-0.73 (p = 0.07-0.40). After CNN harmonization, 17 of 20 (85%) of investigated unknown kernels produced comparable radiomics feature values relative to baseline (median PNFD from 1.10-2.31 to 0.23-1.13).

CONCLUSION

The CNN harmonization effectively improved performance of radiomics models between reconstruction kernels in different clinical tasks, and reduced feature differences between unknown kernels vs. baseline.

KEY POINTS

• The soft (B30f) and sharp (B70f) kernels strongly affect radiomics reproducibility and generalizability. • The convolutional neural network (CNN) harmonization methods performed better than location-scale (ComBat and centering-scaling) and matrix factorization harmonization methods (based on singular value decomposition (SVD) and independent component analysis (ICA)) in both clinical tasks. • The CNN harmonization methods improve feature reproducibility not only between specific kernels (B30f and B70f) from the same scanner, but also between unobserved kernels from different scanners of different vendors.

Multiscale functional connectivity patterns of the aging brain learned from harmonized rsfMRI data of the multi-cohort iSTAGING study

To learn multiscale functional connectivity patterns of the aging brain, we built a brain age prediction model of functional connectivity measures at seven scales on a large fMRI dataset, consisting of resting-state fMRI scans of 4186 individuals with a wide age range (22 to 97 years, with an average of 63) from five cohorts. We computed multiscale functional connectivity measures of individual subjects using a personalized functional network computational method, harmonized the functional connectivity measures of subjects from multiple datasets in order to build a functional brain age model, and finally evaluated how functional brain age gap correlated with cognitive measures of individual subjects. Our study has revealed that functional connectivity measures at multiple scales were more informative than those at any single scale for the brain age prediction, the data harmonization significantly improved the brain age prediction performance, and the data harmonization in the functional connectivity measures' tangent space worked better than in their original space. Moreover, brain age gap scores of individual subjects derived from the brain age prediction model were significantly correlated with clinical and cognitive measures. Overall, these results demonstrated that multiscale functional connectivity patterns learned from a large-scale multi-site rsfMRI dataset were informative for characterizing the aging brain and the derived brain age gap was associated with cognitive and clinical measures.

Machine Learning for Brain MRI Data Harmonisation: A Systematic Review

BACKGROUND

Magnetic Resonance Imaging (MRI) data collected from multiple centres can be heterogeneous due to factors such as the scanner used and the site location. To reduce this heterogeneity, the data needs to be harmonised. In recent years, machine learning (ML) has been used to solve different types of problems related to MRI data, showing great promise.

OBJECTIVE

This study explores how well various ML algorithms perform in harmonising MRI data, both implicitly and explicitly, by summarising the findings in relevant peer-reviewed articles. Furthermore, it provides guidelines for the use of current methods and identifies potential future research directions.

METHOD

This review covers articles published through PubMed, Web of Science, and IEEE databases through June 2022. Data from studies were analysed based on the criteria of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). Quality assessment questions were derived to assess the quality of the included publications.

RESULTS

a total of 41 articles published between 2015 and 2022 were identified and analysed. In the review, MRI data has been found to be harmonised either in an implicit (n = 21) or an explicit (n = 20) way. Three MRI modalities were identified: structural MRI (n = 28), diffusion MRI (n = 7) and functional MRI (n = 6).

CONCLUSION

Various ML techniques have been employed to harmonise different types of MRI data. There is currently a lack of consistent evaluation methods and metrics used across studies, and it is recommended that the issue be addressed in future studies. Harmonisation of MRI data using ML shows promises in improving performance for ML downstream tasks, while caution should be exercised when using ML-harmonised data for direct interpretation.

DOMINO: Domain-aware loss for deep learning calibration

Deep learning has achieved the state-of-the-art performance across medical imaging tasks; however, model calibration is often not considered. Uncalibrated models are potentially dangerous in high-risk applications since the user does not know when they will fail. Therefore, this paper proposes a novel domain-aware loss function to calibrate deep learning models. The proposed loss function applies a class-wise penalty based on the similarity between classes within a given target domain. Thus, the approach improves the calibration while also ensuring that the model makes less risky errors even when incorrect. The code for this software is available at https://github.com/lab-smile/DOMINO.

Deep Learning for Image Enhancement and Correction in Magnetic Resonance Imaging-State-of-the-Art and Challenges

Magnetic resonance imaging (MRI) provides excellent soft-tissue contrast for clinical diagnoses and research which underpin many recent breakthroughs in medicine and biology. The post-processing of reconstructed MR images is often automated for incorporation into MRI scanners by the manufacturers and increasingly plays a critical role in the final image quality for clinical reporting and interpretation. For image enhancement and correction, the post-processing steps include noise reduction, image artefact correction, and image resolution improvements. With the recent success of deep learning in many research fields, there is great potential to apply deep learning for MR image enhancement, and recent publications have demonstrated promising results. Motivated by the rapidly growing literature in this area, in this review paper, we provide a comprehensive overview of deep learning-based methods for post-processing MR images to enhance image quality and correct image artefacts. We aim to provide researchers in MRI or other research fields, including computer vision and image processing, a literature survey of deep learning approaches for MR image enhancement. We discuss the current limitations of the application of artificial intelligence in MRI and highlight possible directions for future developments. In the era of deep learning, we highlight the importance of a critical appraisal of the explanatory information provided and the generalizability of deep learning algorithms in medical imaging.

Trans-Synaptic Degeneration Following Acute Optic Neuritis in Multiple Sclerosis

OBJECTIVE

To explore longitudinal changes in brain volumetric measures and retinal layer thicknesses following acute optic neuritis (AON) in people with multiple sclerosis (PwMS), to investigate the process of trans-synaptic degeneration, and determine its clinical relevance.

METHODS

PwMS were recruited within 40 days of AON onset (n = 49), and underwent baseline retinal optical coherence tomography and brain magnetic resonance imaging followed by longitudinal tracking for up to 5 years. A comparator cohort of PwMS without a recent episode of AON were similarly tracked (n = 73). Mixed-effects linear regression models were used.

RESULTS

Accelerated atrophy of the occipital gray matter (GM), calcarine GM, and thalamus was seen in the AON cohort, as compared with the non-AON cohort (-0.76% vs -0.22% per year [p = 0.01] for occipital GM, -1.83% vs -0.32% per year [p = 0.008] for calcarine GM, -1.17% vs -0.67% per year [p = 0.02] for thalamus), whereas rates of whole-brain, cortical GM, non-occipital cortical GM atrophy, and T2 lesion accumulation did not differ significantly between the cohorts. In the AON cohort, greater AON-induced reduction in ganglion cell+inner plexiform layer thickness over the first year was associated with faster rates of whole-brain (r = 0.32, p = 0.04), white matter (r = 0.32, p = 0.04), and thalamic (r = 0.36, p = 0.02) atrophy over the study period. Significant relationships were identified between faster atrophy of the subcortical GM and thalamus, with worse visual function outcomes after AON.

INTERPRETATION

These results provide in-vivo evidence for anterograde trans-synaptic degeneration following AON in PwMS, and suggest that trans-synaptic degeneration may be related to clinically-relevant visual outcomes. ANN NEUROL 2023;93:76-87.

Challenges for machine learning in clinical translation of big data imaging studies

Combining deep learning image analysis methods and large-scale imaging datasets offers many opportunities to neuroscience imaging and epidemiology. However, despite these opportunities and the success of deep learning when applied to a range of neuroimaging tasks and domains, significant barriers continue to limit the impact of large-scale datasets and analysis tools. Here, we examine the main challenges and the approaches that have been explored to overcome them. We focus on issues relating to data availability, interpretability, evaluation, and logistical challenges and discuss the problems that still need to be tackled to enable the success of "big data" deep learning approaches beyond research.

Semi-Supervised Learning of MRI Synthesis Without Fully-Sampled Ground Truths

Learning-based translation between MRI contrasts involves supervised deep models trained using high-quality source- and target-contrast images derived from fully-sampled acquisitions, which might be difficult to collect under limitations on scan costs or time. To facilitate curation of training sets, here we introduce the first semi-supervised model for MRI contrast translation (ssGAN) that can be trained directly using undersampled k-space data. To enable semi-supervised learning on undersampled data, ssGAN introduces novel multi-coil losses in image, k-space, and adversarial domains. The multi-coil losses are selectively enforced on acquired k-space samples unlike traditional losses in single-coil synthesis models. Comprehensive experiments on retrospectively undersampled multi-contrast brain MRI datasets are provided. Our results demonstrate that ssGAN yields on par performance to a supervised model, while outperforming single-coil models trained on coil-combined magnitude images. It also outperforms cascaded reconstruction-synthesis models where a supervised synthesis model is trained following self-supervised reconstruction of undersampled data. Thus, ssGAN holds great promise to improve the feasibility of learning-based multi-contrast MRI synthesis.

Protocol for mathematical prediction of patient response and survival to immune checkpoint inhibitor immunotherapy

This protocol describes the application of a mechanistic mathematical model of immune checkpoint inhibitor (ICI) immunotherapy to patient tumor imaging data for predicting solid tumor response and patient survival under ICI intervention. We describe steps for data collection and processing, data pipelines, and approaches to increase precision. The protocol is highly predictive as early as the first restaging after treatment start and can be used with standard-of-care imaging measures. For complete details on the use and execution of this protocol, please refer to Butner et al. (2020)¹ and Butner et al. (2021).².

Improved generalized ComBat methods for harmonization of radiomic features

Radiomic approaches in precision medicine are promising, but variation associated with image acquisition factors can result in severe biases and low generalizability. Multicenter datasets used in these studies are often heterogeneous in multiple imaging parameters and/or have missing information, resulting in multimodal radiomic feature distributions. ComBat is a promising harmonization tool, but it only harmonizes by single/known variables and assumes standardized input data are normally distributed. We propose a procedure that sequentially harmonizes for multiple batch effects in an optimized order, called OPNested ComBat. Furthermore, we propose to address bimodality by employing a Gaussian Mixture Model (GMM) grouping considered as either a batch variable (OPNested + GMM) or as a protected clinical covariate (OPNested - GMM). Methods were evaluated on features extracted with CapTK and PyRadiomics from two public lung computed tomography (CT) datasets. We found that OPNested ComBat improved harmonization performance over standard ComBat. OPNested + GMM ComBat exhibited the best harmonization performance but the lowest predictive performance, while OPNested - GMM ComBat showed poorer harmonization performance, but the highest predictive performance. Our findings emphasize that improved harmonization performance is no guarantee of improved predictive performance, and that these methods show promise for superior standardization of datasets heterogeneous in multiple or unknown imaging parameters and greater generalizability.

Harmonization Strategies in Multicenter MRI-Based Radiomics

Radiomics analysis is a powerful tool aiming to provide diagnostic and prognostic patient information directly from images that are decoded into handcrafted features, comprising descriptors of shape, size and textural patterns. Although radiomics is gaining momentum since it holds great promise for accelerating digital diagnostics, it is susceptible to bias and variation due to numerous inter-patient factors (e.g., patient age and gender) as well as inter-scanner ones (different protocol acquisition depending on the scanner center). A variety of image and feature based harmonization methods has been developed to compensate for these effects; however, to the best of our knowledge, none of these techniques has been established as the most effective in the analysis pipeline so far. To this end, this review provides an overview of the challenges in optimizing radiomics analysis, and a concise summary of the most relevant harmonization techniques, aiming to provide a thorough guide to the radiomics harmonization process.

[Applications of Artificial Intelligence in MR Image Acquisition and Reconstruction]

Recently, artificial intelligence (AI) technology has shown potential clinical utility in a wide range of MRI fields. In particular, AI models for improving the efficiency of the image acquisition process and the quality of reconstructed images are being actively developed by the MR research community. AI is expected to further reduce acquisition times in various MRI protocols used in clinical practice when compared to current parallel imaging techniques. Additionally, AI can help with tasks such as planning, parameter optimization, artifact reduction, and quality assessment. Furthermore, AI is being actively applied to automate MR image analysis such as image registration, segmentation, and object detection. For this reason, it is important to consider the effects of protocols or devices in MR image analysis. In this review article, we briefly introduced issues related to AI application of MR image acquisition and reconstruction.

Mitochondrial measures in neuronally enriched extracellular vesicles predict brain and retinal atrophy in multiple sclerosis

BACKGROUND

Mitochondrial dysfunction plays an important role in multiple sclerosis (MS) disease progression. Plasma extracellular vesicles are a potential source of novel biomarkers in MS, and some of these are derived from mitochondria and contain functional mitochondrial components.

OBJECTIVE

To evaluate the relationship between levels of mitochondrial complex IV and V activity in neuronally enriched extracellular vesicles (NEVs) and brain and retinal atrophy as assessed using serial magnetic resonance imaging (MRI) and optical coherence tomography (OCT).

METHODS

Our cohort consisted of 48 people with MS. NEVs were immunocaptured from plasma and mitochondrial complex IV and V activity levels were measured. Subjects underwent OCT every 6 months and brain MRI annually. The associations between baseline mitochondrial complex IV and V activities and brain substructure and retinal thickness changes were estimated utilizing linear mixed-effects models.

RESULTS

We found that higher mitochondrial complex IV activity and lower mitochondrial complex V activity levels were significantly associated with faster whole-brain volume atrophy. Similar results were found with other brain substructures and retinal layer atrophy.

CONCLUSION

Our results suggest that mitochondrial measures in circulating NEVs could serve as potential biomarkers of disease progression and provide the rationale for larger follow-up longitudinal studies.

A comparison of methods to harmonize cortical thickness measurements across scanners and sites

Results of neuroimaging datasets aggregated from multiple sites may be biased by site-specific profiles in participants' demographic and clinical characteristics, as well as MRI acquisition protocols and scanning platforms. We compared the impact of four different harmonization methods on results obtained from analyses of cortical thickness data: (1) linear mixed-effects model (LME) that models site-specific random intercepts (LMEINT), (2) LME that models both site-specific random intercepts and age-related random slopes (LMEINT+SLP), (3) ComBat, and (4) ComBat with a generalized additive model (ComBat-GAM). Our test case for comparing harmonization methods was cortical thickness data aggregated from 29 sites, which included 1,340 cases with posttraumatic stress disorder (PTSD) (6.2-81.8 years old) and 2,057 trauma-exposed controls without PTSD (6.3-85.2 years old). We found that, compared to the other data harmonization methods, data processed with ComBat-GAM was more sensitive to the detection of significant case-control differences (Χ2(3) = 63.704, p < 0.001) as well as case-control differences in age-related cortical thinning (Χ2(3) = 12.082, p = 0.007). Both ComBat and ComBat-GAM outperformed LME methods in detecting sex differences (Χ2(3) = 9.114, p = 0.028) in regional cortical thickness. ComBat-GAM also led to stronger estimates of age-related declines in cortical thickness (corrected p-values < 0.001), stronger estimates of case-related cortical thickness reduction (corrected p-values < 0.001), weaker estimates of age-related declines in cortical thickness in cases than controls (corrected p-values < 0.001), stronger estimates of cortical thickness reduction in females than males (corrected p-values < 0.001), and stronger estimates of cortical thickness reduction in females relative to males in cases than controls (corrected p-values < 0.001). Our results support the use of ComBat-GAM to minimize confounds and increase statistical power when harmonizing data with non-linear effects, and the use of either ComBat or ComBat-GAM for harmonizing data with linear effects.

Site effects how-to and when: An overview of retrospective techniques to accommodate site effects in multi-site neuroimaging analyses

Site differences, or systematic differences in feature distributions across multiple data-acquisition sites, are a known source of heterogeneity that may adversely affect large-scale meta- and mega-analyses of independently collected neuroimaging data. They influence nearly all multi-site imaging modalities and biomarkers, and methods to compensate for them can improve reliability and generalizability in the analysis of genetics, omics, and clinical data. The origins of statistical site effects are complex and involve both technical differences (scanner vendor, head coil, acquisition parameters, imaging processing) and differences in sample characteristics (inclusion/exclusion criteria, sample size, ancestry) between sites. In an age of expanding international consortium research, there is a growing need to disentangle technical site effects from sample characteristics of interest. Numerous statistical and machine learning methods have been developed to control for, model, or attenuate site effects - yet to date, no comprehensive review has discussed the benefits and drawbacks of each for different use cases. Here, we provide an overview of the different existing statistical and machine learning methods developed to remove unwanted site effects from independently collected neuroimaging samples. We focus on linear mixed effect models, the ComBat technique and its variants, adjustments based on image quality metrics, normative modeling, and deep learning approaches such as generative adversarial networks. For each method, we outline the statistical foundation and summarize strengths and weaknesses, including their assumptions and conditions of use. We provide information on software availability and comment on the ease of use and the applicability of these methods to different types of data. We discuss validation and comparative reports, mention caveats and provide guidance on when to use each method, depending on context and specific research questions.

Unraveling schizophrenia replicable functional connectivity disruption patterns across sites

Functional connectivity (FC) disruption is a remarkable characteristic of schizophrenia. However, heterogeneous patterns reported across sites severely hindered its clinical generalization. Based on qualified nodal-based FC of 340 schizophrenia patients (SZ) and 348 normal controls (NC) acquired from seven different scanners, this study compared four commonly used site-effect correction methods in removing the site-related heterogeneities, and then tried to cluster the abnormal FCs into several replicable and independent disrupted subnets across sites, related them to clinical symptoms, and evaluated their potentials in schizophrenia classification. Among the four site-related heterogeneity correction methods, ComBat harmonization (F1 score: 0.806 ± 0.145) achieved the overall best balance between sensitivity and false discovery rate in unraveling the aberrant FCs of schizophrenia in the local and public data sets. Hierarchical clustering analysis identified three replicable FC disruption subnets across the local and public data sets: hypo-connectivity within sensory areas (Net1), hypo-connectivity within thalamus, striatum, and ventral attention network (Net2), and hyper-connectivity between thalamus and sensory processing system (Net3). Notably, the derived composite FC within Net1 was negatively correlated with hostility and disorientation in the public validation set (p < .05). Finally, the three subnet-specific composite FCs (Best area under the receiver operating characteristic curve [AUC] = 0.728) can robustly and meaningfully discriminate the SZ from NC with comparable performance with the full identified FCs features (best AUC = 0.765) in the out-of-sample public data set (Z = -1.583, p = .114). In conclusion, ComBat harmonization was most robust in detecting aberrant connectivity for schizophrenia. Besides, the three subnet-specific composite FC measures might be replicable neuroimaging markers for schizophrenia.

The impact of harmonization on radiomic features in Parkinson’s disease and healthy controls: A multicenter study

Radiomics is a challenging development area in imaging field that is greatly capturing interest of radiologists and neuroscientists. However, radiomics features show a strong non-biological variability determined by different facilities and imaging protocols, limiting the reproducibility and generalizability of analysis frameworks. Our study aimed to investigate the usefulness of harmonization to reduce site-effects on radiomics features over specific brain regions. We selected T1-weighted magnetic resonance imaging (MRI) by using the MRI dataset Parkinson’s Progression Markers Initiative (PPMI) from different sites with healthy controls (HC) and Parkinson’s disease (PD) patients. First, the investigation of radiomics measure discrepancies were assessed on healthy brain regions-of-interest (ROIs) via a classification pipeline based on LASSO feature selection and support vector machine (SVM) model. Then, a ComBat-based harmonization approach was applied to correct site-effects. Finally, a validation step on PD subjects evaluated diagnostic accuracy before and after harmonization of radiomics data. Results on healthy subjects demonstrated a dependence from site-effects that could be corrected with ComBat harmonization. LASSO regressor after harmonization was unable to select any feature to distinguish controls by site. Moreover, harmonized radiomics features achieved an area under the receiving operating characteristic curve (AUC) of 0.77 (compared to AUC of 0.71 for raw radiomics measures) in distinguish Parkinson’s patients from HC. We found a not-negligible site-effect studying radiomics of HC pre- and post-harmonization of features. Our validation study on PD patients demonstrated a significant influence of non-biological noise source in diagnostic performances. Finally, harmonization of multicenter radiomic data represent a necessary step to make analysis pipelines reliable and replicable for multisite neuroimaging studies.

A guide for researchers seeking training in retrospective data harmonization for population neuroscience studies of Alzheimer's disease and related dementias

Due to needs surrounding rigor and reproducibility, subgroup specific disease knowledge, and questions of external validity, data harmonization is an essential tool in population neuroscience of Alzheimer's disease and related dementias (ADRD). Systematic harmonization of data elements is necessary to pool information from heterogeneous samples, and such pooling allows more expansive evaluations of health disparities, more precise effect estimates, and more opportunities to discover effective prevention or treatment strategies. The key goal of this Tutorial in Population Neuroimaging Curriculum, Instruction, and Pedagogy article is to guide researchers in creating a customized population neuroscience of ADRD harmonization training plan to fit their needs or those of their mentees. We provide brief guidance for retrospective data harmonization of multiple data types in this area, including: (1) clinical and demographic, (2) neuropsychological, and (3) neuroimaging data. Core competencies and skills are reviewed, and resources are provided to fill gaps in training as well as data needs. We close with an example study in which harmonization is a critical tool. While several aspects of this tutorial focus specifically on ADRD, the concepts and resources are likely to benefit population neuroscientists working in a range of research areas.

Fast Image-Level MRI Harmonization via Spectrum Analysis

Pooling structural magnetic resonance imaging (MRI) data from different imaging sites helps increase sample size to facilitate machine learning based neuroimage analysis, but usually suffers from significant cross-site and/or cross-scanner data heterogeneity. Existing studies often focus on reducing cross-site and/or cross-scanner heterogeneity at handcrafted feature level targeting specific tasks (e.g., classification or segmentation), limiting their adaptability in clinical practice. Research on image-level MRI harmonization targeting a broad range of applications is very limited. In this paper, we develop a spectrum swapping based image-level MRI harmonization (SSIMH) framework. Different from previous work, our method focuses on alleviating cross-scanner heterogeneity at raw image level. We first construct spectrum analysis to explore the influences of different frequency components on MRI harmonization. We then utilize a spectrum swapping method for the harmonization of raw MRIs acquired by different scanners. Our method does not rely on complex model training, and can be directly applied to fast real-time MRI harmonization. Experimental results on T1- and T2-weighted MRIs of phantom subjects acquired by using different scanners from the public ABCD dataset suggest the effectiveness of our method in structural MRI harmonization at the image level.

Goal-specific brain MRI harmonization

There is significant interest in pooling magnetic resonance image (MRI) data from multiple datasets to enable mega-analysis. Harmonization is typically performed to reduce heterogeneity when pooling MRI data across datasets. Most MRI harmonization algorithms do not explicitly consider downstream application performance during harmonization. However, the choice of downstream application might influence what might be considered as study-specific confounds. Therefore, ignoring downstream applications during harmonization might potentially limit downstream performance. Here we propose a goal-specific harmonization framework that utilizes downstream application performance to regularize the harmonization procedure. Our framework can be integrated with a wide variety of harmonization models based on deep neural networks, such as the recently proposed conditional variational autoencoder (cVAE) harmonization model. Three datasets from three different continents with a total of 2787 participants and 10,085 anatomical T1 scans were used for evaluation. We found that cVAE removed more dataset differences than the widely used ComBat model, but at the expense of removing desirable biological information as measured by downstream prediction of mini mental state examination (MMSE) scores and clinical diagnoses. On the other hand, our goal-specific cVAE (gcVAE) was able to remove as much dataset differences as cVAE, while improving downstream cross-sectional prediction of MMSE scores and clinical diagnoses.

Brain atrophy measurement over a MRI scanner change in multiple sclerosis

BACKGROUND

A change in MRI hardware impacts brain volume measurements. The aim of this study was to use MRI data from multiple sclerosis (MS) patients and healthy control subjects (HCs) to statistically model how to adjust brain atrophy measures in MS patients after a major scanner upgrade.

METHODS

We scanned 20 MS patients and 26 HCs before and three months after a major scanner upgrade (1.5 T Siemens Healthineers Magnetom Avanto to 3 T Siemens Healthineers Skyra Fit). The patient group also underwent standardized serial MRIs before and after the scanner change. Percentage whole brain volume changes (PBVC) measured by Structural Image Evaluation using Normalization of Atrophy (SIENA) in the HCs was used to estimate a corrective term based on a linear model. The factor was internally validated in HCs, and then applied to the MS group.

RESULTS

Mean PBVC during the scanner change was higher in MS than HCs (-4.1 ± 0.8 % versus -3.4 ± 0.6 %). A fixed corrective term of 3.4 (95% confidence interval: 3.13-3.67)% was estimated based on the observed average changes in HCs. Age and gender did not have a significant influence on this corrective term. After adjustment, a linear mixed effects model showed that the brain atrophy measures in MS during the scanner upgrade were not anymore associated with the scanner type (old vs new scanner; p = 0.29).

CONCLUSION

A scanner change affects brain atrophy measures in longitudinal cohorts. The inclusion of a corrective term based on changes observed in HCs helps to adjust for the known and unknown factors associated with a scanner upgrade on a group level.

Impact of image quality on radiomics applications

Radiomics features extracted from medical images have been widely reported to be useful in the patient specific outcome modeling for variety of assessment and prediction purposes. Successful application of radiomics features as imaging biomarkers, however, is dependent on the robustness of the approach to the variation in each step of the modeling workflow. Variation in the input image quality is one of the main sources that impacts the reproducibility of radiomics analysis when a model is applied to broader range of medical imaging data. The quality of medical image is generally affected by both the scanner related factors such as image acquisition/reconstruction settings and the patient related factors such as patient motion. This article aimed to review the published literatures in this field that reported the impact of various imaging factors on the radiomics features through the change in image quality. The literatures were categorized by different imaging modalities and also tabulated based on the imaging parameters and the class of radiomics features included in the study. Strategies for image quality standardization were discussed based on the relevant literatures and recommendations for reducing the impact of image quality variation on the radiomics in multi-institutional clinical trial were summarized at the end of this article.

AutoComBat: a generic method for harmonizing MRI-based radiomic features

The use of multicentric data is becoming essential for developing generalizable radiomic signatures. In particular, Magnetic Resonance Imaging (MRI) data used in brain oncology are often heterogeneous in terms of scanners and acquisitions, which significantly impact quantitative radiomic features. Various methods have been proposed to decrease dependency, including methods acting directly on MR images, i.e., based on the application of several preprocessing steps before feature extraction or the ComBat method, which harmonizes radiomic features themselves. The ComBat method used for radiomics may be misleading and presents some limitations, such as the need to know the labels associated with the "batch effect". In addition, a statistically representative sample is required and the applicability of a signature whose batch label is not present in the train set is not possible. This work aimed to compare a priori and a posteriori radiomic harmonization methods and propose a code adaptation to be machine learning compatible. Furthermore, we have developed AutoComBat, which aims to automatically determine the batch labels, using either MRI metadata or quality metrics as inputs of the proposed constrained clustering. A heterogeneous dataset consisting of high and low-grade gliomas coming from eight different centers was considered. The different methods were compared based on their ability to decrease relative standard deviation of radiomic features extracted from white matter and on their performance on a classification task using different machine learning models. ComBat and AutoComBat using image-derived quality metrics as inputs for batch assignment and preprocessing methods presented promising results on white matter harmonization, but with no clear consensus for all MR images. Preprocessing showed the best results on the T1w-gd images for the grading task. For T2w-flair, AutoComBat, using either metadata plus quality metrics or metadata alone as inputs, performs better than the conventional ComBat, highlighting its potential for data harmonization. Our results are MRI weighting, feature class and task dependent and require further investigations on other datasets.

Self-supervised learning for multi-center magnetic resonance imaging harmonization without traveling phantoms

Objective. With the progress of artificial intelligence (AI) in magnetic resonance imaging (MRI), large-scale multi-center MRI datasets have a great influence on diagnosis accuracy and model performance. However, multi-center images are highly variable due to the variety of scanners or scanning parameters in use, which has a negative effect on the generality of AI-based diagnosis models. To address this problem, we propose a self-supervised harmonization (SSH) method. Approach. Mapping the style of images between centers allows harmonization without traveling phantoms to be formalized as an unpaired image-to-image translation problem between two domains. The mapping is a two-stage transform, consisting of a modified cycle generative adversarial network (cycleGAN) for style transfer and a histogram matching module for structure fidelity. The proposed algorithm is demonstrated using female pelvic MRI images from two 3 T systems and compared with three state-of-the-art methods and one conventional method. In the absence of traveling phantoms, we evaluate harmonization from three perspectives: image fidelity, ability to remove inter-center differences, and influence on the downstream model. Main results. The improved image sharpness and structure fidelity are observed using the proposed harmonization pipeline. It largely decreases the number of features with a significant difference between two systems (from 64 to 45, lower than dualGAN: 57, cycleGAN: 59, ComBat: 64, and CLAHE: 54). In the downstream cervical cancer classification, it yields an area under the receiver operating characteristic curve of 0.894 (higher than dualGAN: 0.828, cycleGAN: 0.812, ComBat: 0.685, and CLAHE: 0.770). Significance. Our SSH method yields superior generality of downstream cervical cancer classification models by significantly decreasing the difference in radiomics features, and it achieves greater image fidelity.

Imaging approaches and radiomics: toward a new era of ultraprecision radioimmunotherapy?

Strong rationale and a growing number of preclinical and clinical studies support combining radiotherapy and immunotherapy to improve patient outcomes. However, several critical questions remain, such as the identification of patients who will benefit from immunotherapy and the identification of the best modalities of treatment to optimize patient response. Imaging biomarkers and radiomics have recently emerged as promising tools for the non-invasive assessment of the whole disease of the patient, allowing comprehensive analysis of the tumor microenvironment, the spatial heterogeneity of the disease and its temporal changes. This review presents the potential applications of medical imaging and the challenges to address, in order to help clinicians choose the optimal modalities of both radiotherapy and immunotherapy, to predict patient’s outcomes and to assess response to these promising combinations.