Reproducibility of Brain Morphometry from Short-Term Repeat Clinical MRI Examinations: A Retrospective Study

Comparing Automated Subcortical Volume Estimation Methods; Amygdala Volumes Estimated by FSL and FreeSurfer Have Poor Consistency

Subcortical volumes are a promising source of biomarkers and features in biosignatures, and automated methods facilitate extracting them in large, phenotypically rich datasets. However, while extensive research has verified that the automated methods produce volumes that are similar to those generated by expert annotation, the consistency of methods with each other is understudied. Using data from the UK Biobank, we compare the estimates of subcortical volumes produced by two popular software suites: FSL and FreeSurfer. Although most subcortical volumes exhibit good to excellent consistency across the methods, the tools produce diverging estimates of amygdalar volume. Through simulation, we show that this poor consistency can lead to conflicting results, where one but not the other tool suggests statistical significance, or where both tools suggest a significant relationship but in opposite directions. Considering these issues, we discuss several ways in which care should be taken when reporting on relationships involving amygdalar volume.

Validation of cross-sectional and longitudinal ComBat harmonization methods for magnetic resonance imaging data on a travelling subject cohort

Background

The growth in multi-center neuroimaging studies generated a need for methods that mitigate the differences in hardware and acquisition protocols across sites i.e., scanner effects. ComBat harmonization methods have shown promise but have not yet been tested on all the data types commonly studied with magnetic resonance imaging (MRI). This study aimed to validate neuroCombat, longCombat and gamCombat on both structural and diffusion metrics in both cross-sectional and longitudinal data.

Methods

We used a travelling subject design whereby 73 healthy volunteers contributed 161 scans across two sites and four machines using one T1 and five diffusion MRI protocols. Scanner was defined as a composite of site, machine and protocol. A common pipeline extracted two structural metrics (volumes and cortical thickness) and two diffusion tensor imaging metrics (mean diffusivity and fractional anisotropy) for seven regions of interest including gray and (except for cortical thickness) white matter regions.

Results

Structural data exhibited no significant scanner effect and therefore did not benefit from harmonization in our particular cohort. Indeed, attempting harmonization obscured the true biological effect for some regions of interest. Diffusion data contained marked scanner effects and was successfully harmonized by all methods, resulting in smaller scanner effects and better detection of true biological effects. LongCombat less effectively reduced the scanner effect for cross-sectional white matter data but had a slightly lower probability of incorrectly finding group differences in simulations, compared to neuroCombat and gamCombat. False positive rates for all methods and all metrics did not significantly exceed 5%.

Conclusions

Statistical harmonization of structural data is not always necessary and harmonization in the absence of a scanner effect may be harmful. Harmonization of diffusion MRI data is highly recommended with neuroCombat, longCombat and gamCombat performing well in cross-sectional and longitudinal settings.

Cortical thickness in Parkinson's disease: a coordinate-based meta-analysis

Parkinson's disease (PD) is a common age-related neurodegenerative disease that affects the structural architecture of the cerebral cortex. Cortical thickness (CTh) via surface-based morphometry (SBM) analysis is a popular measure to assess brain structural alterations in the gray matter in PD. However, the results of CTh analysis in PD lack consistency and have not been systematically reviewed. We conducted a comprehensive coordinate-based meta-analysis (CBMA) of 38 CTh studies (57 comparison datasets) in 1,843 patients with PD using the latest seed-based d mapping software. Compared with 1,172 healthy controls, no significantly consistent CTh alterations were found in patients with PD, suggesting CTh as an unreliable neuroimaging marker for PD. The lack of consistent CTh alterations in PD could be ascribed to the heterogeneity in clinical populations, variations in imaging methods, and underpowered small sample sizes. These results highlight the need to control for potential confounding factors to produce robust and reproducible CTh results in PD.

Cortical Thickness in Migraine: A Coordinate-Based Meta-Analysis

Cortical thickness (CTh) via surface-based morphometry analysis is a popular method to characterize brain morphometry. Many studies have been performed to investigate CTh abnormalities in migraine. However, the results from these studies were not consistent and even conflicting. These divergent results hinder us to obtain a clear picture of brain morphometry regarding CTh alterations in migraine. Coordinate-based meta-analysis (CBMA) is a promising technique to quantitatively pool individual neuroimaging studies to identify consistent brain areas involved. Electronic databases (PubMed, EMBASE, Web of Science, China National Knowledge Infrastructure, WanFang, and SinoMed) and other sources (bioRxiv and reference lists of relevant articles and reviews) were systematically searched for studies that compared regional CTh differences between patients with migraine and healthy controls (HCs) up to May 15, 2020. A CBMA was performed using the Seed-based d Mapping with Permutation of Subject Images approach. In total, we identified 16 studies with 17 datasets reported that were eligible for the CBMA. The 17 datasets included 872 patients with migraine (average sample size 51.3, mean age 39.6 years, 721 females) and 949 HCs (average sample size 59.3, mean age 44.2 years, 680 females). The CBMA detected no statistically significant consistency of CTh alterations in patients with migraine relative to HCs. Sensitivity analysis and subgroup analysis verified this result to be robust. Metaregression analyses revealed that this CBMA result was not confounded by age, gender, aura, attack frequency per month, and illness duration. Our CBMA adds to the evidence of the replication crisis in neuroimaging research that is increasingly recognized. Many potential confounders, such as underpowered sample size, heterogeneous patient selection criteria, and differences in imaging collection and methodology, may contribute to the inconsistencies of CTh alterations in migraine, which merit attention before planning future research on this topic.

A Review of Publicly Available Automatic Brain Segmentation Methodologies, Machine Learning Models, Recent Advancements, and Their Comparison

BACKGROUND

The noninvasive study of the structure and functions of the brain using neuroimaging techniques is increasingly being used for its clinical and research perspective. The morphological and volumetric changes in several regions and structures of brains are associated with the prognosis of neurological disorders such as Alzheimer's disease, epilepsy, schizophrenia, etc. and the early identification of such changes can have huge clinical significance. The accurate segmentation of three-dimensional brain magnetic resonance images into tissue types (i.e., grey matter, white matter, cerebrospinal fluid) and brain structures, thus, has huge importance as they can act as early biomarkers. The manual segmentation though considered the "gold standard" is time-consuming, subjective, and not suitable for bigger neuroimaging studies. Several automatic segmentation tools and algorithms have been developed over the years; the machine learning models particularly those using deep convolutional neural network (CNN) architecture are increasingly being applied to improve the accuracy of automatic methods.

PURPOSE

The purpose of the study is to understand the current and emerging state of automatic segmentation tools, their comparison, machine learning models, their reliability, and shortcomings with an intent to focus on the development of improved methods and algorithms.

METHODS

The study focuses on the review of publicly available neuroimaging tools, their comparison, and emerging machine learning models particularly those based on CNN architecture developed and published during the last five years.

CONCLUSION

Several software tools developed by various research groups and made publicly available for automatic segmentation of the brain show variability in their results in several comparison studies and have not attained the level of reliability required for clinical studies. The machine learning models particularly three dimensional fully convolutional network models can provide a robust and efficient alternative with relation to publicly available tools but perform poorly on unseen datasets. The challenges related to training, computation cost, reproducibility, and validation across distinct scanning modalities for machine learning models need to be addressed.

MAGNIMS consensus recommendations on the use of brain and spinal cord atrophy measures in clinical practice

Early evaluation of treatment response and prediction of disease evolution are key issues in the management of people with multiple sclerosis (MS). In the past 20 years, MRI has become the most useful paraclinical tool in both situations and is used clinically to assess the inflammatory component of the disease, particularly the presence and evolution of focal lesions - the pathological hallmark of MS. However, diffuse neurodegenerative processes that are at least partly independent of inflammatory mechanisms can develop early in people with MS and are closely related to disability. The effects of these neurodegenerative processes at a macroscopic level can be quantified by estimation of brain and spinal cord atrophy with MRI. MRI measurements of atrophy in MS have also been proposed as a complementary approach to lesion assessment to facilitate the prediction of clinical outcomes and to assess treatment responses. In this Consensus statement, the Magnetic Resonance Imaging in MS (MAGNIMS) study group critically review the application of brain and spinal cord atrophy in clinical practice in the management of MS, considering the role of atrophy measures in prognosis and treatment monitoring and the barriers to clinical use of these measures. On the basis of this review, the group makes consensus statements and recommendations for future research.

The effect of the MR pulse sequence on the regional corpus callosum morphometry

Background and purposes

Brain morphometry is an important assessment technique to assess certain morphological brain features of various brain regions, which can be quantified in vivo by using high-resolution structural magnetic resonance (MR) imaging. This study aims to investigate the effect of different types of pulse sequence on regional corpus callosum (CC) morphometry analysis.

Materials and methods

Twenty-one healthy volunteers were scanned twice on the same 3T MRI scanner (Magnetom Trio, Siemens, Erlangen, Germany) equipped with an 8-channel head coil. Two different MR pulse sequences were applied to acquire high-resolution 3D T1-weighted images: magnetization-prepared rapid gradient-echo (MP-RAGE) and modified driven equilibrium Fourier transform (MDEFT) pulse sequence. Image quality measurements such as SNR, contrast-to-noise ratio, and relative contrast were calculated for each pulse sequence images independently. The values of corpus callosum volume were calculated based on the vertex of reconstructed surfaces. The paired dependent t test was applied to compare the means of two matched groups.

Results

Three sub-regional CC, namely anterior, mid-anterior, and posterior, resulted in an estimated volume difference between MDEFT and MP-RAGE pulse sequences. Central and mid-posterior sub-regional CC volume resulted in not significant difference between the two named pulse sequences.

Conclusion

The findings of this study demonstrate that combining data from different pulse sequences in a multisite study could make some variations in the results.

Automated segmentation of cerebral deep gray matter from MRI scans: effect of field strength on sensitivity and reliability

Background

The cerebral subcortical deep gray matter nuclei (DGM) are a common, early, and clinically-relevant site of atrophy in multiple sclerosis (MS). Robust and reliable DGM segmentation could prove useful to evaluate putative neuroprotective MS therapies. The objective of the study was to compare the sensitivity and reliability of DGM volumes obtained from 1.5T vs. 3T MRI.

Methods

Fourteen patients with MS [age (mean, range) 50.2 (32.0–60.8) years, disease duration 18.4 (8.2–35.5) years, Expanded Disability Status Scale score 3.1 (0–6), median 3.0] and 15 normal controls (NC) underwent brain 3D T1-weighted paired scan-rescans at 1.5T and 3T. DGM (caudate, thalamus, globus pallidus, and putamen) segmentation was obtained by the fully automated FSL-FIRST pipeline. Both raw and normalized volumes were derived.

Results

DGM volumes were generally higher at 3T vs. 1.5T in both groups. For raw volumes, 3T showed slightly better sensitivity (thalamus: p = 0.02; caudate: p = 0.10; putamen: p = 0.02; globus pallidus: p = 0.0004; total DGM: p = 0.01) than 1.5T (thalamus: p = 0.05; caudate: p = 0.09; putamen: p = 0.03; globus pallidus: p = 0.0006; total DGM: p = 0.02) for detecting DGM atrophy in MS vs. NC. For normalized volumes, 3T but not 1.5T detected atrophy in the globus pallidus in the MS group. Across all subjects, scan-rescan reliability was generally very high for both platforms, showing slightly higher reliability for some DGM volumes at 3T. Raw volumes showed higher reliability than normalized volumes. Raw DGM volume showed higher reliability than the individual structures.

Conclusions

These results suggest somewhat higher sensitivity and reliability of DGM volumes obtained from 3T vs. 1.5T MRI. Further studies should assess the role of this 3T pipeline in tracking potential MS neurotherapeutic effects.

Reproducibility of quantitative structural and physiological MRI measurements

INTRODUCTION

Quantitative longitudinal magnetic resonance imaging and spectroscopy (MRI/S) is used to assess progress of brain disorders and treatment effects. Understanding the significance of MRI/S changes requires knowledge of the inherent technical and physiological consistency of these measurements. This longitudinal study examined the variance and reproducibility of commonly used quantitative MRI/S measurements in healthy subjects while controlling physiological and technical parameters.

METHODS

Twenty-five subjects were imaged three times over 5 days on a Siemens 3T Verio scanner equipped with a 32-channel phase array coil. Structural (T1, T2-weighted, and diffusion-weighted imaging) and physiological (pseudocontinuous arterial spin labeling, proton magnetic resonance spectroscopy) data were collected. Consistency of repeated images was evaluated with mean relative difference, mean coefficient of variation, and intraclass correlation (ICC). Finally, a "reproducibility rating" was calculated based on the number of subjects needed for a 3% and 10% difference.

RESULTS

Structural measurements generally demonstrated excellent reproducibility (ICCs 0.872-0.998) with a few exceptions. Moderate-to-low reproducibility was observed for fractional anisotropy measurements in fornix and corticospinal tracts, for cortical gray matter thickness in the entorhinal, insula, and medial orbitofrontal regions, and for the count of the periependymal hyperintensive white matter regions. The reproducibility of physiological measurements ranged from excellent for most of the magnetic resonance spectroscopy measurements to moderate for permeability-diffusivity coefficients in cingulate gray matter to low for regional blood flow in gray and white matter.

DISCUSSION

This study demonstrates a high degree of longitudinal consistency across structural and physiological measurements in healthy subjects, defining the inherent variability in these commonly used sequences. Additionally, this study identifies those areas where caution should be exercised in interpretation. Understanding this variability can serve as the basis for interpretation of MRI/S data in the assessment of neurological disorders and treatment effects.