Performance of five research-domain automated WM lesion segmentation methods in a multi-center MS study

Intelligent cholinergic white matter pathways algorithm based on U-net reflects cognitive impairment in patients with silent cerebrovascular disease

BACKGROUND AND OBJECTIVE

The injury of the cholinergic white matter pathway underlies cognition decline in patients with silent cerebrovascular disease (SCD) with white matter hyperintensities (WMH) of vascular origin. However, the evaluation of the cholinergic white matter pathway is complex with poor consistency. We established an intelligent algorithm to evaluate WMH in the cholinergic pathway.

METHODS

Patients with SCD with WMH of vascular origin were enrolled. The Cholinergic Pathways Hyperintensities Scale (CHIPS) was used to measure cholinergic white matter pathway impairment. The intelligent algorithm used a deep learning model based on convolutional neural networks to achieve WMH segmentation and CHIPS scoring. The diagnostic value of the intelligent algorithm for moderate-to-severe cholinergic pathway injury was calculated. The correlation between the WMH in the cholinergic pathway and cognitive function was analysed.

RESULTS

A number of 464 patients with SCD were enrolled in internal training and test set. The algorithm was validated using data from an external cohort comprising 100 patients with SCD. The sensitivity, specificity and area under the curve of the intelligent algorithm to assess moderate and severe cholinergic white matter pathway injury were 91.7%, 87.3%, 0.903 (95% CI 0.861 to 0.952) and 86.5%, 81.3%, 0.868 (95% CI 0.819 to 0.921) for the internal test set and external validation set. for the. The general cognitive function, execution function and attention showed significant differences among the three groups of different CHIPS score (all p<0.05).

DISCUSSION

We have established the first intelligent algorithm to evaluate the cholinergic white matter pathway with good accuracy compared with the gold standard. It helps more easily assess the cognitive function in patients with SCD.

Different inflammatory signatures based on CSF biomarkers relate to preserved or diminished brain structure and cognition

Neuroinflammation is a hallmark of Alzheimer's disease (AD) and both positive and negative associations of individual inflammation-related markers with brain structure and cognitive function have been described. We aimed to identify inflammatory signatures of CSF immune-related markers that relate to changes of brain structure and cognition across the clinical spectrum ranging from normal aging to AD. A panel of 16 inflammatory markers, Aβ42/40 and p-tau181 were measured in CSF at baseline in the DZNE DELCODE cohort (n = 295); a longitudinal observational study focusing on at-risk stages of AD. Volumetric maps of gray and white matter (GM/WM; n = 261) and white matter hyperintensities (WMHs, n = 249) were derived from baseline MRIs. Cognitive decline (n = 204) and the rate of change in GM volume was measured in subjects with at least 3 visits (n = 175). A principal component analysis on the CSF markers revealed four inflammatory components (PCs). Of these, the first component PC1 (highly loading on sTyro3, sAXL, sTREM2, YKL-40, and C1q) was associated with older age and higher p-tau levels, but with less pathological Aβ when controlling for p-tau. PC2 (highly loading on CRP, IL-18, complement factor F/H and C4) was related to male gender, higher body mass index and greater vascular risk. PC1 levels, adjusted for AD markers, were related to higher GM and WM volumes, less WMHs, better baseline memory, and to slower atrophy rates in AD-related areas and less cognitive decline. In contrast, PC2 related to less GM and WM volumes and worse memory at baseline. Similar inflammatory signatures and associations were identified in the independent F.ACE cohort. Our data suggest that there are beneficial and detrimental signatures of inflammatory CSF biomarkers. While higher levels of TAM receptors (sTyro/sAXL) or sTREM2 might reflect a protective glia response to degeneration related to phagocytic clearance, other markers might rather reflect proinflammatory states that have detrimental impact on brain integrity.

Unraveling the link: white matter damage, gray matter atrophy and memory impairment in patients with subcortical ischemic vascular disease

Introduction

Prior MRI studies have shown that patients with subcortical ischemic vascular disease (SIVD) exhibited white matter damage, gray matter atrophy and memory impairment, but the specific characteristics and interrelationships of these abnormal changes have not been fully elucidated.

Materials and methods

We collected the MRI data and memory scores from 29 SIVD patients with cognitive impairment (SIVD-CI), 29 SIVD patients with cognitive unimpaired (SIVD-CU) and 32 normal controls (NC). Subsequently, the thicknesses and volumes of the gray matter regions that are closely related to memory function were automatically assessed using FreeSurfer software. Then, the volume, fractional anisotropy (FA), mean diffusivity (MD), amplitude of low-frequency fluctuation (ALFF) and regional homogeneity (ReHo) values of white matter hyperintensity (WMH) region and normal-appearing white matter (NAWM) were obtained using SPM, DPARSF, and FSL software. Finally, the analysis of covariance, spearman correlation and mediation analysis were used to analyze data.

Results

Compared with NC group, patients in SIVD-CI and SIVD-CU groups showed significantly abnormal volume, FA, MD, ALFF, and ReHo values of WMH region and NAWM, as well as significantly decreased volume and thickness values of gray matter regions, mainly including thalamus, middle temporal gyrus and hippocampal subfields such as cornu ammonis (CA) 1. These abnormal changes were significantly correlated with decreased visual, auditory and working memory scores. Compared with the SIVD-CU group, the significant reductions of the left CA2/3, right amygdala, right parasubiculum and NAWM volumes and the significant increases of the MD values in the WMH region and NAWM were found in the SIVD-CI group. And the increased MD values were significantly related to working memory scores. Moreover, the decreased CA1 and thalamus volumes mediated the correlations between the abnormal microstructure indicators in WMH region and the decreased memory scores in the SIVD-CI group.

Conclusion

Patients with SIVD had structural and functional damages in both WMH and NAWM, along with specific gray matter atrophy, which were closely related to memory impairment, especially CA1 atrophy and thalamic atrophy. More importantly, the volumes of some temporomesial regions and the MD values of WMH regions and NAWM may be potentially helpful neuroimaging indicators for distinguishing between SIVD-CI and SIVD-CU patients.

[Echo Train Length (ETL) of Fluid-attenuated Inversion Recovery (FLAIR) and Extraction Volume of White Matter Hyperintensity Volume in Automated White Matter Signal Analysis]

PURPOSE

To investigate whether the volume of white matter hyperintensity (WMH) extracted from FLAIR images changes when the imaging parameters of the original images are changed.

METHODS

Seven healthy volunteers were imaged by changing the imaging parameter ETL of FLAIR images, and WMHs were extracted and their volumes were calculated by the automatic extraction software. The results were statistically analyzed to examine the relationship (Experiment 1). Simulated images with different SNRs were created by adding white noise to four examples of healthy volunteer images. The SNR of the simulated images simulated the SNR of the measured images of different ETLs. The WMH was extracted from the simulated images and its volume was calculated using the automatic extraction software (Experiment 2).

RESULTS

Experiment 1 showed that there was no significant difference between FLAIR imaging parameters and WMH volume in automatic white matter signal analysis, except for some conditions. Experiment 2 showed that as the SNR of the original image decreased, the volume of high white matter signal extracted decreased.

CONCLUSION

In automatic white matter signal analysis, WMH was shown to be small when the ETL of the FLAIR sequence was larger than normal and/or the SNR of the image was low.

BIANCA-MS: An optimized tool for automated multiple sclerosis lesion segmentation

In this work we present BIANCA-MS, a novel tool for brain white matter lesion segmentation in multiple sclerosis (MS), able to generalize across both the wide spectrum of MRI acquisition protocols and the heterogeneity of manually labeled data. BIANCA-MS is based on the original version of BIANCA and implements two innovative elements: a harmonized setting, tested under different MRI protocols, which avoids the need to further tune algorithm parameters to each dataset; and a cleaning step developed to improve consistency in automated and manual segmentations, thus reducing unwanted variability in output segmentations and validation data. BIANCA-MS was tested on three datasets, acquired with different MRI protocols. First, we compared BIANCA-MS to other widely used tools. Second, we tested how BIANCA-MS performs in separate datasets. Finally, we evaluated BIANCA-MS performance on a pooled dataset where all MRI data were merged. We calculated the overlap using the DICE spatial similarity index (SI) as well as the number of false positive/negative clusters (nFPC/nFNC) in comparison to the manual masks processed with the cleaning step. BIANCA-MS clearly outperformed other available tools in both high- and low-resolution images and provided comparable performance across different scanning protocols, sets of modalities and image resolutions. BIANCA-MS performance on the pooled dataset (SI: 0.72 ± 0.25, nFPC: 13 ± 11, nFNC: 4 ± 8) were comparable to those achieved on each individual dataset (median across datasets SI: 0.72 ± 0.28, nFPC: 14 ± 11, nFNC: 4 ± 8). Our findings suggest that BIANCA-MS is a robust and accurate approach for automated MS lesion segmentation.

Cocaine use is associated with cerebral white matter hyperintensities in HIV disease

BACKGROUND

White matter hyperintensities (WMH), a marker of cerebral small vessel disease and predictor of cognitive decline, are observed at higher rates in persons with HIV (PWH). The use of cocaine, a potent central nervous system stimulant, is disproportionately common in PWH and may contribute to WMH.

METHODS

The sample included of 110 PWH on antiretroviral therapy. Fluid-attenuated inversion recovery (FLAIR) and T1-weighted anatomical MRI scans were collected, along with neuropsychological testing. FLAIR images were processed using the Lesion Segmentation Toolbox. A hierarchical regression model was run to investigate predictors of WMH burden [block 1: demographics; block 2: cerebrovascular disease (CVD) risk; block 3: lesion burden].

RESULTS

The sample was 20% female and 79% African American with a mean age of 45.37. All participants had persistent HIV viral suppression, and the median CD4+ T-cell count was 750. Nearly a third (29%) currently used cocaine regularly, with an average of 23.75 (SD = 20.95) days in the past 90. In the hierarchical linear regression model, cocaine use was a significant predictor of WMH burden (β = .28). WMH burden was significantly correlated with poorer cognitive function (r = -0.27). Finally, higher WMH burden was significantly associated with increased serum concentrations of interferon-γ-inducible protein 10 (IP-10) but lower concentrations of myeloperoxidase (MPO); however, these markers did not differ by COC status.

CONCLUSIONS

WMH burden is associated with poorer cognitive performance in PWH. Cocaine use and CVD risk independently contribute to WMH, and addressing these conditions as part of HIV care may mitigate brain injury underlying neurocognitive impairment.

How far MS lesion detection and segmentation are integrated into the clinical workflow? A systematic review

INTRODUCTION

Over the past few years, the deep learning community has developed and validated a plethora of tools for lesion detection and segmentation in Multiple Sclerosis (MS). However, there is an important gap between validating models technically and clinically. To this end, a six-step framework necessary for the development, validation, and integration of quantitative tools in the clinic was recently proposed under the name of the Quantitative Neuroradiology Initiative (QNI).

AIMS

Investigate to what extent automatic tools in MS fulfill the QNI framework necessary to integrate automated detection and segmentation into the clinical neuroradiology workflow.

METHODS

Adopting the systematic Cochrane literature review methodology, we screened and summarised published scientific articles that perform automatic MS lesions detection and segmentation. We categorised the retrieved studies based on their degree of fulfillment of QNI's six-steps, which include a tool's technical assessment, clinical validation, and integration.

RESULTS

We found 156 studies; 146/156 (94%) fullfilled the first QNI step, 155/156 (99%) the second, 8/156 (5%) the third, 3/156 (2%) the fourth, 5/156 (3%) the fifth and only one the sixth.

CONCLUSIONS

To date, little has been done to evaluate the clinical performance and the integration in the clinical workflow of available methods for MS lesion detection/segmentation. In addition, the socio-economic effects and the impact on patients' management of such tools remain almost unexplored.

The Heterogeneous Multiple Sclerosis Lesion: How Can We Assess and Modify a Degenerating Lesion?

Multiple sclerosis (MS) is a heterogeneous disease of the central nervous system that is governed by neural tissue loss and dystrophy during its progressive phase, with complex reactive pathological cellular changes. The immune-mediated mechanisms that promulgate the demyelinating lesions during relapses of acute episodes are not characteristic of chronic lesions during progressive MS. This has limited our capacity to target the disease effectively as it evolves within the central nervous system white and gray matter, thereby leaving neurologists without effective options to manage individuals as they transition to a secondary progressive phase. The current review highlights the molecular and cellular sequelae that have been identified as cooperating with and/or contributing to neurodegeneration that characterizes individuals with progressive forms of MS. We emphasize the need for appropriate monitoring via known and novel molecular and imaging biomarkers that can accurately detect and predict progression for the purposes of newly designed clinical trials that can demonstrate the efficacy of neuroprotection and potentially neurorepair. To achieve neurorepair, we focus on the modifications required in the reactive cellular and extracellular milieu in order to enable endogenous cell growth as well as transplanted cells that can integrate and/or renew the degenerative MS plaque.

Tract-based white matter hyperintensity patterns in patients with systemic lupus erythematosus using an unsupervised machine learning approach

Currently, little is known about the spatial distribution of white matter hyperintensities (WMH) in the brain of patients with Systemic Lupus erythematosus (SLE). Previous lesion markers, such as number and volume, ignore the strategic location of WMH. The goal of this work was to develop a fully-automated method to identify predominant patterns of WMH across WM tracts based on cluster analysis. A total of 221 SLE patients with and without neuropsychiatric symptoms from two different sites were included in this study. WMH segmentations and lesion locations were acquired automatically. Cluster analysis was performed on the WMH distribution in 20 WM tracts. Our pipeline identified five distinct clusters with predominant involvement of the forceps major, forceps minor, as well as right and left anterior thalamic radiations and the right inferior fronto-occipital fasciculus. The patterns of the affected WM tracts were consistent over the SLE subtypes and sites. Our approach revealed distinct and robust tract-based WMH patterns within SLE patients. This method could provide a basis, to link the location of WMH with clinical symptoms. Furthermore, it could be used for other diseases characterized by presence of WMH to investigate both the clinical relevance of WMH and underlying pathomechanism in the brain.

Higher white matter hyperintensity load adversely affects pre-post proximal cognitive training performance in healthy older adults

Cognitive training has shown promise for improving cognition in older adults. Age-related neuroanatomical changes may affect cognitive training outcomes. White matter hyperintensities are one common brain change in aging reflecting decreased white matter integrity. The current study assessed (1) proximal cognitive training performance following a 3-month randomized control trial and (2) the contribution of baseline whole-brain white matter hyperintensity load, or total lesion volume (TLV), on pre-post proximal training change. Sixty-two healthy older adults were randomized to either adaptive cognitive training or educational training control interventions. Repeated-measures analysis of covariance revealed two-way group × time interactions such that those assigned cognitive training demonstrated greater improvement on proximal composite (total training composite) and sub-composite (processing speed training composite, working memory training composite) measures compared to education training counterparts. Multiple linear regression showed higher baseline TLV associated with lower pre-post change on processing speed training sub-composite (β = -0.19, p = 0.04), but not other composite measures. These findings demonstrate the utility of cognitive training for improving post-intervention proximal performance in older adults. Additionally, pre-post proximal processing speed training change appears to be particularly sensitive to white matter hyperintensity load versus working memory training change. These data suggest that TLV may serve as an important factor for consideration when planning processing speed-based cognitive training interventions for remediation of cognitive decline in older adults.

Automatic brain lesion segmentation on standard magnetic resonance images: a scoping review

OBJECTIVES

Medical image analysis practices face challenges that can potentially be addressed with algorithm-based segmentation tools. In this study, we map the field of automatic MR brain lesion segmentation to understand the clinical applicability of prevalent methods and study designs, as well as challenges and limitations in the field.

DESIGN

Scoping review.

SETTING

Three databases (PubMed, IEEE Xplore and Scopus) were searched with tailored queries. Studies were included based on predefined criteria. Emerging themes during consecutive title, abstract, methods and whole-text screening were identified. The full-text analysis focused on materials, preprocessing, performance evaluation and comparison.

RESULTS

Out of 2990 unique articles identified through the search, 441 articles met the eligibility criteria, with an estimated growth rate of 10% per year. We present a general overview and trends in the field with regard to publication sources, segmentation principles used and types of lesions. Algorithms are predominantly evaluated by measuring the agreement of segmentation results with a trusted reference. Few articles describe measures of clinical validity.

CONCLUSIONS

The observed reporting practices leave room for improvement with a view to studying replication, method comparison and clinical applicability. To promote this improvement, we propose a list of recommendations for future studies in the field.

Frontal White Matter Hyperintensities and Executive Functioning Performance in Older Adults

Frontal lobe structures decline faster than most other brain regions in older adults. Age-related change in the frontal lobe is associated with poorer executive function (e.g., working memory, switching/set-shifting, and inhibitory control). The effects and presence of frontal lobe white matter hyperintensities (WMH) on executive function in normal aging is relatively unknown. The current study assessed relationships between region-specific frontal WMH load and cognitive performance in healthy older adults using three executive function tasks from the NIH Toolbox (NIHTB) Cognition Battery. A cohort of 279 healthy older adults ages 65-88 completed NIHTB and 3T T1-weighted and FLAIR MRI. Lesion Segmentation Toolbox quantified WMH volume and generated lesion probability maps. Individual lesion maps were registered to the Desikan-Killiany atlas in FreeSurfer 6.0 to define regions of interest (ROI). Independent linear regressions assessed relationships between executive function performance and region-specific WMH in frontal lobe ROIs. All models included age, sex, education, estimated total intracranial volume, multi-site scanner differences, and cardiovascular disease risk using Framingham criteria as covariates. Poorer set-shifting performance was associated with greater WMH load in three frontal ROIs including bilateral superior frontal (left β = -0.18, FDR-p = 0.02; right β = -0.20, FDR-p = 0.01) and right medial orbitofrontal (β = -0.17, FDR-p = 0.02). Poorer inhibitory performance associated with higher WMH load in one frontal ROI, the right superior frontal (right β = -0.21, FDR-p = 0.01). There were no significant associations between working memory and WMH in frontal ROIs. Our study demonstrates that location and pattern of frontal WMH may be important to assess when examining age-related differences in cognitive functions involving switching/set-shifting and inhibition. On the other hand, working memory performance was not related to presence of frontal WMH in this sample. These data suggest that WMH may contribute selectively to age-related declines in executive function. Findings emerged beyond predictors known to be associated with WMH presence, including age and cardiovascular disease risk. The spread of WMH within the frontal lobes may play a key role in the neuropsychological profile of cognitive aging. Further research should explore whether early intervention on modifiable vascular factors or cognitive interventions targeted for executive abilities may help mitigate the effect of frontal WMH on executive function.

Accuracy and reproducibility of automated white matter hyperintensities segmentation with lesion segmentation tool: A European multi-site 3T study

Brain vascular damage accumulate in aging and often manifest as white matter hyperintensities (WMHs) on MRI. Despite increased interest in automated methods to segment WMHs, a gold standard has not been achieved and their longitudinal reproducibility has been poorly investigated. The aim of present work is to evaluate accuracy and reproducibility of two freely available segmentation algorithms. A harmonized MRI protocol was implemented in 3T-scanners across 13 European sites, each scanning five volunteers twice (test-retest) using 2D-FLAIR. Automated segmentation was performed using Lesion segmentation tool algorithms (LST): the Lesion growth algorithm (LGA) in SPM8 and 12 and the Lesion prediction algorithm (LPA). To assess reproducibility, we applied the LST longitudinal pipeline to the LGA and LPA outputs for both the test and retest scans. We evaluated volumetric and spatial accuracy comparing LGA and LPA with manual tracing, and for reproducibility the test versus retest. Median volume difference between automated WMH and manual segmentations (mL) was -0.22[IQR = 0.50] for LGA-SPM8, -0.12[0.57] for LGA-SPM12, -0.09[0.53] for LPA, while the spatial accuracy (Dice Coefficient) was 0.29[0.31], 0.33[0.26] and 0.41[0.23], respectively. The reproducibility analysis showed a median reproducibility error of 20%[IQR = 41] for LGA-SPM8, 14% [31] for LGA-SPM12 and 10% [27] with the LPA cross-sectional pipeline. Applying the LST longitudinal pipeline, the reproducibility errors were considerably reduced (LGA: 0%[IQR = 0], p < 0.001; LPA: 0% [3], p < 0.001) compared to those derived using the cross-sectional algorithms. The DC using the longitudinal pipeline was excellent (median = 1) for LGA [IQR = 0] and LPA [0.02]. LST algorithms showed moderate accuracy and good reproducibility. Therefore, it can be used as a reliable cross-sectional and longitudinal tool in multi-site studies.

ExploreASL: An image processing pipeline for multi-center ASL perfusion MRI studies

Arterial spin labeling (ASL) has undergone significant development since its inception, with a focus on improving standardization and reproducibility of its acquisition and quantification. In a community-wide effort towards robust and reproducible clinical ASL image processing, we developed the software package ExploreASL, allowing standardized analyses across centers and scanners. The procedures used in ExploreASL capitalize on published image processing advancements and address the challenges of multi-center datasets with scanner-specific processing and artifact reduction to limit patient exclusion. ExploreASL is self-contained, written in MATLAB and based on Statistical Parameter Mapping (SPM) and runs on multiple operating systems. To facilitate collaboration and data-exchange, the toolbox follows several standards and recommendations for data structure, provenance, and best analysis practice. ExploreASL was iteratively refined and tested in the analysis of >10,000 ASL scans using different pulse-sequences in a variety of clinical populations, resulting in four processing modules: Import, Structural, ASL, and Population that perform tasks, respectively, for data curation, structural and ASL image processing and quality control, and finally preparing the results for statistical analyses on both single-subject and group level. We illustrate ExploreASL processing results from three cohorts: perinatally HIV-infected children, healthy adults, and elderly at risk for neurodegenerative disease. We show the reproducibility for each cohort when processed at different centers with different operating systems and MATLAB versions, and its effects on the quantification of gray matter cerebral blood flow. ExploreASL facilitates the standardization of image processing and quality control, allowing the pooling of cohorts which may increase statistical power and discover between-group perfusion differences. Ultimately, this workflow may advance ASL for wider adoption in clinical studies, trials, and practice.

Performance of five automated white matter hyperintensity segmentation methods in a multicenter dataset

White matter hyperintensities (WMHs) are a common manifestation of cerebral small vessel disease, that is increasingly studied with large, pooled multicenter datasets. This data pooling increases statistical power, but poses challenges for automated WMH segmentation. Although there is extensive literature on the evaluation of automated WMH segmentation methods, such evaluations in a multicenter setting are lacking. We performed WMH segmentations in sixty patients scanned on six different magnetic resonance imaging (MRI) scanners (10 patients per scanner) using five freely available and fully-automated WMH segmentation methods (Cascade, kNN-TTP, Lesion-TOADS, LST-LGA and LST-LPA). Different MRI scanner vendors and field strengths were included. We compared these automated WMH segmentations with manual WMH segmentations as a reference. Performance of each method both within and across scanners was assessed using spatial and volumetric correspondence with the reference segmentations by Dice's similarity coefficient (DSC) and intra-class correlation coefficient (ICC) respectively. We found the best performance, both within and across scanners, for kNN-TTP, followed by LST-LPA and LST-LGA, with worse performance for Lesion-TOADS and Cascade. Our findings can serve as a guide for choosing a method and also highlight the importance to further improve and evaluate consistency of methods in a multicenter setting.

Facing privacy in neuroimaging: removing facial features degrades performance of image analysis methods

Background

Recent studies have created awareness that facial features can be reconstructed from high-resolution MRI. Therefore, data sharing in neuroimaging requires special attention to protect participants’ privacy. Facial features removal (FFR) could alleviate these concerns. We assessed the impact of three FFR methods on subsequent automated image analysis to obtain clinically relevant outcome measurements in three clinical groups.

Methods

FFR was performed using QuickShear, FaceMasking, and Defacing. In 110 subjects of Alzheimer’s Disease Neuroimaging Initiative, normalized brain volumes (NBV) were measured by SIENAX. In 70 multiple sclerosis patients of the MAGNIMS Study Group, lesion volumes (WMLV) were measured by lesion prediction algorithm in lesion segmentation toolbox. In 84 glioblastoma patients of the PICTURE Study Group, tumor volumes (GBV) were measured by BraTumIA. Failed analyses on FFR-processed images were recorded. Only cases in which all image analyses completed successfully were analyzed. Differences between outcomes obtained from FFR-processed and full images were assessed, by quantifying the intra-class correlation coefficient (ICC) for absolute agreement and by testing for systematic differences using paired t tests.

Results

Automated analysis methods failed in 0–19% of cases in FFR-processed images versus 0–2% of cases in full images. ICC for absolute agreement ranged from 0.312 (GBV after FaceMasking) to 0.998 (WMLV after Defacing). FaceMasking yielded higher NBV (p = 0.003) and WMLV (p ≤ 0.001). GBV was lower after QuickShear and Defacing (both p < 0.001).

Conclusions

All three outcome measures were affected differently by FFR, including failure of analysis methods and both “random” variation and systematic differences. Further study is warranted to ensure high-quality neuroimaging research while protecting participants’ privacy.

Key Points

• Protecting participants’ privacy when sharing MRI data is important.

• Impact of three facial features removal methods on subsequent analysis was assessed in three clinical groups.

• Removing facial features degrades performance of image analysis methods.

Electronic supplementary material

The online version of this article (10.1007/s00330-019-06459-3) contains supplementary material, which is available to authorized users.

Comparing lesion segmentation methods in multiple sclerosis: Input from one manually delineated subject is sufficient for accurate lesion segmentation

PURPOSE

Accurate lesion segmentation is important for measurements of lesion load and atrophy in subjects with multiple sclerosis (MS). International MS lesion challenges show a preference of convolutional neural networks (CNN) strategies, such as nicMSlesions. However, since the software is trained on fairly homogenous training data, we aimed to test the performance of nicMSlesions in an independent dataset with manual and other automatic lesion segmentations to determine whether this method is suitable for larger, multi-center studies.

METHODS

Manual lesion segmentation was performed in fourteen subjects with MS on sagittal 3D FLAIR images from a 3T GE whole-body scanner with 8-channel head coil. We compared five different categories of automated lesion segmentation methods for their volumetric and spatial agreement with manual segmentation: (i) unsupervised, untrained (LesionTOADS); (ii) supervised, untrained (LST-LPA and nicMSlesions with default settings); (iii) supervised, untrained with threshold adjustment (LST-LPA optimized for current data); (iv) supervised, trained with leave-one-out cross-validation on fourteen subjects with MS (nicMSlesions and BIANCA); and (v) supervised, trained on a single subject with MS (nicMSlesions). Volumetric accuracy was determined by the intra-class correlation coefficient (ICC) and spatial accuracy by Dice's similarity index (SI). Volumes and SI were compared between methods using repeated measures ANOVA or Friedman tests with post-hoc pairwise comparison.

RESULTS

The best volumetric and spatial agreement with manual was obtained with the supervised and trained methods nicMSlesions and BIANCA (ICC absolute agreement > 0.968 and median SI > 0.643) and the worst with the unsupervised, untrained method LesionTOADS (ICC absolute agreement = 0.140 and median SI = 0.444). Agreement with manual in the single-subject network training of nicMSlesions was poor for input with low lesion volumes (i.e. two subjects with lesion volumes ≤ 3.0 ml). For the other twelve subjects, ICC varied from 0.593 to 0.973 and median SI varied from 0.535 to 0.606. In all cases, the single-subject trained nicMSlesions segmentations outperformed LesionTOADS, and in almost all cases it also outperformed LST-LPA.

CONCLUSION

Input from only one subject to re-train the deep learning CNN nicMSlesions is sufficient for adequate lesion segmentation, with on average higher volumetric and spatial agreement with manual than obtained with the untrained methods LesionTOADS and LST-LPA.

Validation and comparison of two automated methods for quantifying brain white matter hyperintensities of presumed vascular origin

Objectives

White matter hyperintensities (WMH) are a common imaging finding indicative of cerebral small vessel disease. Lesion segmentation algorithms have been developed to overcome issues arising from visual rating scales. In this study, we evaluated two automated methods and compared them to visual and manual segmentation to determine the most robust algorithm provided by the open-source Lesion Segmentation Toolbox (LST).

Methods

We compared WMH data from visual ratings (Scheltens’ scale) with those derived from algorithms provided within LST. We then compared spatial and volumetric WMH data derived from manually-delineated lesion maps with WMH data and lesion maps provided by the LST algorithms.

Results

We identified optimal initial thresholds for algorithms provided by LST compared with visual ratings (Lesion Growth Algorithm (LGA): initial κ and lesion probability thresholds, 0.5; Lesion Probability Algorithm (LPA) lesion probability threshold, 0.65). LGA was found to perform better then LPA compared with manual segmentation.

Conclusion

LGA appeared to be the most suitable algorithm for quantifying WMH in relation to cerebral small vessel disease, compared with Scheltens’ score and manual segmentation. LGA offers a user-friendly, effective WMH segmentation method in the research environment.

FLAIR2 improves LesionTOADS automatic segmentation of multiple sclerosis lesions in non-homogenized, multi-center, 2D clinical magnetic resonance images

Background

Accurate segmentation of MS lesions on MRI is difficult and, if performed manually, time consuming. Automatic segmentations rely strongly on the image contrast and signal-to-noise ratio. Literature examining segmentation tool performances in real-world multi-site data acquisition settings is scarce.

Objective

FLAIR², a combination of T₂-weighted and fluid attenuated inversion recovery (FLAIR) images, improves tissue contrast while suppressing CSF. We compared the use of FLAIR and FLAIR² in LesionTOADS, OASIS and the lesion segmentation toolbox (LST) when applied to non-homogenized, multi-center 2D-imaging data.

Methods

Lesions were segmented on 47 MS patient data sets obtained from 34 sites using LesionTOADS, OASIS and LST, and compared to a semi-automatically generated reference. The performance of FLAIR and FLAIR² was assessed using the relative lesion volume difference (LVD), Dice coefficient (DSC), sensitivity (SEN) and symmetric surface distance (SSD). Performance improvements related to lesion volumes (LVs) were evaluated for all tools. For comparison, LesionTOADS was also used to segment lesions from 3 T single-center MR data of 40 clinically isolated syndrome (CIS) patients.

Results

Compared to FLAIR, the use of FLAIR² in LesionTOADS led to improvements of 31.6% (LVD), 14.0% (DSC), 25.1% (SEN), and 47.0% (SSD) in the multi-center study. DSC and SSD significantly improved for larger LVs, while LVD and SEN were enhanced independent of LV. OASIS showed little difference between FLAIR and FLAIR², likely due to its inherent use of T₂w and FLAIR. LST replicated the benefits of FLAIR² only in part, indicating that further optimization, particularly at low LVs is needed. In the CIS study, LesionTOADS did not benefit from the use of FLAIR² as the segmentation performance for both FLAIR and FLAIR² was heterogeneous.

Conclusions

In this real-world, multi-center experiment, FLAIR² outperformed FLAIR in its ability to segment MS lesions with LesionTOADS. The computation of FLAIR² enhanced lesion detection, at minimally increased computational time or cost, even retrospectively. Further work is needed to determine how LesionTOADS and other tools, such as LST, can optimally benefit from the improved FLAIR² contrast.

•

FLAIR² improves automatic MS lesion segmentation with LesionTOADS compared to FLAIR.

•

Segmentation similarity improves for higher lesion volumes, particularly for FLAIR².

•

FLAIR² provides greater sensitivity independent of lesion volume than FLAIR alone.

•

Other segmentation tools need further optimization to fully benefit from FLAIR².

•

FLAIR² provides immediate benefits at 1.5 T and visually improves segmentation at 3 T.

Brain and lesion segmentation in multiple sclerosis using fully convolutional neural networks: A large-scale study

OBJECTIVE

To investigate the performance of deep learning (DL) based on fully convolutional neural network (FCNN) in segmenting brain tissues in a large cohort of multiple sclerosis (MS) patients.

METHODS

We developed a FCNN model to segment brain tissues, including T2-hyperintense MS lesions. The training, validation, and testing of FCNN were based on ~1000 magnetic resonance imaging (MRI) datasets acquired on relapsing-remitting MS patients, as a part of a phase 3 randomized clinical trial. Multimodal MRI data (dual-echo, FLAIR, and T1-weighted images) served as input to the network. Expert validated segmentation was used as the target for training the FCNN. We cross-validated our results using the leave-one-center-out approach.

RESULTS

We observed a high average (95% confidence limits) Dice similarity coefficient for all the segmented tissues: 0.95 (0.92-0.98) for white matter, 0.96 (0.93-0.98) for gray matter, 0.99 (0.98-0.99) for cerebrospinal fluid, and 0.82 (0.63-1.0) for T2 lesions. High correlations between the DL segmented tissue volumes and ground truth were observed (R² > 0.92 for all tissues). The cross validation showed consistent results across the centers for all tissues.

CONCLUSION

The results from this large-scale study suggest that deep FCNN can automatically segment MS brain tissues, including lesions, with high accuracy.

Validation and Optimization of BIANCA for the Segmentation of Extensive White Matter Hyperintensities

White matter hyperintensities (WMH) are a hallmark of small vessel diseases (SVD). Yet, no automated segmentation method is readily and widely used, especially in patients with extensive WMH where lesions are close to the cerebral cortex. BIANCA (Brain Intensity AbNormality Classification Algorithm) is a new fully automated, supervised method for WMH segmentation. In this study, we optimized and compared BIANCA against a reference method with manual editing in a cohort of patients with extensive WMH. This was achieved in two datasets: a clinical protocol with 90 patients having 2-dimensional FLAIR and an advanced protocol with 66 patients having 3-dimensional FLAIR. We first determined simultaneously which input modalities (FLAIR alone or FLAIR + T1) and which training sets were better compared to the reference. Three strategies for the selection of the threshold that is applied to the probabilistic output of BIANCA were then evaluated: chosen at the group level, based on Fazekas score or determined individually. Accuracy of the segmentation was assessed through measures of spatial agreement and volumetric correspondence with respect to reference segmentation. Based on all our tests, we identified multimodal inputs (FLAIR + T1), mixed WMH load training set and individual threshold selection as the best conditions to automatically segment WMH in our cohort. A median Dice similarity index of 0.80 (0.80) and an intraclass correlation coefficient of 0.97 (0.98) were obtained for the clinical (advanced) protocol. However, Bland-Altman plots identified a difference with the reference method that was linearly related to the total burden of WMH. Our results suggest that BIANCA is a reliable and fast segmentation method to extract masks of WMH in patients with extensive lesions.

Unraveling treatment response in multiple sclerosis: A clinical and MRI challenge

Over the last few decades, the improved diagnostic criteria, the wide use of MRI, and the growing availability of effective pharmacologic treatments have led to substantial advances in the management of multiple sclerosis (MS). The importance of early diagnosis and treatment is now well-established, but there is still no consensus on how to define and monitor response to MS treatments. In particular, the clinical relevance of the detection of minimal MRI activity is controversial and recommendations on how to define and monitor treatment response are warranted. An expert panel of the Magnetic Resonance Imaging in MS Study Group analyzed and discussed published studies on treatment response in MS. The evolving concept of no evidence of disease activity and its effect on predicting long-term prognosis was examined, including the option of defining a more realistic target for daily clinical practice: minimal evidence of disease activity. Advantages and disadvantages associated with the use of MRI activity alone and quantitative scoring systems combining on-treatment clinical relapses and MRI active lesions to detect treatment response in the real-world setting were also discussed. While most published studies on this topic involved patients treated with interferon-β, special attention was given to more recent studies providing evidence based on treatment with other and more efficacious oral and injectable drugs. Finally, the panel identified future directions to pursue in this research field.