One-shot domain adaptation in multiple sclerosis lesion segmentation using convolutional neural networks

Slowly expanding lesions are associated with disease activity and gray matter loss in relapse-onset multiple sclerosis

BACKGROUND AND PURPOSE

Slowly expanding lesions (SELs) have been proposed as novel MRI markers of chronic active lesions in multiple sclerosis (MS). However, the mechanism through which SELs affect brain volume loss in patients with MS remains unknown. Additionally, the prevalence and significance of SELs in Asian patients with MS remain unclear. This study aimed to investigate the association between SELs and no evidence of disease activity (NEDA)-3 status as well as brain volume loss in Japanese patients.

METHODS

A total of 99 patients with relapse-onset MS were retrospectively evaluated. SELs were identified on brain MRI based on local deformation when consecutive scans were registered longitudinally. We developed a logistic regression model and generalized linear mixed models (GLMMs) to evaluate the association between the number of SELs and disease activity and changes in brain volume.

RESULTS

During the observation period (2.0 ± 0.22 years), 35 patients developed at least one SEL. Multivariable logistic regression analysis showed that ≥2 SELs were associated with 0.2 times the risk of achieving a NEDA-3 status. GLMMs revealed that the number of SELs was negatively associated with volume changes in the cortex (p = .00169) and subcortical gray matter (p = .00964) after correction for multiple comparisons.

CONCLUSION

SELs were identified in Japanese patients with MS during the 2-year observation period. The number of SELs is associated with disease activity and brain volume loss, suggesting that the number of SELs could be a biomarker of disease activity in MS.

Consensus of algorithms for lesion segmentation in brain MRI studies of multiple sclerosis

Segmentation of multiple sclerosis (MS) lesions on brain MRI scans is crucial for diagnosis, disease and treatment monitoring but is a time-consuming task. Despite several automated algorithms have been proposed, there is still no consensus on the most effective method. Here, we applied a consensus-based framework to improve lesion segmentation on T1-weighted and FLAIR scans. The framework is designed to combine publicly available state-of-the-art deep learning models, by running multiple segmentation tasks before merging the outputs of each algorithm. To assess the effectiveness of the approach, we applied it to MRI datasets from two different centers, including a private and a public dataset, with 131 and 30 MS patients respectively, with manually segmented lesion masks available. No further training was performed for any of the included algorithms. Overlap and detection scores were improved, with Dice increasing by 4-8% and precision by 3-4% respectively for the private and public dataset. High agreement was obtained between estimated and true lesion load (ρ = 0.92 and ρ = 0.97) and count (ρ = 0.83 and ρ = 0.94). Overall, this framework ensures accurate and reliable results, exploiting complementary features and overcoming some of the limitations of individual algorithms.

Investigating the relationship between thalamic iron concentration and disease severity in secondary progressive multiple sclerosis using quantitative susceptibility mapping: Cross-sectional analysis from the MS-STAT2 randomised controlled trial

Background

Deep grey matter pathology is a key driver of disability worsening in people with multiple sclerosis. Quantitative susceptibility mapping (QSM) is an advanced magnetic resonance imaging (MRI) technique which quantifies local magnetic susceptibility from variations in phase produced by changes in the local magnetic field. In the deep grey matter, susceptibility has previously been validated against tissue iron concentration. However, it currently remains unknown whether susceptibility is abnormal in older progressive MS cohorts, and whether it correlates with disability.

Objectives

To investigate differences in mean regional susceptibility in deep grey matter between people with secondary progressive multiple sclerosis (SPMS) and healthy controls; to examine in patients the relationships between deep grey matter susceptibility and clinical and imaging measures of disease severity.

Methods

Baseline data from a subgroup of the MS-STAT2 trial (simvastatin vs. placebo in SPMS, NCT03387670) were included. The subgroup underwent clinical assessments and an advanced MRI protocol at 3T. A cohort of age-matched healthy controls underwent the same MRI protocol. Susceptibility maps were reconstructed using a robust QSM pipeline from multi-echo 3D gradient-echo sequence. Regions of interest (ROIs) in the thalamus, globus pallidus and putamen were segmented from 3D T1-weighted images, and lesions segmented from 3D fluid-attenuated inversion recovery images. Linear regression was used to compare susceptibility from ROIs between patients and controls, adjusting for age and sex. Where significant differences were found, we further examined the associations between ROI susceptibility and clinical and imaging measures of MS severity.

Results

149 SPMS (77% female; mean age: 53 yrs; median Expanded Disability Status Scale (EDSS): 6.0 [interquartile range 4.5-6.0]) and 33 controls (52% female, mean age: 57) were included.Thalamic susceptibility was significantly lower in SPMS compared to controls: mean (SD) 28.6 (12.8) parts per billion (ppb) in SPMS vs. 39.2 (12.7) ppb in controls; regression coefficient: -12.0 [95% confidence interval: -17.0 to -7.1], p < 0.001. In contrast, globus pallidus and putamen susceptibility were similar between both groups.In SPMS, a 10 ppb lower thalamic susceptibility was associated with a +0.13 [+0.01 to +0.24] point higher EDSS (p < 0.05), a -2.4 [-3.8 to -1.0] point lower symbol digit modality test (SDMT, p = 0.001), and a -2.4 [-3.7 to -1.1] point lower Sloan low contrast acuity, 2.5% (p < 0.01).Lower thalamic susceptibility was also strongly associated with a higher T2 lesion volume (T2LV, p < 0.001) and lower normalised whole brain, deep grey matter and thalamic volumes (all p < 0.001).

Conclusions

The reduced thalamic susceptibility found in SPMS compared to controls suggests that thalamic iron concentrations are lower at this advanced stage of the disease. The observed relationships between lower thalamic susceptibility and more severe physical, cognitive and visual disability suggests that reductions in thalamic iron may correlate with important mechanisms of clinical disease progression. Such mechanisms appear to intimately link reductions in thalamic iron with higher T2LV and the development of thalamic atrophy, encouraging further research into QSM-derived thalamic susceptibility as a biomarker of disease severity in SPMS.

Digital outcome measures are associated with brain atrophy in patients with multiple sclerosis

BACKGROUND

Digital monitoring of people with multiple sclerosis (PwMS) using smartphone-based monitoring tools is a promising method to assess disease activity and progression.

OBJECTIVE

To study cross-sectional and longitudinal associations between active and passive digital monitoring parameters and MRI volume measures in PwMS.

METHODS

In this prospective study, 92 PwMS were included. Clinical tests [Expanded Disability Status Scale (EDSS), Timed 25 Foot Walk test (T25FW), 9-Hole Peg Test (NHPT), and Symbol Digit Modalities Test (SDMT)] and structural MRI scans were performed at baseline (M0) and 12-month follow-up (M12). Active monitoring included the smartphone-based Symbol Digit Modalities Test (sSDMT) and 2 Minute Walk Test (s2MWT), while passive monitoring was based on smartphone keystroke dynamics (KD). Linear regression analyses were used to determine cross-sectional and longitudinal relations between digital and clinical outcomes and brain volumes, with age, disease duration and sex as covariates.

RESULTS

In PwMS, both sSDMT and SDMT were associated with thalamic volumes and lesion volumes. KD were related to brain, ventricular, thalamic and lesion volumes. No relations were found between s2MWT and MRI volumes. NHPT scores were associated with lesion volumes only, while EDSS and T25FW were not related to MRI. No longitudinal associations were found for any of the outcome measures between M0 and M12.

CONCLUSION

Our results show clear cross-sectional correlations between digital biomarkers and brain volumes in PwMS, which were not all present for conventional clinical outcomes, supporting the potential added value of digital monitoring tools.

Assessing the Accuracy and Reproducibility of PARIETAL: A Deep Learning Brain Extraction Algorithm

BACKGROUND

Manual brain extraction from magnetic resonance (MR) images is time-consuming and prone to intra- and inter-rater variability. Several automated approaches have been developed to alleviate these constraints, including deep learning pipelines. However, these methods tend to reduce their performance in unseen magnetic resonance imaging (MRI) scanner vendors and different imaging protocols.

PURPOSE

To present and evaluate for clinical use PARIETAL, a pre-trained deep learning brain extraction method. We compare its reproducibility in a scan/rescan analysis and its robustness among scanners of different manufacturers.

STUDY TYPE

Retrospective.

POPULATION

Twenty-one subjects (12 women) with age range 22-48 years acquired using three different MRI scanner machines including scan/rescan in each of them.

FIELD STRENGTH/SEQUENCE

T1-weighted images acquired in a 3-T Siemens with magnetization prepared rapid gradient-echo sequence and two 1.5 T scanners, Philips and GE, with spin-echo and spoiled gradient-recalled (SPGR) sequences, respectively.

ASSESSMENT

Analysis of the intracranial cavity volumes obtained for each subject on the three different scanners and the scan/rescan acquisitions.

STATISTICAL TESTS

Parametric permutation tests of the differences in volumes to rank and statistically evaluate the performance of PARIETAL compared to state-of-the-art methods.

RESULTS

The mean absolute intracranial volume differences obtained by PARIETAL in the scan/rescan analysis were 1.88 mL, 3.91 mL, and 4.71 mL for Siemens, GE, and Philips scanners, respectively. PARIETAL was the best-ranked method on Siemens and GE scanners, while decreasing to Rank 2 on the Philips images. Intracranial differences for the same subject between scanners were 5.46 mL, 27.16 mL, and 30.44 mL for GE/Philips, Siemens/Philips, and Siemens/GE comparison, respectively. The permutation tests revealed that PARIETAL was always in Rank 1, obtaining the most similar volumetric results between scanners.

DATA CONCLUSION

PARIETAL accurately segments the brain and it generalizes to images acquired at different sites without the need of training or fine-tuning it again. PARIETAL is publicly available.

LEVEL OF EVIDENCE

2 TECHNICAL EFFICACY STAGE: 2.

Diagnostic effectiveness of deep learning-based MRI in predicting multiple sclerosis: A meta-analysis

OBJECTIVES

The brain and spinal cord, constituting the central nervous system (CNS), could be impacted by an inflammatory disease known as multiple sclerosis (MS). The convolutional neural networks (CNN), a machine learning method, can detect lesions early by learning patterns on brain magnetic resonance image (MRI). We performed this study to investigate the diagnostic performance of CNN based MRI in the identification, classification, and segmentation of MS lesions.

METHODS

PubMed, Web of Science, Embase, the Cochrane Library, CINAHL, and Google Scholar were used to retrieve papers reporting the use of CNN based MRI in MS diagnosis. The accuracy, the specificity, the sensitivity, and the Dice Similarity Coefficient (DSC) were evaluated in this study.

RESULTS

In total, 2174 studies were identified and only 15 articles met the inclusion criteria. The 2D-3D CNN presented a high accuracy (98.81, 95% CI: 98.50-99.13), sensitivity (98.76, 95% CI: 98.42-99.10), and specificity (98.67, 95% CI: 98.22-99.12) in the identification of MS lesions. Regarding classification, the overall accuracy rate was significantly high (91.38, 95% CI: 83.23-99.54). A DSC rate of 63.78 (95% CI: 58.29-69.27) showed that 2D-3D CNN-based MRI performed highly in the segmentation of MS lesions. Sensitivity analysis showed that the results are consistent, indicating that this study is robust.

CONCLUSION

This metanalysis revealed that 2D-3D CNN based MRI is an automated system that has high diagnostic performance and can promptly and effectively predict the disease.

What contributes to disability in progressive MS? A brain and cervical cord-matched quantitative MRI study

BACKGROUND

We assessed the ability of a brain-and-cord-matched quantitative magnetic resonance imaging (qMRI) protocol to differentiate patients with progressive multiple sclerosis (PMS) from controls, in terms of normal-appearing (NA) tissue abnormalities, and explain disability.

METHODS

A total of 27 patients and 16 controls were assessed on the Expanded Disability Status Scale (EDSS), 25-foot timed walk (TWT), 9-hole peg (9HPT) and symbol digit modalities (SDMT) tests. All underwent 3T brain and (C2-C3) cord structural imaging and qMRI (relaxometry, quantitative magnetisation transfer, multi-shell diffusion-weighted imaging), using a fast brain-and-cord-matched protocol with brain-and-cord-unified imaging readouts. Lesion and NA-tissue volumes and qMRI metrics reflecting demyelination and axonal loss were obtained. Random forest analyses identified the most relevant volumetric/qMRI measures to clinical outcomes. Confounder-adjusted linear regression estimated the actual MRI-clinical associations.

RESULTS

Several qMRI/volumetric differences between patients and controls were observed (p < 0.01). Higher NA-deep grey matter quantitative-T1 (EDSS: beta = 7.96, p = 0.006; 9HPT: beta = -0.09, p = 0.004), higher NA-white matter orientation dispersion index (TWT: beta = -3.21, p = 0.005; SDMT: beta = -847.10, p < 0.001), lower whole-cord bound pool fraction (9HPT: beta = 0.79, p = 0.001) and higher NA-cortical grey matter quantitative-T1 (SDMT = -94.31, p < 0.001) emerged as particularly relevant predictors of greater disability.

CONCLUSION

Fast brain-and-cord-matched qMRI protocols are feasible and identify demyelination - combined with other mechanisms - as key for disability accumulation in PMS.

LST-AI: a Deep Learning Ensemble for Accurate MS Lesion Segmentation

Automated segmentation of brain white matter lesions is crucial for both clinical assessment and scientific research in multiple sclerosis (MS). Over a decade ago, we introduced an engineered lesion segmentation tool, LST. While recent lesion segmentation approaches have leveraged artificial intelligence (AI), they often remain proprietary and difficult to adopt. As an open-source tool, we present LST-AI, an advanced deep learning-based extension of LST that consists of an ensemble of three 3D-UNets. LST-AI explicitly addresses the imbalance between white matter (WM) lesions and non-lesioned WM. It employs a composite loss function incorporating binary cross-entropy and Tversky loss to improve segmentation of the highly heterogeneous MS lesions. We train the network ensemble on 491 MS pairs of T1w and FLAIR images, collected in-house from a 3T MRI scanner, and expert neuroradiologists manually segmented the utilized lesion maps for training. LST-AI additionally includes a lesion location annotation tool, labeling lesion location according to the 2017 McDonald criteria (periventricular, infratentorial, juxtacortical, subcortical). We conduct evaluations on 103 test cases consisting of publicly available data using the Anima segmentation validation tools and compare LST-AI with several publicly available lesion segmentation models. Our empirical analysis shows that LST-AI achieves superior performance compared to existing methods. Its Dice and F1 scores exceeded 0.62, outperforming LST, SAMSEG (Sequence Adaptive Multimodal SEGmentation), and the popular nnUNet framework, which all scored below 0.56. Notably, LST-AI demonstrated exceptional performance on the MSSEG-1 challenge dataset, an international WM lesion segmentation challenge, with a Dice score of 0.65 and an F1 score of 0.63-surpassing all other competing models at the time of the challenge. With increasing lesion volume, the lesion detection rate rapidly increased with a detection rate of >75% for lesions with a volume between 10mm3 and 100mm3. Given its higher segmentation performance, we recommend that research groups currently using LST transition to LST-AI. To facilitate broad adoption, we are releasing LST-AI as an open-source model, available as a command-line tool, dockerized container, or Python script, enabling diverse applications across multiple platforms.

Assessing robustness and generalization of a deep neural network for brain MS lesion segmentation on real-world data

OBJECTIVES

Evaluate the performance of a deep learning (DL)-based model for multiple sclerosis (MS) lesion segmentation and compare it to other DL and non-DL algorithms.

METHODS

This ambispective, multicenter study assessed the performance of a DL-based model for MS lesion segmentation and compared it to alternative DL- and non-DL-based methods. Models were tested on internal (n = 20) and external (n = 18) datasets from Latin America, and on an external dataset from Europe (n = 49). We also examined robustness by rescanning six patients (n = 6) from our MS clinical cohort. Moreover, we studied inter-human annotator agreement and discussed our findings in light of these results. Performance and robustness were assessed using intraclass correlation coefficient (ICC), Dice coefficient (DC), and coefficient of variation (CV).

RESULTS

Inter-human ICC ranged from 0.89 to 0.95, while spatial agreement among annotators showed a median DC of 0.63. Using expert manual segmentations as ground truth, our DL model achieved a median DC of 0.73 on the internal, 0.66 on the external, and 0.70 on the challenge datasets. The performance of our DL model exceeded that of the alternative algorithms on all datasets. In the robustness experiment, our DL model also achieved higher DC (ranging from 0.82 to 0.90) and lower CV (ranging from 0.7 to 7.9%) when compared to the alternative methods.

CONCLUSION

Our DL-based model outperformed alternative methods for brain MS lesion segmentation. The model also proved to generalize well on unseen data and has a robust performance and low processing times both on real-world and challenge-based data.

CLINICAL RELEVANCE STATEMENT

Our DL-based model demonstrated superior performance in accurately segmenting brain MS lesions compared to alternative methods, indicating its potential for clinical application with improved accuracy, robustness, and efficiency.

KEY POINTS

• Automated lesion load quantification in MS patients is valuable; however, more accurate methods are still necessary. • A novel deep learning model outperformed alternative MS lesion segmentation methods on multisite datasets. • Deep learning models are particularly suitable for MS lesion segmentation in clinical scenarios.

Multimodal MRI study on the relation between WM integrity and connected GM atrophy and its effect on disability in early multiple sclerosis

BACKGROUND

Multiple sclerosis (MS) is characterized by pathology in white matter (WM) and atrophy of grey matter (GM), but it remains unclear how these processes are related, or how they influence clinical progression.

OBJECTIVE

To study the spatial and temporal relationship between GM atrophy and damage in connected WM in relapsing-remitting (RR) MS in relation to clinical progression.

METHODS

Healthy control (HC) and early RRMS subjects visited our center twice with a 1-year interval for MRI and clinical examinations, including the Expanded Disability Status Scale (EDSS) and Multiple Sclerosis Functional Composite (MSFC) scores. RRMS subjects were categorized as MSFC decliners or non-decliners based on ΔMSFC over time. Ten deep (D)GM and 62 cortical (C) GM structures were segmented and probabilistic tractography was performed to identify the connected WM. WM integrity was determined per tract with, amongst others, fractional anisotropy (FA), mean diffusivity (MD), neurite density index (NDI), and myelin water fraction (MWF). Linear mixed models (LMMs) were used to investigate GM and WM differences between HC and RRMS, and between MSFC decliners and non-decliners. LMM was also used to test associations between baseline WM z-scores and changes in connected GM z-scores, and between baseline GM z-scores and changes in connected WM z-scores, in HC/RRMS subjects and in MSFC decliners/non-decliners.

RESULTS

We included 13 HCs and 31 RRMS subjects with an average disease duration of 3.5 years and a median EDSS of 3.0. Fifteen RRMS subjects showed declining MSFC scores over time, and they showed higher atrophy rates and greater WM integrity loss compared to non-decliners. Lower baseline WM integrity was associated with increased CGM atrophy over time in RRMS, but not in HC subjects. This effect was only seen in MSFC decliners, especially when an extended WM z-score was used, which included FA, MD, NDI and MWF. Baseline GM measures were not significantly related to WM integrity changes over time in any of the groups.

DISCUSSION

Lower baseline WM integrity was related to more cortical atrophy in RRMS subjects that showed clinical progression over a 1-year follow-up, while baseline GM did not affect WM integrity changes over time. WM damage, therefore, seems to drive atrophy more than conversely.

BIANCA-MS: An optimized tool for automated multiple sclerosis lesion segmentation

In this work we present BIANCA-MS, a novel tool for brain white matter lesion segmentation in multiple sclerosis (MS), able to generalize across both the wide spectrum of MRI acquisition protocols and the heterogeneity of manually labeled data. BIANCA-MS is based on the original version of BIANCA and implements two innovative elements: a harmonized setting, tested under different MRI protocols, which avoids the need to further tune algorithm parameters to each dataset; and a cleaning step developed to improve consistency in automated and manual segmentations, thus reducing unwanted variability in output segmentations and validation data. BIANCA-MS was tested on three datasets, acquired with different MRI protocols. First, we compared BIANCA-MS to other widely used tools. Second, we tested how BIANCA-MS performs in separate datasets. Finally, we evaluated BIANCA-MS performance on a pooled dataset where all MRI data were merged. We calculated the overlap using the DICE spatial similarity index (SI) as well as the number of false positive/negative clusters (nFPC/nFNC) in comparison to the manual masks processed with the cleaning step. BIANCA-MS clearly outperformed other available tools in both high- and low-resolution images and provided comparable performance across different scanning protocols, sets of modalities and image resolutions. BIANCA-MS performance on the pooled dataset (SI: 0.72 ± 0.25, nFPC: 13 ± 11, nFNC: 4 ± 8) were comparable to those achieved on each individual dataset (median across datasets SI: 0.72 ± 0.28, nFPC: 14 ± 11, nFNC: 4 ± 8). Our findings suggest that BIANCA-MS is a robust and accurate approach for automated MS lesion segmentation.

Cardiovascular risk factors in secondary progressive multiple sclerosis: A cross-sectional analysis from the MS-STAT2 randomized controlled trial

BACKGROUND AND PURPOSE

There is increasing evidence that cardiovascular risk (CVR) contributes to disability progression in multiple sclerosis (MS). CVR is particularly prevalent in secondary progressive MS (SPMS) and can be quantified through validated composite CVR scores. The aim was to examine the cross-sectional relationships between excess modifiable CVR, whole and regional brain atrophy on magnetic resonance imaging, and disability in patients with SPMS.

METHODS

Participants had SPMS, and data were collected at enrolment into the MS-STAT2 trial. Composite CVR scores were calculated using the QRISK3 software. Prematurely achieved CVR due to modifiable risk factors was expressed as QRISK3 premature CVR, derived through reference to the normative QRISK3 dataset and expressed in years. Associations were determined with multiple linear regressions.

RESULTS

For the 218 participants, mean age was 54 years and median Expanded Disability Status Scale was 6.0. Each additional year of prematurely achieved CVR was associated with a 2.7 mL (beta coefficient; 95% confidence interval 0.8-4.7; p = 0.006) smaller normalized whole brain volume. The strongest relationship was seen for the cortical grey matter (beta coefficient 1.6 mL per year; 95% confidence interval 0.5-2.7; p = 0.003), and associations were also found with poorer verbal working memory performance. Body mass index demonstrated the strongest relationships with normalized brain volumes, whilst serum lipid ratios demonstrated strong relationships with verbal and visuospatial working memory performance.

CONCLUSIONS

Prematurely achieved CVR is associated with lower normalized brain volumes in SPMS. Future longitudinal analyses of this clinical trial dataset will be important to determine whether CVR predicts future disease worsening.

Pulmonary nodules segmentation based on domain adaptation

With the development of deep learning, the methods based on transfer learning have promoted the progress of medical image segmentation. However, the domain shift and complex background information of medical images limit the further improvement of the segmentation accuracy. Domain adaptation can compensate for the sample shortage by learning important information from a similar source dataset. Therefore, a segmentation method based on adversarial domain adaptation with background mask (ADAB) is proposed in this paper. Firstly, two ADAB networks are built for the source and target data segmentation, respectively. Next, to extract the foreground features that are the input of the discriminators, the background masks are generated according to the region growth algorithm. Then, to update the parameters in the target network without being affected by the conflict between the distinguishing differences of the discriminator and the domain shift reduction of the adversarial domain adaptation, a gradient reversal layer propagation is embedded in the ADAB model for the target data. Finally, an enhanced boundaries loss is deduced to make the target network sensitive to the edge of the area to be segmented. The performance of the proposed method is evaluated in the segmentation of pulmonary nodules in computed tomography images. Experimental results show that the proposed approach has a potential prospect in medical image processing.

Scanner agnostic large-scale evaluation of MS lesion delineation tool for clinical MRI

Introduction

Patients with MS are MRI scanned continuously throughout their disease course resulting in a large manual workload for radiologists which includes lesion detection and size estimation. Though many models for automatic lesion segmentation have been published, few are used broadly in clinic today, as there is a lack of testing on clinical datasets. By collecting a large, heterogeneous training dataset directly from our MS clinic we aim to present a model which is robust to different scanner protocols and artefacts and which only uses MRI modalities present in routine clinical examinations.

Methods

We retrospectively included 746 patients from routine examinations at our MS clinic. The inclusion criteria included acquisition at one of seven different scanners and an MRI protocol including 2D or 3D T2-w FLAIR, T2-w and T1-w images. Reference lesion masks on the training (n = 571) and validation (n = 70) datasets were generated using a preliminary segmentation model and subsequent manual correction. The test dataset (n = 100) was manually delineated. Our segmentation model https://github.com/CAAI/AIMS/ was based on the popular nnU-Net, which has won several biomedical segmentation challenges. We tested our model against the published segmentation models HD-MS-Lesions, which is also based on nnU-Net, trained with a more homogenous patient cohort. We furthermore tested model robustness to data from unseen scanners by performing a leave-one-scanner-out experiment.

Results

We found that our model was able to segment MS white matter lesions with a performance comparable to literature: DSC = 0.68, precision = 0.90, recall = 0.70, f1 = 0.78. Furthermore, the model outperformed HD-MS-Lesions in all metrics except precision = 0.96. In the leave-one-scanner-out experiment there was no significant change in performance (p < 0.05) between any of the models which were only trained on part of the dataset and the full segmentation model.

Conclusion

In conclusion we have seen, that by including a large, heterogeneous dataset emulating clinical reality, we have trained a segmentation model which maintains a high segmentation performance while being robust to data from unseen scanners. This broadens the applicability of the model in clinic and paves the way for clinical implementation.

Multiple sclerosis lesion segmentation: revisiting weighting mechanisms for federated learning

Background and introduction

Federated learning (FL) has been widely employed for medical image analysis to facilitate multi-client collaborative learning without sharing raw data. Despite great success, FL's applications remain suboptimal in neuroimage analysis tasks such as lesion segmentation in multiple sclerosis (MS), due to variance in lesion characteristics imparted by different scanners and acquisition parameters.

Methods

In this work, we propose the first FL MS lesion segmentation framework via two effective re-weighting mechanisms. Specifically, a learnable weight is assigned to each local node during the aggregation process, based on its segmentation performance. In addition, the segmentation loss function in each client is also re-weighted according to the lesion volume for the data during training.

Results

The proposed method has been validated on two FL MS segmentation scenarios using public and clinical datasets. Specifically, the case-wise and voxel-wise Dice score of the proposed method under the first public dataset is 65.20 and 74.30, respectively. On the second in-house dataset, the case-wise and voxel-wise Dice score is 53.66, and 62.31, respectively.

Discussions and conclusions

The Comparison experiments on two FL MS segmentation scenarios using public and clinical datasets have demonstrated the effectiveness of the proposed method by significantly outperforming other FL methods. Furthermore, the segmentation performance of FL incorporating our proposed aggregation mechanism can achieve comparable performance to that from centralized training with all the raw data.

Contrastive Semi-Supervised Learning for Domain Adaptive Segmentation Across Similar Anatomical Structures

Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for medical image segmentation, yet need plenty of manual annotations for training. Semi-Supervised Learning (SSL) methods are promising to reduce the requirement of annotations, but their performance is still limited when the dataset size and the number of annotated images are small. Leveraging existing annotated datasets with similar anatomical structures to assist training has a potential for improving the model's performance. However, it is further challenged by the cross-anatomy domain shift due to the image modalities and even different organs in the target domain. To solve this problem, we propose Contrastive Semi-supervised learning for Cross Anatomy Domain Adaptation (CS-CADA) that adapts a model to segment similar structures in a target domain, which requires only limited annotations in the target domain by leveraging a set of existing annotated images of similar structures in a source domain. We use Domain-Specific Batch Normalization (DSBN) to individually normalize feature maps for the two anatomical domains, and propose a cross-domain contrastive learning strategy to encourage extracting domain invariant features. They are integrated into a Self-Ensembling Mean-Teacher (SE-MT) framework to exploit unlabeled target domain images with a prediction consistency constraint. Extensive experiments show that our CS-CADA is able to solve the challenging cross-anatomy domain shift problem, achieving accurate segmentation of coronary arteries in X-ray images with the help of retinal vessel images and cardiac MR images with the help of fundus images, respectively, given only a small number of annotations in the target domain. Our code is available at https://github.com/HiLab-git/DAG4MIA.

Commercial volumetric MRI reporting tools in multiple sclerosis: a systematic review of the evidence

PURPOSE

MRI is integral to the diagnosis of multiple sclerosis (MS) and is important for clinical prognostication. Quantitative volumetric reporting tools (QReports) can improve the accuracy and objectivity of MRI-based assessments. Several QReports are commercially available; however, validation can be difficult to establish and does not currently follow a common pathway. To aid evidence-based clinical decision-making, we performed a systematic review of commercial QReports for use in MS including technical details and published reports of validation and in-use evaluation.

METHODS

We categorized studies into three types of testing: technical validation, for example, comparison to manual segmentation, clinical validation by clinicians or interpretation of results alongside clinician-rated variables, and in-use evaluation, such as health economic assessment.

RESULTS

We identified 10 companies, which provide MS lesion and brain segmentation and volume quantification, and 38 relevant publications. Tools received regulatory approval between 2006 and 2020, contextualize results to normative reference populations, ranging from 620 to 8000 subjects, and require T1- and T2-FLAIR-weighted input sequences for longitudinal assessment of whole-brain volume and lesions. In MS, six QReports provided evidence of technical validation, four companies have conducted clinical validation by correlating results with clinical variables, only one has tested their QReport by clinician end-users, and one has performed a simulated in-use socioeconomic evaluation.

CONCLUSION

We conclude that there is limited evidence in the literature regarding clinical validation and in-use evaluation of commercial MS QReports with a particular lack of clinician end-user testing. Our systematic review provides clinicians and institutions with the available evidence when considering adopting a quantitative reporting tool for MS.

Automatic detection of active and inactive multiple sclerosis plaques using the Bayesian approach in susceptibility-weighted imaging

BACKGROUND

Susceptibility-weighted imaging (SWI) is efficient in detecting multiple sclerosis (MS) plaques and evaluating the level of disease activity.

PURPOSE

To automatically detect active and inactive MS plaques in SWI images using a Bayesian approach.

MATERIAL AND METHODS

A 1.5-T scanner was used to evaluate 147 patients with MS. The area of the plaques along with their active or inactive status were automatically identified using a Bayesian approach. Plaques were given an orange color if they were active and a blue color if they were inactive, based on the preset signal intensity.

RESULTS

Experimental findings show that the proposed method has a high accuracy rate of 91% and a sensitivity rate of 76% for identifying the type and area of plaques. Inactive plaques were properly identified in 87% of cases, and active plaques in 76% of cases. The Kappa analysis revealed an 80% agreement between expert diagnoses based on contrast-enhanced and FLAIR images and Bayesian inferences in SWI.

CONCLUSION

The results of our study demonstrated that the proposed method has good accuracy for identifying the MS plaque area as well as for identifying the types of active or inactive plaques in SWI. Therefore, it might be helpful to use the proposed method as a supplemental tool to accelerate the specialist's diagnosis.

Online hard example mining vs. fixed oversampling strategy for segmentation of new multiple sclerosis lesions from longitudinal FLAIR MRI

Detecting new lesions is a key aspect of the radiological follow-up of patients with Multiple Sclerosis (MS), leading to eventual changes in their therapeutics. This paper presents our contribution to the MSSEG-2 MICCAI 2021 challenge. The challenge is focused on the segmentation of new MS lesions using two consecutive Fluid Attenuated Inversion Recovery (FLAIR) Magnetic Resonance Imaging (MRI). In other words, considering longitudinal data composed of two time points as input, the aim is to segment the lesional areas, which are present only in the follow-up scan and not in the baseline. The backbone of our segmentation method is a 3D UNet applied patch-wise to the images, and in which, to take into account both time points, we simply concatenate the baseline and follow-up images along the channel axis before passing them to the 3D UNet. Our key methodological contribution is the use of online hard example mining to address the challenge of class imbalance. Indeed, there are very few voxels belonging to new lesions which makes training deep-learning models difficult. Instead of using handcrafted priors like brain masks or multi-stage methods, we experiment with a novel modification to online hard example mining (OHEM), where we use an exponential moving average (i.e., its weights are updated with momentum) of the 3D UNet to mine hard examples. Using a moving average instead of the raw model should allow smoothing of its predictions and allow it to give more consistent feedback for OHEM.

Cortical lesions, central vein sign, and paramagnetic rim lesions in multiple sclerosis: Emerging machine learning techniques and future avenues

The current diagnostic criteria for multiple sclerosis (MS) lack specificity, and this may lead to misdiagnosis, which remains an issue in present-day clinical practice. In addition, conventional biomarkers only moderately correlate with MS disease progression. Recently, some MS lesional imaging biomarkers such as cortical lesions (CL), the central vein sign (CVS), and paramagnetic rim lesions (PRL), visible in specialized magnetic resonance imaging (MRI) sequences, have shown higher specificity in differential diagnosis. Moreover, studies have shown that CL and PRL are potential prognostic biomarkers, the former correlating with cognitive impairments and the latter with early disability progression. As machine learning-based methods have achieved extraordinary performance in the assessment of conventional imaging biomarkers, such as white matter lesion segmentation, several automated or semi-automated methods have been proposed as well for CL, PRL, and CVS. In the present review, we first introduce these MS biomarkers and their imaging methods. Subsequently, we describe the corresponding machine learning-based methods that were proposed to tackle these clinical questions, putting them into context with respect to the challenges they are facing, including non-standardized MRI protocols, limited datasets, and moderate inter-rater variability. We conclude by presenting the current limitations that prevent their broader deployment and suggesting future research directions.

A domain adaptation benchmark for T1-weighted brain magnetic resonance image segmentation

Accurate brain segmentation is critical for magnetic resonance imaging (MRI) analysis pipelines. Machine-learning-based brain MR image segmentation methods are among the state-of-the-art techniques for this task. Nevertheless, the segmentations produced by machine learning models often degrade in the presence of expected domain shifts between the test and train sets data distributions. These domain shifts are expected due to several factors, such as scanner hardware and software differences, technology updates, and differences in MRI acquisition parameters. Domain adaptation (DA) methods can make machine learning models more resilient to these domain shifts. This paper proposes a benchmark for investigating DA techniques for brain MR image segmentation using data collected across sites with scanners from different vendors (Philips, Siemens, and General Electric). Our work provides labeled data, publicly available source code for a set of baseline and DA models, and a benchmark for assessing different brain MR image segmentation techniques. We applied the proposed benchmark to evaluate two segmentation tasks: skull-stripping; and white-matter, gray-matter, and cerebrospinal fluid segmentation, but the benchmark can be extended to other brain structures. Our main findings during the development of this benchmark are that there is not a single DA technique that consistently outperforms others, and hyperparameter tuning and computational times for these methods still pose a challenge before broader adoption of these methods in the clinical practice.

Multiple sclerosis cortical lesion detection with deep learning at ultra-high-field MRI

Manually segmenting multiple sclerosis (MS) cortical lesions (CLs) is extremely time consuming, and past studies have shown only moderate inter-rater reliability. To accelerate this task, we developed a deep-learning-based framework (CLAIMS: Cortical Lesion AI-Based Assessment in Multiple Sclerosis) for the automated detection and classification of MS CLs with 7 T MRI. Two 7 T datasets, acquired at different sites, were considered. The first consisted of 60 scans that include 0.5 mm isotropic MP2RAGE acquired four times (MP2RAGE×4), 0.7 mm MP2RAGE, 0.5 mm T₂ *-weighted GRE, and 0.5 mm T₂ *-weighted EPI. The second dataset consisted of 20 scans including only 0.75 × 0.75 × 0.9 mm³ MP2RAGE. CLAIMS was first evaluated using sixfold cross-validation with single and multi-contrast 0.5 mm MRI input. Second, the performance of the model was tested on 0.7 mm MP2RAGE images after training with either 0.5 mm MP2RAGE×4, 0.7 mm MP2RAGE, or alternating the two. Third, its generalizability was evaluated on the second external dataset and compared with a state-of-the-art technique based on partial volume estimation and topological constraints (MSLAST). CLAIMS trained only with MP2RAGE×4 achieved results comparable to those of the multi-contrast model, reaching a CL true positive rate of 74% with a false positive rate of 30%. Detection rate was excellent for leukocortical and subpial lesions (83%, and 70%, respectively), whereas it reached 53% for intracortical lesions. The correlation between disability measures and CL count was similar for manual and CLAIMS lesion counts. Applying a domain-scanner adaptation approach and testing CLAIMS on the second dataset, the performance was superior to MSLAST when considering a minimum lesion volume of 6 μL (lesion-wise detection rate of 71% versus 48%). The proposed framework outperforms previous state-of-the-art methods for automated CL detection across scanners and protocols. In the future, CLAIMS may be useful to support clinical decisions at 7 T MRI, especially in the field of diagnosis and differential diagnosis of MS patients.

Prediction of Conversion from CIS to Clinically Definite Multiple Sclerosis Using Convolutional Neural Networks

Multiple sclerosis (MS) is a chronic neurological disease of the central nervous system (CNS). Early diagnosis of MS is highly desirable as treatments are more effective in preventing MS-related disability when given in the early stages of the disease. The main aim of this research is to predict the occurrence of a second MS-related clinical event, which indicates the conversion of clinically isolated syndrome (CIS) to clinically definite MS (CDMS). In this study, we apply a branch of artificial intelligence known as deep learning and develop a fully automated algorithm primed with convolutional neural network (CNN) that has the ability to learn from MRI scan features. The basic architecture of our algorithm is that of the VGG16 CNN model, but amended such that it can handle MRI DICOM images. A dataset comprised of scans acquired using two different scanners was used for the purposes of verification of the algorithm. A group of 49 patients had volumetric MRI scans taken at onset of the disease and then again one year later using one of the two scanners. In total, this yielded 7360 images which were then used for training, validation, and testing of the algorithm. Initially, these raw images were taken through 4 steps of preprocessing. In order to boost the efficiency of the process, we pretrained our algorithm using the publicly available ADNI dataset used to classify Alzheimer's disease. Finally, we used our preprocessed dataset to train and test the algorithm. Clinical evaluation conducted a year after the first time point revealed that 26 of the 49 patients had converted to CDMS, while the remaining 23 had not. Results of testing showed that our algorithm was able to predict the clinical results with an accuracy of 88.8% and with an area under the curve (AUC) of 91%. A highly accurate algorithm was developed using CNN approach to reliably predict conversion of patients with CIS to CDMS using MRI data from two different scanners.

Magnetic susceptibility anisotropy in normal appearing white matter in multiple sclerosis from single-orientation acquisition

Quantitative susceptibility mapping (QSM) has been successfully applied to study changes in deep grey matter nuclei as well as in lesional tissue, but its application to white matter has been complicated by the observed orientation dependence of gradient echo signal. The anisotropic susceptibility tensor is thought to be at the origin of this orientation dependence, and magnetic susceptibility anisotropy (MSA) derived from this tensor has been proposed as a marker of the state and integrity of the myelin sheath and may therefore be of particular interest for the study of demyelinating pathologies such as multiple sclerosis (MS). Reconstruction of the susceptibility tensor, however, requires repeated measurements with multiple head orientations, rendering the approach impractical for clinical applications. In this study, we combined single-orientation QSM with fibre orientation information to assess apparent MSA in three white matter tracts, i.e., optic radiation (OR), splenium of the corpus callosum (SCC), and superior longitudinal fascicle (SLF), in two cohorts of 64 healthy controls and 89 MS patients. The apparent MSA showed a significant decrease in optic radiation in the MS cohort compared with healthy controls. It decreased in the MS cohort with increasing lesion load in OR and with disease duration in the splenium. All of this suggests demyelination in normal appearing white matter. However, the apparent MSA observed in the SLF pointed to potential systematic issues that require further exploration to realize the full potential of the presented approach. Despite the limitations of such single-orientation ROI-specific estimation, we believe that our clinically feasible approach to study degenerative changes in WM is worthy of further investigation.

MAGNIMS recommendations for harmonization of MRI data in MS multicenter studies

•

Sharing data from cooperative studies is essential to develop new biomarkers in MS.

•

Differences in MRI acquisition, analysis, storage represent a substantial constraint.

•

We review the state of the art and developments in the harmonization of MRI.

•

We provide recommendations to harmonize large MRI datasets in the MS field.

There is an increasing need of sharing harmonized data from large, cooperative studies as this is essential to develop new diagnostic and prognostic biomarkers. In the field of multiple sclerosis (MS), the issue has become of paramount importance due to the need to translate into the clinical setting some of the most recent MRI achievements. However, differences in MRI acquisition parameters, image analysis and data storage across sites, with their potential bias, represent a substantial constraint. This review focuses on the state of the art, recent technical advances, and desirable future developments of the harmonization of acquisition, analysis and storage of large-scale multicentre MRI data of MS cohorts. Huge efforts are currently being made to achieve all the requirements needed to provide harmonized MRI datasets in the MS field, as proper management of large imaging datasets is one of our greatest opportunities and challenges in the coming years. Recommendations based on these achievements will be provided here. Despite the advances that have been made, the complexity of these tasks requires further research by specialized academical centres, with dedicated technical and human resources. Such collective efforts involving different professional figures are of crucial importance to offer to MS patients a personalised management while minimizing consumption of resources.

Deep learning kidney segmentation with very limited training data using a cascaded convolution neural network

BACKGROUND

Deep learning segmentation requires large datasets with ground truth. Image annotation is time consuming and leads to shortages of ground truth data for clinical imaging. This study is to investigate the feasibility of kidney segmentation using deep learning convolution neural network (CNN) models trained with MR images from only a few subjects.

METHODS

A total of 60 subjects from two cohorts were included in this study. The first cohort of 20 subjects from publicly available data was used for training and testing. The second cohort of 40 subjects with renal masses from our institution was used for testing only. A few-shot deep learning approach using 3D augmentation was investigated. T1-weighted images in the first cohort were used for training and testing. Cascaded CNN networks were trained using images from one, three, and six subjects, respectively. Images for the remaining subjects were used for testing. Images in the second cohort were utilized for testing only. Dice and Jaccard coefficients were generated to evaluate the performance of CNN models. Statistical analyses for segmentation metrics among different approaches were performed.

RESULTS

Our approach achieved mean Dice coefficients of 0.85 using a single training subject and 0.91 with six training subjects. Compared to a single Unet, the cascaded network significantly improved the results using a single training subject (Dice, 0.759 vs. 0.835; p<0.001) and three subjects (0.864 vs. 0.893; p = 0.015) in the first cohort, and the results for the second cohort (0.821 vs. 0.873; p = 0.008).

CONCLUSION

Our few-shot kidney segmentation approach using 3D augmentation achieved a good performance even using a single Unet. Furthermore, the cascaded network significantly improved the performance of segmentation and was superior to a single Unet in certain cases. Our approach provides a promising solution to segmentation in medical imaging when the number of ground truth masks is limited.

Multiple Sclerosis Lesions Segmentation Using Attention-Based CNNs in FLAIR Images

Objective: Multiple Sclerosis (MS) is an autoimmune and demyelinating disease that leads to lesions in the central nervous system. This disease can be tracked and diagnosed using Magnetic Resonance Imaging (MRI). A multitude of multimodality automatic biomedical approaches are used to segment lesions that are not beneficial for patients in terms of cost, time, and usability. The authors of the present paper propose a method employing just one modality (FLAIR image) to segment MS lesions accurately. Methods: A patch-based Convolutional Neural Network (CNN) is designed, inspired by 3D-ResNet and spatial-channel attention module, to segment MS lesions. The proposed method consists of three stages: (1) the Contrast-Limited Adaptive Histogram Equalization (CLAHE) is applied to the original images and concatenated to the extracted edges to create 4D images; (2) the patches of size [Formula: see text] are randomly selected from the 4D images; and (3) the extracted patches are passed into an attention-based CNN which is used to segment the lesions. Finally, the proposed method was compared to previous studies of the same dataset. Results: The current study evaluates the model with a test set of ISIB challenge data. Experimental results illustrate that the proposed approach significantly surpasses existing methods of Dice similarity and Absolute Volume Difference while the proposed method uses just one modality (FLAIR) to segment the lesions. Conclusion: The authors have introduced an automated approach to segment the lesions, which is based on, at most, two modalities as an input. The proposed architecture comprises convolution, deconvolution, and an SCA-VoxRes module as an attention module. The results show, that the proposed method outperforms well compared to other methods.

Automatic segmentation of white matter hyperintensities: validation and comparison with state-of-the-art methods on both Multiple Sclerosis and elderly subjects

•

Automatic segmentation of MS lesions and age-related WMH from 3D T1 and T2-FLAIR.

•

Comparison to consensus show improved performance of WHASA-3D compared to WHASA.

•

WHASA-3D outperforms available state-of-the-art methods with their default settings.

•

WHASA-3D could be a useful tool for clinical practice and clinical trials.

Different types of white matter hyperintensities (WMH) can be observed through MRI in the brain and spinal cord, especially Multiple Sclerosis (MS) lesions for patients suffering from MS and age-related WMH for subjects with cognitive disorders and/or elderly people. To better diagnose and monitor the disease progression, the quantitative evaluation of WMH load has proven to be useful for clinical routine and trials. Since manual delineation for WMH segmentation is highly time-consuming and suffers from intra and inter observer variability, several methods have been proposed to automatically segment either MS lesions or age-related WMH, but none is validated on both WMH types. Here, we aim at proposing the White matter Hyperintensities Automatic Segmentation Algorithm adapted to 3D T2-FLAIR datasets (WHASA-3D), a fast and robust automatic segmentation tool designed to be implemented in clinical practice for the detection of both MS lesions and age-related WMH in the brain, using both 3D T1-weighted and T2-FLAIR images. In order to increase its robustness for MS lesions, WHASA-3D expands the original WHASA method, which relies on the coupling of non-linear diffusion framework and watershed parcellation, where regions considered as WMH are selected based on intensity and location characteristics, and finally refined with geodesic dilation. The previous validation was performed on 2D T2-FLAIR and subjects with cognitive disorders and elderly subjects. 60 subjects from a heterogeneous database of dementia patients, multiple sclerosis patients and elderly subjects with multiple MRI scanners and a wide range of lesion loads were used to evaluate WHASA and WHASA-3D through volume and spatial agreement in comparison with consensus reference segmentations. In addition, a direct comparison on the MS database with six available supervised and unsupervised state-of-the-art WMH segmentation methods (LST-LGA and LPA, Lesion-TOADS, lesionBrain, BIANCA and nicMSlesions) with default and optimised settings (when feasible) was conducted. WHASA-3D confirmed an improved performance with respect to WHASA, achieving a better spatial overlap (Dice) (0.67 vs 0.63), a reduced absolute volume error (AVE) (3.11 vs 6.2 mL) and an increased volume agreement (intraclass correlation coefficient, ICC) (0.96 vs 0.78). Compared to available state-of-the-art algorithms on the MS database, WHASA-3D outperformed both unsupervised and supervised methods when used with their default settings, showing the highest volume agreement (ICC = 0.95) as well as the highest average Dice (0.58). Optimising and/or retraining LST-LGA, BIANCA and nicMSlesions, using a subset of the MS database as training set, resulted in improved performances on the remaining testing set (average Dice: LST-LGA default/optimized = 0.41/0.51, BIANCA default/optimized = 0.22/0.39, nicMSlesions default/optimized = 0.17/0.63, WHASA-3D = 0.58). Evaluation and comparison results suggest that WHASA-3D is a reliable and easy-to-use method for the automated segmentation of white matter hyperintensities, for both MS lesions and age-related WMH. Further validation on larger datasets would be useful to confirm these first findings.

Domain Adaptation for Medical Image Analysis: A Survey

Machine learning techniques used in computer-aided medical image analysis usually suffer from the domain shift problem caused by different distributions between source/reference data and target data. As a promising solution, domain adaptation has attracted considerable attention in recent years. The aim of this paper is to survey the recent advances of domain adaptation methods in medical image analysis. We first present the motivation of introducing domain adaptation techniques to tackle domain heterogeneity issues for medical image analysis. Then we provide a review of recent domain adaptation models in various medical image analysis tasks. We categorize the existing methods into shallow and deep models, and each of them is further divided into supervised, semi-supervised and unsupervised methods. We also provide a brief summary of the benchmark medical image datasets that support current domain adaptation research. This survey will enable researchers to gain a better understanding of the current status, challenges and future directions of this energetic research field.

Multiple Sclerosis Lesion Analysis in Brain Magnetic Resonance Images: Techniques and Clinical Applications

Multiple sclerosis (MS) is a chronic inflammatory and degenerative disease of the central nervous system, characterized by the appearance of focal lesions in the white and gray matter that topographically correlate with an individual patients neurological symptoms and signs. Magnetic resonance imaging (MRI) provides detailed in-vivo structural information, permitting the quantification and categorization of MS lesions that critically inform disease management. Traditionally, MS lesions have been manually annotated on 2D MRI slices, a process that is inefficient and prone to inter-/intra-observer errors. Recently, automated statistical imaging analysis techniques have been proposed to detect and segment MS lesions based on MRI voxel intensity. However, their effectiveness is limited by the heterogeneity of both MRI data acquisition techniques and the appearance of MS lesions. By learning complex lesion representations directly from images, deep learning techniques have achieved remarkable breakthroughs in the MS lesion segmentation task. Here, we provide a comprehensive review of state-of-the-art automatic statistical and deep-learning MS segmentation methods and discuss current and future clinical applications. Further, we review technical strategies, such as domain adaptation, to enhance MS lesion segmentation in real-world clinical settings.

DeepLesionBrain: Towards a broader deep-learning generalization for multiple sclerosis lesion segmentation

Recently, segmentation methods based on Convolutional Neural Networks (CNNs) showed promising performance in automatic Multiple Sclerosis (MS) lesions segmentation. These techniques have even outperformed human experts in controlled evaluation conditions such as Longitudinal MS Lesion Segmentation Challenge (ISBI Challenge). However, state-of-the-art approaches trained to perform well on highly-controlled datasets fail to generalize on clinical data from unseen datasets. Instead of proposing another improvement of the segmentation accuracy, we propose a novel method robust to domain shift and performing well on unseen datasets, called DeepLesionBrain (DLB). This generalization property results from three main contributions. First, DLB is based on a large group of compact 3D CNNs. This spatially distributed strategy aims to produce a robust prediction despite the risk of generalization failure of some individual networks. Second, we propose a hierarchical specialization learning (HSL) by pre-training a generic network over the whole brain, before using its weights as initialization to locally specialized networks. By this end, DLB learns both generic features extracted at global image level and specific features extracted at local image level. Finally, DLB includes a new image quality data augmentation to reduce dependency to training data specificity (e.g., acquisition protocol). DLB generalization was validated in cross-dataset experiments on MSSEG'16, ISBI challenge, and in-house datasets. During experiments, DLB showed higher segmentation accuracy, better segmentation consistency and greater generalization performance compared to state-of-the-art methods. Therefore, DLB offers a robust framework well-suited for clinical practice.

Opportunities for Understanding MS Mechanisms and Progression With MRI Using Large-Scale Data Sharing and Artificial Intelligence

Patients with multiple sclerosis (MS) have heterogeneous clinical presentations, symptoms, and progression over time, making MS difficult to assess and comprehend in vivo. The combination of large-scale data sharing and artificial intelligence creates new opportunities for monitoring and understanding MS using MRI. First, development of validated MS-specific image analysis methods can be boosted by verified reference, test, and benchmark imaging data. Using detailed expert annotations, artificial intelligence algorithms can be trained on such MS-specific data. Second, understanding disease processes could be greatly advanced through shared data of large MS cohorts with clinical, demographic, and treatment information. Relevant patterns in such data that may be imperceptible to a human observer could be detected through artificial intelligence techniques. This applies from image analysis (lesions, atrophy, or functional network changes) to large multidomain datasets (imaging, cognition, clinical disability, genetics). After reviewing data sharing and artificial intelligence, we highlight 3 areas that offer strong opportunities for making advances in the next few years: crowdsourcing, personal data protection, and organized analysis challenges. Difficulties as well as specific recommendations to overcome them are discussed, in order to best leverage data sharing and artificial intelligence to improve image analysis, imaging, and the understanding of MS.

White matter hyperintensities segmentation using an ensemble of neural networks

White matter hyperintensities (WMHs) represent the most common neuroimaging marker of cerebral small vessel disease (CSVD). The volume and location of WMHs are important clinical measures. We present a pipeline using deep fully convolutional network and ensemble models, combining U‐Net, SE‐Net, and multi‐scale features, to automatically segment WMHs and estimate their volumes and locations. We evaluated our method in two datasets: a clinical routine dataset comprising 60 patients (selected from Chinese National Stroke Registry, CNSR) and a research dataset composed of 60 patients (selected from MICCAI WMH Challenge, MWC). The performance of our pipeline was compared with four freely available methods: LGA, LPA, UBO detector, and U‐Net, in terms of a variety of metrics. Additionally, to access the model generalization ability, another research dataset comprising 40 patients (from Older Australian Twins Study and Sydney Memory and Aging Study, OSM), was selected and tested. The pipeline achieved the best performance in both research dataset and the clinical routine dataset with DSC being significantly higher than other methods (p < .001), reaching .833 and .783, respectively. The results of model generalization ability showed that the model trained on the research dataset (DSC = 0.736) performed higher than that trained on the clinical dataset (DSC = 0.622). Our method outperformed widely used pipelines in WMHs segmentation. This system could generate both image and text outputs for whole brain, lobar and anatomical automatic labeling WMHs. Additionally, software and models of our method are made publicly available at https://www.nitrc.org/projects/what_v1.

Infratentorial lesions in multiple sclerosis patients: intra- and inter-rater variability in comparison to a fully automated segmentation using 3D convolutional neural networks

OBJECTIVE

Automated quantification of infratentorial multiple sclerosis lesions on magnetic resonance imaging is clinically relevant but challenging. To overcome some of these problems, we propose a fully automated lesion segmentation algorithm using 3D convolutional neural networks (CNNs).

METHODS

The CNN was trained on a FLAIR image alone or on FLAIR and T1-weighted images from 1809 patients acquired on 156 different scanners. An additional training using an extra class for infratentorial lesions was implemented. Three experienced raters manually annotated three datasets from 123 MS patients from different scanners.

RESULTS

The inter-rater sensitivity (SEN) was 80% for supratentorial lesions but only 62% for infratentorial lesions. There was no statistically significant difference between the inter-rater SEN and the SEN of the CNN with respect to the raters. For supratentorial lesions, the CNN featured an intra-rater intra-scanner SEN of 0.97 (R1 = 0.90, R2 = 0.84) and for infratentorial lesion a SEN of 0.93 (R1 = 0.61, R2 = 0.73).

CONCLUSION

The performance of the CNN improved significantly for infratentorial lesions when specifically trained on infratentorial lesions using a T1 image as an additional input and matches the detection performance of experienced raters. Furthermore, for infratentorial lesions the CNN was more robust against repeated scans than experienced raters.

KEY POINTS

• A 3D convolutional neural network was trained on MRI data from 1809 patients (156 different scanners) for the quantification of supratentorial and infratentorial multiple sclerosis lesions. • Inter-rater variability was higher for infratentorial lesions than for supratentorial lesions. The performance of the 3D convolutional neural network (CNN) improved significantly for infratentorial lesions when specifically trained on infratentorial lesions using a T1 image as an additional input. • The detection performance of the CNN matches the detection performance of experienced raters.

Assessment of Artificial Intelligence Automatic Multiple Sclerosis Lesion Delineation Tool for Clinical Use

PURPOSE

To implement and validate an existing algorithm for automatic delineation of white matter lesions on magnetic resonance imaging (MRI) in patients with multiple sclerosis (MS) on a local single-center dataset.

METHODS

We implemented a white matter hyperintensity segmentation model, based on a 2D convolutional neural network, using the conventional T2-weighted fluid attenuated inversion recovery (FLAIR) MRI sequence as input. The model was adapted for delineation of MS lesions by further training on a local dataset of 93 MS patients with a total of 3040 lesions. A quantitative evaluation was performed on ten test patients, in which model-generated masks were compared to manually delineated masks from two expert delineators. A subsequent qualitative evaluation of the implemented model was performed by two expert delineators, in which generated delineation masks on a clinical dataset of 53 patients were rated acceptable (< 10% errors) or unacceptable (> 10% errors) based on the total number of true lesions.

RESULTS

The quantitative evaluation resulted in an average accuracy score (F1) of 0.71, recall of 0.77 and dice similarity coefficient of 0.62. Our implemented model obtained the highest scores in all three metrics, when compared to three out of the box lesion segmentation models. In the clinical evaluation an average of 94% of our 53 model-generated masks were rated acceptable.

CONCLUSION

After adaptation to our local dataset, the implemented segmentation model was able to delineate MS lesions with a high clinical value as rated by delineation experts while outperforming popular out of the box applications. This serves as a promising step towards implementation of automatic lesion delineation in our MS clinic.

Emerging deep learning techniques using magnetic resonance imaging data applied in multiple sclerosis and clinical isolated syndrome patients (Review)

Computer-aided diagnosis systems aim to assist clinicians in the early identification of abnormal signs in order to optimize the interpretation of medical images and increase diagnostic precision. Multiple sclerosis (MS) and clinically isolated syndrome (CIS) are chronic inflammatory, demyelinating diseases affecting the central nervous system. Recent advances in deep learning (DL) techniques have led to novel computational paradigms in MS and CIS imaging designed for automatic segmentation and detection of areas of interest and automatic classification of anatomic structures, as well as optimization of neuroimaging protocols. To this end, there are several publications presenting artificial intelligence-based predictive models aiming to increase diagnostic accuracy and to facilitate optimal clinical management in patients diagnosed with MS and/or CIS. The current study presents a thorough review covering DL techniques that have been applied in MS and CIS during recent years, shedding light on their current advances and limitations.

Automatic segmentation of gadolinium-enhancing lesions in multiple sclerosis using deep learning from clinical MRI

Gadolinium-enhancing lesions reflect active disease and are critical for in-patient monitoring in multiple sclerosis (MS). In this work, we have developed the first fully automated method to segment and count the gadolinium-enhancing lesions from routine clinical MRI of MS patients. The proposed method first segments the potential lesions using 2D-UNet from multi-channel scans (T1 post-contrast, T1 pre-contrast, FLAIR, T2, and proton-density) and classifies the lesions using a random forest classifier. The algorithm was trained and validated on 600 MRIs with manual segmentation. We compared the effect of loss functions (Dice, cross entropy, and bootstrapping cross entropy) and number of input contrasts. We compared the lesion counts with those by radiologists using 2,846 images. Dice, lesion-wise sensitivity, and false discovery rate with full 5 contrasts were 0.698, 0.844, and 0.307, which improved to 0.767, 0.969, and 0.00 in large lesions (>100 voxels). The model using bootstrapping loss function provided a statistically significant increase of 7.1% in sensitivity and of 2.3% in Dice compared with the model using cross entropy loss. T1 post/pre-contrast and FLAIR were the most important contrasts. For large lesions, the 2D-UNet model trained using T1 pre-contrast, FLAIR, T2, PD had a lesion-wise sensitivity of 0.688 and false discovery rate 0.083, even without T1 post-contrast. For counting lesions in 2846 routine MRI images, the model with 2D-UNet and random forest, which was trained with bootstrapping cross entropy, achieved accuracy of 87.7% using T1 pre-contrast, T1 post-contrast, and FLAIR when lesion counts were categorized as 0, 1, and 2 or more. The model performs well in routine non-standardized MRI datasets, allows large-scale analysis of clinical datasets, and may have clinical applications.

Applications of deep learning techniques for automated multiple sclerosis detection using magnetic resonance imaging: A review

Multiple Sclerosis (MS) is a type of brain disease which causes visual, sensory, and motor problems for people with a detrimental effect on the functioning of the nervous system. In order to diagnose MS, multiple screening methods have been proposed so far; among them, magnetic resonance imaging (MRI) has received considerable attention among physicians. MRI modalities provide physicians with fundamental information about the structure and function of the brain, which is crucial for the rapid diagnosis of MS lesions. Diagnosing MS using MRI is time-consuming, tedious, and prone to manual errors. Research on the implementation of computer aided diagnosis system (CADS) based on artificial intelligence (AI) to diagnose MS involves conventional machine learning and deep learning (DL) methods. In conventional machine learning, feature extraction, feature selection, and classification steps are carried out by using trial and error; on the contrary, these steps in DL are based on deep layers whose values are automatically learn. In this paper, a complete review of automated MS diagnosis methods performed using DL techniques with MRI neuroimaging modalities is provided. Initially, the steps involved in various CADS proposed using MRI modalities and DL techniques for MS diagnosis are investigated. The important preprocessing techniques employed in various works are analyzed. Most of the published papers on MS diagnosis using MRI modalities and DL are presented. The most significant challenges facing and future direction of automated diagnosis of MS using MRI modalities and DL techniques are also provided.

Deep Learning Application for Analyzing of Constituents and Their Correlations in the Interpretations of Medical Images

The need for time and attention, given by the doctor to the patient, due to the increased volume of medical data to be interpreted and filtered for diagnostic and therapeutic purposes has encouraged the development of the option to support, constructively and effectively, deep learning models. Deep learning (DL) has experienced an exponential development in recent years, with a major impact on interpretations of the medical image. This has influenced the development, diversification and increase of the quality of scientific data, the development of knowledge construction methods and the improvement of DL models used in medical applications. All research papers focus on description, highlighting, classification of one of the constituent elements of deep learning models (DL), used in the interpretation of medical images and do not provide a unified picture of the importance and impact of each constituent in the performance of DL models. The novelty in our paper consists primarily in the unitary approach, of the constituent elements of DL models, namely, data, tools used by DL architectures or specifically constructed DL architecture combinations and highlighting their "key" features, for completion of tasks in current applications in the interpretation of medical images. The use of "key" characteristics specific to each constituent of DL models and the correct determination of their correlations, may be the subject of future research, with the aim of increasing the performance of DL models in the interpretation of medical images.

MRI Natural History of the Leukodystrophy Vanishing White Matter

Background In vanishing white matter (VWM), a form of leukodystrophy, earlier onset is associated with faster clinical progression. MRI typically shows rarefaction and cystic destruction of the cerebral white matter. Information on the evolution of VWM according to age at onset is lacking. Purpose To determine whether nature and progression of cerebral white matter abnormalities in VWM differ according to age at onset. Materials and Methods Patients with genetically confirmed VWM were stratified into six groups according to age at onset: younger than 1 year, 1 year to younger than 2 years, 2 years to younger than 4 years, 4 years to younger than 8 years, 8 years to younger than 18 years, and 18 years or older. With institutional review board approval, all available MRI scans obtained between 1985 and 2019 were retrospectively analyzed with three methods: (a) ratio of the width of the lateral ventricles over the skull (ventricle-to-skull ratio [VSR]) was measured to estimate brain atrophy; (b) cerebral white matter was visually scored as percentage normal, hyperintense, rarefied, or cystic on fluid-attenuated inversion recovery (FLAIR) images and converted into a white matter decay score; and (c) the intracranial volume was segmented into normal-appearing white and gray matter, abnormal but structurally present (FLAIR-hyperintense) and rarefied or cystic (FLAIR-hypointense) white matter, and ventricular and extracerebral cerebrospinal fluid (CSF). Multilevel regression analyses with patient as a clustering variable were performed to account for the nested data structure. Results A total of 461 examinations in 270 patients (median age, 7 years [interquartile range, 3-18 years]; 144 female patients) were evaluated; 112 patients had undergone serial imaging. Patients with later onset had higher VSR [F(5) = 8.42; P < .001] and CSF volume [F(5) = 21.7; P < .001] and lower white matter decay score [F(5) = 4.68; P < .001] and rarefied or cystic white matter volume [F(5) = 13.3; P < .001]. Rate of progression of white matter decay scores [b = -1.6, t(109) = -3.9; P < .001] and VSRs [b = -0.05, t (109) = -3.7; P < .001] were lower with later onset. Conclusion A radiologic spectrum based on age at onset exists in vanishing white matter. The earlier the onset, the faster and more cystic the white matter decay, whereas with later onset, white matter atrophy and gliosis predominate. © RSNA, 2021.

Radius-optimized efficient template matching for lesion detection from brain images

Computer-aided detection of brain lesions from volumetric magnetic resonance imaging (MRI) is in demand for fast and automatic diagnosis of neural diseases. The template-matching technique can provide satisfactory outcome for automatic localization of brain lesions; however, finding the optimal template size that maximizes similarity of the template and the lesion remains challenging. This increases the complexity of the algorithm and the requirement for computational resources, while processing large MRI volumes with three-dimensional (3D) templates. Hence, reducing the computational complexity of template matching is needed. In this paper, we first propose a mathematical framework for computing the normalized cross-correlation coefficient (NCCC) as the similarity measure between the MRI volume and approximated 3D Gaussian template with linear time complexity, [Formula: see text], as opposed to the conventional fast Fourier transform (FFT) based approach with the complexity [Formula: see text], where [Formula: see text] is the number of voxels in the image and [Formula: see text] is the number of tried template radii. We then propose a mathematical formulation to analytically estimate the optimal template radius for each voxel in the image and compute the NCCC with the location-dependent optimal radius, reducing the complexity to [Formula: see text]. We test our methods on one synthetic and two real multiple-sclerosis databases, and compare their performances in lesion detection with FFT and a state-of-the-art lesion prediction algorithm. We demonstrate through our experiments the efficiency of the proposed methods for brain lesion detection and their comparable performance with existing techniques.

Domain adaptation for segmentation of critical structures for prostate cancer therapy

Preoperative assessment of the proximity of critical structures to the tumors is crucial in avoiding unnecessary damage during prostate cancer treatment. A patient-specific 3D anatomical model of those structures, namely the neurovascular bundles (NVB) and the external urethral sphincters (EUS), can enable physicians to perform such assessments intuitively. As a crucial step to generate a patient-specific anatomical model from preoperative MRI in a clinical routine, we propose a multi-class automatic segmentation based on an anisotropic convolutional network. Our specific challenge is to train the network model on a unique source dataset only available at a single clinical site and deploy it to another target site without sharing the original images or labels. As network models trained on data from a single source suffer from quality loss due to the domain shift, we propose a semi-supervised domain adaptation (DA) method to refine the model's performance in the target domain. Our DA method combines transfer learning and uncertainty guided self-learning based on deep ensembles. Experiments on the segmentation of the prostate, NVB, and EUS, show significant performance gain with the combination of those techniques compared to pure TL and the combination of TL with simple self-learning ([Formula: see text] for all structures using a Wilcoxon's signed-rank test). Results on a different task and data (Pancreas CT segmentation) demonstrate our method's generic application capabilities. Our method has the advantage that it does not require any further data from the source domain, unlike the majority of recent domain adaptation strategies. This makes our method suitable for clinical applications, where the sharing of patient data is restricted.

Transfer learning in medical image segmentation: New insights from analysis of the dynamics of model parameters and learned representations

We present a critical assessment of the role of transfer learning in training fully convolutional networks (FCNs) for medical image segmentation. We first show that although transfer learning reduces the training time on the target task, improvements in segmentation accuracy are highly task/data-dependent. Large improvements are observed only when the segmentation task is more challenging and the target training data is smaller. We shed light on these observations by investigating the impact of transfer learning on the evolution of model parameters and learned representations. We observe that convolutional filters change little during training and still look random at convergence. We further show that quite accurate FCNs can be built by freezing the encoder section of the network at random values and only training the decoder section. At least for medical image segmentation, this finding challenges the common belief that the encoder section needs to learn data/task-specific representations. We examine the evolution of FCN representations to gain a deeper insight into the effects of transfer learning on the training dynamics. Our analysis shows that although FCNs trained via transfer learning learn different representations than FCNs trained with random initialization, the variability among FCNs trained via transfer learning can be as high as that among FCNs trained with random initialization. Moreover, feature reuse is not restricted to the early encoder layers; rather, it can be more significant in deeper layers. These findings offer new insights and suggest alternative ways of training FCNs for medical image segmentation.

Transductive Transfer Learning for Domain Adaptation in Brain Magnetic Resonance Image Segmentation

Segmentation of brain images from Magnetic Resonance Images (MRI) is an indispensable step in clinical practice. Morphological changes of sub-cortical brain structures and quantification of brain lesions are considered biomarkers of neurological and neurodegenerative disorders and used for diagnosis, treatment planning, and monitoring disease progression. In recent years, deep learning methods showed an outstanding performance in medical image segmentation. However, these methods suffer from generalisability problem due to inter-centre and inter-scanner variabilities of the MRI images. The main objective of the study is to develop an automated deep learning segmentation approach that is accurate and robust to the variabilities in scanner and acquisition protocols. In this paper, we propose a transductive transfer learning approach for domain adaptation to reduce the domain-shift effect in brain MRI segmentation. The transductive scenario assumes that there are sets of images from two different domains: (1) source-images with manually annotated labels; and (2) target-images without expert annotations. Then, the network is jointly optimised integrating both source and target images into the transductive training process to segment the regions of interest and to minimise the domain-shift effect. We proposed to use a histogram loss in the feature level to carry out the latter optimisation problem. In order to demonstrate the benefit of the proposed approach, the method has been tested in two different brain MRI image segmentation problems using multi-centre and multi-scanner databases for: (1) sub-cortical brain structure segmentation; and (2) white matter hyperintensities segmentation. The experiments showed that the segmentation performance of a pre-trained model could be significantly improved by up to 10%. For the first segmentation problem it was possible to achieve a maximum improvement from 0.680 to 0.799 in average Dice Similarity Coefficient (DSC) metric and for the second problem the average DSC improved from 0.504 to 0.602. Moreover, the improvements after domain adaptation were on par or showed better performance compared to the commonly used traditional unsupervised segmentation methods (FIRST and LST), also achieving faster execution time. Taking this into account, this work presents one more step toward the practical implementation of deep learning algorithms into the clinical routine.

Transfer Learning in Magnetic Resonance Brain Imaging: A Systematic Review

(1) Background: Transfer learning refers to machine learning techniques that focus on acquiring knowledge from related tasks to improve generalization in the tasks of interest. In magnetic resonance imaging (MRI), transfer learning is important for developing strategies that address the variation in MR images from different imaging protocols or scanners. Additionally, transfer learning is beneficial for reutilizing machine learning models that were trained to solve different (but related) tasks to the task of interest. The aim of this review is to identify research directions, gaps in knowledge, applications, and widely used strategies among the transfer learning approaches applied in MR brain imaging; (2) Methods: We performed a systematic literature search for articles that applied transfer learning to MR brain imaging tasks. We screened 433 studies for their relevance, and we categorized and extracted relevant information, including task type, application, availability of labels, and machine learning methods. Furthermore, we closely examined brain MRI-specific transfer learning approaches and other methods that tackled issues relevant to medical imaging, including privacy, unseen target domains, and unlabeled data; (3) Results: We found 129 articles that applied transfer learning to MR brain imaging tasks. The most frequent applications were dementia-related classification tasks and brain tumor segmentation. The majority of articles utilized transfer learning techniques based on convolutional neural networks (CNNs). Only a few approaches utilized clearly brain MRI-specific methodology, and considered privacy issues, unseen target domains, or unlabeled data. We proposed a new categorization to group specific, widely-used approaches such as pretraining and fine-tuning CNNs; (4) Discussion: There is increasing interest in transfer learning for brain MRI. Well-known public datasets have clearly contributed to the popularity of Alzheimer's diagnostics/prognostics and tumor segmentation as applications. Likewise, the availability of pretrained CNNs has promoted their utilization. Finally, the majority of the surveyed studies did not examine in detail the interpretation of their strategies after applying transfer learning, and did not compare their approach with other transfer learning approaches.

Deep learning segmentation of gadolinium-enhancing lesions in multiple sclerosis

OBJECTIVE

The aim of this study is to assess the performance of deep learning convolutional neural networks (CNNs) in segmenting gadolinium-enhancing lesions using a large cohort of multiple sclerosis (MS) patients.

METHODS

A three-dimensional (3D) CNN model was trained for segmentation of gadolinium-enhancing lesions using multispectral magnetic resonance imaging data (MRI) from 1006 relapsing-remitting MS patients. The network performance was evaluated for three combinations of multispectral MRI used as input: (U5) fluid-attenuated inversion recovery (FLAIR), T2-weighted, proton density-weighted, and pre- and post-contrast T1-weighted images; (U2) pre- and post-contrast T1-weighted images; and (U1) only post-contrast T1-weighted images. Segmentation performance was evaluated using the Dice similarity coefficient (DSC) and lesion-wise true-positive (TPR) and false-positive (FPR) rates. Performance was also evaluated as a function of enhancing lesion volume.

RESULTS

The DSC/TPR/FPR values averaged over all the enhancing lesion sizes were 0.77/0.90/0.23 using the U5 model. These values for the largest enhancement volumes (>500 mm3) were 0.81/0.97/0.04. For U2, the average DSC/TPR/FPR values were 0.72/0.86/0.31. Comparable performance was observed with U1. For all types of input, the network performance degraded with decreased enhancement size.

CONCLUSION

Excellent segmentation of enhancing lesions was observed for enhancement volume ⩾70 mm3. The best performance was achieved when the input included all five multispectral image sets.

JOINT SEGMENTATION OF MULTIPLE SCLEROSIS LESIONS AND BRAIN ANATOMY IN MRI SCANS OF ANY CONTRAST AND RESOLUTION WITH CNNs

We present the first deep learning method to segment Multiple Sclerosis lesions and brain structures from MRI scans of any (possibly multimodal) contrast and resolution. Our method only requires segmentations to be trained (no images), as it leverages the generative model of Bayesian segmentation to generate synthetic scans with simulated lesions, which are then used to train a CNN. Our method can be retrained to segment at any resolution by adjusting the amount of synthesised partial volume. By construction, the synthetic scans are perfectly aligned with their labels, which enables training with noisy labels obtained with automatic methods. The training data are generated on the fly, and aggressive augmentation (including artefacts) is applied for improved generalisation. We demonstrate our method on two public datasets, comparing it with a state-of-the-art Bayesian approach implemented in FreeSurfer, and dataset specific CNNs trained on real data. The code is available at https://github.com/BBillot/SynthSeg.