Survey of automated multiple sclerosis lesion segmentation techniques on magnetic resonance imaging

Machine learning for refining interpretation of magnetic resonance imaging scans in the management of multiple sclerosis: a narrative review

Multiple sclerosis (MS) is an autoimmune disease of the brain and spinal cord with both inflammatory and neurodegenerative features. Although advances in imaging techniques, particularly magnetic resonance imaging (MRI), have improved the process of diagnosis, its cause is unknown, a cure remains elusive and the evidence base to guide treatment is lacking. Computational techniques like machine learning (ML) have started to be used to understand MS. Published MS MRI-based computational studies can be divided into five categories: automated diagnosis; differentiation between lesion types and/or MS stages; differential diagnosis; monitoring and predicting disease progression; and synthetic MRI dataset generation. Collectively, these approaches show promise in assisting with MS diagnosis, monitoring of disease activity and prediction of future progression, all potentially contributing to disease management. Analysis quality using ML is highly dependent on the dataset size and variability used for training. Wider public access would mean larger datasets for experimentation, resulting in higher-quality analysis, permitting for more conclusive research. This narrative review provides an outline of the fundamentals of MS pathology and pathogenesis, diagnostic techniques and data types in computational analysis, as well as collating literature pertaining to the application of computational techniques to MRI towards developing a better understanding of MS.

Enhancing classification of active and non-active lesions in multiple sclerosis: machine learning models and feature selection techniques

INTRODUCTION

Gadolinium-based T1-weighted MRI sequence is the gold standard for the detection of active multiple sclerosis (MS) lesions. The performance of machine learning (ML) and deep learning (DL) models in the classification of active and non-active MS lesions from the T2-weighted MRI images has been investigated in this study.

METHODS

107 Features of 75 active and 100 non-active MS lesions were extracted by using SegmentEditor and Radiomics modules of 3D slicer software. Sixteen ML and one sequential DL models were created using the 5-fold cross-validation method and each model with its special optimized parameters trained using the training-validation datasets. Models' performances in test data set were evaluated by metric parameters of accuracy, precision, sensitivity, specificity, AUC, and F1 score.

RESULTS

The sequential DL model achieved the highest AUC of 95.60% on the test dataset, demonstrating its superior ability to distinguish between active and non-active plaques. Among traditional ML models, the Hybrid Gradient Boosting Classifier (HGBC) demonstrated a commendable test AUC of 86.75%, while the Gradient Boosting Classifier (GBC) excelled in cross-validation with an AUC of 87.92%.

CONCLUSION

The performance of sixteen ML and one sequential DL models in the classification of active and non-active MS lesions was evaluated. The results of the study highlight the effectiveness of sequential DL approach and ensemble methods in achieving robust predictive performance, underscoring their potential applications in classifying MS plaques.

Automated assessment of brain MRIs in multiple sclerosis patients significantly reduces reading time

INTRODUCTION

Assessment of multiple sclerosis (MS) lesions on magnetic resonance imaging (MRI) is tedious, time-consuming, and error-prone. We evaluate whether assessment of new, expanding, and contrast-enhancing MS lesions can be done more time-efficiently by radiologists with assistance of artificial intelligence (AI).

METHODS

Baseline and three follow-up (FU) MRIs of thirty-five consecutive patients diagnosed with MS were assessed by a radiologist manually, and with assistance of an AI-tool. Results were discussed with a consultant neuroradiologist and time metrics were evaluated.

RESULTS

The mean reading time for the resident radiologist was 9.05 min (95CI: 6.85-11:25). With AI-assistance, the reading time was reduced by 2.83 min (95CI: 3.28-2.41, p < 0.001). The reading decreased steadily from baseline to FU3 for the resident radiologist (9.85 min baseline, 9.21 FU1, 8.64 FU2 and 8.44 FU3, p < 0.001). Assistance of AI further remarkably decreased reading times during follow-ups (3.29 min FU1, 3.92 FU2, 3.79 FU3, p < 0.001) but not at baseline (0.26 min, p = 0.96). The baseline reading time of the resident radiologist was 5.04 min (p < 0.001), with each lesion adding 0.14 min (p < 0.001). There was a substantial decrease in the baseline reading time from 5.04 min to 1.59 min (p = 0.23) with AI-assistance. Discussion of the reading results of the resident with the neuroradiology consultant (as usual in clinical routine) was exemplary done for FU-3 MRIs and added another 3 min (CI:2.27-3.76) to the reading time without AI-assistance.

CONCLUSION

We found that AI-assisted reading of MRIs of patients with MS may be faster than evaluating these MRIs without AI-assistance.

Perceptual super-resolution in multiple sclerosis MRI

Introduction

Magnetic resonance imaging (MRI) is crucial for diagnosing and monitoring of multiple sclerosis (MS) as it is used to assess lesions in the brain and spinal cord. However, in real-world clinical settings, MRI scans are often acquired with thick slices, limiting their utility for automated quantitative analyses. This work presents a single-image super-resolution (SR) reconstruction framework that leverages SR convolutional neural networks (CNN) to enhance the through-plane resolution of structural MRI in people with MS (PwMS).

Methods

Our strategy involves the supervised fine-tuning of CNN architectures, guided by a content loss function that promotes perceptual quality, as well as reconstruction accuracy, to recover high-level image features.

Results

Extensive evaluation with MRI data of PwMS shows that our SR strategy leads to more accurate MRI reconstructions than competing methods. Furthermore, it improves lesion segmentation on low-resolution MRI, approaching the performance achievable with high-resolution images.

Discussion

Results demonstrate the potential of our SR framework to facilitate the use of low-resolution retrospective MRI from real-world clinical settings to investigate quantitative image-based biomarkers of MS.

Current and future role of MRI in the diagnosis and prognosis of multiple sclerosis

In the majority of cases, multiple sclerosis (MS) is characterized by reversible episodes of neurological dysfunction, often followed by irreversible clinical disability. Accurate diagnostic criteria and prognostic markers are critical to enable early diagnosis and correctly identify patients with MS at increased risk of disease progression. The 2017 McDonald diagnostic criteria, which include magnetic resonance imaging (MRI) as a fundamental paraclinical tool, show high sensitivity and accuracy for the diagnosis of MS allowing early diagnosis and treatment. However, their inappropriate application, especially in the context of atypical clinical presentations, may increase the risk of misdiagnosis. To further improve the diagnostic process, novel imaging markers are emerging, but rigorous validation and standardization is still needed before they can be incorporated into clinical practice. This Series article discusses the current role of MRI in the diagnosis and prognosis of MS, while examining promising MRI markers, which could serve as reliable predictors of subsequent disease progression, helping to optimize the management of individual patients with MS. We also explore the potential of new technologies, such as artificial intelligence and automated quantification tools, to support clinicians in the management of patients. Yet, to ensure consistency and improvement in the use of MRI in MS diagnosis and patient follow-up, it is essential that standardized brain and spinal cord MRI protocols are applied, and that interpretation of results is performed by qualified (neuro)radiologists in all countries.

Artificial intelligence applied to MRI data to tackle key challenges in multiple sclerosis

Artificial intelligence (AI) is the branch of science aiming at creating algorithms able to carry out tasks that typically require human intelligence. In medicine, there has been a tremendous increase in AI applications thanks to increasingly powerful computers and the emergence of big data repositories. Multiple sclerosis (MS) is a chronic autoimmune condition affecting the central nervous system with a complex pathogenesis, a challenging diagnostic process strongly relying on magnetic resonance imaging (MRI) and a high and largely unexplained variability across patients. Therefore, AI applications in MS have the great potential of helping us better support the diagnosis, find markers for prognosis to eventually design more powerful randomised clinical trials and improve patient management in clinical practice and eventually understand the mechanisms of the disease. This topical review aims to summarise the recent advances in AI applied to MRI data in MS to illustrate its achievements, limitations and future directions.

Assessing robustness and generalization of a deep neural network for brain MS lesion segmentation on real-world data

OBJECTIVES

Evaluate the performance of a deep learning (DL)-based model for multiple sclerosis (MS) lesion segmentation and compare it to other DL and non-DL algorithms.

METHODS

This ambispective, multicenter study assessed the performance of a DL-based model for MS lesion segmentation and compared it to alternative DL- and non-DL-based methods. Models were tested on internal (n = 20) and external (n = 18) datasets from Latin America, and on an external dataset from Europe (n = 49). We also examined robustness by rescanning six patients (n = 6) from our MS clinical cohort. Moreover, we studied inter-human annotator agreement and discussed our findings in light of these results. Performance and robustness were assessed using intraclass correlation coefficient (ICC), Dice coefficient (DC), and coefficient of variation (CV).

RESULTS

Inter-human ICC ranged from 0.89 to 0.95, while spatial agreement among annotators showed a median DC of 0.63. Using expert manual segmentations as ground truth, our DL model achieved a median DC of 0.73 on the internal, 0.66 on the external, and 0.70 on the challenge datasets. The performance of our DL model exceeded that of the alternative algorithms on all datasets. In the robustness experiment, our DL model also achieved higher DC (ranging from 0.82 to 0.90) and lower CV (ranging from 0.7 to 7.9%) when compared to the alternative methods.

CONCLUSION

Our DL-based model outperformed alternative methods for brain MS lesion segmentation. The model also proved to generalize well on unseen data and has a robust performance and low processing times both on real-world and challenge-based data.

CLINICAL RELEVANCE STATEMENT

Our DL-based model demonstrated superior performance in accurately segmenting brain MS lesions compared to alternative methods, indicating its potential for clinical application with improved accuracy, robustness, and efficiency.

KEY POINTS

• Automated lesion load quantification in MS patients is valuable; however, more accurate methods are still necessary. • A novel deep learning model outperformed alternative MS lesion segmentation methods on multisite datasets. • Deep learning models are particularly suitable for MS lesion segmentation in clinical scenarios.

BIANCA-MS: An optimized tool for automated multiple sclerosis lesion segmentation

In this work we present BIANCA-MS, a novel tool for brain white matter lesion segmentation in multiple sclerosis (MS), able to generalize across both the wide spectrum of MRI acquisition protocols and the heterogeneity of manually labeled data. BIANCA-MS is based on the original version of BIANCA and implements two innovative elements: a harmonized setting, tested under different MRI protocols, which avoids the need to further tune algorithm parameters to each dataset; and a cleaning step developed to improve consistency in automated and manual segmentations, thus reducing unwanted variability in output segmentations and validation data. BIANCA-MS was tested on three datasets, acquired with different MRI protocols. First, we compared BIANCA-MS to other widely used tools. Second, we tested how BIANCA-MS performs in separate datasets. Finally, we evaluated BIANCA-MS performance on a pooled dataset where all MRI data were merged. We calculated the overlap using the DICE spatial similarity index (SI) as well as the number of false positive/negative clusters (nFPC/nFNC) in comparison to the manual masks processed with the cleaning step. BIANCA-MS clearly outperformed other available tools in both high- and low-resolution images and provided comparable performance across different scanning protocols, sets of modalities and image resolutions. BIANCA-MS performance on the pooled dataset (SI: 0.72 ± 0.25, nFPC: 13 ± 11, nFNC: 4 ± 8) were comparable to those achieved on each individual dataset (median across datasets SI: 0.72 ± 0.28, nFPC: 14 ± 11, nFNC: 4 ± 8). Our findings suggest that BIANCA-MS is a robust and accurate approach for automated MS lesion segmentation.

AI-based detection of contrast-enhancing MRI lesions in patients with multiple sclerosis

BACKGROUND

Contrast-enhancing (CE) lesions are an important finding on brain magnetic resonance imaging (MRI) in patients with multiple sclerosis (MS) but can be missed easily. Automated solutions for reliable CE lesion detection are emerging; however, independent validation of artificial intelligence (AI) tools in the clinical routine is still rare.

METHODS

A three-dimensional convolutional neural network for CE lesion segmentation was trained externally on 1488 datasets of 934 MS patients from 81 scanners using concatenated information from FLAIR and T1-weighted post-contrast imaging. This externally trained model was tested on an independent dataset comprising 504 T1-weighted post-contrast and FLAIR image datasets of MS patients from clinical routine. Two neuroradiologists (R1, R2) labeled CE lesions for gold standard definition in the clinical test dataset. The algorithmic output was evaluated on both patient- and lesion-level.

RESULTS

On a patient-level, recall, specificity, precision, and accuracy of the AI tool to predict patients with CE lesions were 0.75, 0.99, 0.91, and 0.96. The agreement between the AI tool and both readers was within the range of inter-rater agreement (Cohen's kappa; AI vs. R1: 0.69; AI vs. R2: 0.76; R1 vs. R2: 0.76). On a lesion-level, false negative lesions were predominately found in infratentorial location, significantly smaller, and at lower contrast than true positive lesions (p < 0.05).

CONCLUSIONS

AI-based identification of CE lesions on brain MRI is feasible, approaching human reader performance in independent clinical data and might be of help as a second reader in the neuroradiological assessment of active inflammation in MS patients.

CRITICAL RELEVANCE STATEMENT

Al-based detection of contrast-enhancing multiple sclerosis lesions approaches human reader performance, but careful visual inspection is still needed, especially for infratentorial, small and low-contrast lesions.

Scanner agnostic large-scale evaluation of MS lesion delineation tool for clinical MRI

Introduction

Patients with MS are MRI scanned continuously throughout their disease course resulting in a large manual workload for radiologists which includes lesion detection and size estimation. Though many models for automatic lesion segmentation have been published, few are used broadly in clinic today, as there is a lack of testing on clinical datasets. By collecting a large, heterogeneous training dataset directly from our MS clinic we aim to present a model which is robust to different scanner protocols and artefacts and which only uses MRI modalities present in routine clinical examinations.

Methods

We retrospectively included 746 patients from routine examinations at our MS clinic. The inclusion criteria included acquisition at one of seven different scanners and an MRI protocol including 2D or 3D T2-w FLAIR, T2-w and T1-w images. Reference lesion masks on the training (n = 571) and validation (n = 70) datasets were generated using a preliminary segmentation model and subsequent manual correction. The test dataset (n = 100) was manually delineated. Our segmentation model https://github.com/CAAI/AIMS/ was based on the popular nnU-Net, which has won several biomedical segmentation challenges. We tested our model against the published segmentation models HD-MS-Lesions, which is also based on nnU-Net, trained with a more homogenous patient cohort. We furthermore tested model robustness to data from unseen scanners by performing a leave-one-scanner-out experiment.

Results

We found that our model was able to segment MS white matter lesions with a performance comparable to literature: DSC = 0.68, precision = 0.90, recall = 0.70, f1 = 0.78. Furthermore, the model outperformed HD-MS-Lesions in all metrics except precision = 0.96. In the leave-one-scanner-out experiment there was no significant change in performance (p < 0.05) between any of the models which were only trained on part of the dataset and the full segmentation model.

Conclusion

In conclusion we have seen, that by including a large, heterogeneous dataset emulating clinical reality, we have trained a segmentation model which maintains a high segmentation performance while being robust to data from unseen scanners. This broadens the applicability of the model in clinic and paves the way for clinical implementation.

Multiple sclerosis lesion segmentation: revisiting weighting mechanisms for federated learning

Background and introduction

Federated learning (FL) has been widely employed for medical image analysis to facilitate multi-client collaborative learning without sharing raw data. Despite great success, FL's applications remain suboptimal in neuroimage analysis tasks such as lesion segmentation in multiple sclerosis (MS), due to variance in lesion characteristics imparted by different scanners and acquisition parameters.

Methods

In this work, we propose the first FL MS lesion segmentation framework via two effective re-weighting mechanisms. Specifically, a learnable weight is assigned to each local node during the aggregation process, based on its segmentation performance. In addition, the segmentation loss function in each client is also re-weighted according to the lesion volume for the data during training.

Results

The proposed method has been validated on two FL MS segmentation scenarios using public and clinical datasets. Specifically, the case-wise and voxel-wise Dice score of the proposed method under the first public dataset is 65.20 and 74.30, respectively. On the second in-house dataset, the case-wise and voxel-wise Dice score is 53.66, and 62.31, respectively.

Discussions and conclusions

The Comparison experiments on two FL MS segmentation scenarios using public and clinical datasets have demonstrated the effectiveness of the proposed method by significantly outperforming other FL methods. Furthermore, the segmentation performance of FL incorporating our proposed aggregation mechanism can achieve comparable performance to that from centralized training with all the raw data.

Automated Registration and Color Labeling of Serial 3D Double Inversion Recovery MR Imaging for Detection of Lesion Progression in Multiple Sclerosis

Automated co-registration and subtraction techniques have been shown to be useful in the assessment of longitudinal changes in multiple sclerosis (MS) lesion burden, but the majority depend on T2-fluid-attenuated inversion recovery sequences. We aimed to investigate the use of a novel automated temporal color complement imaging (CCI) map overlapped on 3D double inversion recovery (DIR), and to assess its diagnostic performance for detecting disease progression in patients with multiple sclerosis (MS) as compared to standard review of serial 3D DIR images. We developed a fully automated system that co-registers and compares baseline to follow-up 3D DIR images and outputs a pseudo-color RGB map in which red pixels indicate increased intensity values in the follow-up image (i.e., progression; new/enlarging lesion), blue-green pixels represent decreased intensity values (i.e., disappearing/shrinking lesion), and gray-scale pixels reflect unchanged intensity values. Three neuroradiologists blinded to clinical information independently reviewed each patient using standard DIR images alone and using CCI maps based on DIR images at two separate exams. Seventy-six follow-up examinations from 60 consecutive MS patients who underwent standard 3 T MR brain MS protocol that included 3D DIR were included. Median cohort age was 38.5 years, with 46 women, 59 relapsing-remitting type MS, and median follow-up interval of 250 days (interquartile range: 196-394 days). Lesion progression was detected in 67.1% of cases using CCI review versus 22.4% using standard review, with a total of 182 new or enlarged lesions using CCI review versus 28 using standard review. There was a statistically significant difference between the two methods in the rate of all progressive lesions (P < 0.001, McNemar's test) as well as cortical progressive lesions (P < 0.001). Automated CCI maps using co-registered serial 3D DIR, compared to standard review of 3D DIR alone, increased detection rate of MS lesion progression in patients undergoing clinical brain MRI exam.

Combining Clinical and Genetic Data to Predict Response to Fingolimod Treatment in Relapsing Remitting Multiple Sclerosis Patients: A Precision Medicine Approach

A personalized approach is strongly advocated for treatment selection in Multiple Sclerosis patients due to the high number of available drugs. Machine learning methods proved to be valuable tools in the context of precision medicine. In the present work, we applied machine learning methods to identify a combined clinical and genetic signature of response to fingolimod that could support the prediction of drug response. Two cohorts of fingolimod-treated patients from Italy and France were enrolled and divided into training, validation, and test set. Random forest training and robust feature selection were performed in the first two sets respectively, and the independent test set was used to evaluate model performance. A genetic-only model and a combined clinical-genetic model were obtained. Overall, 381 patients were classified according to the NEDA-3 criterion at 2 years; we identified a genetic model, including 123 SNPs, that was able to predict fingolimod response with an AUROC= 0.65 in the independent test set. When combining clinical data, the model accuracy increased to an AUROC= 0.71. Integrating clinical and genetic data by means of machine learning methods can help in the prediction of response to fingolimod, even though further studies are required to definitely extend this approach to clinical applications.

Commercial volumetric MRI reporting tools in multiple sclerosis: a systematic review of the evidence

PURPOSE

MRI is integral to the diagnosis of multiple sclerosis (MS) and is important for clinical prognostication. Quantitative volumetric reporting tools (QReports) can improve the accuracy and objectivity of MRI-based assessments. Several QReports are commercially available; however, validation can be difficult to establish and does not currently follow a common pathway. To aid evidence-based clinical decision-making, we performed a systematic review of commercial QReports for use in MS including technical details and published reports of validation and in-use evaluation.

METHODS

We categorized studies into three types of testing: technical validation, for example, comparison to manual segmentation, clinical validation by clinicians or interpretation of results alongside clinician-rated variables, and in-use evaluation, such as health economic assessment.

RESULTS

We identified 10 companies, which provide MS lesion and brain segmentation and volume quantification, and 38 relevant publications. Tools received regulatory approval between 2006 and 2020, contextualize results to normative reference populations, ranging from 620 to 8000 subjects, and require T1- and T2-FLAIR-weighted input sequences for longitudinal assessment of whole-brain volume and lesions. In MS, six QReports provided evidence of technical validation, four companies have conducted clinical validation by correlating results with clinical variables, only one has tested their QReport by clinician end-users, and one has performed a simulated in-use socioeconomic evaluation.

CONCLUSION

We conclude that there is limited evidence in the literature regarding clinical validation and in-use evaluation of commercial MS QReports with a particular lack of clinician end-user testing. Our systematic review provides clinicians and institutions with the available evidence when considering adopting a quantitative reporting tool for MS.

Tract-based white matter hyperintensity patterns in patients with systemic lupus erythematosus using an unsupervised machine learning approach

Currently, little is known about the spatial distribution of white matter hyperintensities (WMH) in the brain of patients with Systemic Lupus erythematosus (SLE). Previous lesion markers, such as number and volume, ignore the strategic location of WMH. The goal of this work was to develop a fully-automated method to identify predominant patterns of WMH across WM tracts based on cluster analysis. A total of 221 SLE patients with and without neuropsychiatric symptoms from two different sites were included in this study. WMH segmentations and lesion locations were acquired automatically. Cluster analysis was performed on the WMH distribution in 20 WM tracts. Our pipeline identified five distinct clusters with predominant involvement of the forceps major, forceps minor, as well as right and left anterior thalamic radiations and the right inferior fronto-occipital fasciculus. The patterns of the affected WM tracts were consistent over the SLE subtypes and sites. Our approach revealed distinct and robust tract-based WMH patterns within SLE patients. This method could provide a basis, to link the location of WMH with clinical symptoms. Furthermore, it could be used for other diseases characterized by presence of WMH to investigate both the clinical relevance of WMH and underlying pathomechanism in the brain.

Validation of a Denoising Method Using Deep Learning-Based Reconstruction to Quantify Multiple Sclerosis Lesion Load on Fast FLAIR Imaging

BACKGROUND AND PURPOSE

Accurate quantification of WM lesion load is essential for the care of patients with multiple sclerosis. We tested whether the combination of accelerated 3D-FLAIR and denoising using deep learning-based reconstruction could provide a relevant strategy while shortening the imaging examination.

MATERIALS AND METHODS

Twenty-eight patients with multiple sclerosis were prospectively examined using 4 implementations of 3D-FLAIR with decreasing scan times (4 minutes 54 seconds, 2 minutes 35 seconds, 1 minute 40 seconds, and 1 minute 15 seconds). Each FLAIR sequence was reconstructed without and with denoising using deep learning-based reconstruction, resulting in 8 FLAIR sequences per patient. Image quality was assessed with the Likert scale, apparent SNR, and contrast-to-noise ratio. Manual and automatic lesion segmentations, performed randomly and blindly, were quantitatively evaluated against ground truth using the absolute volume difference, true-positive rate, positive predictive value, Dice similarity coefficient, Hausdorff distance, and F1 score based on the lesion count. The Wilcoxon signed-rank test and 2-way ANOVA were performed.

RESULTS

Both image-quality evaluation and the various metrics showed deterioration when the FLAIR scan time was accelerated. However, denoising using deep learning-based reconstruction significantly improved subjective image quality and quantitative performance metrics, particularly for manual segmentation. Overall, denoising using deep learning-based reconstruction helped to recover contours closer to those from the criterion standard and to capture individual lesions otherwise overlooked. The Dice similarity coefficient was equivalent between the 2-minutes-35-seconds-long FLAIR with denoising using deep learning-based reconstruction and the 4-minutes-54-seconds-long reference FLAIR sequence.

CONCLUSIONS

Denoising using deep learning-based reconstruction helps to recognize multiple sclerosis lesions buried in the noise of accelerated FLAIR acquisitions, a possibly useful strategy to efficiently shorten the scan time in clinical practice.

New MS lesion segmentation with deep residual attention gate U-Net utilizing 2D slices of 3D MR images

Multiple sclerosis (MS) is an autoimmune disease that causes lesions in the central nervous system of humans due to demyelinating axons. Magnetic resonance imaging (MRI) is widely used for monitoring and measuring MS lesions. Automated methods for MS lesion segmentation have usually been performed on individual MRI scans. Recently, tracking lesion activity for quantifying and monitoring MS disease progression, especially detecting new lesions, has become an important biomarker. In this study, a unique pipeline with a deep neural network that combines U-Net, attention gate, and residual learning is proposed to perform better new MS lesion segmentation using baseline and follow-up 3D FLAIR MR images. The proposed network has a similar architecture to U-Net and is formed from residual units which facilitate the training of deep networks. Networks with fewer parameters are designed with better performance through the skip connections of U-Net and residual units, which facilitate information propagation without degradation. Attention gates also learn to focus on salient features of the target structures of various sizes and shapes. The MSSEG-2 dataset was used for training and testing the proposed pipeline, and the results were compared with those of other proposed pipelines of the challenge and experts who participated in the same challenge. According to the results over the testing set, the lesion-wise F1 and dice scores were obtained as a mean of 48 and 44.30%. For the no-lesion cases, the number of tested and volume of tested lesions were obtained as a mean of 0.148 and 1.488, respectively. The proposed pipeline outperformed 22 proposed pipelines and ranked 8th in the challenge.

Role of artificial intelligence in MS clinical practice

Machine learning (ML) and its subset, deep learning (DL), are branches of artificial intelligence (AI) showing promising findings in the medical field, especially when applied to imaging data. Given the substantial role of MRI in the diagnosis and management of patients with multiple sclerosis (MS), this disease is an ideal candidate for the application of AI techniques. In this narrative review, we are going to discuss the potential applications of AI for MS clinical practice, together with their limitations. Among their several advantages, ML algorithms are able to automate repetitive tasks, to analyze more data in less time and to achieve higher accuracy and reproducibility than the human counterpart. To date, these algorithms have been applied to MS diagnosis, prognosis, disease and treatment monitoring. Other fields of application have been improvement of MRI protocols as well as automated lesion and tissue segmentation. However, several challenges remain, including a better understanding of the information selected by AI algorithms, appropriate multicenter and longitudinal validations of results and practical aspects regarding hardware and software integration. Finally, one cannot overemphasize the paramount importance of human supervision, in order to optimize the use and take full advantage of the potential of AI approaches.

Automatic segmentation of white matter hyperintensities: validation and comparison with state-of-the-art methods on both Multiple Sclerosis and elderly subjects

•

Automatic segmentation of MS lesions and age-related WMH from 3D T1 and T2-FLAIR.

•

Comparison to consensus show improved performance of WHASA-3D compared to WHASA.

•

WHASA-3D outperforms available state-of-the-art methods with their default settings.

•

WHASA-3D could be a useful tool for clinical practice and clinical trials.

Different types of white matter hyperintensities (WMH) can be observed through MRI in the brain and spinal cord, especially Multiple Sclerosis (MS) lesions for patients suffering from MS and age-related WMH for subjects with cognitive disorders and/or elderly people. To better diagnose and monitor the disease progression, the quantitative evaluation of WMH load has proven to be useful for clinical routine and trials. Since manual delineation for WMH segmentation is highly time-consuming and suffers from intra and inter observer variability, several methods have been proposed to automatically segment either MS lesions or age-related WMH, but none is validated on both WMH types. Here, we aim at proposing the White matter Hyperintensities Automatic Segmentation Algorithm adapted to 3D T2-FLAIR datasets (WHASA-3D), a fast and robust automatic segmentation tool designed to be implemented in clinical practice for the detection of both MS lesions and age-related WMH in the brain, using both 3D T1-weighted and T2-FLAIR images. In order to increase its robustness for MS lesions, WHASA-3D expands the original WHASA method, which relies on the coupling of non-linear diffusion framework and watershed parcellation, where regions considered as WMH are selected based on intensity and location characteristics, and finally refined with geodesic dilation. The previous validation was performed on 2D T2-FLAIR and subjects with cognitive disorders and elderly subjects. 60 subjects from a heterogeneous database of dementia patients, multiple sclerosis patients and elderly subjects with multiple MRI scanners and a wide range of lesion loads were used to evaluate WHASA and WHASA-3D through volume and spatial agreement in comparison with consensus reference segmentations. In addition, a direct comparison on the MS database with six available supervised and unsupervised state-of-the-art WMH segmentation methods (LST-LGA and LPA, Lesion-TOADS, lesionBrain, BIANCA and nicMSlesions) with default and optimised settings (when feasible) was conducted. WHASA-3D confirmed an improved performance with respect to WHASA, achieving a better spatial overlap (Dice) (0.67 vs 0.63), a reduced absolute volume error (AVE) (3.11 vs 6.2 mL) and an increased volume agreement (intraclass correlation coefficient, ICC) (0.96 vs 0.78). Compared to available state-of-the-art algorithms on the MS database, WHASA-3D outperformed both unsupervised and supervised methods when used with their default settings, showing the highest volume agreement (ICC = 0.95) as well as the highest average Dice (0.58). Optimising and/or retraining LST-LGA, BIANCA and nicMSlesions, using a subset of the MS database as training set, resulted in improved performances on the remaining testing set (average Dice: LST-LGA default/optimized = 0.41/0.51, BIANCA default/optimized = 0.22/0.39, nicMSlesions default/optimized = 0.17/0.63, WHASA-3D = 0.58). Evaluation and comparison results suggest that WHASA-3D is a reliable and easy-to-use method for the automated segmentation of white matter hyperintensities, for both MS lesions and age-related WMH. Further validation on larger datasets would be useful to confirm these first findings.

Multiple Sclerosis Lesion Analysis in Brain Magnetic Resonance Images: Techniques and Clinical Applications

Multiple sclerosis (MS) is a chronic inflammatory and degenerative disease of the central nervous system, characterized by the appearance of focal lesions in the white and gray matter that topographically correlate with an individual patients neurological symptoms and signs. Magnetic resonance imaging (MRI) provides detailed in-vivo structural information, permitting the quantification and categorization of MS lesions that critically inform disease management. Traditionally, MS lesions have been manually annotated on 2D MRI slices, a process that is inefficient and prone to inter-/intra-observer errors. Recently, automated statistical imaging analysis techniques have been proposed to detect and segment MS lesions based on MRI voxel intensity. However, their effectiveness is limited by the heterogeneity of both MRI data acquisition techniques and the appearance of MS lesions. By learning complex lesion representations directly from images, deep learning techniques have achieved remarkable breakthroughs in the MS lesion segmentation task. Here, we provide a comprehensive review of state-of-the-art automatic statistical and deep-learning MS segmentation methods and discuss current and future clinical applications. Further, we review technical strategies, such as domain adaptation, to enhance MS lesion segmentation in real-world clinical settings.

Assessment of automatic decision-support systems for detecting active T2 lesions in multiple sclerosis patients

BACKGROUND

Active (new/enlarging) T2 lesion counts are routinely used in the clinical management of multiple sclerosis. Thus, automated tools able to accurately identify active T2 lesions would be of high interest to neuroradiologists for assisting in their clinical activity.

OBJECTIVE

To compare the accuracy in detecting active T2 lesions and of radiologically active patients based on different visual and automated methods.

METHODS

One hundred multiple sclerosis patients underwent two magnetic resonance imaging examinations within 12 months. Four approaches were assessed for detecting active T2 lesions: (1) conventional neuroradiological reports; (2) prospective visual analyses performed by an expert; (3) automated unsupervised tool; and (4) supervised convolutional neural network. As a gold standard, a reference outcome was created by the consensus of two observers.

RESULTS

The automated methods detected a higher number of active T2 lesions, and a higher number of active patients, but a higher number of false-positive active patients than visual methods. The convolutional neural network model was more sensitive in detecting active T2 lesions and active patients than the other automated method.

CONCLUSION

Automated convolutional neural network models show potential as an aid to neuroradiological assessment in clinical practice, although visual supervision of the outcomes is still required.

Infratentorial lesions in multiple sclerosis patients: intra- and inter-rater variability in comparison to a fully automated segmentation using 3D convolutional neural networks

OBJECTIVE

Automated quantification of infratentorial multiple sclerosis lesions on magnetic resonance imaging is clinically relevant but challenging. To overcome some of these problems, we propose a fully automated lesion segmentation algorithm using 3D convolutional neural networks (CNNs).

METHODS

The CNN was trained on a FLAIR image alone or on FLAIR and T1-weighted images from 1809 patients acquired on 156 different scanners. An additional training using an extra class for infratentorial lesions was implemented. Three experienced raters manually annotated three datasets from 123 MS patients from different scanners.

RESULTS

The inter-rater sensitivity (SEN) was 80% for supratentorial lesions but only 62% for infratentorial lesions. There was no statistically significant difference between the inter-rater SEN and the SEN of the CNN with respect to the raters. For supratentorial lesions, the CNN featured an intra-rater intra-scanner SEN of 0.97 (R1 = 0.90, R2 = 0.84) and for infratentorial lesion a SEN of 0.93 (R1 = 0.61, R2 = 0.73).

CONCLUSION

The performance of the CNN improved significantly for infratentorial lesions when specifically trained on infratentorial lesions using a T1 image as an additional input and matches the detection performance of experienced raters. Furthermore, for infratentorial lesions the CNN was more robust against repeated scans than experienced raters.

KEY POINTS

• A 3D convolutional neural network was trained on MRI data from 1809 patients (156 different scanners) for the quantification of supratentorial and infratentorial multiple sclerosis lesions. • Inter-rater variability was higher for infratentorial lesions than for supratentorial lesions. The performance of the 3D convolutional neural network (CNN) improved significantly for infratentorial lesions when specifically trained on infratentorial lesions using a T1 image as an additional input. • The detection performance of the CNN matches the detection performance of experienced raters.

ALL-Net: Anatomical information lesion-wise loss function integrated into neural network for multiple sclerosis lesion segmentation

Accurate detection and segmentation of multiple sclerosis (MS) brain lesions on magnetic resonance images are important for disease diagnosis and treatment. This is a challenging task as lesions vary greatly in size, shape, location, and image contrast. The objective of our study was to develop an algorithm based on deep convolutional neural network integrated with anatomic information and lesion-wise loss function (ALL-Net) for fast and accurate automated segmentation of MS lesions. Distance transformation mapping was used to construct a convolutional module that encoded lesion-specific anatomical information. To overcome the lesion size imbalance during network training and improve the detection of small lesions, a lesion-wise loss function was developed in which individual lesions were modeled as spheres of equal size. On the ISBI-2015 longitudinal MS lesion segmentation challenge dataset (19 subjects in total), ALL-Net achieved an overall score of 93.32 and was amongst the top performing methods. On the larger Cornell MS dataset (176 subjects in total), ALL-Net significantly improved both voxel-wise metrics (Dice improvement of 3.9% to 35.3% with p-values ranging from p < 0.01 to p < 0.0001, and AUC of voxel-wise precision-recall curve improvement of 2.1% to 29.8%) and lesion-wise metrics (lesion-wise F1 score improvement of 12.6% to 29.8% with all p-values p < 0.0001, and AUC of lesion-wise ROC curve improvement of 1.4% to 20.0%) compared to leading publicly available MS lesion segmentation tools.

Deep learning segmentation of gadolinium-enhancing lesions in multiple sclerosis

OBJECTIVE

The aim of this study is to assess the performance of deep learning convolutional neural networks (CNNs) in segmenting gadolinium-enhancing lesions using a large cohort of multiple sclerosis (MS) patients.

METHODS

A three-dimensional (3D) CNN model was trained for segmentation of gadolinium-enhancing lesions using multispectral magnetic resonance imaging data (MRI) from 1006 relapsing-remitting MS patients. The network performance was evaluated for three combinations of multispectral MRI used as input: (U5) fluid-attenuated inversion recovery (FLAIR), T2-weighted, proton density-weighted, and pre- and post-contrast T1-weighted images; (U2) pre- and post-contrast T1-weighted images; and (U1) only post-contrast T1-weighted images. Segmentation performance was evaluated using the Dice similarity coefficient (DSC) and lesion-wise true-positive (TPR) and false-positive (FPR) rates. Performance was also evaluated as a function of enhancing lesion volume.

RESULTS

The DSC/TPR/FPR values averaged over all the enhancing lesion sizes were 0.77/0.90/0.23 using the U5 model. These values for the largest enhancement volumes (>500 mm3) were 0.81/0.97/0.04. For U2, the average DSC/TPR/FPR values were 0.72/0.86/0.31. Comparable performance was observed with U1. For all types of input, the network performance degraded with decreased enhancement size.

CONCLUSION

Excellent segmentation of enhancing lesions was observed for enhancement volume ⩾70 mm3. The best performance was achieved when the input included all five multispectral image sets.

Investigating efficient CNN architecture for multiple sclerosis lesion segmentation

Purpose: The automatic segmentation of multiple sclerosis lesions in magnetic resonance imaging has the potential to reduce radiologists' efforts on a daily time-consuming task and to bring more reproducibility. Almost all new segmentation techniques make use of convolutional neural networks with their own different architecture. Architectural choices are rarely explained. We aimed at presenting the relevance of a U-net-like architecture for our specific task and at building an efficient and simple model. Approach: An experimental study was performed by observing the impact of applying different mutations and deletions to a simple U-net-like architecture. Results: The power of the U-net architecture is explained by the joint benefits of using an encoder-decoder architecture and by linking them with long skip connections. Augmenting the number of convolutional layers and decreasing the number of feature maps allowed us to build an exceptionally light and competitive architecture, the minimally parameterized U-net (MPU-net), with only ∼ 30,000 parameters. Conclusion: The empirical study of the U-net has led to a better understanding of its architecture. It has guided the building of the MPU-net, a model far less parameterized than others (at least by a factor of seven). This neural network achieves a human-level segmentation of multiple sclerosis lesions on fluid-attenuated inversion recovery images only. It shows that this segmentation task does not necessitate overly complicated models to be achieved. This gives the opportunity to build more explainable models that can help such methods to be adopted in a clinical environment.

FLAIR-only joint volumetric analysis of brain lesions and atrophy in clinically isolated syndrome (CIS) suggestive of multiple sclerosis

Background

MRI assessment in multiple sclerosis (MS) focuses on the presence of typical white matter (WM) lesions. Neurodegeneration characterised by brain atrophy is recognised in the research field as an important prognostic factor. It is not routinely reported clinically, in part due to difficulty in achieving reproducible measurements. Automated MRI quantification of WM lesions and brain volume could provide important clinical monitoring data. In general, lesion quantification relies on both T1 and FLAIR input images, while tissue volumetry relies on T1. However, T1-weighted scans are not routinely included in the clinical MS protocol, limiting the utility of automated quantification.

Objectives

We address an aspect of this important translational challenge by assessing the performance of FLAIR-only lesion and brain segmentation, against a conventional approach requiring multi-contrast acquisition. We explore whether FLAIR-only grey matter (GM) segmentation yields more variability in performance compared with two-channel segmentation; whether this is related to field strength; and whether the results meet a level of clinical acceptability demonstrated by the ability to reproduce established biological associations.

Methods

We used a multicentre dataset of subjects with a CIS suggestive of MS scanned at 1.5T and 3T in the same week. WM lesions were manually segmented by two raters, ‘manual 1′ guided by consensus reading of CIS-specific lesions and ‘manual 2′ by any WM hyperintensity. An existing brain segmentation method was adapted for FLAIR-only input. Automated segmentation of WM hyperintensity and brain volumes were performed with conventional (T1/T1 + FLAIR) and FLAIR-only methods.

Results

WM lesion volumes were comparable at 1.5T between ‘manual 2′ and FLAIR-only methods and at 3T between ‘manual 2′, T1 + FLAIR and FLAIR-only methods. For cortical GM volume, linear regression measures between conventional and FLAIR-only segmentation were high (1.5T: α = 1.029, R² = 0.997, standard error (SE) = 0.007; 3T: α = 1.019, R² = 0.998, SE = 0.006). Age-associated change in cortical GM volume was a significant covariate in both T1 (p = 0.001) and FLAIR-only (p = 0.005) methods, confirming the expected relationship between age and GM volume for FLAIR-only segmentations.

Conclusions

FLAIR-only automated segmentation of WM lesions and brain volumes were consistent with results obtained through conventional methods and had the ability to demonstrate biological effects in our study population. Imaging protocol harmonisation and validation with other MS phenotypes could facilitate the integration of automated WM lesion volume and brain atrophy analysis as clinical tools in radiological MS reporting.

Myelin detection in fluorescence microscopy images using machine learning

BACKGROUND

The myelin sheath produced by glial cells insulates the axons, and supports the function of the nervous system. Myelin sheath degeneration causes neurodegenerative disorders, such as multiple sclerosis (MS). There are no therapies for MS that promote remyelination. Drug discovery frequently involves screening thousands of compounds. However, this is not feasible for remyelination drugs, since myelin quantification is a manual labor-intensive endeavor. Therefore, the development of assistive software for expedited myelin detection is instrumental for MS drug discovery by enabling high-content image-based drug screens.

NEW METHOD

In this study, we developed a machine learning based expedited myelin detection approach in fluorescence microscopy images. Multi-channel three-dimensional microscopy images of a mouse stem cell-based myelination assay were labeled by experts. A spectro-spatial feature extraction method was introduced to represent local dependencies of voxels both in spatial and spectral domains. Feature extraction yielded two data set of over forty-seven thousand annotated images in total.

RESULTS

Myelin detection performances of 23 different supervised machine learning techniques including a customized-convolutional neural network (CNN), were assessed using various train/test split ratios of the data sets. The highest accuracy values of 98.84±0.09% and 98.46±0.11% were achieved by Boosted Trees and customized-CNN, respectively.

COMPARISON WITH EXISTING METHODS

Our approach can detect myelin in a common experimental setup. Myelin extending in any orientation in 3 dimensions is segmented from 3 channel z-stack fluorescence images.

CONCLUSIONS

Our results suggest that the proposed expedited myelin detection approach is a feasible and robust method for remyelination drug screening.

Review of Deep Learning Approaches for the Segmentation of Multiple Sclerosis Lesions on Brain MRI

In recent years, there have been multiple works of literature reviewing methods for automatically segmenting multiple sclerosis (MS) lesions. However, there is no literature systematically and individually review deep learning-based MS lesion segmentation methods. Although the previous review also included methods based on deep learning, there are some methods based on deep learning that they did not review. In addition, their review of deep learning methods did not go deep into the specific categories of Convolutional Neural Network (CNN). They only reviewed these methods in a generalized form, such as supervision strategy, input data handling strategy, etc. This paper presents a systematic review of the literature in automated multiple sclerosis lesion segmentation based on deep learning. Algorithms based on deep learning reviewed are classified into two categories through their CNN style, and their strengths and weaknesses will also be given through our investigation and analysis. We give a quantitative comparison of the methods reviewed through two metrics: Dice Similarity Coefficient (DSC) and Positive Predictive Value (PPV). Finally, the future direction of the application of deep learning in MS lesion segmentation will be discussed.

A contrast-adaptive method for simultaneous whole-brain and lesion segmentation in multiple sclerosis

Here we present a method for the simultaneous segmentation of white matter lesions and normal-appearing neuroanatomical structures from multi-contrast brain MRI scans of multiple sclerosis patients. The method integrates a novel model for white matter lesions into a previously validated generative model for whole-brain segmentation. By using separate models for the shape of anatomical structures and their appearance in MRI, the algorithm can adapt to data acquired with different scanners and imaging protocols without retraining. We validate the method using four disparate datasets, showing robust performance in white matter lesion segmentation while simultaneously segmenting dozens of other brain structures. We further demonstrate that the contrast-adaptive method can also be safely applied to MRI scans of healthy controls, and replicate previously documented atrophy patterns in deep gray matter structures in MS. The algorithm is publicly available as part of the open-source neuroimaging package FreeSurfer.

Fully automated longitudinal segmentation of new or enlarged multiple sclerosis lesions using 3D convolutional neural networks

•

A fully automated segmentation of new or enlarged multiple sclerosis (MS) lesions.

•

3D convolutional neural network (CNN) with U-net-like encoder-decoder architecture.

•

Simultaneous processing of baseline and follow-up scan of the same patient.

•

Trained on 3253 patient data from over 103 different MR scanners.

•

Fast (<1min), robust algorithm with segmentation results in inter-rater variability.

The quantification of new or enlarged lesions from follow-up MRI scans is an important surrogate of clinical disease activity in patients with multiple sclerosis (MS). Not only is manual segmentation time consuming, but inter-rater variability is high. Currently, only a few fully automated methods are available. We address this gap in the field by employing a 3D convolutional neural network (CNN) with encoder-decoder architecture for fully automatic longitudinal lesion segmentation.

Input data consist of two fluid attenuated inversion recovery (FLAIR) images (baseline and follow-up) per patient. Each image is entered into the encoder and the feature maps are concatenated and then fed into the decoder. The output is a 3D mask indicating new or enlarged lesions (compared to the baseline scan). The proposed method was trained on 1809 single point and 1444 longitudinal patient data sets and then validated on 185 independent longitudinal data sets from two different scanners. From the two validation data sets, manual segmentations were available from three experienced raters, respectively. The performance of the proposed method was compared to the open source Lesion Segmentation Toolbox (LST), which is a current state-of-art longitudinal lesion segmentation method.

The mean lesion-wise inter-rater sensitivity was 62%, while the mean inter-rater number of false positive (FP) findings was 0.41 lesions per case. The two validated algorithms showed a mean sensitivity of 60% (CNN), 46% (LST) and a mean FP of 0.48 (CNN), 1.86 (LST) per case. Sensitivity and number of FP were not significantly different (p < 0.05) between the CNN and manual raters.

New or enlarged lesions counted by the CNN algorithm appeared to be comparable with manual expert ratings. The proposed algorithm seems to outperform currently available approaches, particularly LST. The high inter-rater variability in case of manual segmentation indicates the complexity of identifying new or enlarged lesions. An automated CNN-based approach can quickly provide an independent and deterministic assessment of new or enlarged lesions from baseline to follow-up scans with acceptable reliability.

Neural Architecture Search for compressed sensing Magnetic Resonance image reconstruction

Recent works have demonstrated that deep learning (DL) based compressed sensing (CS) implementation can accelerate Magnetic Resonance (MR) Imaging by reconstructing MR images from sub-sampled k-space data. However, network architectures adopted in previous methods are all designed by handcraft. Neural Architecture Search (NAS) algorithms can automatically build neural network architectures which have outperformed human designed ones in several vision tasks. Inspired by this, here we proposed a novel and efficient network for the MR image reconstruction problem via NAS instead of manual attempts. Particularly, a specific cell structure, which was integrated into the model-driven MR reconstruction pipeline, was automatically searched from a flexible pre-defined operation search space in a differentiable manner. Experimental results show that our searched network can produce better reconstruction results compared to previous state-of-the-art methods in terms of PSNR and SSIM with 4∼6 times fewer computation resources. Extensive experiments were conducted to analyze how hyper-parameters affect reconstruction performance and the searched structures. The generalizability of the searched architecture was also evaluated on different organ MR datasets. Our proposed method can reach a better trade-off between computation cost and reconstruction performance for MR reconstruction problem with good generalizability and offer insights to design neural networks for other medical image applications. The evaluation code will be available at https://github.com/yjump/NAS-for-CSMRI.

Deep learning to find colorectal polyps in colonoscopy: A systematic literature review

Colorectal cancer has a great incidence rate worldwide, but its early detection significantly increases the survival rate. Colonoscopy is the gold standard procedure for diagnosis and removal of colorectal lesions with potential to evolve into cancer and computer-aided detection systems can help gastroenterologists to increase the adenoma detection rate, one of the main indicators for colonoscopy quality and predictor for colorectal cancer prevention. The recent success of deep learning approaches in computer vision has also reached this field and has boosted the number of proposed methods for polyp detection, localization and segmentation. Through a systematic search, 35 works have been retrieved. The current systematic review provides an analysis of these methods, stating advantages and disadvantages for the different categories used; comments seven publicly available datasets of colonoscopy images; analyses the metrics used for reporting and identifies future challenges and recommendations. Convolutional neural networks are the most used architecture together with an important presence of data augmentation strategies, mainly based on image transformations and the use of patches. End-to-end methods are preferred over hybrid methods, with a rising tendency. As for detection and localization tasks, the most used metric for reporting is the recall, while Intersection over Union is highly used in segmentation. One of the major concerns is the difficulty for a fair comparison and reproducibility of methods. Even despite the organization of challenges, there is still a need for a common validation framework based on a large, annotated and publicly available database, which also includes the most convenient metrics to report results. Finally, it is also important to highlight that efforts should be focused in the future on proving the clinical value of the deep learning based methods, by increasing the adenoma detection rate.

AI in Radiology: Where are we today in Multiple Sclerosis Imaging?

Background MR imaging is an essential component in managing patients with Multiple sclerosis (MS). This holds true for the initial diagnosis as well as for assessing the clinical course of MS. In recent years, a growing number of computer tools were developed to analyze imaging data in MS. This review gives an overview of the most important applications with special emphasis on artificial intelligence (AI).

Methods Relevant studies were identified through a literature search in recognized databases, and through parsing the references in studies found this way. Literature published as of November 2019 was included with a special focus on recent studies from 2018 and 2019.

Results There are a number of studies which focus on optimizing lesion visualization and lesion segmentation. Some of these studies accomplished these tasks with high accuracy, enabling a reproducible quantitative analysis of lesion loads. Some studies took a radiomics approach and aimed at predicting clinical endpoints such as the conversion from a clinically isolated syndrome to definite MS. Moreover, recent studies investigated synthetic imaging, i. e. imaging data that is not measured during an MR scan but generated by a computer algorithm to optimize the contrast between MS lesions and brain parenchyma.

Conclusion Computer-based image analysis and AI are hot topics in imaging MS. Some applications are ready for use in clinical routine. A major challenge for the future is to improve prediction of expected disease courses and thereby helping to find optimal treatment decisions on an individual level. With technical improvements, more questions arise about the integration of new tools into the radiological workflow.

Key Points: 

Citation Format

A Global Inhomogeneous Intensity Clustering- (GINC-) Based Active Contour Model for Image Segmentation and Bias Correction

Image segmentation is still an open problem especially when intensities of the objects of interest are overlapped due to the presence of intensity inhomogeneities. A bias correction embedded level set model is proposed in this paper where inhomogeneities are estimated by orthogonal primary functions. First, an inhomogeneous intensity clustering energy is defined based on global distribution characteristics of the image intensities, and membership functions of the clusters described by the level set function are then introduced to define the data term energy of the proposed model. Second, a regularization term and an arc length term are also included to regularize the level set function and smooth its zero-level set contour, respectively. Third, the proposed model is extended to multichannel and multiphase patterns to segment colorful images and images with multiple objects, respectively. Experimental results and comparison with relevant models demonstrate the advantages of the proposed model in terms of bias correction and segmentation accuracy on widely used synthetic and real images and the BrainWeb and the IBSR image repositories.

TAPAS: A Thresholding Approach for Probability Map Automatic Segmentation in Multiple Sclerosis

Total brain white matter lesion (WML) volume is the most widely established magnetic resonance imaging (MRI) outcome measure in studies of multiple sclerosis (MS). To estimate WML volume, there are a number of automatic segmentation methods available, yet manual delineation remains the gold standard approach. Automatic approaches often yield a probability map to which a threshold is applied to create lesion segmentation masks. Unfortunately, few approaches systematically determine the threshold employed; many methods use a manually selected threshold, thus introducing human error and bias into the automated procedure. In this study, we propose and validate an automatic thresholding algorithm, Thresholding Approach for Probability Map Automatic Segmentation in Multiple Sclerosis (TAPAS), to obtain subject-specific threshold estimates for probability map automatic segmentation of T2-weighted (T2) hyperintense WMLs. Using multimodal MRI, the proposed method applies an automatic segmentation algorithm to obtain probability maps. We obtain the true subject-specific threshold that maximizes the Sørensen-Dice similarity coefficient (DSC). Then the subject-specific thresholds are modeled on a naive estimate of volume using a generalized additive model. Applying this model, we predict a subject-specific threshold in data not used for training. We ran a Monte Carlo-resampled split-sample cross-validation (100 validation sets) using two data sets: the first obtained from the Johns Hopkins Hospital (JHH) on a Philips 3 Tesla (3T) scanner (n = 94) and a second collected at the Brigham and Women's Hospital (BWH) using a Siemens 3T scanner (n = 40). By means of the proposed automated technique, in the JHH data we found an average reduction in subject-level absolute error of 0.1 mL per one mL increase in manual volume. Using Bland-Altman analysis, we found that volumetric bias associated with group-level thresholding was mitigated when applying TAPAS. The BWH data showed similar absolute error estimates using group-level thresholding or TAPAS likely since Bland-Altman analyses indicated no systematic biases associated with group or TAPAS volume estimates. The current study presents the first validated fully automated method for subject-specific threshold prediction to segment brain lesions.

Deep-Learning-Based Neural Tissue Segmentation of MRI in Multiple Sclerosis: Effect of Training Set Size

BACKGROUND

The dependence of deep-learning (DL)-based segmentation accuracy of brain MRI on the training size is not known.

PURPOSE

To determine the required training size for a desired accuracy in brain MRI segmentation in multiple sclerosis (MS) using DL.

STUDY TYPE

Retrospective analysis of MRI data acquired as part of a multicenter clinical trial.

STUDY POPULATION

In all, 1008 patients with clinically definite MS.

FIELD STRENGTH/SEQUENCE

MRIs were acquired at 1.5T and 3T scanners manufactured by GE, Philips, and Siemens with dual turbo spin echo, FLAIR, and T1 -weighted turbo spin echo sequences.

ASSESSMENT

Segmentation results using an automated analysis pipeline and validated by two neuroimaging experts served as the ground truth. A DL model, based on a fully convolutional neural network, was trained separately using 16 different training sizes. The segmentation accuracy as a function of the training size was determined. These data were fitted to the learning curve for estimating the required training size for desired accuracy.

STATISTICAL TESTS

The performance of the network was evaluated by calculating the Dice similarity coefficient (DSC), and lesion true-positive and false-positive rates.

RESULTS

The DSC for lesions showed much stronger dependency on the sample size than gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF). When the training size was increased from 10 to 800 the DSC values varied from 0.00 to 0.86 ± 0.016 for T2 lesions, 0.87 ± 009 to 0.94 ± 0.004 for GM, 0.86 ± 0.08 to 0.94 ± 0.005 for WM, and 0.91 ± 0.009 to 0.96 ± 0.003 for CSF.

DATA CONCLUSION

Excellent segmentation was achieved with a training size as small as 10 image volumes for GM, WM, and CSF. In contrast, a training size of at least 50 image volumes was necessary for adequate lesion segmentation.

LEVEL OF EVIDENCE

1 Technical Efficacy Stage: 1 J. Magn. Reson. Imaging 2020;51:1487-1496.

Automated volumetric assessment with artificial neural networks might enable a more accurate assessment of disease burden in patients with multiple sclerosis

OBJECTIVES

Patients with multiple sclerosis (MS) regularly undergo MRI for assessment of disease burden. However, interpretation may be time consuming and prone to intra- and interobserver variability. Here, we evaluate the potential of artificial neural networks (ANN) for automated volumetric assessment of MS disease burden and activity on MRI.

METHODS

A single-institutional dataset with 334 MS patients (334 MRI exams) was used to develop and train an ANN for automated identification and volumetric segmentation of T2/FLAIR-hyperintense and contrast-enhancing (CE) lesions. Independent testing was performed in a single-institutional longitudinal dataset with 82 patients (266 MRI exams). We evaluated lesion detection performance (F1 scores), lesion segmentation agreement (DICE coefficients), and lesion volume agreement (concordance correlation coefficients [CCC]). Independent evaluation was performed on the public ISBI-2015 challenge dataset.

RESULTS

The F1 score was maximized in the training set at a detection threshold of 7 mm³ for T2/FLAIR lesions and 14 mm³ for CE lesions. In the training set, mean F1 scores were 0.867 for T2/FLAIR lesions and 0.636 for CE lesions, as compared to 0.878 for T2/FLAIR lesions and 0.715 for CE lesions in the test set. Using these thresholds, the ANN yielded mean DICE coefficients of 0.834 and 0.878 for segmentation of T2/FLAIR and CE lesions in the training set (fivefold cross-validation). Corresponding DICE coefficients in the test set were 0.846 for T2/FLAIR lesions and 0.908 for CE lesions, and the CCC was ≥ 0.960 in each dataset.

CONCLUSIONS

Our results highlight the capability of ANN for quantitative state-of-the-art assessment of volumetric lesion load on MRI and potentially enable a more accurate assessment of disease burden in patients with MS.

KEY POINTS

• Artificial neural networks (ANN) can accurately detect and segment both T2/FLAIR and contrast-enhancing MS lesions in MRI data. • Performance of the ANN was consistent in a clinically derived dataset, with patients presenting all possible disease stages in MRI scans acquired from standard clinical routine rather than with high-quality research sequences. • Computer-aided evaluation of MS with ANN could streamline both clinical and research procedures in the volumetric assessment of MS disease burden as well as in lesion detection.

Are multi-contrast magnetic resonance images necessary for segmenting multiple sclerosis brains? A large cohort study based on deep learning

BACKGROUND

Magnetic resonance images with multiple contrasts or sequences are commonly used for segmenting brain tissues, including lesions, in multiple sclerosis (MS). However, acquisition of images with multiple contrasts increases the scan time and complexity of the analysis, possibly introducing factors that could compromise segmentation quality.

OBJECTIVE

To investigate the effect of various combinations of multi-contrast images as input on the segmented volumes of gray (GM) and white matter (WM), cerebrospinal fluid (CSF), and lesions using a deep neural network.

METHODS

U-net, a fully convolutional neural network was used to automatically segment GM, WM, CSF, and lesions in 1000 MS patients. The input to the network consisted of 15 combinations of FLAIR, T1-, T2-, and proton density-weighted images. The Dice similarity coefficient (DSC) was evaluated to assess the segmentation performance. For lesions, true positive rate (TPR) and false positive rate (FPR) were also evaluated. In addition, the effect of lesion size on lesion segmentation was investigated.

RESULTS

Highest DSC was observed for all the tissue volumes, including lesions, when the input was combination of all four image contrasts. All other input combinations that included FLAIR also provided high DSC for all tissue classes. However, the quality of lesion segmentation showed strong dependence on the input images. The DSC and TPR values for inputs with the four contrast combination and FLAIR alone were very similar, but FLAIR showed a moderately higher FPR for lesion size <100 μl. For lesions smaller than 20 μl all image combinations resulted in poor performance. The segmentation quality improved with lesion size.

CONCLUSIONS

Best performance for segmented tissue volumes was obtained with all four image contrasts as the input, and comparable performance was attainable with FLAIR only as the input, albeit with a moderate increase in FPR for small lesions. This implies that acquisition of only FLAIR images provides satisfactory tissue segmentation. Lesion segmentation was poor for very small lesions and improved rapidly with lesion size.

Automated Segmentation of Tissues Using CT and MRI: A Systematic Review

RATIONALE AND OBJECTIVES

The automated segmentation of organs and tissues throughout the body using computed tomography and magnetic resonance imaging has been rapidly increasing. Research into many medical conditions has benefited greatly from these approaches by allowing the development of more rapid and reproducible quantitative imaging markers. These markers have been used to help diagnose disease, determine prognosis, select patients for therapy, and follow responses to therapy. Because some of these tools are now transitioning from research environments to clinical practice, it is important for radiologists to become familiar with various methods used for automated segmentation.

MATERIALS AND METHODS

The Radiology Research Alliance of the Association of University Radiologists convened an Automated Segmentation Task Force to conduct a systematic review of the peer-reviewed literature on this topic.

RESULTS

The systematic review presented here includes 408 studies and discusses various approaches to automated segmentation using computed tomography and magnetic resonance imaging for neurologic, thoracic, abdominal, musculoskeletal, and breast imaging applications.

CONCLUSION

These insights should help prepare radiologists to better evaluate automated segmentation tools and apply them not only to research, but eventually to clinical practice.

Comparing lesion segmentation methods in multiple sclerosis: Input from one manually delineated subject is sufficient for accurate lesion segmentation

PURPOSE

Accurate lesion segmentation is important for measurements of lesion load and atrophy in subjects with multiple sclerosis (MS). International MS lesion challenges show a preference of convolutional neural networks (CNN) strategies, such as nicMSlesions. However, since the software is trained on fairly homogenous training data, we aimed to test the performance of nicMSlesions in an independent dataset with manual and other automatic lesion segmentations to determine whether this method is suitable for larger, multi-center studies.

METHODS

Manual lesion segmentation was performed in fourteen subjects with MS on sagittal 3D FLAIR images from a 3T GE whole-body scanner with 8-channel head coil. We compared five different categories of automated lesion segmentation methods for their volumetric and spatial agreement with manual segmentation: (i) unsupervised, untrained (LesionTOADS); (ii) supervised, untrained (LST-LPA and nicMSlesions with default settings); (iii) supervised, untrained with threshold adjustment (LST-LPA optimized for current data); (iv) supervised, trained with leave-one-out cross-validation on fourteen subjects with MS (nicMSlesions and BIANCA); and (v) supervised, trained on a single subject with MS (nicMSlesions). Volumetric accuracy was determined by the intra-class correlation coefficient (ICC) and spatial accuracy by Dice's similarity index (SI). Volumes and SI were compared between methods using repeated measures ANOVA or Friedman tests with post-hoc pairwise comparison.

RESULTS

The best volumetric and spatial agreement with manual was obtained with the supervised and trained methods nicMSlesions and BIANCA (ICC absolute agreement > 0.968 and median SI > 0.643) and the worst with the unsupervised, untrained method LesionTOADS (ICC absolute agreement = 0.140 and median SI = 0.444). Agreement with manual in the single-subject network training of nicMSlesions was poor for input with low lesion volumes (i.e. two subjects with lesion volumes ≤ 3.0 ml). For the other twelve subjects, ICC varied from 0.593 to 0.973 and median SI varied from 0.535 to 0.606. In all cases, the single-subject trained nicMSlesions segmentations outperformed LesionTOADS, and in almost all cases it also outperformed LST-LPA.

CONCLUSION

Input from only one subject to re-train the deep learning CNN nicMSlesions is sufficient for adequate lesion segmentation, with on average higher volumetric and spatial agreement with manual than obtained with the untrained methods LesionTOADS and LST-LPA.

Follow-up MRI in multiple sclerosis patients: automated co-registration and lesion color-coding improves diagnostic accuracy and reduces reading time

OBJECTIVES

In multiple sclerosis (MS), the heterogeneous and numerous appearances of lesions may impair diagnostic accuracy. This study investigates if a combined automated co-registration and lesion color-coding method (AC) improves assessment of MS follow-up MRI compared with conventional reading (CR).

METHODS

We retrospectively assessed 70 follow-up MRI of 53 patients. Heterogeneous datasets of diverse scanners and institutions were used. Two readers determined presence of (a) progression, (b) regression, (c) mixed change, or (d) stable disease between the two examinations using corresponding FLAIR sequences in CR and AC-assisted reading. Consensus reference reading was provided by two blinded radiologists. Kappa statistics tested interrater agreement, McNemar's test dichotomous variables, and Wilcoxon's test continuous variables (statistical significance p ≤ 0.05).

RESULTS

The cohort comprised 41 female and 12 male patients with a mean age of 40 (± 14) years. Average rating time was reduced from 78 (± 36) to 44 (±22) s with the AC approach (p < 0.001). The time needed to start and match datasets with AC was 14 (± 1) s. Compared with CR, AC improved interrater agreement, both between raters (0.52 vs. 0.67) and between raters and consensus reference reading (0.47/0.5 vs. 0.83/0.78). Compared with CR, the diagnostic accuracy increased from 67 to 90% (reader 1, p < 0.01) and from 70 to 87% (reader 2, p < 0.05) in the AC-assisted reading.

CONCLUSIONS

Compared with CR, automated co-registration and lesion color-coding of MS-associated FLAIR-lesions in follow-up MRI increased diagnostic accuracy and reduced the time required for follow-up evaluation significantly. The AC algorithm therefore appears to be helpful to improve MS follow-up assessments in clinical routine.

KEY POINTS

• Automated co-registration and lesion color-coding increases diagnostic accuracy in the assessment of MRI follow-up examinations in patients with multiple sclerosis. • Automated co-registration and lesion color-coding reduces reading time of MRI follow-up examinations in patients with multiple sclerosis. • Automated co-registration and lesion color-coding improved interrater agreement in the assessment of MRI follow-up examinations in patients with multiple sclerosis.

Brain and lesion segmentation in multiple sclerosis using fully convolutional neural networks: A large-scale study

OBJECTIVE

To investigate the performance of deep learning (DL) based on fully convolutional neural network (FCNN) in segmenting brain tissues in a large cohort of multiple sclerosis (MS) patients.

METHODS

We developed a FCNN model to segment brain tissues, including T2-hyperintense MS lesions. The training, validation, and testing of FCNN were based on ~1000 magnetic resonance imaging (MRI) datasets acquired on relapsing-remitting MS patients, as a part of a phase 3 randomized clinical trial. Multimodal MRI data (dual-echo, FLAIR, and T1-weighted images) served as input to the network. Expert validated segmentation was used as the target for training the FCNN. We cross-validated our results using the leave-one-center-out approach.

RESULTS

We observed a high average (95% confidence limits) Dice similarity coefficient for all the segmented tissues: 0.95 (0.92-0.98) for white matter, 0.96 (0.93-0.98) for gray matter, 0.99 (0.98-0.99) for cerebrospinal fluid, and 0.82 (0.63-1.0) for T2 lesions. High correlations between the DL segmented tissue volumes and ground truth were observed (R² > 0.92 for all tissues). The cross validation showed consistent results across the centers for all tissues.

CONCLUSION

The results from this large-scale study suggest that deep FCNN can automatically segment MS brain tissues, including lesions, with high accuracy.

Automated segmentation of changes in FLAIR-hyperintense white matter lesions in multiple sclerosis on serial magnetic resonance imaging

Longitudinal analysis of white matter lesion changes on serial MRI has become an important parameter to study diseases with white-matter lesions. Here, we build on earlier work on cross-sectional lesion segmentation; we present a fully automatic pipeline for serial analysis of FLAIR-hyperintense white matter lesions. Our algorithm requires three-dimensional gradient echo T1- and FLAIR- weighted images at 3 Tesla as well as available cross-sectional lesion segmentations of both time points. Preprocessing steps include lesion filling and intrasubject registration. For segmentation of lesion changes, initial lesion maps of different time points are fused; herein changes in intensity are analyzed at the voxel level. Significance of lesion change is estimated by comparison with the difference distribution of FLAIR intensities within normal appearing white matter. The method is validated on MRI data of two time points from 40 subjects with multiple sclerosis derived from two different scanners (20 subjects per scanner). Manual segmentation of lesion increases served as gold standard. Across all lesion increases, voxel-wise Dice coefficient (0.7) as well as lesion-wise detection rate (0.8) and false-discovery rate (0.2) indicate good overall performance. Analysis of scans from a repositioning experiment in a single patient with multiple sclerosis did not yield a single false positive lesion. We also introduce the lesion change plot as a descriptive tool for the lesion change of individual patients with regard to both number and volume. An open source implementation of the algorithm is available at http://www.statistical-modeling.de/lst.html.

•

Quantification of white matter lesion changes is important in multiple sclerosis.

•

We developed and validated an algorithm for automated detection of lesion changes.

•

Our algorithm requires T1-weighted and FLAIR images derived at 3 T as well as available cross-sectional lesion segmentations.

•

With data from 2 different scanners, the tool showed good agreement with manual tracing.

•

An open-source application is available.