Automatic detection of lesion load change in Multiple Sclerosis using convolutional neural networks with segmentation confidence

Clinical applications of deep learning in neuroinflammatory diseases: A scoping review

BACKGROUND

Deep learning (DL) is an artificial intelligence technology that has aroused much excitement for predictive medicine due to its ability to process raw data modalities such as images, text, and time series of signals.

OBJECTIVES

Here, we intend to give the clinical reader elements to understand this technology, taking neuroinflammatory diseases as an illustrative use case of clinical translation efforts. We reviewed the scope of this rapidly evolving field to get quantitative insights about which clinical applications concentrate the efforts and which data modalities are most commonly used.

METHODS

We queried the PubMed database for articles reporting DL algorithms for clinical applications in neuroinflammatory diseases and the radiology.healthairegister.com website for commercial algorithms.

RESULTS

The review included 148 articles published between 2018 and 2024 and five commercial algorithms. The clinical applications could be grouped as computer-aided diagnosis, individual prognosis, functional assessment, the segmentation of radiological structures, and the optimization of data acquisition. Our review highlighted important discrepancies in efforts. The segmentation of radiological structures and computer-aided diagnosis currently concentrate most efforts with an overrepresentation of imaging. Various model architectures have addressed different applications, relatively low volume of data, and diverse data modalities. We report the high-level technical characteristics of the algorithms and synthesize narratively the clinical applications. Predictive performances and some common a priori on this topic are finally discussed.

CONCLUSION

The currently reported efforts position DL as an information processing technology, enhancing existing modalities of paraclinical investigations and bringing perspectives to make innovative ones actionable for healthcare.

The macular retinal ganglion cell layer as a biomarker for diagnosis and prognosis in multiple sclerosis: A deep learning approach

PURPOSE

The macular ganglion cell layer (mGCL) is a strong potential biomarker of axonal degeneration in multiple sclerosis (MS). For this reason, this study aims to develop a computer-aided method to facilitate diagnosis and prognosis in MS.

METHODS

This paper combines a cross-sectional study of 72 MS patients and 30 healthy control subjects for diagnosis and a 10-year longitudinal study of the same MS patients for the prediction of disability progression, during which the mGCL was measured using optical coherence tomography (OCT). Deep neural networks were used as an automatic classifier.

RESULTS

For MS diagnosis, greatest accuracy (90.3%) was achieved using 17 features as inputs. The neural network architecture comprised the input layer, two hidden layers and the output layer with softmax activation. For the prediction of disability progression 8 years later, accuracy of 81.9% was achieved with a neural network comprising two hidden layers and 400 epochs.

CONCLUSION

We present evidence that by applying deep learning techniques to clinical and mGCL thickness data it is possible to identify MS and predict the course of the disease. This approach potentially constitutes a non-invasive, low-cost, easy-to-implement and effective method.

An update on the role of magnetic resonance imaging in predicting and monitoring multiple sclerosis progression

INTRODUCTION

While magnetic resonance imaging (MRI) is established in diagnosing and monitoring disease activity in multiple sclerosis (MS), its utility in predicting and monitoring disease progression is less clear.

AREAS COVERED

The authors consider changing concepts in the phenotypic classification of MS, including progression independent of relapses; pathological processes underpinning progression; advances in MRI measures to assess them; how well MRI features explain and predict clinical outcomes, including models that assess disease effects on neural networks, and the potential role for machine learning.

EXPERT OPINION

Relapsing-remitting and progressive MS have evolved from being viewed as mutually exclusive to having considerable overlap. Progression is likely the consequence of several pathological elements, each important in building more holistic prognostic models beyond conventional phenotypes. MRI is well placed to assess pathogenic processes underpinning progression, but we need to bridge the gap between MRI measures and clinical outcomes. Mapping pathological effects on specific neural networks may help and machine learning methods may be able to optimize predictive markers while identifying new, or previously overlooked, clinically relevant features. The ever-increasing ability to measure features on MRI raises the dilemma of what to measure and when, and the challenge of translating research methods into clinically useable tools.

Impact of automatic tools for detecting new lesions on therapeutic strategies offered to patients with MS by neurologists

BACKGROUND

Automatic tools for detecting new lesions in patients with MS between two MRI scans are now available to clinicians. They have been assessed from the radiologist's point of view, but their impact on the therapeutic strategies that neurologists offer their patients has not yet been documented.

OBJECTIVES

To compare neurologist's decisions according to whether a lesion detection support system had been used and describe variability between neurologists on decision-making for the same clinical cases.

METHODS

We submitted 28 clinical cases associated with pairs of MRI images and radiological reports (produced by the same radiologist without vs. with the help of a system to detect new lesions) to 10 neurologists who regularly follow patients with MS. They examined each clinical case twice (without vs. with support system) in two sessions several weeks apart, and their patient management decisions were recorded.

RESULTS

There was considerable variability between neurologists on decision-making (both with and without support system). When the support system had been used, neurologists more often made changes to patient management (75 % vs. 68 % of cases, p = 0.01) and spent significantly less time analyzing the clinical cases (249 s vs. 216 s, p == 3.10-4).

CONCLUSION

The use of a lesion detection support system has an impact not only on radiologists' reports, but also on neurologists' subsequent decision-making. This observation constitutes another strong argument for promoting the wider use of such systems in clinical routine. However, despite their use, there is still considerable variability in decision-making across neurologists, which should encourage us to refine the guidelines.

3D U-Net for automated detection of multiple sclerosis lesions: utility of transfer learning from other pathologies

Background and purpose

Deep learning algorithms for segmentation of multiple sclerosis (MS) plaques generally require training on large datasets. This manuscript evaluates the effect of transfer learning from segmentation of another pathology to facilitate use of smaller MS-specific training datasets. That is, a model trained for detection of one type of pathology was re-trained to identify MS lesions and active demyelination.

Materials and methods

In this retrospective study using MRI exams from 149 patients spanning 4/18/2014 to 7/8/2021, 3D convolutional neural networks were trained with a variable number of manually-segmented MS studies. Models were trained for FLAIR lesion segmentation at a single timepoint, new FLAIR lesion segmentation comparing two timepoints, and enhancing (actively demyelinating) lesion segmentation on T1 post-contrast imaging. Models were trained either de-novo or fine-tuned with transfer learning applied to a pre-existing model initially trained on non-MS data. Performance was evaluated with lesionwise sensitivity and positive predictive value (PPV).

Results

For single timepoint FLAIR lesion segmentation with 10 training studies, a fine-tuned model demonstrated improved performance [lesionwise sensitivity 0.55 ± 0.02 (mean ± standard error), PPV 0.66 ± 0.02] compared to a de-novo model (sensitivity 0.49 ± 0.02, p = 0.001; PPV 0.32 ± 0.02, p < 0.001). For new lesion segmentation with 30 training studies and their prior comparisons, a fine-tuned model demonstrated similar sensitivity (0.49 ± 0.05) and significantly improved PPV (0.60 ± 0.05) compared to a de-novo model (sensitivity 0.51 ± 0.04, p = 0.437; PPV 0.43 ± 0.04, p = 0.002). For enhancement segmentation with 20 training studies, a fine-tuned model demonstrated significantly improved overall performance (sensitivity 0.74 ± 0.06, PPV 0.69 ± 0.05) compared to a de-novo model (sensitivity 0.44 ± 0.09, p = 0.001; PPV 0.37 ± 0.05, p = 0.001).

Conclusion

By fine-tuning models trained for other disease pathologies with MS-specific data, competitive models identifying existing MS plaques, new MS plaques, and active demyelination can be built with substantially smaller datasets than would otherwise be required to train new models.

A Multicenter Longitudinal MRI Study Assessing LeMan-PV Software Accuracy in the Detection of White Matter Lesions in Multiple Sclerosis Patients

BACKGROUND

Detecting new and enlarged lesions in multiple sclerosis (MS) patients is needed to determine their disease activity. LeMan-PV is a software embedded in the scanner reconstruction system of one vendor, which automatically assesses new and enlarged white matter lesions (NELs) in the follow-up of MS patients; however, multicenter validation studies are lacking.

PURPOSE

To assess the accuracy of LeMan-PV for the longitudinal detection NEL white-matter MS lesions in a multicenter clinical setting.

STUDY TYPE

Retrospective, longitudinal.

SUBJECTS

A total of 206 patients with a definitive MS diagnosis and at least two follow-up MRI studies from five centers participating in the Swiss Multiple Sclerosis Cohort study. Mean age at first follow-up = 45.2 years (range: 36.9-52.8 years); 70 males.

FIELD STRENGTH/SEQUENCE

Fluid attenuated inversion recovery (FLAIR) and T1-weighted magnetization prepared rapid gradient echo (T1-MPRAGE) sequences at 1.5 T and 3 T.

ASSESSMENT

The study included 313 MRI pairs of datasets. Data were analyzed with LeMan-PV and compared with a manual "reference standard" provided by a neuroradiologist. A second rater (neurologist) performed the same analysis in a subset of MRI pairs to evaluate the rating-accuracy. The Sensitivity (Se), Specificity (Sp), Accuracy (Acc), F1-score, lesion-wise False-Positive-Rate (aFPR), and other measures were used to assess LeMan-PV performance for the detection of NEL at 1.5 T and 3 T. The performance was also evaluated in the subgroup of 123 MRI pairs at 3 T.

STATISTICAL TESTS

Intraclass correlation coefficient (ICC) and Cohen's kappa (CK) were used to evaluate the agreement between readers.

RESULTS

The interreader agreement was high for detecting new lesions (ICC = 0.97, Pvalue < 10-20 , CK = 0.82, P value = 0) and good (ICC = 0.75, P value < 10-12 , CK = 0.68, P value = 0) for detecting enlarged lesions. Across all centers, scanner field strengths (1.5 T, 3 T), and for NEL, LeMan-PV achieved: Acc = 61%, Se = 65%, Sp = 60%, F1-score = 0.44, aFPR = 1.31. When both follow-ups were acquired at 3 T, LeMan-PV accuracy was higher (Acc = 66%, Se = 66%, Sp = 66%, F1-score = 0.28, aFPR = 3.03).

DATA CONCLUSION

In this multicenter study using clinical data settings acquired at 1.5 T and 3 T, and variations in MRI protocols, LeMan-PV showed similar sensitivity in detecting NEL with respect to other recent 3 T multicentric studies based on neural networks. While LeMan-PV performance is not optimal, its main advantage is that it provides automated clinical decision support integrated into the radiological-routine flow.

EVIDENCE LEVEL

4 TECHNICAL EFFICACY: Stage 2.

Estimation of white matter hyperintensities with synthetic MRI myelin volume fraction in patients with multiple sclerosis and non-multiple-sclerosis white matter hyperintensities: A pilot study among the Indian population

AIM

Synthetic MRI (SyMRI) works on the MDME sequence, which acquires the relaxation properties of the brain and helps to measure the accurate tissue properties in 6 minutes. The aim of this study was to evaluate the synthetic MRI (SyMRI)-generated myelin (MyC) to white matter (WM) ratio, the WM fraction (WMF), MyC partial maps performing normative brain volumetry to investigate MyC loss in multiple sclerosis (MS) patients with white-matter hyperintensites (WMHs) and non-MS patients with WMHs in a clinical setting.

MATERIALS and METHODS

Synthetic MRI images were acquired from 15 patients with MS, and from 15 non-MS patients on a 3T MRI scanner (Discovery MR750w; GE Healthcare; Milwaukee, USA) using MAGiC, a customized version of SyntheticMR's SyMRI® IMAGE software marketed by GE Healthcare under a license agreement. Fast multi-delay multi-echo acquisition was performed with a 2D axial pulse sequence with different combinations of echo time (TEs) and saturation delay times. The total image acquisition time was 6 minutes. SyMRI image analysis was done using SyMRI software (SyMRI Version: 11.3.6; Synthetic MR, Linköping, Sweden). SyMRI data were used to generate the MyC partial maps and WMFs to quantify the signal intensities of test group and control group, andcontrol group , and their mean values were recorded. All patients also underwent conventional diffusion-weighted imaging, i.e., T1w and T2w imaging.

RESULTS

The results showed that the WMF was significantly lower in the test group than in the control group (38.8% vs 33.2%, p < 0.001). The Mann-Whitney U nonparametric t-test revealed a significant difference in the mean myelin volume between the test group and the control group (158.66 ± 32.31 vs. 138.29 ± 29.28, p = 0.044). Also, there were no significant differences in the gray matter fraction and intracranial volume between the test group and the control group.

CONCLUSIONS

We observed MyC loss in test group using quantitative SyMRI. Thus, myelin loss in MS patients can be quantitatively evaluated using SyMRI.

Multiple sclerosis lesion segmentation: revisiting weighting mechanisms for federated learning

Background and introduction

Federated learning (FL) has been widely employed for medical image analysis to facilitate multi-client collaborative learning without sharing raw data. Despite great success, FL's applications remain suboptimal in neuroimage analysis tasks such as lesion segmentation in multiple sclerosis (MS), due to variance in lesion characteristics imparted by different scanners and acquisition parameters.

Methods

In this work, we propose the first FL MS lesion segmentation framework via two effective re-weighting mechanisms. Specifically, a learnable weight is assigned to each local node during the aggregation process, based on its segmentation performance. In addition, the segmentation loss function in each client is also re-weighted according to the lesion volume for the data during training.

Results

The proposed method has been validated on two FL MS segmentation scenarios using public and clinical datasets. Specifically, the case-wise and voxel-wise Dice score of the proposed method under the first public dataset is 65.20 and 74.30, respectively. On the second in-house dataset, the case-wise and voxel-wise Dice score is 53.66, and 62.31, respectively.

Discussions and conclusions

The Comparison experiments on two FL MS segmentation scenarios using public and clinical datasets have demonstrated the effectiveness of the proposed method by significantly outperforming other FL methods. Furthermore, the segmentation performance of FL incorporating our proposed aggregation mechanism can achieve comparable performance to that from centralized training with all the raw data.

An open-source tool for longitudinal whole-brain and white matter lesion segmentation

In this paper we describe and validate a longitudinal method for whole-brain segmentation of longitudinal MRI scans. It builds upon an existing whole-brain segmentation method that can handle multi-contrast data and robustly analyze images with white matter lesions. This method is here extended with subject-specific latent variables that encourage temporal consistency between its segmentation results, enabling it to better track subtle morphological changes in dozens of neuroanatomical structures and white matter lesions. We validate the proposed method on multiple datasets of control subjects and patients suffering from Alzheimer's disease and multiple sclerosis, and compare its results against those obtained with its original cross-sectional formulation and two benchmark longitudinal methods. The results indicate that the method attains a higher test-retest reliability, while being more sensitive to longitudinal disease effect differences between patient groups. An implementation is publicly available as part of the open-source neuroimaging package FreeSurfer.

Online hard example mining vs. fixed oversampling strategy for segmentation of new multiple sclerosis lesions from longitudinal FLAIR MRI

Detecting new lesions is a key aspect of the radiological follow-up of patients with Multiple Sclerosis (MS), leading to eventual changes in their therapeutics. This paper presents our contribution to the MSSEG-2 MICCAI 2021 challenge. The challenge is focused on the segmentation of new MS lesions using two consecutive Fluid Attenuated Inversion Recovery (FLAIR) Magnetic Resonance Imaging (MRI). In other words, considering longitudinal data composed of two time points as input, the aim is to segment the lesional areas, which are present only in the follow-up scan and not in the baseline. The backbone of our segmentation method is a 3D UNet applied patch-wise to the images, and in which, to take into account both time points, we simply concatenate the baseline and follow-up images along the channel axis before passing them to the 3D UNet. Our key methodological contribution is the use of online hard example mining to address the challenge of class imbalance. Indeed, there are very few voxels belonging to new lesions which makes training deep-learning models difficult. Instead of using handcrafted priors like brain masks or multi-stage methods, we experiment with a novel modification to online hard example mining (OHEM), where we use an exponential moving average (i.e., its weights are updated with momentum) of the 3D UNet to mine hard examples. Using a moving average instead of the raw model should allow smoothing of its predictions and allow it to give more consistent feedback for OHEM.

New lesion segmentation for multiple sclerosis brain images with imaging and lesion-aware augmentation

Multiple sclerosis (MS) is an inflammatory and demyelinating neurological disease of the central nervous system. Image-based biomarkers, such as lesions defined on magnetic resonance imaging (MRI), play an important role in MS diagnosis and patient monitoring. The detection of newly formed lesions provides crucial information for assessing disease progression and treatment outcome. Here, we propose a deep learning-based pipeline for new MS lesion detection and segmentation, which is built upon the nnU-Net framework. In addition to conventional data augmentation, we employ imaging and lesion-aware data augmentation methods, axial subsampling and CarveMix, to generate diverse samples and improve segmentation performance. The proposed pipeline is evaluated on the MICCAI 2021 MS new lesion segmentation challenge (MSSEG-2) dataset. It achieves an average Dice score of 0.510 and F 1 score of 0.552 on cases with new lesions, and an average false positive lesion number n FP of 0.036 and false positive lesion volume V FP of 0.192 mm 3 on cases with no new lesions. Our method outperforms other participating methods in the challenge and several state-of-the-art network architectures.

Evaluating the use of synthetic T1-w images in new T2 lesion detection in multiple sclerosis

The assessment of disease activity using serial brain MRI scans is one of the most valuable strategies for monitoring treatment response in patients with multiple sclerosis (MS) receiving disease-modifying treatments. Recently, several deep learning approaches have been proposed to improve this analysis, obtaining a good trade-off between sensitivity and specificity, especially when using T1-w and T2-FLAIR images as inputs. However, the need to acquire two different types of images is time-consuming, costly and not always available in clinical practice. In this paper, we investigate an approach to generate synthetic T1-w images from T2-FLAIR images and subsequently analyse the impact of using original and synthetic T1-w images on the performance of a state-of-the-art approach for longitudinal MS lesion detection. We evaluate our approach on a dataset containing 136 images from MS patients, and 73 images with lesion activity (the appearance of new T2 lesions in follow-up scans). To evaluate the synthesis of the images, we analyse the structural similarity index metric and the median absolute error and obtain consistent results. To study the impact of synthetic T1-w images, we evaluate the performance of the new lesion detection approach when using (1) both T2-FLAIR and T1-w original images, (2) only T2-FLAIR images, and (3) both T2-FLAIR and synthetic T1-w images. Sensitivities of 0.75, 0.63, and 0.81, respectively, were obtained at the same false-positive rate (0.14) for all experiments. In addition, we also present the results obtained when using the data from the international MSSEG-2 challenge, showing also an improvement when including synthetic T1-w images. In conclusion, we show that the use of synthetic images can support the lack of data or even be used instead of the original image to homogenize the contrast of the different acquisitions in new T2 lesions detection algorithms.

Evaluation of Disability Progression in Multiple Sclerosis via Magnetic-Resonance-Based Deep Learning Techniques

Short-term disability progression was predicted from a baseline evaluation in patients with multiple sclerosis (MS) using their three-dimensional T1-weighted (3DT1) magnetic resonance images (MRI). One-hundred-and-eighty-one subjects diagnosed with MS underwent 3T-MRI and were followed up for two to six years at two sites, with disability progression defined according to the expanded-disability-status-scale (EDSS) increment at the follow-up. The patients’ 3DT1 images were bias-corrected, brain-extracted, registered onto MNI space, and divided into slices along coronal, sagittal, and axial projections. Deep learning image classification models were applied on slices and devised as ResNet50 fine-tuned adaptations at first on a large independent dataset and secondly on the study sample. The final classifiers’ performance was evaluated via the area under the curve (AUC) of the false versus true positive diagram. Each model was also tested against its null model, obtained by reshuffling patients’ labels in the training set. Informative areas were found by intersecting slices corresponding to models fulfilling the disability progression prediction criteria. At follow-up, 34% of patients had disability progression. Five coronal and five sagittal slices had one classifier surviving the AUC evaluation and null test and predicted disability progression (AUC > 0.72 and AUC > 0.81, respectively). Likewise, fifteen combinations of classifiers and axial slices predicted disability progression in patients (AUC > 0.69). Informative areas were the frontal areas, mainly within the grey matter. Briefly, 3DT1 images may give hints on disability progression in MS patients, exploiting the information hidden in the MRI of specific areas of the brain.

Image registration and appearance adaptation in non-correspondent image regions for new MS lesions detection

Manual detection of newly formed lesions in multiple sclerosis is an important but tedious and difficult task. Several approaches for automating the detection of new lesions have recently been proposed, but they tend to either overestimate the actual amount of new lesions or to miss many lesions. In this paper, an image registration convolutional neural network (CNN) that adapts the baseline image to the follow-up image by spatial deformations and simulation of new lesions is proposed. Simultaneously, segmentations of new lesions are generated, which are shown to reliably estimate the real new lesion load and to separate stable and progressive patients. Several applications of the proposed network emerge: image registration, detection and segmentation of new lesions, and modeling of new MS lesions. The modeled lesions offer the possibility to investigate the intensity profile of new lesions.

Triplanar U-Net with lesion-wise voting for the segmentation of new lesions on longitudinal MRI studies

We present a deep learning method for the segmentation of new lesions in longitudinal FLAIR MRI sequences acquired at two different time points. In our approach, the 3D volumes are processed slice-wise across the coronal, axial, and sagittal planes and the predictions from the three orientations are merged using an optimized voting strategy. Our method achieved best F1 score (0.541) among all participating methods in the MICCAI 2021 challenge Multiple sclerosis new lesions segmentation (MSSEG-2). Moreover, we show that our method is on par with the challenge's expert neuroradiologists: on an unbiased ground truth, our method achieves results comparable to those of the four experts in terms of detection (F1 score) and segmentation accuracy (Dice score).

New MS lesion segmentation with deep residual attention gate U-Net utilizing 2D slices of 3D MR images

Multiple sclerosis (MS) is an autoimmune disease that causes lesions in the central nervous system of humans due to demyelinating axons. Magnetic resonance imaging (MRI) is widely used for monitoring and measuring MS lesions. Automated methods for MS lesion segmentation have usually been performed on individual MRI scans. Recently, tracking lesion activity for quantifying and monitoring MS disease progression, especially detecting new lesions, has become an important biomarker. In this study, a unique pipeline with a deep neural network that combines U-Net, attention gate, and residual learning is proposed to perform better new MS lesion segmentation using baseline and follow-up 3D FLAIR MR images. The proposed network has a similar architecture to U-Net and is formed from residual units which facilitate the training of deep networks. Networks with fewer parameters are designed with better performance through the skip connections of U-Net and residual units, which facilitate information propagation without degradation. Attention gates also learn to focus on salient features of the target structures of various sizes and shapes. The MSSEG-2 dataset was used for training and testing the proposed pipeline, and the results were compared with those of other proposed pipelines of the challenge and experts who participated in the same challenge. According to the results over the testing set, the lesion-wise F1 and dice scores were obtained as a mean of 48 and 44.30%. For the no-lesion cases, the number of tested and volume of tested lesions were obtained as a mean of 0.148 and 1.488, respectively. The proposed pipeline outperformed 22 proposed pipelines and ranked 8th in the challenge.

Recent advances in the longitudinal segmentation of multiple sclerosis lesions on magnetic resonance imaging: a review

Multiple sclerosis (MS) is a chronic autoimmune disease characterized by demyelinating lesions that are often visible on magnetic resonance imaging (MRI). Segmentation of these lesions can provide imaging biomarkers of disease burden that can help monitor disease progression and the imaging response to treatment. Manual delineation of MRI lesions is tedious and prone to subjective bias, while automated lesion segmentation methods offer objectivity and speed, the latter being particularly important when analysing large datasets. Lesion segmentation can be broadly categorised into two groups: cross-sectional methods, which use imaging data acquired at a single time-point to characterise MRI lesions; and longitudinal methods, which use imaging data from the same subject acquired at two or more different time-points to characterise lesions over time. The main objective of longitudinal segmentation approaches is to more accurately detect the presence of new MS lesions and the growth or remission of existing lesions, which may be effective biomarkers of disease progression and treatment response. This paper reviews articles on longitudinal MS lesion segmentation methods published over the past 10 years. These are divided into traditional machine learning methods and deep learning techniques. PubMed articles using longitudinal information and comparing fully automatic two time point segmentations in any step of the process were selected. Nineteen articles were reviewed. There is an increasing number of deep learning techniques for longitudinal MS lesion segmentation that are promising to help better understand disease progression.

The current role and future directions of imaging in failed back surgery syndrome patients: an educational review

BACKGROUND

Failed back surgery syndrome (FBSS) is an umbrella term referring to painful sensations experienced by patients after spinal surgery, mostly of neuropathic nature. Adequate treatment of FBSS is challenging, as its etiology is believed to be multifactorial and still not fully clarified. Accurate identification of the source of pain is difficult but pivotal to establish the most appropriate treatment strategy. Although the clinical utility of imaging in FBSS patients is still contentious, objective parameters are highly warranted to map different phenotypes of FBSS and tailor each subsequent therapy.

MAIN BODY

Since technological developments have weakened the applicability of prior research, this educational review outlined the recent evidence (i.e., from January 2005 onwards) after a systematic literature search. The state of the art on multiple imaging modalities in FBSS patients was reviewed. Future directions related to functional MRI and the development of imaging biomarkers have also been discussed.

CONCLUSION

Besides the fact that more imaging studies correlated with symptomatology in the postoperative setting are warranted, the current educational review outlined that contrast-enhanced MRI and MR neurography have been suggested as valuable imaging protocols to assess alterations in the spine of FBSS patients. The use of imaging biomarkers to study correlations between imaging features and symptomatology might hold future potential; however, more research is required before any promising hypotheses can be drawn.

The Need for Medical Artificial Intelligence That Incorporates Prior Images

The use of artificial intelligence (AI) has grown dramatically in the past few years in the United States and worldwide, with more than 300 AI-enabled devices approved by the U.S. Food and Drug Administration (FDA). Most of these AI-enabled applications focus on helping radiologists with detection, triage, and prioritization of tasks by using data from a single point, but clinical practice often encompasses a dynamic scenario wherein physicians make decisions on the basis of longitudinal information. Unfortunately, benchmark data sets incorporating clinical and radiologic data from several points are scarce, and, therefore, the machine learning community has not focused on developing methods and architectures suitable for these tasks. Current AI algorithms are not suited to tackle key image interpretation tasks that require comparisons to previous examinations. Focusing on the curation of data sets and algorithm development that allow for comparisons at different points will be required to advance the range of relevant tasks covered by future AI-enabled FDA-cleared devices.

Deep Transfer Learning for Automatic Prediction of Hemorrhagic Stroke on CT Images

Intracerebral hemorrhage (ICH) is the most common type of hemorrhagic stroke which occurs due to ruptures of weakened blood vessel in brain tissue. It is a serious medical emergency issues that needs immediate treatment. Large numbers of noncontrast-computed tomography (NCCT) brain images are analyzed manually by radiologists to diagnose the hemorrhagic stroke, which is a difficult and time-consuming process. In this study, we propose an automated transfer deep learning method that combines ResNet-50 and dense layer for accurate prediction of intracranial hemorrhage on NCCT brain images. A total of 1164 NCCT brain images were collected from 62 patients with hemorrhagic stroke from Kalinga Institute of Medical Science, Bhubaneswar and used for evaluating the model. The proposed model takes individual CT images as input and classifies them as hemorrhagic or normal. This deep transfer learning approach reached 99.6% accuracy, 99.7% specificity, and 99.4% sensitivity which are better results than that of ResNet-50 only. It is evident that the deep transfer learning model has advantages for automatic diagnosis of hemorrhagic stroke and has the potential to be used as a clinical decision support tool to assist radiologists in stroke diagnosis.

Predictive MRI Biomarkers in MS—A Critical Review

Background and Objectives: In this critical review, we explore the potential use of MRI measurements as prognostic biomarkers in multiple sclerosis (MS) patients, for both conventional measurements and more novel techniques such as magnetization transfer, diffusion tensor, and proton spectroscopy MRI. Materials and Methods: All authors individually and comprehensively reviewed each of the aspects listed below in PubMed, Medline, and Google Scholar. Results: There are numerous MRI metrics that have been proven by clinical studies to hold important prognostic value for MS patients, most of which can be readily obtained from standard 1.5T MRI scans. Conclusions: While some of these parameters have passed the test of time and seem to be associated with a reliable predictive power, some are still better interpreted with caution. We hope this will serve as a reminder of how vast a resource we have on our hands in this versatile tool—it is up to us to make use of it.

Multiple Sclerosis Lesion Analysis in Brain Magnetic Resonance Images: Techniques and Clinical Applications

Multiple sclerosis (MS) is a chronic inflammatory and degenerative disease of the central nervous system, characterized by the appearance of focal lesions in the white and gray matter that topographically correlate with an individual patients neurological symptoms and signs. Magnetic resonance imaging (MRI) provides detailed in-vivo structural information, permitting the quantification and categorization of MS lesions that critically inform disease management. Traditionally, MS lesions have been manually annotated on 2D MRI slices, a process that is inefficient and prone to inter-/intra-observer errors. Recently, automated statistical imaging analysis techniques have been proposed to detect and segment MS lesions based on MRI voxel intensity. However, their effectiveness is limited by the heterogeneity of both MRI data acquisition techniques and the appearance of MS lesions. By learning complex lesion representations directly from images, deep learning techniques have achieved remarkable breakthroughs in the MS lesion segmentation task. Here, we provide a comprehensive review of state-of-the-art automatic statistical and deep-learning MS segmentation methods and discuss current and future clinical applications. Further, we review technical strategies, such as domain adaptation, to enhance MS lesion segmentation in real-world clinical settings.

Early Diagnosis of Multiple Sclerosis Using Swept-Source Optical Coherence Tomography and Convolutional Neural Networks Trained with Data Augmentation

BACKGROUND

The aim of this paper is to implement a system to facilitate the diagnosis of multiple sclerosis (MS) in its initial stages. It does so using a convolutional neural network (CNN) to classify images captured with swept-source optical coherence tomography (SS-OCT).

METHODS

SS-OCT images from 48 control subjects and 48 recently diagnosed MS patients have been used. These images show the thicknesses (45 × 60 points) of the following structures: complete retina, retinal nerve fiber layer, two ganglion cell layers (GCL+, GCL++) and choroid. The Cohen distance is used to identify the structures and the regions within them with greatest discriminant capacity. The original database of OCT images is augmented by a deep convolutional generative adversarial network to expand the CNN's training set.

RESULTS

The retinal structures with greatest discriminant capacity are the GCL++ (44.99% of image points), complete retina (26.71%) and GCL+ (22.93%). Thresholding these images and using them as inputs to a CNN comprising two convolution modules and one classification module obtains sensitivity = specificity = 1.0.

CONCLUSIONS

Feature pre-selection and the use of a convolutional neural network may be a promising, nonharmful, low-cost, easy-to-perform and effective means of assisting the early diagnosis of MS based on SS-OCT thickness data.

A Clinically-Compatible Workflow for Computer-Aided Assessment of Brain Disease Activity in Multiple Sclerosis Patients

Over the last 10 years, the number of approved disease modifying drugs acting on the focal inflammatory process in Multiple Sclerosis (MS) has increased from 3 to 10. This wide choice offers the opportunity of a personalized medicine with the objective of no clinical and radiological activity for each patient. This new paradigm requires the optimization of the detection of new FLAIR lesions on longitudinal MRI. In this paper, we describe a complete workflow—that we developed, implemented, deployed, and evaluated—to facilitate the monitoring of new FLAIR lesions on longitudinal MRI of MS patients. This workflow has been designed to be usable by both hospital and private neurologists and radiologists in France. It consists of three main components: (i) a software component that allows for automated and secured anonymization and transfer of MRI data from the clinical Picture Archive and Communication System (PACS) to a processing server (and vice-versa); (ii) a fully automated segmentation core that enables detection of focal longitudinal changes in patients from T1-weighted, T2-weighted and FLAIR brain MRI scans, and (iii) a dedicated web viewer that provides an intuitive visualization of new lesions to radiologists and neurologists. We first present these different components. Then, we evaluate the workflow on 54 pairs of longitudinal MRI scans that were analyzed by 3 experts (1 neuroradiologist, 1 radiologist, and 1 neurologist) with and without the proposed workflow. We show that our workflow provided a valuable aid to clinicians in detecting new MS lesions both in terms of accuracy (mean number of detected lesions per patient and per expert 1.8 without the workflow vs. 2.3 with the workflow, p = 5.10⁻⁴) and of time dedicated by the experts (mean time difference 2′45″, p = 10⁻⁴). This increase in the number of detected lesions has implications in the classification of MS patients as stable or active, even for the most experienced neuroradiologist (mean sensitivity was 0.74 without the workflow and 0.90 with the workflow, p-value for no difference = 0.003). It therefore has potential consequences on the therapeutic management of MS patients.

Opportunities for Understanding MS Mechanisms and Progression With MRI Using Large-Scale Data Sharing and Artificial Intelligence

Patients with multiple sclerosis (MS) have heterogeneous clinical presentations, symptoms, and progression over time, making MS difficult to assess and comprehend in vivo. The combination of large-scale data sharing and artificial intelligence creates new opportunities for monitoring and understanding MS using MRI. First, development of validated MS-specific image analysis methods can be boosted by verified reference, test, and benchmark imaging data. Using detailed expert annotations, artificial intelligence algorithms can be trained on such MS-specific data. Second, understanding disease processes could be greatly advanced through shared data of large MS cohorts with clinical, demographic, and treatment information. Relevant patterns in such data that may be imperceptible to a human observer could be detected through artificial intelligence techniques. This applies from image analysis (lesions, atrophy, or functional network changes) to large multidomain datasets (imaging, cognition, clinical disability, genetics). After reviewing data sharing and artificial intelligence, we highlight 3 areas that offer strong opportunities for making advances in the next few years: crowdsourcing, personal data protection, and organized analysis challenges. Difficulties as well as specific recommendations to overcome them are discussed, in order to best leverage data sharing and artificial intelligence to improve image analysis, imaging, and the understanding of MS.

A deep learning algorithm for white matter hyperintensity lesion detection and segmentation

PURPOSE

White matter hyperintensity (WMHI) lesions on MR images are an important indication of various types of brain diseases that involve inflammation and blood vessel abnormalities. Automated quantification of the WMHI can be valuable for the clinical management of patients, but existing automated software is often developed for a single type of disease and may not be applicable for clinical scans with thick slices and different scanning protocols. The purpose of the study is to develop and validate an algorithm for automatic quantification of white matter hyperintensity suitable for heterogeneous MRI data with different disease types.

METHODS

We developed and evaluated "DeepWML", a deep learning method for fully automated white matter lesion (WML) segmentation of multicentre FLAIR images. We used MRI from 507 patients, including three distinct white matter diseases, obtained in 9 centres, with a wide range of scanners and acquisition protocols. The automated delineation tool was evaluated through quantitative parameters of Dice similarity, sensitivity and precision compared to manual delineation (gold standard).

RESULTS

The overall median Dice similarity coefficient was 0.78 (range 0.64 ~ 0.86) across the three disease types and multiple centres. The median sensitivity and precision were 0.84 (range 0.67 ~ 0.94) and 0.81 (range 0.64 ~ 0.92), respectively. The tool's performance increased with larger lesion volumes.

CONCLUSION

DeepWML was successfully applied to a wide spectrum of MRI data in the three white matter disease types, which has the potential to improve the practical workflow of white matter lesion delineation.

Emerging deep learning techniques using magnetic resonance imaging data applied in multiple sclerosis and clinical isolated syndrome patients (Review)

Computer-aided diagnosis systems aim to assist clinicians in the early identification of abnormal signs in order to optimize the interpretation of medical images and increase diagnostic precision. Multiple sclerosis (MS) and clinically isolated syndrome (CIS) are chronic inflammatory, demyelinating diseases affecting the central nervous system. Recent advances in deep learning (DL) techniques have led to novel computational paradigms in MS and CIS imaging designed for automatic segmentation and detection of areas of interest and automatic classification of anatomic structures, as well as optimization of neuroimaging protocols. To this end, there are several publications presenting artificial intelligence-based predictive models aiming to increase diagnostic accuracy and to facilitate optimal clinical management in patients diagnosed with MS and/or CIS. The current study presents a thorough review covering DL techniques that have been applied in MS and CIS during recent years, shedding light on their current advances and limitations.

Applications of deep learning techniques for automated multiple sclerosis detection using magnetic resonance imaging: A review

Multiple Sclerosis (MS) is a type of brain disease which causes visual, sensory, and motor problems for people with a detrimental effect on the functioning of the nervous system. In order to diagnose MS, multiple screening methods have been proposed so far; among them, magnetic resonance imaging (MRI) has received considerable attention among physicians. MRI modalities provide physicians with fundamental information about the structure and function of the brain, which is crucial for the rapid diagnosis of MS lesions. Diagnosing MS using MRI is time-consuming, tedious, and prone to manual errors. Research on the implementation of computer aided diagnosis system (CADS) based on artificial intelligence (AI) to diagnose MS involves conventional machine learning and deep learning (DL) methods. In conventional machine learning, feature extraction, feature selection, and classification steps are carried out by using trial and error; on the contrary, these steps in DL are based on deep layers whose values are automatically learn. In this paper, a complete review of automated MS diagnosis methods performed using DL techniques with MRI neuroimaging modalities is provided. Initially, the steps involved in various CADS proposed using MRI modalities and DL techniques for MS diagnosis are investigated. The important preprocessing techniques employed in various works are analyzed. Most of the published papers on MS diagnosis using MRI modalities and DL are presented. The most significant challenges facing and future direction of automated diagnosis of MS using MRI modalities and DL techniques are also provided.

Lesion probability mapping in MS patients using a regression network on MR fingerprinting

BACKGROUND

To develop a regression neural network for the reconstruction of lesion probability maps on Magnetic Resonance Fingerprinting using echo-planar imaging (MRF-EPI) in addition to [Formula: see text], [Formula: see text], NAWM, and GM- probability maps.

METHODS

We performed MRF-EPI measurements in 42 patients with multiple sclerosis and 6 healthy volunteers along two sites. A U-net was trained to reconstruct the denoised and distortion corrected [Formula: see text] and [Formula: see text] maps, and to additionally generate NAWM-, GM-, and WM lesion probability maps.

RESULTS

WM lesions were predicted with a dice coefficient of [Formula: see text] and a lesion detection rate of [Formula: see text] for a threshold of 33%. The network jointly enabled accurate [Formula: see text] and [Formula: see text] times with relative deviations of 5.2% and 5.1% and average dice coefficients of [Formula: see text] and [Formula: see text] for NAWM and GM after binarizing with a threshold of 80%.

CONCLUSION

DL is a promising tool for the prediction of lesion probability maps in a fraction of time. These might be of clinical interest for the WM lesion analysis in MS patients.

JOINT SEGMENTATION OF MULTIPLE SCLEROSIS LESIONS AND BRAIN ANATOMY IN MRI SCANS OF ANY CONTRAST AND RESOLUTION WITH CNNs

We present the first deep learning method to segment Multiple Sclerosis lesions and brain structures from MRI scans of any (possibly multimodal) contrast and resolution. Our method only requires segmentations to be trained (no images), as it leverages the generative model of Bayesian segmentation to generate synthetic scans with simulated lesions, which are then used to train a CNN. Our method can be retrained to segment at any resolution by adjusting the amount of synthesised partial volume. By construction, the synthetic scans are perfectly aligned with their labels, which enables training with noisy labels obtained with automatic methods. The training data are generated on the fly, and aggressive augmentation (including artefacts) is applied for improved generalisation. We demonstrate our method on two public datasets, comparing it with a state-of-the-art Bayesian approach implemented in FreeSurfer, and dataset specific CNNs trained on real data. The code is available at https://github.com/BBillot/SynthSeg.

Simultaneous lesion and brain segmentation in multiple sclerosis using deep neural networks

Segmentation of white matter lesions and deep grey matter structures is an important task in the quantification of magnetic resonance imaging in multiple sclerosis. In this paper we explore segmentation solutions based on convolutional neural networks (CNNs) for providing fast, reliable segmentations of lesions and grey-matter structures in multi-modal MR imaging, and the performance of these methods when applied to out-of-centre data. We trained two state-of-the-art fully convolutional CNN architectures on the 2016 MSSEG training dataset, which was annotated by seven independent human raters: a reference implementation of a 3D Unet, and a more recently proposed 3D-to-2D architecture (DeepSCAN). We then retrained those methods on a larger dataset from a single centre, with and without labels for other brain structures. We quantified changes in performance owing to dataset shift, and changes in performance by adding the additional brain-structure labels. We also compared performance with freely available reference methods. Both fully-convolutional CNN methods substantially outperform other approaches in the literature when trained and evaluated in cross-validation on the MSSEG dataset, showing agreement with human raters in the range of human inter-rater variability. Both architectures showed drops in performance when trained on single-centre data and tested on the MSSEG dataset. When trained with the addition of weak anatomical labels derived from Freesurfer, the performance of the 3D Unet degraded, while the performance of the DeepSCAN net improved. Overall, the DeepSCAN network predicting both lesion and anatomical labels was the best-performing network examined.

Deep learning of brain magnetic resonance images: A brief review

Magnetic resonance imaging (MRI) is one of the most popular techniques in brain science and is important for understanding brain function and neuropsychiatric disorders. However, the processing and analysis of MRI is not a trivial task with lots of challenges. Recently, deep learning has shown superior performance over traditional machine learning approaches in image analysis. In this survey, we give a brief review of the recent popular deep learning approaches and their applications in brain MRI analysis. Furthermore, popular brain MRI databases and deep learning tools are also introduced. The strength and weaknesses of different approaches are addressed, and challenges as well as future directions are also discussed.