Simultaneous lesion and brain segmentation in multiple sclerosis using deep neural networks

The impact of image contrast, resolution and reader expertise on black hole identification in Multiple Sclerosis

OBJECTIVES

In the neuroradiological work-up of Multiple Sclerosis (MS), the detection of "black holes" (BH) represent an information of undeniable importance. Nevertheless, different sequences can be used in clinical practice to evaluate BH in MS. Aim of this study was to investigate the possible impact of different sequences, resolutions, and levels of expertise on the intra- and inter-rater reliability identification of BH in MS.

METHODS

Brain MRI scans of 85 MS patients (M/F = 22/63; mean age = 36.0 ± 10.2 years) were evaluated in this prospective single-center study. The acquisition protocol included a 3 mm SE-T1w sequence, a 1 mm 3D-GrE-T1w sequence from which a resliced 3 mm sequence was also obtained. Images were evaluated independently by two readers of different expertise at baseline and after a wash-out period of 30 days. The intraclass correlation coefficient (ICC) was calculated as an index of intra and inter-reader reliability.

RESULTS

For both readers, the intra-reader ICC analysis showed that the 3 mm SE-T1w and 3 mm resliced GrE-T1w images achieved an excellent performance (both with an ICC ≥ 0.95), while 1 mm 3D-GrE-T1w scans achieved a moderate one (ICC < 0.90). The inter-reader analysis showed that each of the three sequences achieved a moderate performance (all ICCs < 0.90).

CONCLUSIONS

The 1 mm 3D-GrE-T1w sequence seems to be prone to a greater intra-reader variability compared to the 3 mm SE-T1w, with this effect being driven by the higher spatial resolution of the first sequence. To ensure reliability levels comparable with the standard SE-T1w in BH count, an assessment on a 3 mm resliced GrE-T1w sequence should be recommended.

Image preprocessing with contrast-limited adaptive histogram equalization improves the segmentation performance of deep learning for the articular disk of the temporomandibular joint on magnetic resonance images

OBJECTIVES

The objective was to evaluate the robustness of deep learning (DL)-based encoder-decoder convolutional neural networks (ED-CNNs) for segmenting temporomandibular joint (TMJ) articular disks using data sets acquired from 2 different 3.0-T magnetic resonance imaging (MRI) scanners using original images and images subjected to contrast-limited adaptive histogram equalization (CLAHE).

STUDY DESIGN

In total, 536 MR images from 49 individuals were examined. An expert orthodontist identified and manually segmented the disks in all images, which were then reviewed by another expert orthodontist and 2 expert oral and maxillofacial radiologists. These images were used to evaluate a DL-based semantic segmentation approach using an ED-CNN. Original and preprocessed CLAHE images were used to train and validate the models whose performances were compared.

RESULTS

Original and CLAHE images acquired on 1 scanner had pixel values that were significantly darker and with lower contrast. The values of 3 metrics-the Dice similarity coefficient, sensitivity, and positive predictive value-were low when the original MR images were used for model training and validation. However, these metrics significantly improved when images were preprocessed with CLAHE.

CONCLUSIONS

The robustness of the ED-CNN model trained on a dataset obtained from a single device is low but can be improved with CLAHE preprocessing. The proposed system provides promising results for a DL-based, fully automated segmentation method for TMJ articular disks on MRI.

LST-AI: A deep learning ensemble for accurate MS lesion segmentation

Automated segmentation of brain white matter lesions is crucial for both clinical assessment and scientific research in multiple sclerosis (MS). Over a decade ago, we introduced an engineered lesion segmentation tool, LST. While recent lesion segmentation approaches have leveraged artificial intelligence (AI), they often remain proprietary and difficult to adopt. As an open-source tool, we present LST-AI, an advanced deep learning-based extension of LST that consists of an ensemble of three 3D U-Nets. LST-AI explicitly addresses the imbalance between white matter (WM) lesions and non-lesioned WM. It employs a composite loss function incorporating binary cross-entropy and Tversky loss to improve segmentation of the highly heterogeneous MS lesions. We train the network ensemble on 491 MS pairs of T1-weighted and FLAIR images, collected in-house from a 3T MRI scanner, and expert neuroradiologists manually segmented the utilized lesion maps for training. LST-AI also includes a lesion location annotation tool, labeling lesions as periventricular, infratentorial, and juxtacortical according to the 2017 McDonald criteria, and, additionally, as subcortical. We conduct evaluations on 103 test cases consisting of publicly available data using the Anima segmentation validation tools and compare LST-AI with several publicly available lesion segmentation models. Our empirical analysis shows that LST-AI achieves superior performance compared to existing methods. Its Dice and F1 scores exceeded 0.62, outperforming LST, SAMSEG (Sequence Adaptive Multimodal SEGmentation), and the popular nnUNet framework, which all scored below 0.56. Notably, LST-AI demonstrated exceptional performance on the MSSEG-1 challenge dataset, an international WM lesion segmentation challenge, with a Dice score of 0.65 and an F1 score of 0.63-surpassing all other competing models at the time of the challenge. With increasing lesion volume, the lesion detection rate rapidly increased with a detection rate of >75% for lesions with a volume between 10 mm3 and 100 mm3. Given its higher segmentation performance, we recommend that research groups currently using LST transition to LST-AI. To facilitate broad adoption, we are releasing LST-AI as an open-source model, available as a command-line tool, dockerized container, or Python script, enabling diverse applications across multiple platforms.

Deep Multimodal Fusion of Data With Heterogeneous Dimensionality via Projective Networks

The use of multimodal imaging has led to significant improvements in the diagnosis and treatment of many diseases. Similar to clinical practice, some works have demonstrated the benefits of multimodal fusion for automatic segmentation and classification using deep learning-based methods. However, current segmentation methods are limited to fusion of modalities with the same dimensionality (e.g., 3D + 3D, 2D + 2D), which is not always possible, and the fusion strategies implemented by classification methods are incompatible with localization tasks. In this work, we propose a novel deep learning-based framework for the fusion of multimodal data with heterogeneous dimensionality (e.g., 3D + 2D) that is compatible with localization tasks. The proposed framework extracts the features of the different modalities and projects them into the common feature subspace. The projected features are then fused and further processed to obtain the final prediction. The framework was validated on the following tasks: segmentation of geographic atrophy (GA), a late-stage manifestation of age-related macular degeneration, and segmentation of retinal blood vessels (RBV) in multimodal retinal imaging. Our results show that the proposed method outperforms the state-of-the-art monomodal methods on GA and RBV segmentation by up to 3.10% and 4.64% Dice, respectively.

LST-AI: a Deep Learning Ensemble for Accurate MS Lesion Segmentation

Automated segmentation of brain white matter lesions is crucial for both clinical assessment and scientific research in multiple sclerosis (MS). Over a decade ago, we introduced an engineered lesion segmentation tool, LST. While recent lesion segmentation approaches have leveraged artificial intelligence (AI), they often remain proprietary and difficult to adopt. As an open-source tool, we present LST-AI, an advanced deep learning-based extension of LST that consists of an ensemble of three 3D-UNets. LST-AI explicitly addresses the imbalance between white matter (WM) lesions and non-lesioned WM. It employs a composite loss function incorporating binary cross-entropy and Tversky loss to improve segmentation of the highly heterogeneous MS lesions. We train the network ensemble on 491 MS pairs of T1w and FLAIR images, collected in-house from a 3T MRI scanner, and expert neuroradiologists manually segmented the utilized lesion maps for training. LST-AI additionally includes a lesion location annotation tool, labeling lesion location according to the 2017 McDonald criteria (periventricular, infratentorial, juxtacortical, subcortical). We conduct evaluations on 103 test cases consisting of publicly available data using the Anima segmentation validation tools and compare LST-AI with several publicly available lesion segmentation models. Our empirical analysis shows that LST-AI achieves superior performance compared to existing methods. Its Dice and F1 scores exceeded 0.62, outperforming LST, SAMSEG (Sequence Adaptive Multimodal SEGmentation), and the popular nnUNet framework, which all scored below 0.56. Notably, LST-AI demonstrated exceptional performance on the MSSEG-1 challenge dataset, an international WM lesion segmentation challenge, with a Dice score of 0.65 and an F1 score of 0.63-surpassing all other competing models at the time of the challenge. With increasing lesion volume, the lesion detection rate rapidly increased with a detection rate of >75% for lesions with a volume between 10mm3 and 100mm3. Given its higher segmentation performance, we recommend that research groups currently using LST transition to LST-AI. To facilitate broad adoption, we are releasing LST-AI as an open-source model, available as a command-line tool, dockerized container, or Python script, enabling diverse applications across multiple platforms.

An update on the role of magnetic resonance imaging in predicting and monitoring multiple sclerosis progression

INTRODUCTION

While magnetic resonance imaging (MRI) is established in diagnosing and monitoring disease activity in multiple sclerosis (MS), its utility in predicting and monitoring disease progression is less clear.

AREAS COVERED

The authors consider changing concepts in the phenotypic classification of MS, including progression independent of relapses; pathological processes underpinning progression; advances in MRI measures to assess them; how well MRI features explain and predict clinical outcomes, including models that assess disease effects on neural networks, and the potential role for machine learning.

EXPERT OPINION

Relapsing-remitting and progressive MS have evolved from being viewed as mutually exclusive to having considerable overlap. Progression is likely the consequence of several pathological elements, each important in building more holistic prognostic models beyond conventional phenotypes. MRI is well placed to assess pathogenic processes underpinning progression, but we need to bridge the gap between MRI measures and clinical outcomes. Mapping pathological effects on specific neural networks may help and machine learning methods may be able to optimize predictive markers while identifying new, or previously overlooked, clinically relevant features. The ever-increasing ability to measure features on MRI raises the dilemma of what to measure and when, and the challenge of translating research methods into clinically useable tools.

3D U-Net for automated detection of multiple sclerosis lesions: utility of transfer learning from other pathologies

Background and purpose

Deep learning algorithms for segmentation of multiple sclerosis (MS) plaques generally require training on large datasets. This manuscript evaluates the effect of transfer learning from segmentation of another pathology to facilitate use of smaller MS-specific training datasets. That is, a model trained for detection of one type of pathology was re-trained to identify MS lesions and active demyelination.

Materials and methods

In this retrospective study using MRI exams from 149 patients spanning 4/18/2014 to 7/8/2021, 3D convolutional neural networks were trained with a variable number of manually-segmented MS studies. Models were trained for FLAIR lesion segmentation at a single timepoint, new FLAIR lesion segmentation comparing two timepoints, and enhancing (actively demyelinating) lesion segmentation on T1 post-contrast imaging. Models were trained either de-novo or fine-tuned with transfer learning applied to a pre-existing model initially trained on non-MS data. Performance was evaluated with lesionwise sensitivity and positive predictive value (PPV).

Results

For single timepoint FLAIR lesion segmentation with 10 training studies, a fine-tuned model demonstrated improved performance [lesionwise sensitivity 0.55 ± 0.02 (mean ± standard error), PPV 0.66 ± 0.02] compared to a de-novo model (sensitivity 0.49 ± 0.02, p = 0.001; PPV 0.32 ± 0.02, p < 0.001). For new lesion segmentation with 30 training studies and their prior comparisons, a fine-tuned model demonstrated similar sensitivity (0.49 ± 0.05) and significantly improved PPV (0.60 ± 0.05) compared to a de-novo model (sensitivity 0.51 ± 0.04, p = 0.437; PPV 0.43 ± 0.04, p = 0.002). For enhancement segmentation with 20 training studies, a fine-tuned model demonstrated significantly improved overall performance (sensitivity 0.74 ± 0.06, PPV 0.69 ± 0.05) compared to a de-novo model (sensitivity 0.44 ± 0.09, p = 0.001; PPV 0.37 ± 0.05, p = 0.001).

Conclusion

By fine-tuning models trained for other disease pathologies with MS-specific data, competitive models identifying existing MS plaques, new MS plaques, and active demyelination can be built with substantially smaller datasets than would otherwise be required to train new models.

Application of uncertainty quantification to artificial intelligence in healthcare: A review of last decade (2013-2023)

Uncertainty estimation in healthcare involves quantifying and understanding the inherent uncertainty or variability associated with medical predictions, diagnoses, and treatment outcomes. In this era of Artificial Intelligence (AI) models, uncertainty estimation becomes vital to ensure safe decision-making in the medical field. Therefore, this review focuses on the application of uncertainty techniques to machine and deep learning models in healthcare. A systematic literature review was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Our analysis revealed that Bayesian methods were the predominant technique for uncertainty quantification in machine learning models, with Fuzzy systems being the second most used approach. Regarding deep learning models, Bayesian methods emerged as the most prevalent approach, finding application in nearly all aspects of medical imaging. Most of the studies reported in this paper focused on medical images, highlighting the prevalent application of uncertainty quantification techniques using deep learning models compared to machine learning models. Interestingly, we observed a scarcity of studies applying uncertainty quantification to physiological signals. Thus, future research on uncertainty quantification should prioritize investigating the application of these techniques to physiological signals. Overall, our review highlights the significance of integrating uncertainty techniques in healthcare applications of machine learning and deep learning models. This can provide valuable insights and practical solutions to manage uncertainty in real-world medical data, ultimately improving the accuracy and reliability of medical diagnoses and treatment recommendations.

BIANCA-MS: An optimized tool for automated multiple sclerosis lesion segmentation

In this work we present BIANCA-MS, a novel tool for brain white matter lesion segmentation in multiple sclerosis (MS), able to generalize across both the wide spectrum of MRI acquisition protocols and the heterogeneity of manually labeled data. BIANCA-MS is based on the original version of BIANCA and implements two innovative elements: a harmonized setting, tested under different MRI protocols, which avoids the need to further tune algorithm parameters to each dataset; and a cleaning step developed to improve consistency in automated and manual segmentations, thus reducing unwanted variability in output segmentations and validation data. BIANCA-MS was tested on three datasets, acquired with different MRI protocols. First, we compared BIANCA-MS to other widely used tools. Second, we tested how BIANCA-MS performs in separate datasets. Finally, we evaluated BIANCA-MS performance on a pooled dataset where all MRI data were merged. We calculated the overlap using the DICE spatial similarity index (SI) as well as the number of false positive/negative clusters (nFPC/nFNC) in comparison to the manual masks processed with the cleaning step. BIANCA-MS clearly outperformed other available tools in both high- and low-resolution images and provided comparable performance across different scanning protocols, sets of modalities and image resolutions. BIANCA-MS performance on the pooled dataset (SI: 0.72 ± 0.25, nFPC: 13 ± 11, nFNC: 4 ± 8) were comparable to those achieved on each individual dataset (median across datasets SI: 0.72 ± 0.28, nFPC: 14 ± 11, nFNC: 4 ± 8). Our findings suggest that BIANCA-MS is a robust and accurate approach for automated MS lesion segmentation.

Serum neurofilament light chain is more strongly associated with T2 lesion volume than with number of T2 lesions in patients with multiple sclerosis

BACKGROUND AND PURPOSE

MR imaging provides information on the number and extend of focal lesions in multiple sclerosis (MS) patients. This study explores whether total brain T2 lesion volume or lesion number shows a better correlation with serum and cerebrospinal fluid (CSF) biomarkers of disease activity.

MATERIALS AND METHODS

In total, 52 patients suffering from clinically isolated syndrome (CIS)/relapsing-remitting multiple sclerosis (RRMS) were assessed including MRI markers (total brain T2 lesion volume semi-automatically outlined on 3D DIR/FLAIR sequences, number of lesions), serum and CSF biomarkers at the time of neuroimaging (neurofilament light chain (NfL), glial fibrillary acidic protein (GFAP)), and clinical parameters. After log-transformation and partial correlations adjusted for the covariates patients' age, BMI, EDSS-score and diagnosis, the Fisher's r-to-Z transformation was used to compare different correlation coefficients.

RESULTS

The correlation between lesion volume and serum NfL (r = 0.6, p < 0.001) was stronger compared to the association between the number of T2 lesions and serum NfL (r = 0.4, p < 0.01) (z = -2.0, p < 0.05). With regard to CSF NfL, there was a moderate, positive relationship for both number of T2 lesions and lesion volume (r = 0.5 respectively, p < 0.01). We found no significant association between MRI markers and GFAP levels.

CONCLUSION

Our findings suggest that there is a stronger association between serum NfL and T2 lesion volume, than there is between serum NfL and T2 lesion number. Improving robustness and accuracy of fully-automated lesion volume segmentation tools can expedite implementation into clinical routine and trials.

Scanner agnostic large-scale evaluation of MS lesion delineation tool for clinical MRI

Introduction

Patients with MS are MRI scanned continuously throughout their disease course resulting in a large manual workload for radiologists which includes lesion detection and size estimation. Though many models for automatic lesion segmentation have been published, few are used broadly in clinic today, as there is a lack of testing on clinical datasets. By collecting a large, heterogeneous training dataset directly from our MS clinic we aim to present a model which is robust to different scanner protocols and artefacts and which only uses MRI modalities present in routine clinical examinations.

Methods

We retrospectively included 746 patients from routine examinations at our MS clinic. The inclusion criteria included acquisition at one of seven different scanners and an MRI protocol including 2D or 3D T2-w FLAIR, T2-w and T1-w images. Reference lesion masks on the training (n = 571) and validation (n = 70) datasets were generated using a preliminary segmentation model and subsequent manual correction. The test dataset (n = 100) was manually delineated. Our segmentation model https://github.com/CAAI/AIMS/ was based on the popular nnU-Net, which has won several biomedical segmentation challenges. We tested our model against the published segmentation models HD-MS-Lesions, which is also based on nnU-Net, trained with a more homogenous patient cohort. We furthermore tested model robustness to data from unseen scanners by performing a leave-one-scanner-out experiment.

Results

We found that our model was able to segment MS white matter lesions with a performance comparable to literature: DSC = 0.68, precision = 0.90, recall = 0.70, f1 = 0.78. Furthermore, the model outperformed HD-MS-Lesions in all metrics except precision = 0.96. In the leave-one-scanner-out experiment there was no significant change in performance (p < 0.05) between any of the models which were only trained on part of the dataset and the full segmentation model.

Conclusion

In conclusion we have seen, that by including a large, heterogeneous dataset emulating clinical reality, we have trained a segmentation model which maintains a high segmentation performance while being robust to data from unseen scanners. This broadens the applicability of the model in clinic and paves the way for clinical implementation.

Multi-arm U-Net with dense input and skip connectivity for T2 lesion segmentation in clinical trials of multiple sclerosis

T2 lesion quantification plays a crucial role in monitoring disease progression and evaluating treatment response in multiple sclerosis (MS). We developed a 3D, multi-arm U-Net for T2 lesion segmentation, which was trained on a large, multicenter clinical trial dataset of relapsing MS. We investigated its generalization to other relapsing and primary progressive MS clinical trial datasets, and to an external dataset from the MICCAI 2016 MS lesion segmentation challenge. Additionally, we assessed the model's ability to reproduce the separation of T2 lesion volumes between treatment and control arms; and the association of baseline T2 lesion volumes with clinical disability scores compared with manual lesion annotations. The trained model achieved a mean dice coefficient of ≥ 0.66 and a lesion detection sensitivity of ≥ 0.72 across the internal test datasets. On the external test dataset, the model achieved a mean dice coefficient of 0.62, which is comparable to 0.59 from the best model in the challenge, and a lesion detection sensitivity of 0.68. Lesion detection performance was reduced for smaller lesions (≤ 30 μL, 3-10 voxels). The model successfully maintained the separation of the longitudinal changes in T2 lesion volumes between the treatment and control arms. Such tools could facilitate semi-automated MS lesion quantification; and reduce rater burden in clinical trials.

RAAGR2-Net: A brain tumor segmentation network using parallel processing of multiple spatial frames

Brain tumors are one of the most fatal cancers. Magnetic Resonance Imaging (MRI) is a non-invasive method that provides multi-modal images containing important information regarding the tumor. Many contemporary techniques employ four modalities: T1-weighted (T1), T1-weighted with contrast (T1c), T2-weighted (T2), and fluid-attenuation-inversion-recovery (FLAIR), each of which provides unique and important characteristics for the location of each tumor. Although several modern procedures provide decent segmentation results on the multimodal brain tumor image segmentation benchmark (BraTS) dataset, they lack performance when evaluated simultaneously on all the regions of MRI images. Furthermore, there is still room for improvement due to parameter limitations and computational complexity. Therefore, in this work, a novel encoder-decoder-based architecture is proposed for the effective segmentation of brain tumor regions. Data pre-processing is performed by applying N4 bias field correction, z-score, and 0 to 1 resampling to facilitate model training. To minimize the loss of location information in different modules, a residual spatial pyramid pooling (RASPP) module is proposed. RASPP is a set of parallel layers using dilated convolution. In addition, an attention gate (AG) module is used to efficiently emphasize and restore the segmented output from extracted feature maps. The proposed modules attempt to acquire rich feature representations by combining knowledge from diverse feature maps and retaining their local information. The performance of the proposed deep network based on RASPP, AG, and recursive residual (R2) block termed RAAGR2-Net is evaluated on the BraTS benchmarks. The experimental results show that the suggested network outperforms existing networks that exhibit the usefulness of the proposed modules for "fine" segmentation. The code for this work is made available online at: https://github.com/Rehman1995/RAAGR2-Net.

A joint ventricle and WMH segmentation from MRI for evaluation of healthy and pathological changes in the aging brain

Age-related changes in brain structure include atrophy of the brain parenchyma and white matter changes of presumed vascular origin. Enlargement of the ventricles may occur due to atrophy or impaired cerebrospinal fluid (CSF) circulation. The co-occurrence of these changes in neurodegenerative diseases and in aging brains often requires investigators to take both into account when studying the brain, however, automated segmentation of enlarged ventricles and white matter hyperintensities (WMHs) can be a challenging task. Here, we present a hybrid multi-atlas segmentation and convolutional autoencoder approach for joint ventricle parcellation and WMH segmentation from magnetic resonance images (MRIs). Our fully automated approach uses a convolutional autoencoder to generate a standardized image of grey matter, white matter, CSF, and WMHs, which, in conjunction with labels generated by a multi-atlas segmentation approach, is then fed into a convolutional neural network to parcellate the ventricular system. Hence, our approach does not depend on manually delineated training data for new data sets. The segmentation pipeline was validated on both healthy elderly subjects and subjects with normal pressure hydrocephalus using ground truth manual labels and compared with state-of-the-art segmentation methods. We then applied the method to a cohort of 2401 elderly brains to investigate associations of ventricle volume and WMH load with various demographics and clinical biomarkers, using a multiple regression model. Our results indicate that the ventricle volume and WMH load are both highly variable in a cohort of elderly subjects and there is an independent association between the two, which highlights the importance of taking both the possibility of enlarged ventricles and WMHs into account when studying the aging brain.

Multiple sclerosis cortical lesion detection with deep learning at ultra-high-field MRI

Manually segmenting multiple sclerosis (MS) cortical lesions (CLs) is extremely time consuming, and past studies have shown only moderate inter-rater reliability. To accelerate this task, we developed a deep-learning-based framework (CLAIMS: Cortical Lesion AI-Based Assessment in Multiple Sclerosis) for the automated detection and classification of MS CLs with 7 T MRI. Two 7 T datasets, acquired at different sites, were considered. The first consisted of 60 scans that include 0.5 mm isotropic MP2RAGE acquired four times (MP2RAGE×4), 0.7 mm MP2RAGE, 0.5 mm T₂ *-weighted GRE, and 0.5 mm T₂ *-weighted EPI. The second dataset consisted of 20 scans including only 0.75 × 0.75 × 0.9 mm³ MP2RAGE. CLAIMS was first evaluated using sixfold cross-validation with single and multi-contrast 0.5 mm MRI input. Second, the performance of the model was tested on 0.7 mm MP2RAGE images after training with either 0.5 mm MP2RAGE×4, 0.7 mm MP2RAGE, or alternating the two. Third, its generalizability was evaluated on the second external dataset and compared with a state-of-the-art technique based on partial volume estimation and topological constraints (MSLAST). CLAIMS trained only with MP2RAGE×4 achieved results comparable to those of the multi-contrast model, reaching a CL true positive rate of 74% with a false positive rate of 30%. Detection rate was excellent for leukocortical and subpial lesions (83%, and 70%, respectively), whereas it reached 53% for intracortical lesions. The correlation between disability measures and CL count was similar for manual and CLAIMS lesion counts. Applying a domain-scanner adaptation approach and testing CLAIMS on the second dataset, the performance was superior to MSLAST when considering a minimum lesion volume of 6 μL (lesion-wise detection rate of 71% versus 48%). The proposed framework outperforms previous state-of-the-art methods for automated CL detection across scanners and protocols. In the future, CLAIMS may be useful to support clinical decisions at 7 T MRI, especially in the field of diagnosis and differential diagnosis of MS patients.

CEL-Unet: a novel CNN architecture for 3D Segmentation of Knee Bones affected by Severe Osteoarthritis for PSI-Based Surgical Planning

Unet architectures are promising deep learning networks exploited to perform the automatic segmentation of bone CT images, in line with their ability to deal with pathological deformations and size-varying anatomies. However, bone degeneration, like the development of irregular osteophytes as well as mineral density alterations might interfere with this automated process and demand extensive manual refinement. The aim of this work is to implement an innovative Unet variant, the CEL-Unet, to improve the femur and tibia segmentation outcomes in osteoarthritic knee joints. In this network the decoding path is split into a region and contour-aware branch to increase the prediction reliability in such pathological conditions. The comparison between the segmentation results achieved with a standard Unet and its novel variant (CEL-Unet) was performed as follows: the Unet was trained with 5 different loss functions: Dice Loss, Focal Loss, Exponential Logarithmic Loss, Double Cross Entropy Loss and Distanced Cross Entropy loss. The CEL-Unet was instead trained with two loss functions, one for each of the network outputs, namely Mask and Edge, yielding the so-called Combined Edge Loss (CEL) function. A set of 259 knee CT scans was used to train the model and test segmentation performance. The CEL-Unet outperformed all other Unet-based models, reaching the highest Jaccard values of about 0.97 and 0.96 on femur and tibia, respectively. Clinical Relevance- With the increasing rate of Total Knee Arthoplasty deep learning-based methods can achieve fast accurate and automatic 3D segmentation of the knee joint bones to enhance new costumized pre-operative planning.

MAGNIMS recommendations for harmonization of MRI data in MS multicenter studies

•

Sharing data from cooperative studies is essential to develop new biomarkers in MS.

•

Differences in MRI acquisition, analysis, storage represent a substantial constraint.

•

We review the state of the art and developments in the harmonization of MRI.

•

We provide recommendations to harmonize large MRI datasets in the MS field.

There is an increasing need of sharing harmonized data from large, cooperative studies as this is essential to develop new diagnostic and prognostic biomarkers. In the field of multiple sclerosis (MS), the issue has become of paramount importance due to the need to translate into the clinical setting some of the most recent MRI achievements. However, differences in MRI acquisition parameters, image analysis and data storage across sites, with their potential bias, represent a substantial constraint. This review focuses on the state of the art, recent technical advances, and desirable future developments of the harmonization of acquisition, analysis and storage of large-scale multicentre MRI data of MS cohorts. Huge efforts are currently being made to achieve all the requirements needed to provide harmonized MRI datasets in the MS field, as proper management of large imaging datasets is one of our greatest opportunities and challenges in the coming years. Recommendations based on these achievements will be provided here. Despite the advances that have been made, the complexity of these tasks requires further research by specialized academical centres, with dedicated technical and human resources. Such collective efforts involving different professional figures are of crucial importance to offer to MS patients a personalised management while minimizing consumption of resources.

Delve into Multiple Sclerosis (MS) lesion exploration: A modified attention U-Net for MS lesion segmentation in Brain MRI

Multiple Sclerosis (MS) is a Central Nervous System (CNS) disease that Magnetic Resonance Imaging (MRI) system can detect and segment its lesions. Artificial Neural Networks (ANNs) recently reached a noticeable performance in finding MS lesions from MRI. U-Net and Attention U-Net are two of the most successful ANNs in the field of MS lesion segmentation. In this work, we proposed a framework to segment MS lesions in Fluid-Attenuated Inversion Recovery (FLAIR) and T2 MRI images by modified U-Net and modified Attention U-Net. For this purpose, we developed some extra preprocessing on MRI scans, made modifications in the loss function of U-Net and Attention U-Net, and proposed using the union of FLAIR and T2 predictions to reach a better performance. Results show that the union of FLAIR and T2 predicted masks by the modified Attention U-Net reaches the performance of 82.30% in terms of Dice Similarity Coefficient (DSC) in the test dataset, which is a considerable improvement compared to the previous works.

Medical-Blocks: A Platform for Exploration, Management, Analysis, and Sharing of Data in Biomedical Research

BACKGROUND

Biomedical research requires healthcare institutions to provide sensitive clinical data to leverage data science and artificial intelligence technologies. However, providing healthcare data to researchers simple and secure, proves to be challenging for healthcare institutions.

OBJECTIVE

We describe and introduce Medical-Blocks, a platform for data exploration, data management, data analysis, and data sharing in biomedical research.

METHODS

The specification requirements for Medical-Blocks included: i) Connection to data sources of healthcare institutions with an interface for data exploration, ii) management of data in an internal file storage system, iii) data analysis through visualization and classification of data, and iv) data sharing via a file hosting service for collaboration. Medical-Blocks should be simple to use via a web-based user interface and extensible with new functionalities by a modular design via microservices ("blocks"). The scalability of the platform should be ensured by containerization. Security and legal regulations were considered during the development.

RESULTS

Medical-Blocks is a web application that runs in the cloud or as a local instance at a healthcare institution. Local instances of Medical-Blocks access data sources such as electronic health records and picture archiving and communications system (PACS) at healthcare institutions. Researchers and clinicians can explore, manage, and analyze the available data through Medical-Blocks. The data analysis involves classification of data for metadata extraction and the formation of cohorts. In collaborations, metadata (e.g., number of patients per cohort) and/or the data itself can be shared through Medical-Blocks locally or via a cloud instance to other researchers and clinicians.

CONCLUSIONS

Medical-Blocks facilitates biomedical research by providing a centralized platform to interact with medical data in collaborative research projects. The access to and management of medical data is simplified. Data can be swiftly analyzed to form cohorts for research and be shared among researchers. The modularity of Medical-Blocks makes the platform feasible for biomedical research where heterogenous medical data is needed.

CLINICALTRIAL

Multiple Sclerosis Lesion Analysis in Brain Magnetic Resonance Images: Techniques and Clinical Applications

Multiple sclerosis (MS) is a chronic inflammatory and degenerative disease of the central nervous system, characterized by the appearance of focal lesions in the white and gray matter that topographically correlate with an individual patients neurological symptoms and signs. Magnetic resonance imaging (MRI) provides detailed in-vivo structural information, permitting the quantification and categorization of MS lesions that critically inform disease management. Traditionally, MS lesions have been manually annotated on 2D MRI slices, a process that is inefficient and prone to inter-/intra-observer errors. Recently, automated statistical imaging analysis techniques have been proposed to detect and segment MS lesions based on MRI voxel intensity. However, their effectiveness is limited by the heterogeneity of both MRI data acquisition techniques and the appearance of MS lesions. By learning complex lesion representations directly from images, deep learning techniques have achieved remarkable breakthroughs in the MS lesion segmentation task. Here, we provide a comprehensive review of state-of-the-art automatic statistical and deep-learning MS segmentation methods and discuss current and future clinical applications. Further, we review technical strategies, such as domain adaptation, to enhance MS lesion segmentation in real-world clinical settings.

Construction of Community Life Service in the Sharing Economy Based on Deep Neural Network

Currently, the development of sharing economy and interconnection also has a profound impact on community life services. This study is based on the deep neural network theory, combined with the evolution mechanism of the commercial network of the community life service industry, link prediction theory, and the latest deep neural network algorithm, referring to the evolution model of merger and stripping, and the network structure is optimized on this basis. Through simulation experiments and result analysis, the model is used to deeply study the evolution trend and dynamics of the community life service business network from the perspective of quantitative analysis. Then the business network structure is optimized and development is promoted at the same time. At the same time, it can also upgrade those old scattered industries and provide theoretical and decision-making guidance for the future transformation and upgrading of the innovative community life service industry.

Optimization Algorithm of Tourism Security Early Warning Information System Based on Long Short-Term Memory (LSTM)

Tourism safety is the focus of the tourism industry. It is not only related to the safety of tourists' lives and property, but also related to social stability and sustainable development of the tourism industry. However, the security early warning of many scenic spots focuses on the response measures and remedial plans after the occurrence of security incidents, and the staff of many scenic spots have limited security awareness and information analysis ability, which is prone to lag in information release, and do not pay attention to the information of potential security problems. Therefore, this paper studies the optimization algorithm of the tourism security early warning information system based on the LSTM model and uses the recurrent neural network and LSTM to improve the processing and prediction ability of time-series data. The experimental results show that the number of three hidden layers in the tourism security early warning information system based on the LSTM model can reduce the training time of the model and improve the performance. Compared with the tourism safety early warning information system based on the BP neural network, it has better accuracy and stability, has better processing and prediction ability for time series data, and can monitor and analyze data scientifically in real-time and dynamically analyze data.

Assessment of Artificial Intelligence Automatic Multiple Sclerosis Lesion Delineation Tool for Clinical Use

PURPOSE

To implement and validate an existing algorithm for automatic delineation of white matter lesions on magnetic resonance imaging (MRI) in patients with multiple sclerosis (MS) on a local single-center dataset.

METHODS

We implemented a white matter hyperintensity segmentation model, based on a 2D convolutional neural network, using the conventional T2-weighted fluid attenuated inversion recovery (FLAIR) MRI sequence as input. The model was adapted for delineation of MS lesions by further training on a local dataset of 93 MS patients with a total of 3040 lesions. A quantitative evaluation was performed on ten test patients, in which model-generated masks were compared to manually delineated masks from two expert delineators. A subsequent qualitative evaluation of the implemented model was performed by two expert delineators, in which generated delineation masks on a clinical dataset of 53 patients were rated acceptable (< 10% errors) or unacceptable (> 10% errors) based on the total number of true lesions.

RESULTS

The quantitative evaluation resulted in an average accuracy score (F1) of 0.71, recall of 0.77 and dice similarity coefficient of 0.62. Our implemented model obtained the highest scores in all three metrics, when compared to three out of the box lesion segmentation models. In the clinical evaluation an average of 94% of our 53 model-generated masks were rated acceptable.

CONCLUSION

After adaptation to our local dataset, the implemented segmentation model was able to delineate MS lesions with a high clinical value as rated by delineation experts while outperforming popular out of the box applications. This serves as a promising step towards implementation of automatic lesion delineation in our MS clinic.

Emerging deep learning techniques using magnetic resonance imaging data applied in multiple sclerosis and clinical isolated syndrome patients (Review)

Computer-aided diagnosis systems aim to assist clinicians in the early identification of abnormal signs in order to optimize the interpretation of medical images and increase diagnostic precision. Multiple sclerosis (MS) and clinically isolated syndrome (CIS) are chronic inflammatory, demyelinating diseases affecting the central nervous system. Recent advances in deep learning (DL) techniques have led to novel computational paradigms in MS and CIS imaging designed for automatic segmentation and detection of areas of interest and automatic classification of anatomic structures, as well as optimization of neuroimaging protocols. To this end, there are several publications presenting artificial intelligence-based predictive models aiming to increase diagnostic accuracy and to facilitate optimal clinical management in patients diagnosed with MS and/or CIS. The current study presents a thorough review covering DL techniques that have been applied in MS and CIS during recent years, shedding light on their current advances and limitations.

Human Skeleton Detection and Extraction in Dance Video Based on PSO-Enabled LSTM Neural Network

With the significant increase of social informatization, the emerging information technology represented by machine vision has been applied to more and more scenes. Among them, the detection and extraction of human skeleton in a dance video based on this technology has a huge market demand in education and training. However, the existing detection and extraction technology has the problems of slow recognition speed and low extraction accuracy. Therefore, this paper proposes a neural network based on particle swarm optimization to detect and extract human skeletons in a dance video. Through the research and test on different data sets, it is found that the neural network based on particle swarm optimization algorithm has good detection and extraction ability and has high accuracy for the detection and recognition of human skeleton points. Among them, on all MPII data sets, the average accuracy of PSO-LSTM proposed in this paper is 3.9% higher than that of other optimal algorithms; on the PoseTrack data set, the average accuracy of detection and extraction is improved by 2.3%. The above results show that the neural network based on particle swarm optimization has fast detection speed and good extraction accuracy and can be used for the detection and extraction of human skeleton in a dance video.

Applications of deep learning techniques for automated multiple sclerosis detection using magnetic resonance imaging: A review

Multiple Sclerosis (MS) is a type of brain disease which causes visual, sensory, and motor problems for people with a detrimental effect on the functioning of the nervous system. In order to diagnose MS, multiple screening methods have been proposed so far; among them, magnetic resonance imaging (MRI) has received considerable attention among physicians. MRI modalities provide physicians with fundamental information about the structure and function of the brain, which is crucial for the rapid diagnosis of MS lesions. Diagnosing MS using MRI is time-consuming, tedious, and prone to manual errors. Research on the implementation of computer aided diagnosis system (CADS) based on artificial intelligence (AI) to diagnose MS involves conventional machine learning and deep learning (DL) methods. In conventional machine learning, feature extraction, feature selection, and classification steps are carried out by using trial and error; on the contrary, these steps in DL are based on deep layers whose values are automatically learn. In this paper, a complete review of automated MS diagnosis methods performed using DL techniques with MRI neuroimaging modalities is provided. Initially, the steps involved in various CADS proposed using MRI modalities and DL techniques for MS diagnosis are investigated. The important preprocessing techniques employed in various works are analyzed. Most of the published papers on MS diagnosis using MRI modalities and DL are presented. The most significant challenges facing and future direction of automated diagnosis of MS using MRI modalities and DL techniques are also provided.

icobrain ms 5.1: Combining unsupervised and supervised approaches for improving the detection of multiple sclerosis lesions

Multiple sclerosis (MS) is a chronic autoimmune, inflammatory neurological disease of the central nervous system. Its diagnosis nowadays commonly includes performing an MRI scan, as it is the most sensitive imaging test for MS. MS plaques are commonly identified from fluid-attenuated inversion recovery (FLAIR) images as hyperintense regions that are highly varying in terms of their shapes, sizes and locations, and are routinely classified in accordance to the McDonald criteria. Recent years have seen an increase in works that aimed at development of various semi-automatic and automatic methods for detection, segmentation and classification of MS plaques. In this paper, we present an automatic combined method, based on two pipelines: a traditional unsupervised machine learning technique and a deep-learning attention-gate 3D U-net network. The deep-learning network is specifically trained to address the weaker points of the traditional approach, namely difficulties in segmenting infratentorial and juxtacortical plaques in real-world clinical MRIs. It was trained and validated on a multi-center multi-scanner dataset that contains 159 cases, each with T1 weighted (T1w) and FLAIR images, as well as manual delineations of the MS plaques, segmented and validated by a panel of raters. The detection rate was quantified using lesion-wise Dice score. A simple label fusion is implemented to combine the output segmentations of the two pipelines. This combined method improves the detection of infratentorial and juxtacortical lesions by 14% and 31% respectively, in comparison to the unsupervised machine learning pipeline that was used as a performance assessment baseline.

Radius-optimized efficient template matching for lesion detection from brain images

Computer-aided detection of brain lesions from volumetric magnetic resonance imaging (MRI) is in demand for fast and automatic diagnosis of neural diseases. The template-matching technique can provide satisfactory outcome for automatic localization of brain lesions; however, finding the optimal template size that maximizes similarity of the template and the lesion remains challenging. This increases the complexity of the algorithm and the requirement for computational resources, while processing large MRI volumes with three-dimensional (3D) templates. Hence, reducing the computational complexity of template matching is needed. In this paper, we first propose a mathematical framework for computing the normalized cross-correlation coefficient (NCCC) as the similarity measure between the MRI volume and approximated 3D Gaussian template with linear time complexity, [Formula: see text], as opposed to the conventional fast Fourier transform (FFT) based approach with the complexity [Formula: see text], where [Formula: see text] is the number of voxels in the image and [Formula: see text] is the number of tried template radii. We then propose a mathematical formulation to analytically estimate the optimal template radius for each voxel in the image and compute the NCCC with the location-dependent optimal radius, reducing the complexity to [Formula: see text]. We test our methods on one synthetic and two real multiple-sclerosis databases, and compare their performances in lesion detection with FFT and a state-of-the-art lesion prediction algorithm. We demonstrate through our experiments the efficiency of the proposed methods for brain lesion detection and their comparable performance with existing techniques.

Investigating efficient CNN architecture for multiple sclerosis lesion segmentation

Purpose: The automatic segmentation of multiple sclerosis lesions in magnetic resonance imaging has the potential to reduce radiologists' efforts on a daily time-consuming task and to bring more reproducibility. Almost all new segmentation techniques make use of convolutional neural networks with their own different architecture. Architectural choices are rarely explained. We aimed at presenting the relevance of a U-net-like architecture for our specific task and at building an efficient and simple model. Approach: An experimental study was performed by observing the impact of applying different mutations and deletions to a simple U-net-like architecture. Results: The power of the U-net architecture is explained by the joint benefits of using an encoder-decoder architecture and by linking them with long skip connections. Augmenting the number of convolutional layers and decreasing the number of feature maps allowed us to build an exceptionally light and competitive architecture, the minimally parameterized U-net (MPU-net), with only ∼ 30,000 parameters. Conclusion: The empirical study of the U-net has led to a better understanding of its architecture. It has guided the building of the MPU-net, a model far less parameterized than others (at least by a factor of seven). This neural network achieves a human-level segmentation of multiple sclerosis lesions on fluid-attenuated inversion recovery images only. It shows that this segmentation task does not necessitate overly complicated models to be achieved. This gives the opportunity to build more explainable models that can help such methods to be adopted in a clinical environment.

A contrast-adaptive method for simultaneous whole-brain and lesion segmentation in multiple sclerosis

Here we present a method for the simultaneous segmentation of white matter lesions and normal-appearing neuroanatomical structures from multi-contrast brain MRI scans of multiple sclerosis patients. The method integrates a novel model for white matter lesions into a previously validated generative model for whole-brain segmentation. By using separate models for the shape of anatomical structures and their appearance in MRI, the algorithm can adapt to data acquired with different scanners and imaging protocols without retraining. We validate the method using four disparate datasets, showing robust performance in white matter lesion segmentation while simultaneously segmenting dozens of other brain structures. We further demonstrate that the contrast-adaptive method can also be safely applied to MRI scans of healthy controls, and replicate previously documented atrophy patterns in deep gray matter structures in MS. The algorithm is publicly available as part of the open-source neuroimaging package FreeSurfer.