A multi-path 2.5 dimensional convolutional neural network system for segmenting stroke lesions in brain MRI images

Robust Ensemble of Two Different Multimodal Approaches to Segment 3D Ischemic Stroke Segmentation Using Brain Tumor Representation Among Multiple Center Datasets

Ischemic stroke segmentation at an acute stage is vital in assessing the severity of patients' impairment and guiding therapeutic decision-making for reperfusion. Although many deep learning studies have shown attractive performance in medical segmentation, it is difficult to use these models trained on public data with private hospitals' datasets. Here, we demonstrate an ensemble model that employs two different multimodal approaches for generalization, a more effective way to perform on external datasets. First, after we jointly train a segmentation model on diffusion-weighted imaging (DWI) and apparent diffusion coefficient (ADC) MR modalities, the model is inferred on the DWI images. Second, a channel-wise segmentation model is trained by concatenating the DWI and ADC images as input, and then is inferred using both MR modalities. Before training with ischemic stroke data, we utilized BraTS 2021, a public brain tumor dataset, for transfer learning. An extensive ablation study evaluates which strategy learns better representations for ischemic stroke segmentation. In our study, nnU-Net well-known for robustness is selected as our baseline model. Our proposed method is evaluated on three different datasets: the Asan Medical Center (AMC) I and II, and the 2022 Ischemic Stroke Lesion Segmentation (ISLES). Our experiments are widely validated over a large, multi-center, and multi-scanner dataset with a huge amount of 846 scans. Not only stroke lesion models can benefit from transfer learning using brain tumor data, but combining the MR modalities using different training schemes also highly improves segmentation performance. The method achieved a top-1 ranking in the ongoing ISLES'22 challenge and performed particularly well on lesion-wise metrics of interest to neuroradiologists, achieving a Dice coefficient of 78.69% and a lesion-wise F1 score of 82.46%. Also, the method was relatively robust on the AMC I (Dice, 60.35%; lesion-wise F1, 68.30%) and II (Dice; 74.12%; lesion-wise F1, 67.53%) datasets in different settings. The high segmentation accuracy of our proposed method could improve radiologists' ability to detect ischemic stroke lesions in MRI images. Our model weights and inference code are available on https://github.com/MDOpx/ISLES22-model-inference .

Enhancing automatic prediction of clinically significant prostate cancer with deep transfer learning 2.5-dimensional segmentation on bi-parametric magnetic resonance imaging (bp-MRI)

Background

The aggressiveness of prostate cancer (PCa) is crucial in determining treatment method. The purpose of this study was to establish a 2.5-dimensional (2.5D) deep transfer learning (DTL) detection model for the automatic detection of clinically significant PCa (csPCa) based on bi-parametric magnetic resonance imaging (bp-MRI).

Methods

A total of 231 patients, including 181 with csPCa and 50 with non-clinically significant PCa (non-csPCa), were enrolled. Stratified random sampling was then employed to divide all participants into a training set [185] and a test set [46]. The DTL model was obtained through image acquisition, image segmentation, and model construction. Finally, the diagnostic performance of the 2.5D and 2-dimensional (2D) models in predicting the aggressiveness of PCa was evaluated and compared using receiver operating characteristic (ROC) curves.

Results

DTL models based on 2D and 2.5D segmentation were established and validated to assess the aggressiveness of PCa. The results demonstrated that the diagnostic efficiency of the DTL model based on 2.5D was superior to that of the 2D model, regardless of whether in a single or combined sequence. Particularly, the 2.5D combined model outperformed other models in differentiating csPCa from non-csPCa. The area under the curve (AUC) values for the 2.5D combined model in the training and test sets were 0.960 and 0.949, respectively. Furthermore, the T2-weighted imaging (T2WI) model showed superiority over the apparent diffusion coefficient (ADC) model, but was not as effective as the combined model, whether based on 2.5D or 2D.

Conclusions

A DTL model based on 2.5D segmentation was developed to automatically evaluate PCa aggressiveness on bp-MRI, improving the diagnostic performance of the 2D model. The results indicated that the continuous information between adjacent layers can enhance the detection rate of lesions and reduce the misjudgment rate based on the DTL model.

Distinct brain morphometry patterns revealed by deep learning improve prediction of post-stroke aphasia severity

BACKGROUND

Emerging evidence suggests that post-stroke aphasia severity depends on the integrity of the brain beyond the lesion. While measures of lesion anatomy and brain integrity combine synergistically to explain aphasic symptoms, substantial interindividual variability remains unaccounted. One explanatory factor may be the spatial distribution of morphometry beyond the lesion (e.g., atrophy), including not just specific brain areas, but distinct three-dimensional patterns.

METHODS

Here, we test whether deep learning with Convolutional Neural Networks (CNNs) on whole brain morphometry (i.e., segmented tissue volumes) and lesion anatomy better predicts chronic stroke individuals with severe aphasia (N = 231) than classical machine learning (Support Vector Machines; SVMs), evaluating whether encoding spatial dependencies identifies uniquely predictive patterns.

RESULTS

CNNs achieve higher balanced accuracy and F1 scores, even when SVMs are nonlinear or integrate linear or nonlinear dimensionality reduction. Parity only occurs when SVMs access features learned by CNNs. Saliency maps demonstrate that CNNs leverage distributed morphometry patterns, whereas SVMs focus on the area around the lesion. Ensemble clustering of CNN saliencies reveals distinct morphometry patterns unrelated to lesion size, consistent across individuals, and which implicate unique networks associated with different cognitive processes as measured by the wider neuroimaging literature. Individualized predictions depend on both ipsilateral and contralateral features outside the lesion.

CONCLUSIONS

Three-dimensional network distributions of morphometry are directly associated with aphasia severity, underscoring the potential for CNNs to improve outcome prognostication from neuroimaging data, and highlighting the prospective benefits of interrogating spatial dependence at different scales in multivariate feature space.

Taxonomy of Acute Stroke: Imaging, Processing, and Treatment

Stroke management employs a variety of diagnostic imaging modalities, image processing and analysis methods, and treatment procedures. This work categorizes methods for stroke imaging, image processing and analysis, and treatment, and provides their taxonomies illustrated by a state-of-the-art review. Imaging plays a critical role in stroke management, and the most frequently employed modalities are computed tomography (CT) and magnetic resonance (MR). CT includes unenhanced non-contrast CT as the first-line diagnosis, CT angiography, and CT perfusion. MR is the most complete method to examine stroke patients. MR angiography is useful to evaluate the severity of artery stenosis, vascular occlusion, and collateral flow. Diffusion-weighted imaging is the gold standard for evaluating ischemia. MR perfusion-weighted imaging assesses the penumbra. The stroke image processing methods are divided into non-atlas/template-based and atlas/template-based. The non-atlas/template-based methods are subdivided into intensity and contrast transformations, local segmentation-related, anatomy-guided, global density-guided, and artificial intelligence/deep learning-based. The atlas/template-based methods are subdivided into intensity templates and atlases with three atlas types: anatomy atlases, vascular atlases, and lesion-derived atlases. The treatment procedures for arterial and venous strokes include intravenous and intraarterial thrombolysis and mechanical thrombectomy. This work captures the state-of-the-art in stroke management summarized in the form of comprehensive and straightforward taxonomy diagrams. All three introduced taxonomies in diagnostic imaging, image processing and analysis, and treatment are widely illustrated and compared against other state-of-the-art classifications.

Distinct brain morphometry patterns revealed by deep learning improve prediction of aphasia severity

Emerging evidence suggests that post-stroke aphasia severity depends on the integrity of the brain beyond the stroke lesion. While measures of lesion anatomy and brain integrity combine synergistically to explain aphasic symptoms, significant interindividual variability remains unaccounted for. A possible explanatory factor may be the spatial distribution of brain atrophy beyond the lesion. This includes not just the specific brain areas showing atrophy, but also distinct three-dimensional patterns of atrophy. Here, we tested whether deep learning with Convolutional Neural Networks (CNN) on whole brain morphometry (i.e., segmented tissue volumes) and lesion anatomy can better predict which individuals with chronic stroke (N=231) have severe aphasia, and whether encoding spatial dependencies in the data might be capable of improving predictions by identifying unique individualized spatial patterns. We observed that CNN achieves significantly higher accuracy and F1 scores than Support Vector Machine (SVM), even when the SVM is nonlinear or integrates linear and nonlinear dimensionality reduction techniques. Performance parity was only achieved when the SVM was directly trained on the latent features learned by the CNN. Saliency maps demonstrated that the CNN leveraged widely distributed patterns of brain atrophy predictive of aphasia severity, whereas the SVM focused almost exclusively on the area around the lesion. Ensemble clustering of CNN saliency maps revealed distinct morphometry patterns that were unrelated to lesion size, highly consistent across individuals, and implicated unique brain networks associated with different cognitive processes as measured by the wider neuroimaging literature. Individualized predictions of severity depended on both ipsilateral and contralateral features outside of the location of stroke. Our findings illustrate that three-dimensional network distributions of atrophy in individuals with aphasia are directly associated with aphasia severity, underscoring the potential for deep learning to improve prognostication of behavioral outcomes from neuroimaging data, and highlighting the prospective benefits of interrogating spatial dependence at different scales in multivariate feature space.

Fine-grained brain tissue segmentation for brain modeling of stroke patient

Brain segmentation of stroke patients can facilitate brain modeling for electrical non-invasive brain stimulation, a therapy for stimulating brain function using an electric current. However, it remains challenging owing to its time-consuming, labor-dependent, and complicated pipeline. In addition, conventional tools that define lesions into one region rather than distinguishing between the stroke-affected regions and cerebrospinal fluid can lead to inaccurate treatment results. In this study, we first define a novel stroke-affected region as a detailed sub-region of the conventionally defined lesion. Subsequently, a novel comprehensive framework is proposed to segment head-brain and fine-level stroke-affected regions for normal controls and chronic stroke patients. The proposed framework consists of a time-efficient and precise deep learning-based segmentation model. The experiment results indicate that the proposed method perform better than the conventional deep learning-based segmentation model in terms of the evaluation metrics. The proposed method would be a valuable addition to brain modeling for non-invasive neuromodulation.

Multislice input for 2D and 3D residual convolutional neural network noise reduction in CT

Purpose

Deep convolutional neural network (CNN)-based methods are increasingly used for reducing image noise in computed tomography (CT). Current attempts at CNN denoising are based on 2D or 3D CNN models with a single- or multiple-slice input. Our work aims to investigate if the multiple-slice input improves the denoising performance compared with the single-slice input and if a 3D network architecture is better than a 2D version at utilizing the multislice input.

Approach

Two categories of network architectures can be used for the multislice input. First, multislice images can be stacked channel-wise as the multichannel input to a 2D CNN model. Second, multislice images can be employed as the 3D volumetric input to a 3D CNN model, in which the 3D convolution layers are adopted. We make performance comparisons among 2D CNN models with one, three, and seven input slices and two versions of 3D CNN models with seven input slices and one or three output slices. Evaluation was performed on liver CT images using three quantitative metrics with full-dose images as reference. Visual assessment was made by an experienced radiologist.

Results

When the input channels of the 2D CNN model increases from one to three to seven, a trend of improved performance was observed. Comparing the three models with the seven-slice input, the 3D CNN model with a one-slice output outperforms the other models in terms of noise texture and homogeneity in liver parenchyma as well as subjective visualization of vessels.

Conclusions

We conclude the that multislice input is an effective strategy for improving performance for 2D deep CNN denoising models. The pure 3D CNN model tends to have a better performance than the other models in terms of continuity across axial slices, but the difference was not significant compared with the 2D CNN model with the same number of slices as the input.

Predicting final ischemic stroke lesions from initial diffusion-weighted images using a deep neural network

BACKGROUND

For prognosis of stroke, measurement of the diffusion-perfusion mismatch is a common practice for estimating tissue at risk of infarction in the absence of timely reperfusion. However, perfusion-weighted imaging (PWI) adds time and expense to the acute stroke imaging workup. We explored whether a deep convolutional neural network (DCNN) model trained with diffusion-weighted imaging obtained at admission could predict final infarct volume and location in acute stroke patients.

METHODS

In 445 patients, we trained and validated an attention-gated (AG) DCNN to predict final infarcts as delineated on follow-up studies obtained 3 to 7 days after stroke. The input channels consisted of MR diffusion-weighted imaging (DWI), apparent diffusion coefficients (ADC) maps, and thresholded ADC maps with values less than 620 × 10-6 mm2/s, while the output was a voxel-by-voxel probability map of tissue infarction. We evaluated performance of the model using the area under the receiver-operator characteristic curve (AUC), the Dice similarity coefficient (DSC), absolute lesion volume error, and the concordance correlation coefficient (ρc) of the predicted and true infarct volumes.

RESULTS

The model obtained a median AUC of 0.91 (IQR: 0.84-0.96). After thresholding at an infarction probability of 0.5, the median sensitivity and specificity were 0.60 (IQR: 0.16-0.84) and 0.97 (IQR: 0.93-0.99), respectively, while the median DSC and absolute volume error were 0.50 (IQR: 0.17-0.66) and 27 ml (IQR: 7-60 ml), respectively. The model's predicted lesion volumes showed high correlation with ground truth volumes (ρc = 0.73, p < 0.01).

CONCLUSION

An AG-DCNN using diffusion information alone upon admission was able to predict infarct volumes at 3-7 days after stroke onset with comparable accuracy to models that consider both DWI and PWI. This may enable treatment decisions to be made with shorter stroke imaging protocols.

Application of Machine Learning Techniques for Characterization of Ischemic Stroke with MRI Images: A Review

Magnetic resonance imaging (MRI) is a standard tool for the diagnosis of stroke, but its manual interpretation by experts is arduous and time-consuming. Thus, there is a need for computer-aided-diagnosis (CAD) models for the automatic segmentation and classification of stroke on brain MRI. The heterogeneity of stroke pathogenesis, morphology, image acquisition modalities, sequences, and intralesional tissue signal intensity, as well as lesion-to-normal tissue contrast, pose significant challenges to the development of such systems. Machine learning (ML) is increasingly being used in predictive neuroimaging diagnosis and prognostication. This paper reviews image processing and machine learning techniques that have been applied to detect ischemic stroke on brain MRI, including details on image acquisition, pre-processing, techniques to segment, extraction of features, and classification into stroke types. The main objective of this work is to find the state-of-art machine learning techniques used to predict the ischemic stroke and their application in clinical set-up. The article selection is performed according to PRISMA guideline. The state-of-the-art on automated MRI stroke diagnosis, with a focus on machine learning, is discussed, along with its advantages and limitations. We found that the various machine learning models discussed in this article are able to detect the infarcts with an acceptable accuracy of 70–90%. However, no one has highlighted the time complexity to predict the stroke in the model developed, which is an important factor. The work concludes with proposals for future recommendations for building efficient and robust deep learning (DL) models for quantitative brain MRI analysis. In recent work, with the application of DL approaches, using large datasets to train the models has improved the detection accuracy and reduced computational complexity. We suggest that the design of a decision support system based on artificial intelligence (AI) and clinical data presenting symptoms is essential to support clinicians to accelerate diagnosis and timeous therapy in the emergency management of stroke.

Automatic Segmentation and Quantitative Assessment of Stroke Lesions on MR Images

Lesion studies are crucial in establishing brain-behavior relationships, and accurately segmenting the lesion represents the first step in achieving this. Manual lesion segmentation is the gold standard for chronic strokes. However, it is labor-intensive, subject to bias, and limits sample size. Therefore, our objective is to develop an automatic segmentation algorithm for chronic stroke lesions on T1-weighted MR images. Methods: To train our model, we utilized an open-source dataset: ATLAS v2.0 (Anatomical Tracings of Lesions After Stroke). We partitioned the dataset of 655 T1 images with manual segmentation labels into five subsets and performed a 5-fold cross-validation to avoid overfitting of the model. We used a deep neural network (DNN) architecture for model training. Results: To evaluate the model performance, we used three metrics that pertain to diverse aspects of volumetric segmentation, including shape, location, and size. The Dice similarity coefficient (DSC) compares the spatial overlap between manual and machine segmentation. The average DSC was 0.65 (0.61−0.67; 95% bootstrapped CI). Average symmetric surface distance (ASSD) measures contour distances between the two segmentations. ASSD between manual and automatic segmentation was 12 mm. Finally, we compared the total lesion volumes and the Pearson correlation coefficient (ρ) between the manual and automatically segmented lesion volumes, which was 0.97 (p-value < 0.001). Conclusions: We present the first automated segmentation model trained on a large multicentric dataset. This model will enable automated on-demand processing of MRI scans and quantitative chronic stroke lesion assessment.

Artificial intelligence in multi-parametric magnetic resonance imaging: A review

Multi-parametric magnetic resonance imaging (mpMRI) is an indispensable tool in the clinical workflow for the diagnosis and treatment planning of various diseases. Machine learning-based artificial intelligence (AI) methods, especially those adopting the deep learning technique, have been extensively employed to perform mpMRI image classification, segmentation, registration, detection, reconstruction, and super-resolution. The current availability of increasing computational power and fast-improving AI algorithms have empowered numerous computer-based systems for applying mpMRI to disease diagnosis, imaging-guided radiotherapy, patient risk and overall survival time prediction, and the development of advanced quantitative imaging technology for magnetic resonance fingerprinting. However, the wide application of these developed systems in the clinic is still limited by a number of factors, including robustness, reliability, and interpretability. This survey aims to provide an overview for new researchers in the field as well as radiologists with the hope that they can understand the general concepts, main application scenarios, and remaining challenges of AI in mpMRI. This article is protected by copyright. All rights reserved.

A large, curated, open-source stroke neuroimaging dataset to improve lesion segmentation algorithms

Accurate lesion segmentation is critical in stroke rehabilitation research for the quantification of lesion burden and accurate image processing. Current automated lesion segmentation methods for T1-weighted (T1w) MRIs, commonly used in stroke research, lack accuracy and reliability. Manual segmentation remains the gold standard, but it is time-consuming, subjective, and requires neuroanatomical expertise. We previously released an open-source dataset of stroke T1w MRIs and manually-segmented lesion masks (ATLAS v1.2, N = 304) to encourage the development of better algorithms. However, many methods developed with ATLAS v1.2 report low accuracy, are not publicly accessible or are improperly validated, limiting their utility to the field. Here we present ATLAS v2.0 (N = 1271), a larger dataset of T1w MRIs and manually segmented lesion masks that includes training (n = 655), test (hidden masks, n = 300), and generalizability (hidden MRIs and masks, n = 316) datasets. Algorithm development using this larger sample should lead to more robust solutions; the hidden datasets allow for unbiased performance evaluation via segmentation challenges. We anticipate that ATLAS v2.0 will lead to improved algorithms, facilitating large-scale stroke research.

Automatic kidney segmentation using 2.5D ResUNet and 2.5D DenseUNet for malignant potential analysis in complex renal cyst based on CT images

Bosniak renal cyst classification has been widely used in determining the complexity of a renal cyst. However, it turns out that about half of patients undergoing surgery for Bosniak category III, take surgical risks that reward them with no clinical benefit at all. This is because their pathological results reveal that the cysts are actually benign not malignant. This problem inspires us to use recently popular deep learning techniques and study alternative analytics methods for precise binary classification (benign or malignant tumor) on Computerized Tomography (CT) images. To achieve our goal, two consecutive steps are required-segmenting kidney organs or lesions from CT images then classifying the segmented kidneys. In this paper, we propose a study of kidney segmentation using 2.5D ResUNet and 2.5D DenseUNet for efficiently extracting intra-slice and inter-slice features. Our models are trained and validated on the public data set from Kidney Tumor Segmentation (KiTS19) challenge in two different training environments. As a result, all experimental models achieve high mean kidney Dice scores of at least 95% on the KiTS19 validation set consisting of 60 patients. Apart from the KiTS19 data set, we also conduct separate experiments on abdomen CT images of four Thai patients. Based on the four Thai patients, our experimental models show a drop in performance, where the best mean kidney Dice score is 87.60%.

Cross-Attention and Deep Supervision UNet for Lesion Segmentation of Chronic Stroke

Stroke is an acute cerebrovascular disease with high incidence, high mortality, and high disability rate. Determining the location and volume of the disease in MR images promotes accurate stroke diagnosis and surgical planning. Therefore, the automatic recognition and segmentation of stroke lesions has important clinical significance for large-scale stroke imaging analysis. There are some problems in the segmentation of stroke lesions, such as imbalance of the front and back scenes, uncertainty of position, and unclear boundary. To meet this challenge, this paper proposes a cross-attention and deep supervision UNet (CADS-UNet) to segment chronic stroke lesions from T1-weighted MR images. Specifically, we propose a cross-spatial attention module, which is different from the usual self-attention module. The location information interactively selects encode features and decode features to enrich the lost spatial focus. At the same time, the channel attention mechanism is used to screen the channel characteristics. Finally, combined with deep supervision and mixed loss, the model is supervised more accurately. We compared and verified the model on the authoritative open dataset "Anatomical Tracings of Lesions After Stroke" (Atlas), which fully proved the effectiveness of our model.

Toward automated segmentation for acute ischemic stroke using non-contrast computed tomography

PURPOSE

Non-contrast computed tomography (NCCT) is a first-line imaging technique for determining treatment options for acute ischemic stroke (AIS). However, its poor contrast and signal-to-noise ratio limit the diagnosis accuracy for radiologists, and automated AIS lesion segmentation using NCCT also remains a challenge. In this paper, we propose R2U-RNet, a novel model for AIS lesion segmentation using NCCT.

METHODS

We used an in-house retrospective NCCT dataset with 261 AIS patients with manual lesion segmentation using follow-up diffusion-weighted images. R2U-RNet is based on an R2U-Net backbone with a novel residual refinement unit. Each input image contains two image channels from separate preprocessing procedures. The proposed model incorporates multiscale focal loss to mitigate the class imbalance problem and to leverage the importance of different levels of details. A proposed noisy-label training scheme is utilized to account for uncertainties in the manual annotations.

RESULTS

The proposed model outperformed several iconic segmentation models in AIS lesion segmentation using NCCT, and our ablation study demonstrated the efficacy of the proposed model. Statistical analysis of segmentation performance revealed significant effects of regional stroke occurrence and side of the stroke, suggesting the importance of region-specific information for automated segmentation, and the potential influence of the hemispheric difference in clinical data.

CONCLUSION

This study demonstrated the potentials of R2U-RNet model for automated NCCT AIS lesion segmentation. The proposed model can serve as a tool for accelerating AIS diagnoses and improving the treatment quality of AIS patients.

A Review on Computer Aided Diagnosis of Acute Brain Stroke

Amongst the most common causes of death globally, stroke is one of top three affecting over 100 million people worldwide annually. There are two classes of stroke, namely ischemic stroke (due to impairment of blood supply, accounting for ~70% of all strokes) and hemorrhagic stroke (due to bleeding), both of which can result, if untreated, in permanently damaged brain tissue. The discovery that the affected brain tissue (i.e., 'ischemic penumbra') can be salvaged from permanent damage and the bourgeoning growth in computer aided diagnosis has led to major advances in stroke management. Abiding to the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines, we have surveyed a total of 177 research papers published between 2010 and 2021 to highlight the current status and challenges faced by computer aided diagnosis (CAD), machine learning (ML) and deep learning (DL) based techniques for CT and MRI as prime modalities for stroke detection and lesion region segmentation. This work concludes by showcasing the current requirement of this domain, the preferred modality, and prospective research areas.

Deep learning-based detection and segmentation of diffusion abnormalities in acute ischemic stroke

Background

Accessible tools to efficiently detect and segment diffusion abnormalities in acute strokes are highly anticipated by the clinical and research communities.

Methods

We developed a tool with deep learning networks trained and tested on a large dataset of 2,348 clinical diffusion weighted MRIs of patients with acute and sub-acute ischemic strokes, and further tested for generalization on 280 MRIs of an external dataset (STIR).

Results

Our proposed model outperforms generic networks and DeepMedic, particularly in small lesions, with lower false positive rate, balanced precision and sensitivity, and robustness to data perturbs (e.g., artefacts, low resolution, technical heterogeneity). The agreement with human delineation rivals the inter-evaluator agreement; the automated lesion quantification of volume and contrast has virtually total agreement with human quantification.

Conclusion

Our tool is fast, public, accessible to non-experts, with minimal computational requirements, to detect and segment lesions via a single command line. Therefore, it fulfills the conditions to perform large scale, reliable and reproducible clinical and translational research.

Asymmetric Ensemble of Asymmetric U-Net Models for Brain Tumor Segmentation With Uncertainty Estimation

Accurate brain tumor segmentation is crucial for clinical assessment, follow-up, and subsequent treatment of gliomas. While convolutional neural networks (CNN) have become state of the art in this task, most proposed models either use 2D architectures ignoring 3D contextual information or 3D models requiring large memory capacity and extensive learning databases. In this study, an ensemble of two kinds of U-Net-like models based on both 3D and 2.5D convolutions is proposed to segment multimodal magnetic resonance images (MRI). The 3D model uses concatenated data in a modified U-Net architecture. In contrast, the 2.5D model is based on a multi-input strategy to extract low-level features from each modality independently and on a new 2.5D Multi-View Inception block that aims to merge features from different views of a 3D image aggregating multi-scale features. The Asymmetric Ensemble of Asymmetric U-Net (AE AU-Net) based on both is designed to find a balance between increasing multi-scale and 3D contextual information extraction and keeping memory consumption low. Experiments on 2019 dataset show that our model improves enhancing tumor sub-region segmentation. Overall, performance is comparable with state-of-the-art results, although with less learning data or memory requirements. In addition, we provide voxel-wise and structure-wise uncertainties of the segmentation results, and we have established qualitative and quantitative relationships between uncertainty and prediction errors. Dice similarity coefficient for the whole tumor, tumor core, and tumor enhancing regions on BraTS 2019 validation dataset were 0.902, 0.815, and 0.773. We also applied our method in BraTS 2018 with corresponding Dice score values of 0.908, 0.838, and 0.800.

Robust performance of deep learning for automatic detection and segmentation of brain metastases using three-dimensional black-blood and three-dimensional gradient echo imaging

OBJECTIVES

To evaluate whether a deep learning (DL) model using both three-dimensional (3D) black-blood (BB) imaging and 3D gradient echo (GRE) imaging may improve the detection and segmentation performance of brain metastases compared to that using only 3D GRE imaging.

METHODS

A total of 188 patients with brain metastases (917 lesions) who underwent a brain metastasis MRI protocol including contrast-enhanced 3D BB and 3D GRE were included in the training set. DL models based on 3D U-net were constructed. The models were validated in the test set consisting of 45 patients with brain metastases (203 lesions) and 49 patients without brain metastases.

RESULTS

The combined 3D BB and 3D GRE model yielded better performance than the 3D GRE model (sensitivities of 93.1% vs 76.8%, p < 0.001), and this effect was significantly stronger in subgroups with small metastases (p interaction < 0.001). For metastases < 3 mm, ≥ 3 mm and < 10 mm, and ≥ 10 mm, the sensitivities were 82.4%, 93.2%, and 100%, respectively. The combined 3D BB and 3D GRE model showed a false-positive per case of 0.59 in the test set. The combined 3D BB and 3D GRE model showed a Dice coefficient of 0.822, while 3D GRE model showed a lower Dice coefficient of 0.756.

CONCLUSIONS

The combined 3D BB and 3D GRE DL model may improve the detection and segmentation performance of brain metastases, especially in detecting small metastases.

KEY POINTS

• The combined 3D BB and 3D GRE model yielded better performance for the detection of brain metastases than the 3D GRE model (p < 0.001), with sensitivities of 93.1% and 76.8%, respectively. • The combined 3D BB and 3D GRE model showed a false-positive rate per case of 0.59 in the test set. • The combined 3D BB and 3D GRE model showed a Dice coefficient of 0.822, while the 3D GRE model showed a lower Dice coefficient of 0.756.

Automated Segmentation of Infarct Lesions in T1-Weighted MRI Scans Using Variational Mode Decomposition and Deep Learning

Automated segmentation methods are critical for early detection, prompt actions, and immediate treatments in reducing disability and death risks of brain infarction. This paper aims to develop a fully automated method to segment the infarct lesions from T1-weighted brain scans. As a key novelty, the proposed method combines variational mode decomposition and deep learning-based segmentation to take advantages of both methods and provide better results. There are three main technical contributions in this paper. First, variational mode decomposition is applied as a pre-processing to discriminate the infarct lesions from unwanted non-infarct tissues. Second, overlapped patches strategy is proposed to reduce the workload of the deep-learning-based segmentation task. Finally, a three-dimensional U-Net model is developed to perform patch-wise segmentation of infarct lesions. A total of 239 brain scans from a public dataset is utilized to develop and evaluate the proposed method. Empirical results reveal that the proposed automated segmentation can provide promising performances with an average dice similarity coefficient (DSC) of 0.6684, intersection over union (IoU) of 0.5022, and average symmetric surface distance (ASSD) of 0.3932, respectively.

Acute and sub-acute stroke lesion segmentation from multimodal MRI

BACKGROUND AND OBJECTIVE

Acute stroke lesion segmentation tasks are of great clinical interest as they can help doctors make better informed time-critical treatment decisions. Magnetic resonance imaging (MRI) is time demanding but can provide images that are considered the gold standard for diagnosis. Automated stroke lesion segmentation can provide with an estimate of the location and volume of the lesioned tissue, which can help in the clinical practice to better assess and evaluate the risks of each treatment.

METHODS

We propose a deep learning methodology for acute and sub-acute stroke lesion segmentation using multimodal MR imaging. We pre-process the data to facilitate learning features based on the symmetry of brain hemispheres. The issue of class imbalance is tackled using small patches with a balanced training patch sampling strategy and a dynamically weighted loss function. Moreover, a combination of whole patch predictions, using a U-Net based CNN architecture, and high degree of overlapping patches reduces the need for additional post-processing.

RESULTS

The proposed method is evaluated using two public datasets from the 2015 Ischemic Stroke Lesion Segmentation challenge (ISLES 2015). These involve the tasks of sub-acute stroke lesion segmentation (SISS) and acute stroke penumbra estimation (SPES) from multiple diffusion, perfusion and anatomical MRI modalities. The performance is compared against state-of-the-art methods with a blind online testing set evaluation on each of the challenges. At the time of submitting this manuscript, our approach is the first method in the online rankings for the SISS (DSC=0.59 ± 0.31) and SPES sub-tasks (DSC=0.84 ± 0.10). When compared with the rest of submitted strategies, we achieve top rank performance with a lower Hausdorff distance.

CONCLUSIONS

Better segmentation results are obtained by leveraging the anatomy and pathophysiology of acute stroke lesions and using a combined approach to minimize the effects of class imbalance. The same training procedure is used for both tasks, showing the proposed methodology can generalize well enough to deal with different unrelated tasks and imaging modalities without hyper-parameter tuning. In order to promote the reproducibility of our results, a public version of the proposed method has been released to the scientific community.

Automatic post-stroke lesion segmentation on MR images using 3D residual convolutional neural network

In this paper, we demonstrate the feasibility and performance of deep residual neural networks for volumetric segmentation of irreversibly damaged brain tissue lesions on T1-weighted MRI scans for chronic stroke patients. A total of 239 T1-weighted MRI scans of chronic ischemic stroke patients from a public dataset were retrospectively analyzed by 3D deep convolutional segmentation models with residual learning, using a novel zoom-in&out strategy. Dice similarity coefficient (DSC), average symmetric surface distance (ASSD), and Hausdorff distance (HD) of the identified lesions were measured by using manual tracing of lesions as the reference standard. Bootstrapping was employed for all metrics to estimate 95% confidence intervals. The models were assessed on a test set of 31 scans. The average DSC was 0.64 (0.51-0.76) with a median of 0.78. ASSD and HD were 3.6 mm (1.7-6.2 mm) and 20.4 mm (10.0-33.3 mm), respectively. The latest deep learning architecture and techniques were applied with 3D segmentation on MRI scans and demonstrated effectiveness for volumetric segmentation of chronic ischemic stroke lesions.