3D multimodal MRI brain glioma tumor and edema segmentation: a graph cut distribution matching approach

Automated identification and quantification of metastatic brain tumors and perilesional edema based on a deep learning neural network

PURPOSE

This paper presents a deep learning model for use in the automated segmentation of metastatic brain tumors and associated perilesional edema.

METHODS

The model was trained using Gamma Knife surgical data (90 MRI sets from 46 patients), including the initial treatment plan and follow-up images (T1-weighted contrast-enhanced (T1cWI) and T2-weighted images (T2WI)) manually annotated by neurosurgeons to indicate the target tumor and edema regions. A mask region-based convolutional neural network was used to extract brain parenchyma, after which the DeepMedic 3D convolutional neural network was in the segmentation of tumors and edemas.

RESULTS

Five-fold cross-validation demonstrated the efficacy of the brain parenchyma extraction model, achieving a Dice similarity coefficient of 96.4%. The segmentation models used for metastatic tumors and brain edema achieved Dice similarity coefficients of 71.6% and 85.1%, respectively. This study also presents an intuitive graphical user interface to facilitate the use of these models in clinical analysis.

CONCLUSION

This paper introduces a deep learning model for the automated segmentation and quantification of brain metastatic tumors and perilesional edema trained using only T1cWI and T2WI. This technique could facilitate further research on metastatic tumors and perilesional edema as well as other intracranial lesions.

A review on brain tumor segmentation based on deep learning methods with federated learning techniques

Brain tumors have become a severe medical complication in recent years due to their high fatality rate. Radiologists segment the tumor manually, which is time-consuming, error-prone, and expensive. In recent years, automated segmentation based on deep learning has demonstrated promising results in solving computer vision problems such as image classification and segmentation. Brain tumor segmentation has recently become a prevalent task in medical imaging to determine the tumor location, size, and shape using automated methods. Many researchers have worked on various machine and deep learning approaches to determine the most optimal solution using the convolutional methodology. In this review paper, we discuss the most effective segmentation techniques based on the datasets that are widely used and publicly available. We also proposed a survey of federated learning methodologies to enhance global segmentation performance and ensure privacy. A comprehensive literature review is suggested after studying more than 100 papers to generalize the most recent techniques in segmentation and multi-modality information. Finally, we concentrated on unsolved problems in brain tumor segmentation and a client-based federated model training strategy. Based on this review, future researchers will understand the optimal solution path to solve these issues.

The Algorithm of Watershed Color Image Segmentation Based on Morphological Gradient

The traditional watershed algorithm has the disadvantage of over-segmentation and interference with an image by reflected light. We propose an improved watershed color image segmentation algorithm. It is based on a morphological gradient. This method obtains the component gradient of a color image in a new color space is not disturbed by the reflected light. The gradient image is reconstructed by opening and closing. Therefore, the final gradient image is obtained. The maximum inter-class variance algorithm is used to obtain the threshold automatically for the final gradient image. The original gradient image is forcibly calibrated with the obtained binary labeled image, and the modified gradient image is segmented by watershed. Experimental results show that the proposed method can obtain an accurate and continuous target contour. It will achieve the minimum number of segmentation regions following human vision. Compared with similar algorithms, this way can suppress the meaningless area generated by the reflected light. It will maintain the edge information of the object well. It will improve the robustness and applicability. From the experimental results, it can be seen that compared with the region-growing method and the automatic threshold method; the proposed algorithm has a great improvement in operation efficiency, which increased by 10%. The accuracy and recall rate of the proposed algorithm is more than 0.98. Through the experimental comparison, the advantages of the proposed algorithm in object segmentation can be more intuitively illustrated.

Magnetic resonance image-based brain tumour segmentation methods: A systematic review

Background

Image segmentation is an essential step in the analysis and subsequent characterisation of brain tumours through magnetic resonance imaging. In the literature, segmentation methods are empowered by open-access magnetic resonance imaging datasets, such as the brain tumour segmentation dataset. Moreover, with the increased use of artificial intelligence methods in medical imaging, access to larger data repositories has become vital in method development.

Purpose

To determine what automated brain tumour segmentation techniques can medical imaging specialists and clinicians use to identify tumour components, compared to manual segmentation.

Methods

We conducted a systematic review of 572 brain tumour segmentation studies during 2015-2020. We reviewed segmentation techniques using T1-weighted, T2-weighted, gadolinium-enhanced T1-weighted, fluid-attenuated inversion recovery, diffusion-weighted and perfusion-weighted magnetic resonance imaging sequences. Moreover, we assessed physics or mathematics-based methods, deep learning methods, and software-based or semi-automatic methods, as applied to magnetic resonance imaging techniques. Particularly, we synthesised each method as per the utilised magnetic resonance imaging sequences, study population, technical approach (such as deep learning) and performance score measures (such as Dice score).

Statistical tests

We compared median Dice score in segmenting the whole tumour, tumour core and enhanced tumour.

Results

We found that T1-weighted, gadolinium-enhanced T1-weighted, T2-weighted and fluid-attenuated inversion recovery magnetic resonance imaging are used the most in various segmentation algorithms. However, there is limited use of perfusion-weighted and diffusion-weighted magnetic resonance imaging. Moreover, we found that the U-Net deep learning technology is cited the most, and has high accuracy (Dice score 0.9) for magnetic resonance imaging-based brain tumour segmentation.

Conclusion

U-Net is a promising deep learning technology for magnetic resonance imaging-based brain tumour segmentation. The community should be encouraged to contribute open-access datasets so training, testing and validation of deep learning algorithms can be improved, particularly for diffusion- and perfusion-weighted magnetic resonance imaging, where there are limited datasets available.

Brain Tumor Segmentation From Multi-Modal MR Images via Ensembling UNets

Glioma is a type of severe brain tumor, and its accurate segmentation is useful in surgery planning and progression evaluation. Based on different biological properties, the glioma can be divided into three partially-overlapping regions of interest, including whole tumor (WT), tumor core (TC), and enhancing tumor (ET). Recently, UNet has identified its effectiveness in automatically segmenting brain tumor from multi-modal magnetic resonance (MR) images. In this work, instead of network architecture, we focus on making use of prior knowledge (brain parcellation), training and testing strategy (joint 3D+2D), ensemble and post-processing to improve the brain tumor segmentation performance. We explore the accuracy of three UNets with different inputs, and then ensemble the corresponding three outputs, followed by post-processing to achieve the final segmentation. Similar to most existing works, the first UNet uses 3D patches of multi-modal MR images as the input. The second UNet uses brain parcellation as an additional input. And the third UNet is inputted by 2D slices of multi-modal MR images, brain parcellation, and probability maps of WT, TC, and ET obtained from the second UNet. Then, we sequentially unify the WT segmentation from the third UNet and the fused TC and ET segmentation from the first and the second UNets as the complete tumor segmentation. Finally, we adopt a post-processing strategy by labeling small ET as non-enhancing tumor to correct some false-positive ET segmentation. On one publicly-available challenge validation dataset (BraTS2018), the proposed segmentation pipeline yielded average Dice scores of 91.03/86.44/80.58% and average 95% Hausdorff distances of 3.76/6.73/2.51 mm for WT/TC/ET, exhibiting superior segmentation performance over other state-of-the-art methods. We then evaluated the proposed method on the BraTS2020 training data through five-fold cross validation, with similar performance having also been observed. The proposed method was finally evaluated on 10 in-house data, the effectiveness of which has been established qualitatively by professional radiologists.

Role of the noninvasive imaging techniques in monitoring and understanding the evolution of brain edema

Human brain injury elicits accumulation of water within the brain due to a variety of pathophysiological processes. As our understanding of edema emerged two temporally (and cellular) distinct processes were identified, cytotoxic and vasogenic edema. The emergence of both types of edema is reflected by the temporal evolution and is influenced by the underlying pathology (type and extent). However, this two-edema compartment model does not adequately describe the transition between cytotoxic and vasogenic edema. Hence, a third category has been proposed, termed ionic edema, that is observed in the transition between cytotoxic and vasogenic edema. Magnetic resonance neuroimaging of edema today primarily utilizes T2-weighted (T2WI) and diffusion-weighted imaging (DWI). Clinical diagnostics and translational science studies have clearly demonstrated the temporal ability of both T2WI and DWI to monitor edema content and evolution. DWI measures water mobility within the brain reflecting cytotoxic edema. T2WI at later time points when vasogenic edema develops visualizes increased water content in the brain. Clinically relevant imaging modalities, including ultrasound and positron emission tomography, are not typically used to assess edema. In sum, edema imaging is an important cornerstone of clinical diagnostics and translational studies and can guide effective therapeutics manage edema and improve patient outcomes.

Automatic brain lesion segmentation on standard magnetic resonance images: a scoping review

OBJECTIVES

Medical image analysis practices face challenges that can potentially be addressed with algorithm-based segmentation tools. In this study, we map the field of automatic MR brain lesion segmentation to understand the clinical applicability of prevalent methods and study designs, as well as challenges and limitations in the field.

DESIGN

Scoping review.

SETTING

Three databases (PubMed, IEEE Xplore and Scopus) were searched with tailored queries. Studies were included based on predefined criteria. Emerging themes during consecutive title, abstract, methods and whole-text screening were identified. The full-text analysis focused on materials, preprocessing, performance evaluation and comparison.

RESULTS

Out of 2990 unique articles identified through the search, 441 articles met the eligibility criteria, with an estimated growth rate of 10% per year. We present a general overview and trends in the field with regard to publication sources, segmentation principles used and types of lesions. Algorithms are predominantly evaluated by measuring the agreement of segmentation results with a trusted reference. Few articles describe measures of clinical validity.

CONCLUSIONS

The observed reporting practices leave room for improvement with a view to studying replication, method comparison and clinical applicability. To promote this improvement, we propose a list of recommendations for future studies in the field.

A Fully Automatic Procedure for Brain Tumor Segmentation from Multi-Spectral MRI Records Using Ensemble Learning and Atlas-Based Data Enhancement

The accurate and reliable segmentation of gliomas from magnetic resonance image (MRI) data has an important role in diagnosis, intervention planning, and monitoring the tumor’s evolution during and after therapy. Segmentation has serious anatomical obstacles like the great variety of the tumor’s location, size, shape, and appearance and the modified position of normal tissues. Other phenomena like intensity inhomogeneity and the lack of standard intensity scale in MRI data represent further difficulties. This paper proposes a fully automatic brain tumor segmentation procedure that attempts to handle all the above problems. Having its foundations on the MRI data provided by the MICCAI Brain Tumor Segmentation (BraTS) Challenges, the procedure consists of three main phases. The first pre-processing phase prepares the MRI data to be suitable for supervised classification, by attempting to fix missing data, suppressing the intensity inhomogeneity, normalizing the histogram of observed data channels, generating additional morphological, gradient-based, and Gabor-wavelet features, and optionally applying atlas-based data enhancement. The second phase accomplishes the main classification process using ensembles of binary decision trees and provides an initial, intermediary labeling for each pixel of test records. The last phase reevaluates these intermediary labels using a random forest classifier, then deploys a spatial region growing-based structural validation of suspected tumors, thus achieving a high-quality final segmentation result. The accuracy of the procedure is evaluated using the multi-spectral MRI records of the BraTS 2015 and BraTS 2019 training data sets. The procedure achieves high-quality segmentation results, characterized by average Dice similarity scores of up to 86%.

An Intensity Variation Pattern Analysis Based Machine Learning Classifier for MRI Brain Tumor Detection

BACKGROUND

An accurate detection of tumor from the Magnetic Resonance Images (MRIs) is a critical and demanding task in medical image processing, due to the varying shape and structure of brain. So, different segmentation approaches such as manual, semi-automatic, and fully automatic are developed in the traditional works. Among them, the fully automatic segmentation techniques are increasingly used by the medical experts for an efficient disease diagnosis. But, it has the limitations of over segmentation, increased complexity, and time consumption.

OBJECTIVE

In order to solve these problems, this paper aims to develop an efficient segmentation and classification system by incorporating a novel image processing techniques.

METHODS

Here, the Distribution based Adaptive Median Filtering (DMAF) technique is employed for preprocessing the image. Then, skull removal is performed to extract the tumor portion from the filtered image. Further, the Neighborhood Differential Edge Detection (NDED) technique is implemented to cluster the tumor affected pixels, and it is segmented by the use of Intensity Variation Pattern Analysis (IVPA) technique. Finally, the normal and abnormal images are classified by using the Weighted Machine Learning (WML) technique.

RESULTS

During experiments, the results of the existing and proposed segmentation and classification techniques are evaluated based on different performance measures. To prove the superiority of the proposed technique, it is compared with the existing techniques.

CONCLUSION

From the analysis, it is observed that the proposed IVPA-WML techniques provide the better results compared than the existing techniques.

Relative Ectopic Kidney Function Quantification Using DMSA Tomoscintigraphy Modality

The ectopic renal function estimation based on a manual region of interest (ROI) extraction could be considered as time consuming. It could also affect the clinical interpretation and thus deviate the therapeutic attitude. For this purpose, we propose an advanced tool to evaluate such function through the dimercaptosuccinic acid (DMSA) kidney scintigraphy scans. Methods. The proposed study has been performed on one hundred patients (fifty cases with normal kidneys and fifty cases with ectopic kidneys). We present our segmentation problems as several cost functions' optimization, each containing two terms: (i) a distribution matching prior, which evaluates a global similarity between distributions, and (ii) a smoothness prior to avoid the occurrence of small, isolated regions in the solution. Obtained following recent bound-relaxation results, the optima of the cost functions yield each kidney region in near real time. The Dice Metric (DM), the Jaccard Index (JI), and the correlation parameter have been adopted as validation parameters in order to evaluate the segmentation results. The obtained relative function of both kidneys has been then compared with that evaluated in clinical routine (planar projection) and then validated statistically by the Bland-Altman plots and the Interclass Correlation Coefficient (ICC). Results. Compared to the expert's manual kidney segmentation, the obtained results have been judged to be acceptable for clinical use with high Mean Dice Metric (MDM) value and high Jaccard Index (JI). The evaluated relative renal function has been different from those calculated by the projection planar method usually used in clinical routines. Conclusion. The proposed system could efficiently extract the renal region. The relative function estimation could be considered as more accurate. In fact, the background noise correction and the attenuation phenomenon, which could yield an error measure for renal ectopia, have been avoided. Our clinical staff members have validated the results and have suggested using such tool in their clinical routines.

Toward Effective Medical Image Analysis Using Hybrid Approaches—Review, Challenges and Applications

Accurate medical images analysis plays a vital role for several clinical applications. Nevertheless, the immense and complex data volume to be processed make difficult the design of effective algorithms. The first aim of this paper is to examine this area of research and to provide some relevant reference sources related to the context of medical image analysis. Then, an effective hybrid solution to further improve the expected results is proposed here. It allows to consider the benefits of the cooperation of different complementary approaches such as statistical-based, variational-based and atlas-based techniques and to reduce their drawbacks. In particular, a pipeline framework that involves different steps such as a preprocessing step, a classification step and a refinement step with variational-based method is developed to identify accurately pathological regions in biomedical images. The preprocessing step has the role to remove noise and improve the quality of the images. Then the classification is based on both symmetry axis detection step and non linear learning with SVM algorithm. Finally, a level set-based model is performed to refine the boundary detection of the region of interest. In this work we will show that an accurate initialization step could enhance final performances. Some obtained results are exposed which are related to the challenging application of brain tumor segmentation.

A Feature Ranking and Selection Algorithm for Brain Tumor Segmentation in Multi-Spectral Magnetic Resonance Image Data

Accuracy is the most important quality marker in medical image segmentation. However, when the task is to handle large volumes of data, the relevance of processing speed rises. In machine learning solutions the optimization of the feature set can significantly reduce the computational load. This paper presents a method for feature selection and applies it in the context of a brain tumor detection and segmentation problem in multi-spectral magnetic resonance image data. Starting from an initial set of 104 features involved in an existing ensemble learning solution that employs binary decision trees, a reduced set of features is obtained using a iterative algorithm based on a composite criterion. In each iteration, features are ranked according to the frequency of usage and the correctness of the decisions to which they contribute. Lowest ranked features are iteratively eliminated as long as the segmentation accuracy is not damaged. The final reduced set of 13 features provide the same accuracy in the whole tumor segmentation process as the initial one, but three times faster.

Supervised learning based multimodal MRI brain tumour segmentation using texture features from supervoxels

BACKGROUND

Accurate segmentation of brain tumour in magnetic resonance images (MRI) is a difficult task due to various tumour types. Using information and features from multimodal MRI including structural MRI and isotropic (p) and anisotropic (q) components derived from the diffusion tensor imaging (DTI) may result in a more accurate analysis of brain images.

METHODS

We propose a novel 3D supervoxel based learning method for segmentation of tumour in multimodal MRI brain images (conventional MRI and DTI). Supervoxels are generated using the information across the multimodal MRI dataset. For each supervoxel, a variety of features including histograms of texton descriptor, calculated using a set of Gabor filters with different sizes and orientations, and first order intensity statistical features are extracted. Those features are fed into a random forests (RF) classifier to classify each supervoxel into tumour core, oedema or healthy brain tissue.

RESULTS

The method is evaluated on two datasets: 1) Our clinical dataset: 11 multimodal images of patients and 2) BRATS 2013 clinical dataset: 30 multimodal images. For our clinical dataset, the average detection sensitivity of tumour (including tumour core and oedema) using multimodal MRI is 86% with balanced error rate (BER) 7%; while the Dice score for automatic tumour segmentation against ground truth is 0.84. The corresponding results of the BRATS 2013 dataset are 96%, 2% and 0.89, respectively.

CONCLUSION

The method demonstrates promising results in the segmentation of brain tumour. Adding features from multimodal MRI images can largely increase the segmentation accuracy. The method provides a close match to expert delineation across all tumour grades, leading to a faster and more reproducible method of brain tumour detection and delineation to aid patient management.

A novel fully automatic multilevel thresholding technique based on optimized intuitionistic fuzzy sets and tsallis entropy for MR brain tumor image segmentation

In the present paper, a hybrid multilevel thresholding technique that combines intuitionistic fuzzy sets and tsallis entropy has been proposed for the automatic delineation of the tumor from magnetic resonance images having vague boundaries and poor contrast. This novel technique takes into account both the image histogram and the uncertainty information for the computation of multiple thresholds. The benefit of the methodology is that it provides fast and improved segmentation for the complex tumorous images with imprecise gray levels. To further boost the computational speed, the mutation based particle swarm optimization is used that selects the most optimal threshold combination. The accuracy of the proposed segmentation approach has been validated on simulated, real low-grade glioma tumor volumes taken from MICCAI brain tumor segmentation (BRATS) challenge 2012 dataset and the clinical tumor images, so as to corroborate its generality and novelty. The designed technique achieves an average Dice overlap equal to 0.82010, 0.78610 and 0.94170 for three datasets. Further, a comparative analysis has also been made between the eight existing multilevel thresholding implementations so as to show the superiority of the designed technique. In comparison, the results indicate a mean improvement in Dice by an amount equal to 4.00% (p < 0.005), 9.60% (p < 0.005) and 3.58% (p < 0.005), respectively in contrast to the fuzzy tsallis approach.

Brain tumor segmentation using holistically nested neural networks in MRI images

PURPOSE

Gliomas are rapidly progressive, neurologically devastating, largely fatal brain tumors. Magnetic resonance imaging (MRI) is a widely used technique employed in the diagnosis and management of gliomas in clinical practice. MRI is also the standard imaging modality used to delineate the brain tumor target as part of treatment planning for the administration of radiation therapy. Despite more than 20 yr of research and development, computational brain tumor segmentation in MRI images remains a challenging task. We are presenting a novel method of automatic image segmentation based on holistically nested neural networks that could be employed for brain tumor segmentation of MRI images.

METHODS

Two preprocessing techniques were applied to MRI images. The N4ITK method was employed for correction of bias field distortion. A novel landmark-based intensity normalization method was developed so that tissue types have a similar intensity scale in images of different subjects for the same MRI protocol. The holistically nested neural networks (HNN), which extend from the convolutional neural networks (CNN) with a deep supervision through an additional weighted-fusion output layer, was trained to learn the multiscale and multilevel hierarchical appearance representation of the brain tumor in MRI images and was subsequently applied to produce a prediction map of the brain tumor on test images. Finally, the brain tumor was obtained through an optimum thresholding on the prediction map.

RESULTS

The proposed method was evaluated on both the Multimodal Brain Tumor Image Segmentation (BRATS) Benchmark 2013 training datasets, and clinical data from our institute. A dice similarity coefficient (DSC) and sensitivity of 0.78 and 0.81 were achieved on 20 BRATS 2013 training datasets with high-grade gliomas (HGG), based on a two-fold cross-validation. The HNN model built on the BRATS 2013 training data was applied to ten clinical datasets with HGG from a locally developed database. DSC and sensitivity of 0.83 and 0.85 were achieved. A quantitative comparison indicated that the proposed method outperforms the popular fully convolutional network (FCN) method. In terms of efficiency, the proposed method took around 10 h for training with 50,000 iterations, and approximately 30 s for testing of a typical MRI image in the BRATS 2013 dataset with a size of 160 × 216 × 176, using a DELL PRECISION workstation T7400, with an NVIDIA Tesla K20c GPU.

CONCLUSIONS

An effective brain tumor segmentation method for MRI images based on a HNN has been developed. The high level of accuracy and efficiency make this method practical in brain tumor segmentation. It may play a crucial role in both brain tumor diagnostic analysis and in the treatment planning of radiation therapy.

Automatic brain tumour detection and neovasculature assessment with multiseries MRI analysis

In this paper a novel multi-stage automatic method for brain tumour detection and neovasculature assessment is presented. First, the brain symmetry is exploited to register the magnetic resonance (MR) series analysed. Then, the intracranial structures are found and the region of interest (ROI) is constrained within them to tumour and peritumoural areas using the Fluid Light Attenuation Inversion Recovery (FLAIR) series. Next, the contrast-enhanced lesions are detected on the basis of T1-weighted (T1W) differential images before and after contrast medium administration. Finally, their vascularisation is assessed based on the Regional Cerebral Blood Volume (RCBV) perfusion maps. The relative RCBV (rRCBV) map is calculated in relation to a healthy white matter, also found automatically, and visualised on the analysed series. Three main types of brain tumours, i.e. HG gliomas, metastases and meningiomas have been subjected to the analysis. The results of contrast enhanced lesions detection have been compared with manual delineations performed independently by two experts, yielding 64.84% sensitivity, 99.89% specificity and 71.83% Dice Similarity Coefficient (DSC) for twenty analysed studies of subjects with brain tumours diagnosed.