Semi-automated brain tumor segmentation on multi-parametric MRI using regularized non-negative matrix factorization

Unsupervised learning for medical data: A review of probabilistic factorization methods

We review popular unsupervised learning methods for the analysis of high-dimensional data encountered in, for example, genomics, medical imaging, cohort studies, and biobanks. We show that four commonly used methods, principal component analysis, K-means clustering, nonnegative matrix factorization, and latent Dirichlet allocation, can be written as probabilistic models underpinned by a low-rank matrix factorization. In addition to highlighting their similarities, this formulation clarifies the various assumptions and restrictions of each approach, which eases identifying the appropriate method for specific applications for applied medical researchers. We also touch upon the most important aspects of inference and model selection for the application of these methods to health data.

Is pre-radiotherapy metabolic heterogeneity of glioblastoma predictive of progression-free survival?

BACKGROUND AND PURPOSE

All glioblastoma subtypes share the hallmark of aggressive invasion, meaning that it is crucial to identify their different components if we are to ensure effective treatment and improve survival. Proton MR spectroscopic imaging (MRSI) is a noninvasive technique that yields metabolic information and is able to identify pathological tissue with high accuracy. The aim of the present study was to identify clusters of metabolic heterogeneity, using a large MRSI dataset, and determine which of these clusters are predictive of progression-free survival (PFS).

MATERIALS AND METHODS

MRSI data of 180 patients acquired in a pre-radiotherapy examination were included in the prospective SPECTRO-GLIO trial. Eight features were extracted for each spectrum: Cho/NAA, NAA/Cr, Cho/Cr, Lac/NAA, and the ratio of each metabolite to the sum of all the metabolites. Clustering of data was performed using a mini-batch k-means algorithm. The Cox model and logrank test were used for PFS analysis.

RESULTS

Five clusters were identified as sharing similar metabolic information and being predictive of PFS. Two clusters revealed metabolic abnormalities. PFS was lower when Cluster 2 was the dominant cluster in patients' MRSI data. Among the metabolites, lactate (present in this cluster and in Cluster 5) was the most statistically significant predictor of poor outcome.

CONCLUSION

Results showed that pre-radiotherapy MRSI can be used to reveal tumor heterogeneity. Groups of spectra, which have the same metabolic information, reflect the different tissue components representative of tumor burden proliferation and hypoxia. Clusters with metabolic abnormalities and high lactate are predictive of PFS.

Factorizer: A scalable interpretable approach to context modeling for medical image segmentation

Convolutional Neural Networks (CNNs) with U-shaped architectures have dominated medical image segmentation, which is crucial for various clinical purposes. However, the inherent locality of convolution makes CNNs fail to fully exploit global context, essential for better recognition of some structures, e.g., brain lesions. Transformers have recently proven promising performance on vision tasks, including semantic segmentation, mainly due to their capability of modeling long-range dependencies. Nevertheless, the quadratic complexity of attention makes existing Transformer-based models use self-attention layers only after somehow reducing the image resolution, which limits the ability to capture global contexts present at higher resolutions. Therefore, this work introduces a family of models, dubbed Factorizer, which leverages the power of low-rank matrix factorization for constructing an end-to-end segmentation model. Specifically, we propose a linearly scalable approach to context modeling, formulating Nonnegative Matrix Factorization (NMF) as a differentiable layer integrated into a U-shaped architecture. The shifted window technique is also utilized in combination with NMF to effectively aggregate local information. Factorizers compete favorably with CNNs and Transformers in terms of accuracy, scalability, and interpretability, achieving state-of-the-art results on the BraTS dataset for brain tumor segmentation and ISLES'22 dataset for stroke lesion segmentation. Highly meaningful NMF components give an additional interpretability advantage to Factorizers over CNNs and Transformers. Moreover, our ablation studies reveal a distinctive feature of Factorizers that enables a significant speed-up in inference for a trained Factorizer without any extra steps and without sacrificing much accuracy. The code and models are publicly available at https://github.com/pashtari/factorizer.

Application of non-negative matrix factorization in oncology: one approach for establishing precision medicine

The increase in the expectations of artificial intelligence (AI) technology has led to machine learning technology being actively used in the medical field. Non-negative matrix factorization (NMF) is a machine learning technique used for image analysis, speech recognition, and language processing; recently, it is being applied to medical research. Precision medicine, wherein important information is extracted from large-scale medical data to provide optimal medical care for every individual, is considered important in medical policies globally, and the application of machine learning techniques to this end is being handled in several ways. NMF is also introduced differently because of the characteristics of its algorithms. In this review, the importance of NMF in the field of medicine, with a focus on the field of oncology, is described by explaining the mathematical science of NMF and the characteristics of the algorithm, providing examples of how NMF can be used to establish precision medicine, and presenting the challenges of NMF. Finally, the direction regarding the effective use of NMF in the field of oncology is also discussed.

Magnetic resonance image-based brain tumour segmentation methods: A systematic review

Background

Image segmentation is an essential step in the analysis and subsequent characterisation of brain tumours through magnetic resonance imaging. In the literature, segmentation methods are empowered by open-access magnetic resonance imaging datasets, such as the brain tumour segmentation dataset. Moreover, with the increased use of artificial intelligence methods in medical imaging, access to larger data repositories has become vital in method development.

Purpose

To determine what automated brain tumour segmentation techniques can medical imaging specialists and clinicians use to identify tumour components, compared to manual segmentation.

Methods

We conducted a systematic review of 572 brain tumour segmentation studies during 2015-2020. We reviewed segmentation techniques using T1-weighted, T2-weighted, gadolinium-enhanced T1-weighted, fluid-attenuated inversion recovery, diffusion-weighted and perfusion-weighted magnetic resonance imaging sequences. Moreover, we assessed physics or mathematics-based methods, deep learning methods, and software-based or semi-automatic methods, as applied to magnetic resonance imaging techniques. Particularly, we synthesised each method as per the utilised magnetic resonance imaging sequences, study population, technical approach (such as deep learning) and performance score measures (such as Dice score).

Statistical tests

We compared median Dice score in segmenting the whole tumour, tumour core and enhanced tumour.

Results

We found that T1-weighted, gadolinium-enhanced T1-weighted, T2-weighted and fluid-attenuated inversion recovery magnetic resonance imaging are used the most in various segmentation algorithms. However, there is limited use of perfusion-weighted and diffusion-weighted magnetic resonance imaging. Moreover, we found that the U-Net deep learning technology is cited the most, and has high accuracy (Dice score 0.9) for magnetic resonance imaging-based brain tumour segmentation.

Conclusion

U-Net is a promising deep learning technology for magnetic resonance imaging-based brain tumour segmentation. The community should be encouraged to contribute open-access datasets so training, testing and validation of deep learning algorithms can be improved, particularly for diffusion- and perfusion-weighted magnetic resonance imaging, where there are limited datasets available.

Brain tumor detection and classification using machine learning: a comprehensive survey

Brain tumor occurs owing to uncontrolled and rapid growth of cells. If not treated at an initial phase, it may lead to death. Despite many significant efforts and promising outcomes in this domain, accurate segmentation and classification remain a challenging task. A major challenge for brain tumor detection arises from the variations in tumor location, shape, and size. The objective of this survey is to deliver a comprehensive literature on brain tumor detection through magnetic resonance imaging to help the researchers. This survey covered the anatomy of brain tumors, publicly available datasets, enhancement techniques, segmentation, feature extraction, classification, and deep learning, transfer learning and quantum machine learning for brain tumors analysis. Finally, this survey provides all important literature for the detection of brain tumors with their advantages, limitations, developments, and future trends.

Image Features of Magnetic Resonance Imaging under the Deep Learning Algorithm in the Diagnosis and Nursing of Malignant Tumors

In order to explore the effect of convolutional neural network (CNN) algorithm based on deep learning on magnetic resonance imaging (MRI) images of brain tumor patients and evaluate the practical value of MRI image features based on deep learning algorithm in the clinical diagnosis and nursing of malignant tumors, in this study, a brain tumor MRI image model based on the CNN algorithm was constructed, and 80 patients with brain tumors were selected as the research objects. They were divided into an experimental group (CNN algorithm) and a control group (traditional algorithm). The patients were nursed in the whole process. The macroscopic characteristics and imaging index of the MRI image and anxiety of patients in two groups were compared and analyzed. In addition, the image quality after nursing was checked. The results of the study revealed that the MRI characteristics of brain tumors based on CNN algorithm were clearer and more accurate in the fluid-attenuated inversion recovery (FLAIR), MRI T1, T1c, and T2; in terms of accuracy, sensitivity, and specificity, the mean value was 0.83, 0.84, and 0.83, which had obvious advantages compared with the traditional algorithm (P < 0.05). The patients in the nursing group showed lower depression scores and better MRI images in contrast to the control group (P < 0.05). Therefore, the deep learning algorithm can further accurately analyze the MRI image characteristics of brain tumor patients on the basis of conventional algorithms, showing high sensitivity and specificity, which improved the application value of MRI image characteristics in the diagnosis of malignant tumors. In addition, effective nursing for patients undergoing analysis and diagnosis on brain tumor MRI image characteristics can alleviate the patient's anxiety and ensure that high-quality MRI images were obtained after the examination.

Hahn-PCNN-CNN: an end-to-end multi-modal brain medical image fusion framework useful for clinical diagnosis

Background

In medical diagnosis of brain, the role of multi-modal medical image fusion is becoming more prominent. Among them, there is no lack of filtering layered fusion and newly emerging deep learning algorithms. The former has a fast fusion speed but the fusion image texture is blurred; the latter has a better fusion effect but requires higher machine computing capabilities. Therefore, how to find a balanced algorithm in terms of image quality, speed and computing power is still the focus of all scholars.

Methods

We built an end-to-end Hahn-PCNN-CNN. The network is composed of feature extraction module, feature fusion module and image reconstruction module. We selected 8000 multi-modal brain medical images downloaded from the Harvard Medical School website to train the feature extraction layer and image reconstruction layer to enhance the network’s ability to reconstruct brain medical images. In the feature fusion module, we use the moments of the feature map combined with the pulse-coupled neural network to reduce the information loss caused by convolution in the previous fusion module and save time.

Results

We choose eight sets of registered multi-modal brain medical images in four diease to verify our model. The anatomical structure images are from MRI and the functional metabolism images are SPECT and 18F-FDG. At the same time, we also selected eight representative fusion models as comparative experiments. In terms of objective quality evaluation, we select six evaluation metrics in five categories to evaluate our model.

Conclusions

The fusion image obtained by our model can retain the effective information in source images to the greatest extent. In terms of image fusion evaluation metrics, our model is superior to other comparison algorithms. In terms of time computational efficiency, our model also performs well. In terms of robustness, our model is very stable and can be generalized to multi-modal image fusion of other organs.

Deep semi-supervised learning for brain tumor classification

Background

This paper addresses issues of brain tumor, glioma, classification from four modalities of Magnetic Resonance Image (MRI) scans (i.e., T1 weighted MRI, T1 weighted MRI with contrast-enhanced, T2 weighted MRI and FLAIR). Currently, many available glioma datasets often contain some unlabeled brain scans, and many datasets are moderate in size.

Methods

We propose to exploit deep semi-supervised learning to make full use of the unlabeled data. Deep CNN features were incorporated into a new graph-based semi-supervised learning framework for learning the labels of the unlabeled data, where a new 3D-2D consistent constraint is added to make consistent classifications for the 2D slices from the same 3D brain scan. A deep-learning classifier is then trained to classify different glioma types using both labeled and unlabeled data with estimated labels. To alleviate the overfitting caused by moderate-size datasets, synthetic MRIs generated by Generative Adversarial Networks (GANs) are added in the training of CNNs.

Results

The proposed scheme has been tested on two glioma datasets, TCGA dataset for IDH-mutation prediction (molecular-based glioma subtype classification) and MICCAI dataset for glioma grading. Our results have shown good performance (with test accuracies 86.53% on TCGA dataset and 90.70% on MICCAI dataset).

Conclusions

The proposed scheme is effective for glioma IDH-mutation prediction and glioma grading, and its performance is comparable to the state-of-the-art.

Data Augmentation for Brain-Tumor Segmentation: A Review

Data augmentation is a popular technique which helps improve generalization capabilities of deep neural networks, and can be perceived as implicit regularization. It plays a pivotal role in scenarios in which the amount of high-quality ground-truth data is limited, and acquiring new examples is costly and time-consuming. This is a very common problem in medical image analysis, especially tumor delineation. In this paper, we review the current advances in data-augmentation techniques applied to magnetic resonance images of brain tumors. To better understand the practical aspects of such algorithms, we investigate the papers submitted to the Multimodal Brain Tumor Segmentation Challenge (BraTS 2018 edition), as the BraTS dataset became a standard benchmark for validating existent and emerging brain-tumor detection and segmentation techniques. We verify which data augmentation approaches were exploited and what was their impact on the abilities of underlying supervised learners. Finally, we highlight the most promising research directions to follow in order to synthesize high-quality artificial brain-tumor examples which can boost the generalization abilities of deep models.

Machine learning analyses can differentiate meningioma grade by features on magnetic resonance imaging

OBJECTIVEPrognostication and surgical planning for WHO grade I versus grade II meningioma requires thoughtful decision-making based on radiographic evidence, among other factors. Although conventional statistical models such as logistic regression are useful, machine learning (ML) algorithms are often more predictive, have higher discriminative ability, and can learn from new data. The authors used conventional statistical models and an array of ML algorithms to predict atypical meningioma based on radiologist-interpreted preoperative MRI findings. The goal of this study was to compare the performance of ML algorithms to standard statistical methods when predicting meningioma grade.METHODSThe cohort included patients aged 18-65 years with WHO grade I (n = 94) and II (n = 34) meningioma in whom preoperative MRI was obtained between 1998 and 2010. A board-certified neuroradiologist, blinded to histological grade, interpreted all MR images for tumor volume, degree of peritumoral edema, presence of necrosis, tumor location, presence of a draining vein, and patient sex. The authors trained and validated several binary classifiers: k-nearest neighbors models, support vector machines, naïve Bayes classifiers, and artificial neural networks as well as logistic regression models to predict tumor grade. The area under the curve-receiver operating characteristic curve was used for comparison across and within model classes. All analyses were performed in MATLAB using a MacBook Pro.RESULTSThe authors included 6 preoperative imaging and demographic variables: tumor volume, degree of peritumoral edema, presence of necrosis, tumor location, patient sex, and presence of a draining vein to construct the models. The artificial neural networks outperformed all other ML models across the true-positive versus false-positive (receiver operating characteristic) space (area under curve = 0.8895).CONCLUSIONSML algorithms are powerful computational tools that can predict meningioma grade with great accuracy.

Embedding MRI information into MRSI data source extraction improves brain tumour delineation in animal models

Glioblastoma is the most frequent malignant intra-cranial tumour. Magnetic resonance imaging is the modality of choice in diagnosis, aggressiveness assessment, and follow-up. However, there are examples where it lacks diagnostic accuracy. Magnetic resonance spectroscopy enables the identification of molecules present in the tissue, providing a precise metabolomic signature. Previous research shows that combining imaging and spectroscopy information results in more accurate outcomes and superior diagnostic value. This study proposes a method to combine them, which builds upon a previous methodology whose main objective is to guide the extraction of sources. To this aim, prior knowledge about class-specific information is integrated into the methodology by setting the metric of a latent variable space where Non-negative Matrix Factorisation is performed. The former methodology, which only used spectroscopy and involved combining spectra from different subjects, was adapted to use selected areas of interest that arise from segmenting the T2-weighted image. Results showed that embedding imaging information into the source extraction (the proposed semi-supervised analysis) improved the quality of the tumour delineation, as compared to those obtained without this information (unsupervised analysis). Both approaches were applied to pre-clinical data, involving thirteen brain tumour-bearing mice, and tested against histopathological data. On results of twenty-eight images, the proposed Semi-Supervised Source Extraction (SSSE) method greatly outperformed the unsupervised one, as well as an alternative semi-supervised approach from the literature, with differences being statistically significant. SSSE has proven successful in the delineation of the tumour, while bringing benefits such as 1) not constricting the metabolomic-based prediction to the image-segmented area, 2) ability to deal with signal-to-noise issues, 3) opportunity to answer specific questions by allowing researchers/radiologists define areas of interest that guide the source extraction, 4) creation of an intra-subject model and avoiding contamination from inter-subject overlaps, and 5) extraction of meaningful, good-quality sources that adds interpretability, conferring validation and better understanding of each case.

Analysis of metabolic abnormalities in high-grade glioma using MRSI and convex NMF

Clinical use of MRSI is limited by the level of experience required to properly translate MRSI examinations into relevant clinical information. To solve this, several methods have been proposed to automatically recognize a predefined set of reference metabolic patterns. Given the variety of metabolic patterns seen in glioma patients, the decision on the optimal number of patterns that need to be used to describe the data is not trivial. In this paper, we propose a novel framework to (1) separate healthy from abnormal metabolic patterns and (2) retrieve an optimal number of reference patterns describing the most important types of abnormality. Using 41 MRSI examinations (1.5 T, PRESS, T_E 135 ms) from 22 glioma patients, four different patterns describing different types of abnormality were detected: edema, healthy without Glx, active tumor and necrosis. The identified patterns were then evaluated on 17 MRSI examinations from nine different glioma patients. The results were compared against BraTumIA, an automatic segmentation method trained to identify different tumor compartments on structural MRI data. Finally, the ability to predict future contrast enhancement using the proposed approach was also evaluated.

An Improved Full Convolutional Network Combined with Conditional Random Fields for Brain MR Image Segmentation Algorithm and its 3D Visualization Analysis

Existing brain region segmentation algorithms based on deep convolutional neural networks (CNN) are inefficient for object boundary segmentation. In order to enhance the segmentation accuracy of brain tissue, this paper proposed an object region segmentation algorithm that combines pixel-level information and semantic information. Firstly, we extract semantic information by CNN with the attention module and get the coarse segmentation results through a specific pixel-level classifier. Then, we exploit conditional random fields to model the relationship between the underlying pixels so as to get local features. Finally, the semantic information and the local pixel-level information are respectively used as the unary potential and the binary potential of the Gibbs distribution, and the combination of both can obtain the fine region segmentation algorithm based on the fusion of pixel-level information and the semantic information. A large number of qualitative and quantitative test results show that our proposed algorithm has higher precision than the existing state-of-the-art deep feature models, which can better solve the problem of rough edge segmentation and produce good 3D visualization effect.

Artificial intelligence in cancer imaging: Clinical challenges and applications

Judgement, as one of the core tenets of medicine, relies upon the integration of multilayered data with nuanced decision making. Cancer offers a unique context for medical decisions given not only its variegated forms with evolution of disease but also the need to take into account the individual condition of patients, their ability to receive treatment, and their responses to treatment. Challenges remain in the accurate detection, characterization, and monitoring of cancers despite improved technologies. Radiographic assessment of disease most commonly relies upon visual evaluations, the interpretations of which may be augmented by advanced computational analyses. In particular, artificial intelligence (AI) promises to make great strides in the qualitative interpretation of cancer imaging by expert clinicians, including volumetric delineation of tumors over time, extrapolation of the tumor genotype and biological course from its radiographic phenotype, prediction of clinical outcome, and assessment of the impact of disease and treatment on adjacent organs. AI may automate processes in the initial interpretation of images and shift the clinical workflow of radiographic detection, management decisions on whether or not to administer an intervention, and subsequent observation to a yet to be envisioned paradigm. Here, the authors review the current state of AI as applied to medical imaging of cancer and describe advances in 4 tumor types (lung, brain, breast, and prostate) to illustrate how common clinical problems are being addressed. Although most studies evaluating AI applications in oncology to date have not been vigorously validated for reproducibility and generalizability, the results do highlight increasingly concerted efforts in pushing AI technology to clinical use and to impact future directions in cancer care.

Rank-Two NMF Clustering for Glioblastoma Characterization

This study investigates a novel classification method for 3D multimodal MRI glioblastomas tumor characterization. We formulate our segmentation problem as a linear mixture model (LMM). Thus, we provide a nonnegative matrix M from every MRI slice in every segmentation process' step. This matrix will be used as an input for the first segmentation process to extract the edema region from T2 and FLAIR modalities. After that, in the rest of segmentation processes, we extract the edema region from T1c modality, generate the matrix M, and segment the necrosis, the enhanced tumor, and the nonenhanced tumor regions. In the segmentation process, we apply a rank-two NMF clustering. We have executed our tumor characterization method on BraTS 2015 challenge dataset. Quantitative and qualitative evaluations over the publicly training and testing dataset from the MICCAI 2015 multimodal brain segmentation challenge (BraTS 2015) attested that the proposed algorithm could yield a competitive performance for brain glioblastomas characterization (necrosis, tumor core, and edema) among several competing methods.

Fully automated detection and segmentation of meningiomas using deep learning on routine multiparametric MRI

Objectives

Magnetic resonance imaging (MRI) is the method of choice for imaging meningiomas. Volumetric assessment of meningiomas is highly relevant for therapy planning and monitoring. We used a multiparametric deep-learning model (DLM) on routine MRI data including images from diverse referring institutions to investigate DLM performance in automated detection and segmentation of meningiomas in comparison to manual segmentations.

Methods

We included 56 of 136 consecutive preoperative MRI datasets [T1/T2-weighted, T1-weighted contrast-enhanced (T1CE), FLAIR] of meningiomas that were treated surgically at the University Hospital Cologne and graded histologically as tumour grade I (n = 38) or grade II (n = 18). The DLM was trained on an independent dataset of 249 glioma cases and segmented different tumour classes as defined in the brain tumour image segmentation benchmark (BRATS benchmark). The DLM was based on the DeepMedic architecture. Results were compared to manual segmentations by two radiologists in a consensus reading in FLAIR and T1CE.

Results

The DLM detected meningiomas in 55 of 56 cases. Further, automated segmentations correlated strongly with manual segmentations: average Dice coefficients were 0.81 ± 0.10 (range, 0.46-0.93) for the total tumour volume (union of tumour volume in FLAIR and T1CE) and 0.78 ± 0.19 (range, 0.27-0.95) for contrast-enhancing tumour volume in T1CE.

Conclusions

The DLM yielded accurate automated detection and segmentation of meningioma tissue despite diverse scanner data and thereby may improve and facilitate therapy planning as well as monitoring of this highly frequent tumour entity.

Key Points

• Deep learning allows for accurate meningioma detection and segmentation

• Deep learning helps clinicians to assess patients with meningiomas

• Meningioma monitoring and treatment planning can be improved

Electronic supplementary material

The online version of this article (10.1007/s00330-018-5595-8) contains supplementary material, which is available to authorized users.

Automatic brain tissue segmentation based on graph filter

Background

Accurate segmentation of brain tissues from magnetic resonance imaging (MRI) is of significant importance in clinical applications and neuroscience research. Accurate segmentation is challenging due to the tissue heterogeneity, which is caused by noise, bias filed and partial volume effects.

Methods

To overcome this limitation, this paper presents a novel algorithm for brain tissue segmentation based on supervoxel and graph filter. Firstly, an effective supervoxel method is employed to generate effective supervoxels for the 3D MRI image. Secondly, the supervoxels are classified into different types of tissues based on filtering of graph signals.

Results

The performance is evaluated on the BrainWeb 18 dataset and the Internet Brain Segmentation Repository (IBSR) 18 dataset. The proposed method achieves mean dice similarity coefficient (DSC) of 0.94, 0.92 and 0.90 for the segmentation of white matter (WM), grey matter (GM) and cerebrospinal fluid (CSF) for BrainWeb 18 dataset, and mean DSC of 0.85, 0.87 and 0.57 for the segmentation of WM, GM and CSF for IBSR18 dataset.

Conclusions

The proposed approach can well discriminate different types of brain tissues from the brain MRI image, which has high potential to be applied for clinical applications.