Machine Learning Models for Multiparametric Glioma Grading With Quantitative Result Interpretations

Artificial intelligence in histopathological image analysis of central nervous system tumours: A systematic review

The convergence of digital pathology and artificial intelligence could assist histopathology image analysis by providing tools for rapid, automated morphological analysis. This systematic review explores the use of artificial intelligence for histopathological image analysis of digitised central nervous system (CNS) tumour slides. Comprehensive searches were conducted across EMBASE, Medline and the Cochrane Library up to June 2023 using relevant keywords. Sixty-eight suitable studies were identified and qualitatively analysed. The risk of bias was evaluated using the Prediction model Risk of Bias Assessment Tool (PROBAST) criteria. All the studies were retrospective and preclinical. Gliomas were the most frequently analysed tumour type. The majority of studies used convolutional neural networks or support vector machines, and the most common goal of the model was for tumour classification and/or grading from haematoxylin and eosin-stained slides. The majority of studies were conducted when legacy World Health Organisation (WHO) classifications were in place, which at the time relied predominantly on histological (morphological) features but have since been superseded by molecular advances. Overall, there was a high risk of bias in all studies analysed. Persistent issues included inadequate transparency in reporting the number of patients and/or images within the model development and testing cohorts, absence of external validation, and insufficient recognition of batch effects in multi-institutional datasets. Based on these findings, we outline practical recommendations for future work including a framework for clinical implementation, in particular, better informing the artificial intelligence community of the needs of the neuropathologist.

Neuropathologist-level integrated classification of adult-type diffuse gliomas using deep learning from whole-slide pathological images

Current diagnosis of glioma types requires combining both histological features and molecular characteristics, which is an expensive and time-consuming procedure. Determining the tumor types directly from whole-slide images (WSIs) is of great value for glioma diagnosis. This study presents an integrated diagnosis model for automatic classification of diffuse gliomas from annotation-free standard WSIs. Our model is developed on a training cohort (n = 1362) and a validation cohort (n = 340), and tested on an internal testing cohort (n = 289) and two external cohorts (n = 305 and 328, respectively). The model can learn imaging features containing both pathological morphology and underlying biological clues to achieve the integrated diagnosis. Our model achieves high performance with area under receiver operator curve all above 0.90 in classifying major tumor types, in identifying tumor grades within type, and especially in distinguishing tumor genotypes with shared histological features. This integrated diagnosis model has the potential to be used in clinical scenarios for automated and unbiased classification of adult-type diffuse gliomas.

Texture feature analysis of MRI-ADC images to differentiate glioma grades using machine learning techniques

Apparent diffusion coefficient (ADC) of magnetic resonance imaging (MRI) is an indispensable imaging technique in clinical neuroimaging that quantitatively assesses the diffusivity of water molecules within tissues using diffusion-weighted imaging (DWI). This study focuses on developing a robust machine learning (ML) model to predict the aggressiveness of gliomas according to World Health Organization (WHO) grading by analyzing patients' demographics, higher-order moments, and grey level co-occurrence matrix (GLCM) texture features of ADC. A population of 722 labeled MRI-ADC brain image slices from 88 human subjects was selected, where gliomas are labeled as glioblastoma multiforme (WHO-IV), high-grade glioma (WHO-III), and low-grade glioma (WHO I-II). Images were acquired using 3T-MR systems and a region of interest (ROI) was delineated manually over tumor areas. Skewness, kurtosis, and statistical texture features of GLCM (mean, variance, energy, entropy, contrast, homogeneity, correlation, prominence, and shade) were calculated using ADC values within ROI. The ANOVA f-test was utilized to select the best features to train an ML model. The data set was split into training (70%) and testing (30%) sets. The train set was fed into several ML algorithms and selected most promising ML algorithm using K-fold cross-validation. The hyper-parameters of the selected algorithm were optimized using random grid search technique. Finally, the performance of the developed model was assessed by calculating accuracy, precision, recall, and F1 values reported for the test set. According to the ANOVA f-test, three attributes; patient gender (1.48), GLCM energy (9.48), and correlation (13.86) that performed minimum scores were excluded from the dataset. Among the tested algorithms, the random forest classifier(0.8772 ± 0.0237) performed the highest mean-cross-validation score and selected to build the ML model which was able to predict tumor categories with an accuracy of 88.14% over the test set. The study concludes that the developed ML model using the above features except for patient gender, GLCM energy, and correlation, has high prediction accuracy in glioma grading. Therefore, the outcomes of this study enable to development of advanced tumor classification applications that assist in the decision-making process in a real-time clinical environment.

Prediction of metastatic pheochromocytoma and paraganglioma: a machine learning modelling study using data from a cross-sectional cohort

BACKGROUND

Pheochromocytomas and paragangliomas have up to a 20% rate of metastatic disease that cannot be reliably predicted. This study prospectively assessed whether the dopamine metabolite, methoxytyramine, might predict metastatic disease, whether predictions might be improved using machine learning models that incorporate other features, and how machine learning-based predictions compare with predictions made by specialists in the field.

METHODS

In this machine learning modelling study, we used cross-sectional cohort data from the PMT trial, based in Germany, Poland, and the Netherlands, to prospectively examine the utility of methoxytyramine to predict metastatic disease in 267 patients with pheochromocytoma or paraganglioma and positive biochemical test results at initial screening. Another retrospective dataset of 493 patients with these tumors enrolled under clinical protocols at National Institutes of Health (00-CH-0093) and the Netherlands (PRESCRIPT trial) was used to train and validate machine learning models according to selections of additional features. The best performing machine learning models were then externally validated using data for all patients in the PMT trial. For comparison, 12 specialists provided predictions of metastatic disease using data from the training and external validation datasets.

FINDINGS

Prospective predictions indicated that plasma methoxytyramine could identify metastatic disease at sensitivities of 52% and specificities of 85%. The best performing machine learning model was based on an ensemble tree classifier algorithm that used nine features: plasma methoxytyramine, metanephrine, normetanephrine, age, sex, previous history of pheochromocytoma or paraganglioma, location and size of primary tumours, and presence of multifocal disease. This model had an area under the receiver operating characteristic curve of 0·942 (95% CI 0·894-0·969) that was larger (p<0·0001) than that of the best performing specialist before (0·815, 0·778-0·853) and after (0·812, 0·781-0·854) provision of SDHB variant data. Sensitivity for prediction of metastatic disease in the external validation cohort reached 83% at a specificity of 92%.

INTERPRETATION

Although methoxytyramine has some utility for prediction of metastatic pheochromocytomas and paragangliomas, sensitivity is limited. Predictive value is considerably enhanced with machine learning models that incorporate our nine recommended features. Our final model provides a preoperative approach to predict metastases in patients with pheochromocytomas and paragangliomas, and thereby guide individualised patient management and follow-up.

FUNDING

Deutsche Forschungsgemeinschaft.

Artificial Intelligence-Assisted Classification of Gliomas Using Whole Slide Images

CONTEXT.—

Glioma is the most common primary brain tumor in adults. The diagnosis and grading of different pathological subtypes of glioma is essential in treatment planning and prognosis.

OBJECTIVE.—

To propose a deep learning-based approach for the automated classification of glioma histopathology images. Two classification methods, the ensemble method based on 2 binary classifiers and the multiclass method using a single multiclass classifier, were implemented to classify glioma images into astrocytoma, oligodendroglioma, and glioblastoma, according to the 5th edition of the World Health Organization classification of central nervous system tumors, published in 2021.

DESIGN.—

We tested 2 different deep neural network architectures (VGG19 and ResNet50) and extensively validated the proposed approach based on The Cancer Genome Atlas data set (n = 700). We also studied the effects of stain normalization and data augmentation on the glioma classification task.

RESULTS.—

With the binary classifiers, our model could distinguish astrocytoma and oligodendroglioma (combined) from glioblastoma with an accuracy of 0.917 (area under the curve [AUC] = 0.976) and astrocytoma from oligodendroglioma (accuracy = 0.821, AUC = 0.865). The multiclass method (accuracy = 0.861, AUC = 0.961) outperformed the ensemble method (accuracy = 0.847, AUC = 0.933) with the best performance displayed by the ResNet50 architecture.

CONCLUSIONS.—

With the high performance of our model (>80%), the proposed method can assist pathologists and physicians to support examination and differential diagnosis of glioma histopathology images, with the aim to expedite personalized medical care.

Multiple therapeutic approaches of glioblastoma multiforme: From terminal to therapy

Glioblastoma multiforme (GBM) is an aggressive brain cancer showing poor prognosis. Currently, treatment methods of GBM are limited with adverse outcomes and low survival rate. Thus, advancements in the treatment of GBM are of utmost importance, which can be achieved in recent decades. However, despite aggressive initial treatment, most patients develop recurrent diseases, and the overall survival rate of patients is impossible to achieve. Currently, researchers across the globe target signaling events along with tumor microenvironment (TME) through different drug molecules to inhibit the progression of GBM, but clinically they failed to demonstrate much success. Herein, we discuss the therapeutic targets and signaling cascades along with the role of the organoids model in GBM research. Moreover, we systematically review the traditional and emerging therapeutic strategies in GBM. In addition, we discuss the implications of nanotechnologies, AI, and combinatorial approach to enhance GBM therapeutics.

Artificial intelligence techniques for neuropathological diagnostics and research

Artificial intelligence (AI) research began in theoretical neurophysiology, and the resulting classical paper on the McCulloch-Pitts mathematical neuron was written in a psychiatry department almost 80 years ago. However, the application of AI in digital neuropathology is still in its infancy. Rapid progress is now being made, which prompted this article. Human brain diseases represent distinct system states that fall outside the normal spectrum. Many differ not only in functional but also in structural terms, and the morphology of abnormal nervous tissue forms the traditional basis of neuropathological disease classifications. However, only a few countries have the medical specialty of neuropathology, and, given the sheer number of newly developed histological tools that can be applied to the study of brain diseases, a tremendous shortage of qualified hands and eyes at the microscope is obvious. Similarly, in neuroanatomy, human observers no longer have the capacity to process the vast amounts of connectomics data. Therefore, it is reasonable to assume that advances in AI technology and, especially, whole-slide image (WSI) analysis will greatly aid neuropathological practice. In this paper, we discuss machine learning (ML) techniques that are important for understanding WSI analysis, such as traditional ML and deep learning, introduce a recently developed neuropathological AI termed PathoFusion, and present thoughts on some of the challenges that must be overcome before the full potential of AI in digital neuropathology can be realized.

A deep learning framework integrating MRI image preprocessing methods for brain tumor segmentation and classification

Glioma grading is critical in treatment planning and prognosis. This study aims to address this issue through MRI-based classification to develop an accurate model for glioma diagnosis. Here, we employed a deep learning pipeline with three essential steps: (1) MRI images were segmented using preprocessing approaches and UNet architecture, (2) brain tumor regions were extracted using segmentation, then (3) high-grade gliomas and low-grade gliomas were classified using the VGG and GoogleNet implementations. Among the additional preprocessing techniques used in conjunction with the segmentation task, the combination of data augmentation and Window Setting Optimization was found to be the most effective tool, resulting in the Dice coefficient of 0.82, 0.91, and 0.72 for enhancing tumor, whole tumor, and tumor core, respectively. While most of the proposed models achieve comparable accuracies of about 93 % on the testing dataset, the pipeline of VGG combined with UNet segmentation obtains the highest accuracy of 97.44 %. In conclusion, the presented architecture illustrates a realistic model for detecting gliomas; moreover, it emphasizes the significance of data augmentation and segmentation in improving model performance.

Artificial intelligence-based preoperative prediction system for diagnosis and prognosis in epithelial ovarian cancer: A multicenter study

Background

Ovarian cancer (OC) is the most lethal gynecological malignancy, with limited early screening methods and poor prognosis. Artificial intelligence technology has made a great breakthrough in cancer diagnosis.

Purpose

We aim to develop a specific interpretable machine learning (ML) prediction model for the diagnosis and prognosis of epithelial ovarian cancer (EOC) based on a variety of biomarkers.

Methods

A total of 521 patients with EOC and 144 patients with benign gynecological diseases were enrolled including derivation datasets and an external validation cohort. The predicted information was acquired by 9 supervised ML methods, through 34 parameters. Behind predicted reasons for the best ML were improved by using the SHapley Additive exPlanations (SHAP) algorithm. In addition, the prognosis of EOC was analyzed by unsupervised clustering and Kaplan–Meier (KM) survival analysis.

Results

ML technology was superior to conventional logistic regression in predicting EOC diagnosis and XGBoost performed best in the external validation datasets. The AUC values of distinguishing EOC and benign disease patients, determining pathological type, grade and clinical stage were 0.958 (0.926-0.989), 0.792 (0.701-0.8834), 0.819 (0.687-0.950) and 0.68 (0.573-0.788) respectively. For negative CA-125 EOC patients, the AUC performance of XGBoost model was 0.835(0.763-0.907). We used unsupervised cluster analysis to identify EOC subgroups with significantly poor overall survival (p-value <0.0001) and recurrence-free survival (p-value <0.0001).

Conclusions

Based on the preoperative characteristics, we proved that ML algorithm can provide an acceptable diagnosis and prognosis prediction model for EOC patients. Meanwhile, SHAP analysis can improve the interpretability of ML models and contribute to precision medicine.

Machine learning as a clinical decision support tool for patients with acromegaly

OBJECTIVE

To develop machine learning (ML) models that predict postoperative remission, remission at last visit, and resistance to somatostatin receptor ligands (SRL) in patients with acromegaly and to determine the clinical features associated with the prognosis.

METHODS

We studied outcomes using the area under the receiver operating characteristics (AUROC) values, which were reported as the performance metric. To determine the importance of each feature and easy interpretation, Shapley Additive explanations (SHAP) values, which help explain the outputs of ML models, are used.

RESULTS

One-hundred fifty-two patients with acromegaly were included in the final analysis. The mean AUROC values resulting from 100 independent replications were 0.728 for postoperative 3 months remission status classification, 0.879 for remission at last visit classification, and 0.753 for SRL resistance status classification. Extreme gradient boosting model demonstrated that preoperative growth hormone (GH) level, age at operation, and preoperative tumor size were the most important predictors for early remission; resistance to SRL and preoperative tumor size represented the most important predictors of remission at last visit, and postoperative 3-month insulin-like growth factor 1 (IGF1) and GH levels (random and nadir) together with the sparsely granulated somatotroph adenoma subtype served as the most important predictors of SRL resistance.

CONCLUSIONS

ML models may serve as valuable tools in the prediction of remission and SRL resistance.

A weakly supervised deep learning-based method for glioma subtype classification using WSI and mpMRIs

Accurate glioma subtype classification is critical for the treatment management of patients with brain tumors. Developing an automatically computer-aided algorithm for glioma subtype classification is challenging due to many factors. One of the difficulties is the label constraint. Specifically, each case is simply labeled the glioma subtype without precise annotations of lesion regions information. In this paper, we propose a novel hybrid fully convolutional neural network (CNN)-based method for glioma subtype classification using both whole slide imaging (WSI) and multiparametric magnetic resonance imagings (mpMRIs). It is comprised of two methods: a WSI-based method and a mpMRIs-based method. For the WSI-based method, we categorize the glioma subtype using a 2D CNN on WSIs. To overcome the label constraint issue, we extract the truly representative patches for the glioma subtype classification in a weakly supervised fashion. For the mpMRIs-based method, we develop a 3D CNN-based method by analyzing the mpMRIs. The mpMRIs-based method consists of brain tumor segmentation and classification. Finally, to enhance the robustness of the predictions, we fuse the WSI-based and mpMRIs-based results guided by a confidence index. The experimental results on the validation dataset in the competition of CPM-RadPath 2020 show the comprehensive judgments from both two modalities can achieve better performance than the ones by solely using WSI or mpMRIs. Furthermore, our result using the proposed method ranks the third place in the CPM-RadPath 2020 in the testing phase. The proposed method demonstrates a competitive performance, which is creditable to the success of weakly supervised approach and the strategy of label agreement from multi-modality data.

Combining Radiology and Pathology for Automatic Glioma Classification

Subtype classification is critical in the treatment of gliomas because different subtypes lead to different treatment options and postoperative care. Although many radiological- or histological-based glioma classification algorithms have been developed, most of them focus on single-modality data. In this paper, we propose an innovative two-stage model to classify gliomas into three subtypes (i.e., glioblastoma, oligodendroglioma, and astrocytoma) based on radiology and histology data. In the first stage, our model classifies each image as having glioblastoma or not. Based on the obtained non-glioblastoma images, the second stage aims to accurately distinguish astrocytoma and oligodendroglioma. The radiological images and histological images pass through the two-stage design with 3D and 2D models, respectively. Then, an ensemble classification network is designed to automatically integrate the features of the two modalities. We have verified our method by participating in the MICCAI 2020 CPM-RadPath Challenge and won 1st place. Our proposed model achieves high performance on the validation set with a balanced accuracy of 0.889, Cohen’s Kappa of 0.903, and an F1-score of 0.943. Our model could advance multimodal-based glioma research and provide assistance to pathologists and neurologists in diagnosing glioma subtypes. The code has been publicly available online at https://github.com/Xiyue-Wang/1st-in-MICCAI2020-CPM.

QuPath Digital Immunohistochemical Analysis of Placental Tissue

Background:

QuPath is an open-source digital image analyzer notable for its user-friendly design, cross-platform compatibility, and customizable functionality. Since it was first released in 2016, at least 624 publications have reported its use, and it has been applied in a wide spectrum of settings. However, there are currently limited reports of its use in placental tissue. Here, we present the use of QuPath to quantify staining of G-protein coupled receptor 18 (GPR18), the receptor for the pro-resolving lipid mediator Resolvin D2, in placental tissue.

Methods:

Whole slide images of vascular smooth muscle (VSM) and extravillous trophoblast (EVT) cells stained for GPR18 were annotated for areas of interest. Visual scoring was performed on these images by trained and in-training pathologists, while QuPath scoring was performed with the methodology described herein.

Results:

Bland–Altman analyses showed that, for the VSM category, the two methods were comparable across all staining levels. For EVT cells, the high-intensity staining level was comparable across methods, but the medium and low staining levels were not comparable.

Conclusions:

Digital image analysis programs offer great potential to revolutionize pathology practice and research by increasing accuracy and decreasing the time and cost of analysis. Careful study is needed to optimize this methodology further.

Explainable machine learning model for predicting the occurrence of postoperative malnutrition in children with congenital heart disease

BACKGROUND & AIMS

Malnutrition is persistent in 50%-75% of children with congenital heart disease (CHD) after surgery, and early prediction is crucial for nutritional intervention. The aim of this study was to develop and validate machine learning (ML) models to predict the malnutrition status of children with CHD. We used explainable ML methods to provide insight into the model's predictions and outcomes.

METHODS

This prospective cohort study included consecutive children with CHD admitted to the hospital from December 2017 to May 2020. The cohort data were divided into the training and test data sets based on the follow-up time. The outcome of the study was CHD child malnutrition 1 year after surgery, the primary outcome was an underweight status, and the secondary outcomes were stunted and wasting status. We used five ML algorithms with multiple features to construct prediction models, and the performance of these ML models was measured by an area under the receiver operating characteristic curve (AUC) analysis. We also used the permutation importance and SHapley Additive exPlanations (SHAP) to determine the importance of the selected features and interpret the ML models.

RESULTS

We enrolled 536 children with CHD who underwent complete repair. The proportions of children with an underweight, stunted, or wasting status 1 year after surgery were 18.1% (97/536), 12.1% (65/536), and 17.5% (94/536), respectively. All patients contributed to the generation of 115 useable features, which allowed us to build models to predict malnutrition. Five prediction algorithms were used, and the XGBoost model achieved the greatest AUC in all outcomes. The results obtained from the permutation importance and SHAP analyses showed that the 1-month postoperative WAZ-score, discharge WAZ score and preoperative WAZ score were the top 3 important features in predicting an underweight status in the XGBoost algorithm. Regarding the stunted status, the top 3 important features were the 1-month postoperative HAZ score, discharge HAZ score, and aortic clamping time. Regarding the wasting status, the top 3 important features were the hospital length of stay, formula intake, and discharge WHZ-score. We also used a narrative case report as an example to describe the clinical manifestations and predicted the primary outcomes of two children.

CONCLUSIONS

We developed an ML model (XGBoost) that provides accurate early predictions of malnutrition 1-year postoperatively in children with CHD. Because the ML model is explainable, it may better enable clinicians to better understand the reasoning underlying the outcome. Our study could aid in determining individual treatment and nutritional follow-up strategies for children with CHD.

A Review of Radiomics and Deep Predictive Modeling in Glioma Characterization

Recent developments in glioma categorization based on biological genotypes and application of computational machine learning or deep learning based predictive models using multi-modal MRI biomarkers to assess these genotypes provides potential assurance for optimal and personalized treatment plans and efficacy. Artificial intelligence based quantified assessment of glioma using MRI derived hand-crafted or auto-extracted features have become crucial as genomic alterations can be associated with MRI based phenotypes. This survey integrates all the recent work carried out in state-of-the-art radiomics, and Artificial Intelligence based learning solutions related to molecular diagnosis, prognosis, and treatment monitoring with the aim to create a structured resource on radiogenomic analysis of glioma. Challenges such as inter-scanner variability, requirement of benchmark datasets, prospective validations for clinical applicability are discussed with further scope for designing optimal solutions for glioma stratification with immediate recommendations for further diagnostic decisions and personalized treatment plans for glioma patients.

Artificial Intelligence in Brain Tumour Surgery-An Emerging Paradigm

Artificial intelligence (AI) platforms have the potential to cause a paradigm shift in brain tumour surgery. Brain tumour surgery augmented with AI can result in safer and more effective treatment. In this review article, we explore the current and future role of AI in patients undergoing brain tumour surgery, including aiding diagnosis, optimising the surgical plan, providing support during the operation, and better predicting the prognosis. Finally, we discuss barriers to the successful clinical implementation, the ethical concerns, and we provide our perspective on how the field could be advanced.

Ensemble based machine learning approach for prediction of glioma and multi-grade classification

Glioma is the most pernicious cancer of the nervous system, with histological grade influencing the survival of patients. Despite many studies on the multimodal treatment approach, survival time remains brief. In this study, a novel two-stage ensemble of an ensemble-type machine learning-based predictive framework for glioma detection and its histograde classification is proposed. In the proposed framework, five characteristics belonging to 135 subjects were considered: human telomerase reverse transcriptase (hTERT), chitinase-like protein (YKL-40), interleukin 6 (IL-6), tissue inhibitor of metalloproteinase-1 (TIMP-1) and neutrophil/lymphocyte ratio (NLR). These characteristics were examined using distinctive ensemble-based machine learning classifiers and combination strategies to develop a computer-aided diagnostic system for the non-invasive prediction of glioma cases and their grade. In the first stage, the analysis was conducted to classify glioma cases and control subjects. Machine learning approaches were applied in the second stage to classify the recognised glioma cases into three grades, from grade II, which has a good prognosis, to grade IV, which is also known as glioblastoma. All experiments were evaluated with a five-fold cross-validation method, and the classification results were analysed using different statistical parameters. The proposed approach obtained a high value of accuracy and other statistical parameters compared with other state-of-the-art machine learning classifiers. Therefore, the proposed framework can be utilised for designing other intervention strategies for the prediction of glioma cases and their grades.

Multi-parametric MRI phenotype with trustworthy machine learning for differentiating CNS demyelinating diseases

Background

Misdiagnosis of multiple sclerosis (MS) and neuromyelitis optica (NMO) may delay the treatment, resulting in poor prognosis. However, the precise identification of these two diseases is still challenging in clinical practice. We aimed to evaluate the value of quantitative radiomic features extracted from the brain white matter lesions for differential diagnosis of MS and NMO.

Methods

We recruited 116 CNS demyelinating patients including 78 MS, and 38 NMO. Three neuroradiologists performed visual differential diagnosis based on brain MRI for comparison purpose. A multi-level scheme was designed to harness the selection of discriminative and stable radiomics features extracted from brain while mater lesions in T1-MPRAGE, T2 sequences and clinical factors. Based on the imaging phenotype composed of the selected radiomic and clinical features, Multi-parametric Multivariate Random Forest (MM-RF) model was constructed and verified with both 10-fold cross-validation and independent testing. Result interpretation was provided to build trust in diagnostic decisions.

Results

Eighty-six patients were randomly selected to form the training set while the rest 30 patients for independent testing. On the training set, our MM-RF model achieved accuracy 0.849 and AUC 0.826 in 10-fold cross-validation, which were significantly higher than clinical visual analysis (0.709 and 0.683, p < 0.05). In the independent testing, the MM-RF model achieved AUC 0.902, accuracy 0.871, sensitivity 0.873, specificity 0.869, respectively. Furthermore, age, sex and EDSS were found mildly correlated with the radiomic features (p of all < 0.05).

Conclusions

Multi-parametric radiomic features have potential as practical quantitative imaging biomarkers for differentiating MS from NMO.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12967-021-03015-w.

Combining MRI and Histologic Imaging Features for Predicting Overall Survival in Patients with Glioma

Purpose To test the hypothesis that combined features from MR and digital histopathologic images more accurately predict overall survival (OS) in patients with glioma compared with MRI or histopathologic features alone. Materials and Methods Multiparametric MR and histopathologic images in patients with a diagnosis of glioma (high- or low-grade glioma [HGG or LGG]) were obtained from The Cancer Imaging Archive (original images acquired 1983-2008). An extensive set of engineered features such as intensity, histogram, and texture were extracted from delineated tumor regions in MR and histopathologic images. Cox proportional hazard regression and support vector machine classification (SVC) models were applied to (a) MRI features only (MRI_cox/svc), histopathologic features only (HistoPath_cox/svc), and (c) combined MRI and histopathologic features (MRI+HistoPath_cox/svc) and evaluated in a split train-test configuration. Results A total of 171 patients (mean age, 51 years ± 15; 91 men) were included with HGG (n = 75) and LGG (n = 96). Median OS was 467 days (range, 3-4752 days) for all patients, 350 days (range, 15-1561 days) for HGG, and 595 days (range, 3-4752 days) for LGG. The MRI+HistoPath_cox model demonstrated higher concordance index (C-index) compared with MRI_cox and HistoPath_cox models on all patients (C-index, 0.79 vs 0.70 [P = .02; MRI_cox] and 0.67 [P = .01; HistoPath_cox]), patients with HGG (C-index, 0.78 vs 0.68 [P = .03; MRI_cox] and 0.64 [P = .01; HistoPath_cox]), and patients with LGG (C-index, 0.88 vs 0.62 [P = .008; MRI_cox] and 0.62 [P = .006; HistoPath_cox]). In binary classification, the MRI+HistoPath_svc model (area under the receiver operating characteristic curve [AUC], 0.86 [95% CI: 0.80, 0.95]) had higher performance than the MRI_svc model (AUC, 0.68 [95% CI: 0.50, 0.81]; P = .01) and the HistoPath_svc model (AUC, 0.72 [95% CI: 0.60, 0.85]; P = .04). Conclusion The model combining features from MR and histopathologic images had higher accuracy in predicting OS compared with the models with MR or histopathologic images alone. Keywords: Survival Prediction, Gliomas, Digital Pathology Imaging, MR Imaging, Machine Learning Supplemental material is available for this article.

Deep Neural Network Analysis of Pathology Images With Integrated Molecular Data for Enhanced Glioma Classification and Grading

Gliomas are primary brain tumors that originate from glial cells. Classification and grading of these tumors is critical to prognosis and treatment planning. The current criteria for glioma classification in central nervous system (CNS) was introduced by World Health Organization (WHO) in 2016. This criteria for glioma classification requires the integration of histology with genomics. In 2017, the Consortium to Inform Molecular and Practical Approaches to CNS Tumor Taxonomy (cIMPACT-NOW) was established to provide up-to-date recommendations for CNS tumor classification, which in turn the WHO is expected to adopt in its upcoming edition. In this work, we propose a novel glioma analytical method that, for the first time in the literature, integrates a cellularity feature derived from the digital analysis of brain histopathology images integrated with molecular features following the latest WHO criteria. We first propose a novel over-segmentation strategy for region-of-interest (ROI) selection in large histopathology whole slide images (WSIs). A Deep Neural Network (DNN)-based classification method then fuses molecular features with cellularity features to improve tumor classification performance. We evaluate the proposed method with 549 patient cases from The Cancer Genome Atlas (TCGA) dataset for evaluation. The cross validated classification accuracies are 93.81% for lower-grade glioma (LGG) and high-grade glioma (HGG) using a regular DNN, and 73.95% for LGG II and LGG III using a residual neural network (ResNet) DNN, respectively. Our experiments suggest that the type of deep learning has a significant impact on tumor subtype discrimination between LGG II vs. LGG III. These results outperform state-of-the-art methods in classifying LGG II vs. LGG III and offer competitive performance in distinguishing LGG vs. HGG in the literature. In addition, we also investigate molecular subtype classification using pathology images and cellularity information. Finally, for the first time in literature this work shows promise for cellularity quantification to predict brain tumor grading for LGGs with IDH mutations.

Can artificial intelligence overtake human intelligence on the bumpy road towards glioma therapy?

Gliomas are one of the most devastating primary brain tumors which impose significant management challenges to the clinicians. The aggressive behaviour of gliomas is mainly attributed to their rapid proliferation, unravelled genomics and the blood-brain barrier which protects the tumor cells from chemotherapeutic regimens. Suspects of brain tumors are usually assessed by magnetic resonance imaging and computed tomography. These images allow surgeons to decide on the tumor grading, intra-operative pathology, feasibility of surgery, and treatment planning. All these data are compiled manually by physicians, wherein it takes time for the validation of results and concluding the treatment modality. In this context, the arrival of artificial intelligence in this era of personalized medicine, has proven promising performance in the diagnosis and management of gliomas. Starting from grading prediction till outcome evaluation, artificial intelligence-based forefronts have revolutionized oncological research. Interestingly, this approach has also been able to precisely differentiate tumor lesion from healthy tissues. However, till date, their utility in neuro-oncological field remains limited due to the issues pertaining to their reliability and transparency. Hence, to shed novel insights on the "clinical utility of this novel approach on glioma management" and to reveal "the black-boxes that have to be solved for fruitful application of artificial intelligence in neuro-oncology research", we provide in this review, a succinct description of the potential gear of artificial intelligence-based avenues in glioma treatment and the barriers that impede their rapid implementation in neuro-oncology.

Development of Machine Learning Models for Predicting Postoperative Delayed Remission in Patients With Cushing's Disease

CONTEXT

Postoperative hypercortisolemia mandates further therapy in patients with Cushing's disease (CD). Delayed remission (DR) is defined as not achieving postoperative immediate remission (IR), but having spontaneous remission during long-term follow-up.

OBJECTIVE

We aimed to develop and validate machine learning (ML) models for predicting DR in non-IR patients with CD.

METHODS

We enrolled 201 CD patients, and randomly divided them into training and test datasets. We then used the recursive feature elimination (RFE) algorithm to select features and applied 5 ML algorithms to construct DR prediction models. We used permutation importance and local interpretable model-agnostic explanation (LIME) algorithms to determine the importance of the selected features and interpret the ML models.

RESULTS

Eighty-eight (43.8%) of the 201 CD patients met the criteria for DR. Overall, patients who were younger, had a low body mass index, a Knosp grade of III-IV, and a tumor not found by pathological examination tended to achieve a lower rate of DR. After RFE feature selection, the Adaboost model, which comprised 18 features, had the greatest discriminatory ability, and its predictive ability was significantly better than using Knosp grading and postoperative immediate morning serum cortisol (PoC). The results obtained from permutation importance and LIME algorithms showed that preoperative 24-hour urine free cortisol, PoC, and age were the most important features, and showed the reliability and clinical practicability of the Adaboost model in DC prediction.

CONCLUSIONS

Machine learning-based models could serve as an effective noninvasive approach to predicting DR, and could aid in determining individual treatment and follow-up strategies for CD patients.

Identification of the potential biomarkers in patients with glioma: a weighted gene co-expression network analysis

Glioma is the most common brain tumor with high mortality. However, there are still challenges for the timely and accurate diagnosis and effective treatment of the tumor. One hundred and twenty-one samples with grades II, III and IV from the Gene Expression Omnibus database were used to construct gene co-expression networks to identify hub modules closely related to glioma grade, and performed pathway enrichment analysis on genes from significant modules. In gene co-expression network constructed by 2345 differentially expressed genes from 121 gene expression profiles for glioma, we identified the black and blue modules that associated with grading. The module preservation analysis based on 118 samples indicates that the two modules were replicable. Enrichment analysis showed that the extracellular matrix genes were enriched for blue module, while cell division genes were enriched for black module. According to survival analysis, 21 hub genes were significantly up-regulated and one gene was significantly down-regulated. What’s more, IKBIP, SEC24D, and FAM46A are the genes with little attention among the 22 hub genes. In this study, IKBIP, SEC24D, and FAM46A related to glioma were mentioned for the first time to the current knowledge, which might provide a new idea for us to study the disease in the future. IKBIP, SEC24D and FAM46A among the 22 hub genes identified that are related to the malignancy degree of glioma might be used as new biomarkers to improve the diagnosis, treatment and prognosis of glioma.

Automated glioma grading on conventional MRI images using deep convolutional neural networks

PURPOSE

Gliomas are the most common primary tumor of the brain and are classified into grades I-IV of the World Health Organization (WHO), based on their invasively histological appearance. Gliomas grading plays an important role to determine the treatment plan and prognosis prediction. In this study we propose two novel methods for automatic, non-invasively distinguishing low-grade (Grades II and III) glioma (LGG) and high-grade (grade IV) glioma (HGG) on conventional MRI images by using deep convolutional neural networks (CNNs).

METHODS

All MRI images have been preprocessed first by rigid image registration and intensity inhomogeneity correction. Both proposed methods consist of two steps: (a) three-dimensional (3D) brain tumor segmentation based on a modification of the popular U-Net model; (b) tumor classification on segmented brain tumor. In the first method, the slice with largest area of tumor is determined and the state-of-the-art mask R-CNN model is employed for tumor grading. To improve the performance of the grading model, a two-dimensional (2D) data augmentation has been implemented to increase both the amount and the diversity of the training images. In the second method, denoted as 3DConvNet, a 3D volumetric CNNs is applied directly on bounding image regions of segmented tumor for classification, which can fully leverage the 3D spatial contextual information of volumetric image data.

RESULTS

The proposed schemes were evaluated on The Cancer Imaging Archive (TCIA) low grade glioma (LGG) data, and the Multimodal Brain Tumor Image Segmentation (BraTS) Benchmark 2018 training datasets with fivefold cross validation. All data are divided into training, validation, and test sets. Based on biopsy-proven ground truth, the performance metrics of sensitivity, specificity, and accuracy are measured on the test sets. The results are 0.935 (sensitivity), 0.972 (specificity), and 0.963 (accuracy) for the 2D Mask R-CNN based method, and 0.947 (sensitivity), 0.968 (specificity), and 0.971 (accuracy) for the 3DConvNet method, respectively. In regard to efficiency, for 3D brain tumor segmentation, the program takes around ten and a half hours for training with 300 epochs on BraTS 2018 dataset and takes only around 50 s for testing of a typical image with a size of 160 × 216 × 176. For 2D Mask R-CNN based tumor grading, the program takes around 4 h for training with around 60 000 iterations, and around 1 s for testing of a 2D slice image with size of 128 × 128. For 3DConvNet based tumor grading, the program takes around 2 h for training with 10 000 iterations, and 0.25 s for testing of a 3D cropped image with size of 64 × 64 × 64, using a DELL PRECISION Tower T7910, with two NVIDIA Titan Xp GPUs.

CONCLUSIONS

Two effective glioma grading methods on conventional MRI images using deep convolutional neural networks have been developed. Our methods are fully automated without manual specification of region-of-interests and selection of slices for model training, which are common in traditional machine learning based brain tumor grading methods. This methodology may play a crucial role in selecting effective treatment options and survival predictions without the need for surgical biopsy.

Emerging MRI Techniques to Redefine Treatment Response in Patients With Glioblastoma

Glioblastoma is the most common and most malignant primary brain tumor. Despite aggressive multimodal treatment, its prognosis remains poor. Even with continuous developments in MRI, which has provided us with newer insights into the diagnosis and understanding of tumor biology, response assessment in the posttherapy setting remains challenging. We believe that the integration of additional information from advanced neuroimaging techniques can further improve the diagnostic accuracy of conventional MRI. In this article, we review the utility of advanced neuroimaging techniques such as diffusion-weighted imaging, diffusion tensor imaging, perfusion-weighted imaging, proton magnetic resonance spectroscopy, and chemical exchange saturation transfer in characterizing and evaluating treatment response in patients with glioblastoma. We will also discuss the existing challenges and limitations of using these techniques in clinical settings and possible solutions to avoiding pitfalls in study design, data acquisition, and analysis for future studies. LEVEL OF EVIDENCE: 2 TECHNICAL EFFICACY STAGE: 3 J. Magn. Reson. Imaging 2020;52:978-997.

Segmentation and Classification in Digital Pathology for Glioma Research: Challenges and Deep Learning Approaches

Biomedical imaging Is an important source of information in cancer research. Characterizations of cancer morphology at onset, progression, and in response to treatment provide complementary information to that gleaned from genomics and clinical data. Accurate extraction and classification of both visual and latent image features Is an increasingly complex challenge due to the increased complexity and resolution of biomedical image data. In this paper, we present four deep learning-based image analysis methods from the Computational Precision Medicine (CPM) satellite event of the 21st International Medical Image Computing and Computer Assisted Intervention (MICCAI 2018) conference. One method Is a segmentation method designed to segment nuclei in whole slide tissue images (WSIs) of adult diffuse glioma cases. It achieved a Dice similarity coefficient of 0.868 with the CPM challenge datasets. Three methods are classification methods developed to categorize adult diffuse glioma cases into oligodendroglioma and astrocytoma classes using radiographic and histologic image data. These methods achieved accuracy values of 0.75, 0.80, and 0.90, measured as the ratio of the number of correct classifications to the number of total cases, with the challenge datasets. The evaluations of the four methods indicate that (1) carefully constructed deep learning algorithms are able to produce high accuracy in the analysis of biomedical image data and (2) the combination of radiographic with histologic image information improves classification performance.

Optimization of deep learning methods for visualization of tumor heterogeneity and brain tumor grading through digital pathology

BACKGROUND

Variations in prognosis and treatment options for gliomas are dependent on tumor grading. When tissue is available for analysis, grade is established based on histological criteria. However, histopathological diagnosis is not always reliable or straight-forward due to tumor heterogeneity, sampling error, and subjectivity, and hence there is great interobserver variability in readings.

METHODS

We trained convolutional neural network models to classify digital whole-slide histopathology images from The Cancer Genome Atlas. We tested a number of optimization parameters.

RESULTS

Data augmentation did not improve model training, while a smaller batch size helped to prevent overfitting and led to improved model performance. There was no significant difference in performance between a modular 2-class model and a single 3-class model system. The best models trained achieved a mean accuracy of 73% in classifying glioblastoma from other grades and 53% between WHO grade II and III gliomas. A visualization method was developed to convey the model output in a clinically relevant manner by overlaying color-coded predictions over the original whole-slide image.

CONCLUSIONS

Our developed visualization method reflects the clinical decision-making process by highlighting the intratumor heterogeneity and may be used in a clinical setting to aid diagnosis. Explainable artificial intelligence techniques may allow further evaluation of the model and underline areas for improvements such as biases. Due to intratumor heterogeneity, data annotation for training was imprecise, and hence performance was lower than expected. The models may be further improved by employing advanced data augmentation strategies and using more precise semiautomatic or manually labeled training data.

Development and Interpretation of Multiple Machine Learning Models for Predicting Postoperative Delayed Remission of Acromegaly Patients During Long-Term Follow-Up

Background: Some patients with acromegaly do not reach the remission standard in the short term after surgery but achieve remission without additional postoperative treatment during long-term follow-up; this phenomenon is defined as postoperative delayed remission (DR). DR may complicate the interpretation of surgical outcomes in patients with acromegaly and interfere with decision-making regarding postoperative adjuvant therapy. Objective: We aimed to develop and validate machine learning (ML) models for predicting DR in acromegaly patients who have not achieved remission within 6 months of surgery. Methods: We enrolled 306 acromegaly patients and randomly divided them into training and test datasets. We used the recursive feature elimination (RFE) algorithm to select features and applied six ML algorithms to construct DR prediction models. The performance of these ML models was validated using receiver operating characteristics analysis. We used permutation importance, SHapley Additive exPlanations (SHAP), and local interpretable model-agnostic explanation (LIME) algorithms to determine the importance of the selected features and interpret the ML models. Results: Fifty-five (17.97%) acromegaly patients met the criteria for DR, and five features (post-1w rGH, post-1w nGH, post-6m rGH, post-6m IGF-1, and post-6m nGH) were significantly associated with DR in both the training and the test datasets. After the RFE feature selection, the XGboost model, which comprised the 15 important features, had the greatest discriminatory ability (area under the curve = 0.8349, sensitivity = 0.8889, Youden's index = 0.6842). The XGboost model showed good discrimination ability and provided significantly better estimates of DR of patients with acromegaly compared with using only the Knosp grade. The results obtained from permutation importance, SHAP, and LIME algorithms showed that post-6m IGF-1 is the most important feature in XGboost algorithm prediction and showed the reliability and the clinical practicability of the XGboost model in DR prediction. Conclusions: ML-based models can serve as an effective non-invasive approach to predicting DR and could aid in determining individual treatment and follow-up strategies for acromegaly patients who have not achieved remission within 6 months of surgery.

Artificial Intelligence in the Management of Glioma: Era of Personalized Medicine

Purpose: Artificial intelligence (AI) has accelerated novel discoveries across multiple disciplines including medicine. Clinical medicine suffers from a lack of AI-based applications, potentially due to lack of awareness of AI methodology. Future collaboration between computer scientists and clinicians is critical to maximize the benefits of transformative technology in this field for patients. To illustrate, we describe AI-based advances in the diagnosis and management of gliomas, the most common primary central nervous system (CNS) malignancy.

Methods: Presented is a succinct description of foundational concepts of AI approaches and their relevance to clinical medicine, geared toward clinicians without computer science backgrounds. We also review novel AI approaches in the diagnosis and management of glioma.

Results: Novel AI approaches in gliomas have been developed to predict the grading and genomics from imaging, automate the diagnosis from histopathology, and provide insight into prognosis.

Conclusion: Novel AI approaches offer acceptable performance in gliomas. Further investigation is necessary to improve the methodology and determine the full clinical utility of these novel approaches.

Nested Dilation Networks for Brain Tumor Segmentation Based on Magnetic Resonance Imaging

Aim: Brain tumors are among the most fatal cancers worldwide. Diagnosing and manually segmenting tumors are time-consuming clinical tasks, and success strongly depends on the doctor's experience. Automatic quantitative analysis and accurate segmentation of brain tumors are greatly needed for cancer diagnosis. Methods:This paper presents an advanced three-dimensional multimodal segmentation algorithm called nested dilation networks (NDNs). It is inspired by the U-Net architecture, a convolutional neural network (CNN) developed for biomedical image segmentation and is modified to achieve better performance for brain tumor segmentation. Thus, we propose residual blocks nested with dilations (RnD) in the encoding part to enrich the low-level features and use squeeze-and-excitation (SE) blocks in both the encoding and decoding parts to boost significant features. To prove the reliability of the network structure, we compare our results with those of the standard U-Net and its transmutation networks. Different loss functions are considered to cope with class imbalance problems to maximize the brain tumor segmentation results. A cascade training strategy is employed to run NDNs for coarse-to-fine tumor segmentation. This strategy decomposes the multiclass segmentation problem into three binary segmentation problems and trains each task sequentially. Various augmentation techniques are utilized to increase the diversity of the data to avoid overfitting. Results: This approach achieves Dice similarity scores of 0.6652, 0.5880, and 0.6682 for edema, non-enhancing tumors, and enhancing tumors, respectively, in which the Dice loss is used for single-pass training. After cascade training, the Dice similarity scores rise to 0.7043, 0.5889, and 0.7206, respectively. Conclusion: Experiments show that the proposed deep learning algorithm outperforms other U-Net transmutation networks for brain tumor segmentation. Moreover, applying cascade training to NDNs facilitates better performance than other methods. The findings of this study provide considerable insight into the automatic and accurate segmentation of brain tumors.