Combining MRI and Histologic Imaging Features for Predicting Overall Survival in Patients with Glioma

Artificial intelligence in histopathological image analysis of central nervous system tumours: A systematic review

The convergence of digital pathology and artificial intelligence could assist histopathology image analysis by providing tools for rapid, automated morphological analysis. This systematic review explores the use of artificial intelligence for histopathological image analysis of digitised central nervous system (CNS) tumour slides. Comprehensive searches were conducted across EMBASE, Medline and the Cochrane Library up to June 2023 using relevant keywords. Sixty-eight suitable studies were identified and qualitatively analysed. The risk of bias was evaluated using the Prediction model Risk of Bias Assessment Tool (PROBAST) criteria. All the studies were retrospective and preclinical. Gliomas were the most frequently analysed tumour type. The majority of studies used convolutional neural networks or support vector machines, and the most common goal of the model was for tumour classification and/or grading from haematoxylin and eosin-stained slides. The majority of studies were conducted when legacy World Health Organisation (WHO) classifications were in place, which at the time relied predominantly on histological (morphological) features but have since been superseded by molecular advances. Overall, there was a high risk of bias in all studies analysed. Persistent issues included inadequate transparency in reporting the number of patients and/or images within the model development and testing cohorts, absence of external validation, and insufficient recognition of batch effects in multi-institutional datasets. Based on these findings, we outline practical recommendations for future work including a framework for clinical implementation, in particular, better informing the artificial intelligence community of the needs of the neuropathologist.

Pathomic model based on histopathological features and machine learning to predict IDO1 status and its association with breast cancer prognosis

PURPOSE

To establish a pathomic model using histopathological image features for predicting indoleamine 2,3-dioxygenase 1 (IDO1) status and its relationship with overall survival (OS) in breast cancer.

METHODS

A pathomic model was constructed using machine learning and histopathological images obtained from The Cancer Genome Atlas database to predict IDO1 expression. The model performance was evaluated based on the area under the curve, calibration curve, and decision curve analysis (DCA). Prediction scores (PSes) were generated from the model and applied to divide the patients into two groups. Survival outcomes, gene set enrichment, immune microenvironment, and tumor mutations were assessed between the two groups.

RESULTS

Survival analysis followed by multivariate correction revealed that high IDO1 is a protective factor for OS. Further, the model was calibrated, and it exhibited good discrimination. Additionally, the DCA showed that the proposed model provided a good clinical net benefit. The Kaplan-Meier analysis revealed a positive correlation between high PS and improved OS. Univariate and multivariate Cox regression analyses demonstrated that PS is an independent protective factor for OS. Moreover, differentially expressed genes were enriched in various essential biological processes, including extracellular matrix receptor interaction, angiogenesis, transforming growth factor β signaling, epithelial mesenchymal transition, cell junction, tryptophan metabolism, and heme metabolic processes. PS was positively correlated with M1 macrophages, CD8 + T cells, T follicular helper cells, and tumor mutational burden.

CONCLUSION

These results indicate the potential ability of the proposed pathomic model to predict IDO1 status and the OS of breast cancer patients.

An integrated radiology-pathology machine learning classifier for outcome prediction following radical prostatectomy: Preliminary findings

Objectives

To evaluate the added benefit of integrating features from pre-treatment MRI (radiomics) and digitized post-surgical pathology slides (pathomics) in prostate cancer (PCa) patients for prognosticating outcomes post radical-prostatectomy (RP) including a) rising prostate specific antigen (PSA), and b) extraprostatic-extension (EPE).

Methods

Multi-institutional data (N = 58) of PCa patients who underwent pre-treatment 3-T MRI prior to RP were included in this retrospective study. Radiomic and pathomic features were extracted from PCa regions on MRI and RP specimens delineated by expert clinicians. On training set (D1, N = 44), Cox Proportional-Hazards models MR, MP and MRaP were trained using radiomics, pathomics, and their combination, respectively, to prognosticate rising PSA (PSA > 0.03 ng/mL). Top features from MRaP were used to train a model to predict EPE on D1 and test on external dataset (D2, N = 14). C-index, Kalplan-Meier curves were used for survival analysis, and area under ROC (AUC) was used for EPE. MRaP was compared with the existing post-treatment risk-calculator, CAPRA (MC).

Results

Patients had median follow-up of 34 months. MRaP (c-index = 0.685 ± 0.05) significantly outperformed MR (c-index = 0.646 ± 0.05), MP (c-index = 0.631 ± 0.06) and MC (c-index = 0.601 ± 0.071) (p < 0.0001). Cross-validated Kaplan-Meier curves showed significant separation among risk groups for rising PSA for MRaP (p < 0.005, Hazard Ratio (HR) = 11.36) as compared to MR (p = 0.64, HR = 1.33), MP (p = 0.19, HR = 2.82) and MC (p = 0.10, HR = 3.05). Integrated radio-pathomic model MRaP (AUC = 0.80) outperformed MR (AUC = 0.57) and MP (AUC = 0.76) in predicting EPE on external-data (D2).

Conclusions

Results from this preliminary study suggest that a combination of radiomic and pathomic features can better predict post-surgical outcomes (rising PSA and EPE) compared to either of them individually as well as extant prognostic nomogram (CAPRA).

Added prognostic value of 3D deep learning-derived features from preoperative MRI for adult-type diffuse gliomas

BACKGROUND

To investigate the prognostic value of spatial features from whole-brain MRI using a three-dimensional (3D) convolutional neural network for adult-type diffuse gliomas.

METHODS

In a retrospective, multicenter study, 1925 diffuse glioma patients were enrolled from 5 datasets: SNUH (n = 708), UPenn (n = 425), UCSF (n = 500), TCGA (n = 160), and Severance (n = 132). The SNUH and Severance datasets served as external test sets. Precontrast and postcontrast 3D T1-weighted, T2-weighted, and T2-FLAIR images were processed as multichannel 3D images. A 3D-adapted SE-ResNeXt model was trained to predict overall survival. The prognostic value of the deep learning-based prognostic index (DPI), a spatial feature-derived quantitative score, and established prognostic markers were evaluated using Cox regression. Model evaluation was performed using the concordance index (C-index) and Brier score.

RESULTS

The MRI-only median DPI survival prediction model achieved C-indices of 0.709 and 0.677 (BS = 0.142 and 0.215) and survival differences (P < 0.001 and P = 0.002; log-rank test) for the SNUH and Severance datasets, respectively. Multivariate Cox analysis revealed DPI as a significant prognostic factor, independent of clinical and molecular genetic variables: hazard ratio = 0.032 and 0.036 (P < 0.001 and P = 0.004) for the SNUH and Severance datasets, respectively. Multimodal prediction models achieved higher C-indices than models using only clinical and molecular genetic variables: 0.783 vs. 0.774, P = 0.001, SNUH; 0.766 vs. 0.748, P = 0.023, Severance.

CONCLUSIONS

The global morphologic feature derived from 3D CNN models using whole-brain MRI has independent prognostic value for diffuse gliomas. Combining clinical, molecular genetic, and imaging data yields the best performance.

A radio-pathologic integrated model for prediction of lymph node metastasis stage in patients with gastric cancer

BACKGROUND

Accurate prediction of lymph node metastasis stage (LNMs) facilitates precision therapy for gastric cancer. We aimed to develop and validate a deep learning-based radio-pathologic model to predict the LNM stage in patients with gastric cancer by integrating CT images and histopathological whole-slide images (WSIs).

METHODS

A total of 252 patients were enrolled and randomly divided into a training set (n = 202) and a testing set (n = 50). Both pretreatment contrast-enhanced abdominal CT and WSI of biopsy specimens were collected for each patient. The deep radiologic and pathologic features were extracted from CT and WSI using ResNet-50 and Vision Transformer (ViT) network, respectively. By fusing both radiologic and pathologic features, a radio-pathologic integrated model was constructed to predict the five LNM stages. For comparison, four single-modality models using CT images or WSIs were also constructed, respectively. All models were trained on the training set and validated on the testing set.

RESULTS

The radio-pathologic integrated mode achieved an overall accuracy of 84.0% and a kappa coefficient of 0.795 on the testing set. The areas under the curves (AUCs) of the integrated model in predicting the five LNM stages were 0.978 (95% Confidence Interval (CI 0.917-1.000), 0.946 (95% CI 0.867-1.000), 0.890 (95% CI 0.718-1.000), 0.971 (95% CI 0.920-1.000), and 0.982 (95% CI 0.911-1.000), respectively. Moreover, the integrated model achieved an AUC of 0.978 (95% CI 0.912-1.000) in predicting the binary status of nodal metastasis.

CONCLUSION

Our study suggests that radio-pathologic integrated model that combined both macroscale radiologic image and microscale pathologic image can better predict lymph node metastasis stage in patients with gastric cancer.

Pretreatment MR-based radiomics in patients with glioblastoma: A systematic review and meta-analysis of prognostic endpoints

PURPOSE

Recent studies have shown promise of MR-based radiomics in predicting the survival of patients with untreated glioblastoma. This study aimed to comprehensively collate evidence to assess the prognostic value of radiomics in glioblastoma.

METHODS

PubMed-MEDLINE, Embase, and Web of Science were searched to find original articles investigating the prognostic value of MR-based radiomics in glioblastoma published up to July 14, 2023. Concordance indexes (C-indexes) and Cox proportional hazards ratios (HRs) of overall survival (OS) and progression-free survival (PFS) were pooled via random-effects modeling. For studies aimed at classifying long-term and short-term PFS, a hierarchical regression model was used to calculate pooled sensitivity and specificity. Between-study heterogeneity was assessed using the Higgin inconsistency index (I2). Subgroup regression analysis was performed to find potential factors contributing to heterogeneity. Publication bias was assessed via funnel plots and the Egger test.

RESULTS

Among 1371 abstracts, 18 and 17 studies were included for qualitative and quantitative data synthesis, respectively. Respective pooled C-indexes and HRs for OS were 0.65 (95 % confidence interval [CI], 0.58-0.72) and 2.88 (95 % CI, 2.28-3.64), whereas those for PFS were 0.61 (95 % CI, 0.55-0.66) and 2.78 (95 % CI, 1.91-4.03). Among 4 studies that predicted short-term PFS, the pooled sensitivity and specificity were 0.77 (95 % CI, 0.58-0.89) and 0.60 (95 % CI, 0.45-0.73), respectively. There was a substantial between-study heterogeneity among studies with the survival endpoint of OS C-index (n = 9, I2 = 83.8 %). Publication bias was not observed overall.

CONCLUSION

Pretreatment MR-based radiomics provided modest prognostic value in both OS and PFS in patients with glioblastoma.

New insights into glioma frequency maps: From genetic and transcriptomic correlate to survival prediction

Increasing evidence indicates that glioma topographic location is linked to the cellular origin, molecular alterations and genetic profile. This research aims to (a) reveal the underlying mechanisms of tumor location predilection in glioblastoma multiforme (GBM) and lower-grade glioma (LGG) and (b) leverage glioma location features to predict prognosis. MRI images from 396 GBM and 190 LGG (115 astrocytoma and 75 oligodendroglioma) patients were standardized to construct frequency maps and analyzed by voxel-based lesion-symptom mapping. We then investigated the spatial correlation between glioma distribution with gene expression in healthy brains. We also evaluated transcriptomic differences in tumor tissue from predilection and nonpredilection sites. Furthermore, we quantitively characterized tumor anatomical localization and explored whether it was significantly related to overall survival. Finally, we employed a support vector machine to build a survival prediction model for GBM patients. GBMs exhibited a distinct location predilection from LGGs. GBMs were nearer to the subventricular zone and more likely to be localized to regions enriched with synaptic signaling, whereas astrocytoma and oligodendroglioma tended to occur in areas associated with the immune response. Synapse, neurotransmitters and calcium ion channel-related genes were all activated in GBM tissues coming from predilection regions. Furthermore, we characterized tumor location features in terms of a series of tumor-to-predilection distance metrics, which were able to predict GBM 1-year survival status with an accuracy of 0.71. These findings provide new perspectives on our understanding of tumor anatomic localization. The spatial features of glioma are of great value in individual therapy and prognosis prediction.

Radio-pathomic approaches in pediatric neuro-oncology: Opportunities and challenges

With medical software platforms moving to cloud environments with scalable storage and computing, the translation of predictive artificial intelligence (AI) models to aid in clinical decision-making and facilitate personalized medicine for cancer patients is becoming a reality. Medical imaging, namely radiologic and histologic images, has immense analytical potential in neuro-oncology, and models utilizing integrated radiomic and pathomic data may yield a synergistic effect and provide a new modality for precision medicine. At the same time, the ability to harness multi-modal data is met with challenges in aggregating data across medical departments and institutions, as well as significant complexity in modeling the phenotypic and genotypic heterogeneity of pediatric brain tumors. In this paper, we review recent pathomic and integrated pathomic, radiomic, and genomic studies with clinical applications. We discuss current challenges limiting translational research on pediatric brain tumors and outline technical and analytical solutions. Overall, we propose that to empower the potential residing in radio-pathomics, systemic changes in cross-discipline data management and end-to-end software platforms to handle multi-modal data sets are needed, in addition to embracing modern AI-powered approaches. These changes can improve the performance of predictive models, and ultimately the ability to advance brain cancer treatments and patient outcomes through the development of such models.

Predicting Survival in Patients with Brain Tumors: Current State-of-the-Art of AI Methods Applied to MRI

Given growing clinical needs, in recent years Artificial Intelligence (AI) techniques have increasingly been used to define the best approaches for survival assessment and prediction in patients with brain tumors. Advances in computational resources, and the collection of (mainly) public databases, have promoted this rapid development. This narrative review of the current state-of-the-art aimed to survey current applications of AI in predicting survival in patients with brain tumors, with a focus on Magnetic Resonance Imaging (MRI). An extensive search was performed on PubMed and Google Scholar using a Boolean research query based on MeSH terms and restricting the search to the period between 2012 and 2022. Fifty studies were selected, mainly based on Machine Learning (ML), Deep Learning (DL), radiomics-based methods, and methods that exploit traditional imaging techniques for survival assessment. In addition, we focused on two distinct tasks related to survival assessment: the first on the classification of subjects into survival classes (short and long-term or eventually short, mid and long-term) to stratify patients in distinct groups. The second focused on quantification, in days or months, of the individual survival interval. Our survey showed excellent state-of-the-art methods for the first, with accuracy up to ∼98%. The latter task appears to be the most challenging, but state-of-the-art techniques showed promising results, albeit with limitations, with C-Index up to ∼0.91. In conclusion, according to the specific task, the available computational methods perform differently, and the choice of the best one to use is non-univocal and dependent on many aspects. Unequivocally, the use of features derived from quantitative imaging has been shown to be advantageous for AI applications, including survival prediction. This evidence from the literature motivates further research in the field of AI-powered methods for survival prediction in patients with brain tumors, in particular, using the wealth of information provided by quantitative MRI techniques.

Investigating Useful Features for Overall Survival Prediction in Patients with Low-Grade Glioma Using Histology Slides

Glioma, characterized by neoplastic growth in the brain, is a life-threatening condition that, in most cases, ultimately leads to death. Typical analysis of glioma development involves observation of brain tissue in the form of a histology slide under a microscope. Although brain histology images have much potential for predicting patient outcomes such as overall survival (OS), they are rarely used as the sole predictors due challenges presented by unique characteristics of brain tissue histology. However, utilizing histology in predicting overall survival can be useful for treatment and quality-of-life for patients with early-stage glioma. In this study, we investigate the use of deep learning models on histology slides combined with simple descriptor data (age and glioma subtype) as a predictor of (OS) in patients with low-grade glioma (LGG). Using novel clinical data, we show that models which are more attentive to discriminative features of the image will confer better predictions than generic models (82.7 and 65.3 AUC RFD-Net and Baseline VGG16 model, respectively). Additionally, we show that adding age and subtype information to a histology image-based model may provide greater robustness in the model than using the image alone (3.8 and 4.3 stds for RFD-Net and Baseline VGG16 model with 3-fold CV, respectively), while a model based on image and age but not subtype may confer the best predictive results (83.7 and 82.0 AUC for RFD-Net + age and RFD-Net + age + subtype, respectively). Clinical relevance- This study establishes important criteria for deep learning models which predict OS using histology and basic clinical data from LGG patients.

Ensembles of Convolutional Neural Networks for Survival Time Estimation of High-Grade Glioma Patients from Multimodal MRI

Glioma is the most common type of primary malignant brain tumor. Accurate survival time prediction for glioma patients may positively impact treatment planning. In this paper, we develop an automatic survival time prediction tool for glioblastoma patients along with an effective solution to the limited availability of annotated medical imaging datasets. Ensembles of snapshots of three dimensional (3D) deep convolutional neural networks (CNN) are applied to Magnetic Resonance Image (MRI) data to predict survival time of high-grade glioma patients. Additionally, multi-sequence MRI images were used to enhance survival prediction performance. A novel way to leverage the potential of ensembles to overcome the limitation of labeled medical image availability is shown. This new classification method separates glioblastoma patients into long- and short-term survivors. The BraTS (Brain Tumor Image Segmentation) 2019 training dataset was used in this work. Each patient case consisted of three MRI sequences (T1CE, T2, and FLAIR). Our training set contained 163 cases while the test set included 46 cases. The best known prediction accuracy of 74% for this type of problem was achieved on the unseen test set.

AI-Driven Image Analysis in Central Nervous System Tumors-Traditional Machine Learning, Deep Learning and Hybrid Models

The interpretation of imaging in medicine in general and in oncology specifically remains problematic due to several limitations which include the need to incorporate detailed clinical history, patient and disease-specific history, clinical exam features, previous and ongoing treatment, and account for the dependency on reproducible human interpretation of multiple factors with incomplete data linkage. To standardize reporting, minimize bias, expedite management, and improve outcomes, the use of Artificial Intelligence (AI) has gained significant prominence in imaging analysis. In oncology, AI methods have as a result been explored in most cancer types with ongoing progress in employing AI towards imaging for oncology treatment, assessing treatment response, and understanding and communicating prognosis. Challenges remain with limited available data sets, variability in imaging changes over time augmented by a growing heterogeneity in analysis approaches. We review the imaging analysis workflow and examine how hand-crafted features also referred to as traditional Machine Learning (ML), Deep Learning (DL) approaches, and hybrid analyses, are being employed in AI-driven imaging analysis in central nervous system tumors. ML, DL, and hybrid approaches coexist, and their combination may produce superior results although data in this space is as yet novel, and conclusions and pitfalls have yet to be fully explored. We note the growing technical complexities that may become increasingly separated from the clinic and enforce the acute need for clinician engagement to guide progress and ensure that conclusions derived from AI-driven imaging analysis reflect that same level of scrutiny lent to other avenues of clinical research.