Unsupervised Resolution of Histomorphologic Heterogeneity in Renal Cell Carcinoma Using a Brain Tumor-Educated Neural Network

Clinical Application of Digital and Computational Pathology in Renal Cell Carcinoma: A Systematic Review

CONTEXT

Computational pathology is a new interdisciplinary field that combines traditional pathology with modern technologies such as digital imaging and machine learning to better understand the diagnosis, prognosis, and natural history of many diseases.

OBJECTIVE

To provide an overview of digital and computational pathology and its current and potential applications in renal cell carcinoma (RCC).

EVIDENCE ACQUISITION

A systematic review of the English-language literature was conducted using the PubMed, Web of Science, and Scopus databases in December 2022 according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (PROSPERO ID: CRD42023389282). Risk of bias was assessed according to the Prediction Model Study Risk of Bias Assessment Tool.

EVIDENCE SYNTHESIS

In total, 20 articles were included in the review. All the studies used a retrospective design, and all digital pathology techniques were implemented retrospectively. The studies were classified according to their primary objective: detection, tumor characterization, and patient outcome. Regarding the transition to clinical practice, several studies showed promising potential. However, none presented a comprehensive assessment of clinical utility and implementation. Notably, there was substantial heterogeneity for both the strategies used for model building and the performance metrics reported.

CONCLUSIONS

This review highlights the vast potential of digital and computational pathology for the detection, classification, and assessment of oncological outcomes in RCC. Preliminary work in this field has yielded promising results. However, these models have not yet reached a stage where they can be integrated into routine clinical practice.

PATIENT SUMMARY

Computational pathology combines traditional pathology and technologies such as digital imaging and artificial intelligence to improve diagnosis of disease and identify prognostic factors and new biomarkers. The number of studies exploring its potential in kidney cancer is rapidly increasing. However, despite the surge in research activity, computational pathology is not yet ready for widespread routine use.

Empowering Renal Cancer Management with AI and Digital Pathology: Pathology, Diagnostics and Prognosis

Renal cell carcinoma is a significant health burden worldwide, necessitating accurate and efficient diagnostic methods to guide treatment decisions. Traditional pathology practices have limitations, including interobserver variability and time-consuming evaluations. In recent years, digital pathology tools emerged as a promising solution to enhance the diagnosis and management of renal cancer. This review aims to provide a comprehensive overview of the current state and potential of digital pathology in the context of renal cell carcinoma. Through advanced image analysis algorithms, artificial intelligence (AI) technologies facilitate quantification of cellular and molecular markers, leading to improved accuracy and reproducibility in renal cancer diagnosis. Digital pathology platforms empower remote collaboration between pathologists and help with the creation of comprehensive databases for further research and machine learning applications. The integration of digital pathology tools with other diagnostic modalities, such as radiology and genomics, enables a novel multimodal characterization of different types of renal cell carcinoma. With continuous advancements and refinement, AI technologies are expected to play an integral role in diagnostics and clinical decision-making, improving patient outcomes. In this article, we explored the digital pathology instruments available for clear cell, papillary and chromophobe renal cancers from pathologist and data analyst perspectives.

HAVOC: Small-scale histomic mapping of cancer biodiversity across large tissue distances using deep neural networks

Intratumoral heterogeneity can wreak havoc on current precision medicine strategies because of challenges in sufficient sampling of geographically separated areas of biodiversity distributed across centimeter-scale tumor distances. To address this gap, we developed a deep learning pipeline that leverages histomorphologic fingerprints of tissue to create "Histomic Atlases of Variation Of Cancers" (HAVOC). Using a number of objective molecular readouts, we demonstrate that HAVOC can define regional cancer boundaries with distinct biology. Using larger tumor specimens, we show that HAVOC can map biodiversity even across multiple tissue sections. By guiding profiling of 19 partitions across six high-grade gliomas, HAVOC revealed that distinct differentiation states can often coexist and be regionally distributed within these tumors. Last, to highlight generalizability, we benchmark HAVOC on additional tumor types. Together, we establish HAVOC as a versatile tool to generate small-scale maps of tissue heterogeneity and guide regional deployment of molecular resources to relevant biodiverse niches.

Standardizing analysis of intra-tumoral heterogeneity with computational pathology

Many malignant cancers like glioblastoma are highly adaptive diseases that dynamically change their regional biology to survive and thrive under diverse microenvironmental and therapeutic pressures. While the concept of intra-tumoral heterogeneity has become a major paradigm in cancer research and care, systematic approaches to assess and document bio-variation in cancer are still in their infancy. Here we discuss existing approaches and challenges to documenting intra-tumoral heterogeneity and emerging computational approaches that leverage artificial intelligence to begin to overcome these limitations. We propose how these emerging techniques can be coupled with a diversity of molecular tools to address intra-tumoral heterogeneity more systematically in research and in practice, especially across larger specimens and longitudinal analyses. Systematic documentation and characterization of heterogeneity across entire tumor specimens and their longitudinal evolution has the potential to improve our understanding and treatment of cancer.

Artificial Intelligence in Renal Cell Carcinoma Histopathology: Current Applications and Future Perspectives

Renal cell carcinoma (RCC) is characterized by its diverse histopathological features, which pose possible challenges to accurate diagnosis and prognosis. A comprehensive literature review was conducted to explore recent advancements in the field of artificial intelligence (AI) in RCC pathology. The aim of this paper is to assess whether these advancements hold promise in improving the precision, efficiency, and objectivity of histopathological analysis for RCC, while also reducing costs and interobserver variability and potentially alleviating the labor and time burden experienced by pathologists. The reviewed AI-powered approaches demonstrate effective identification and classification abilities regarding several histopathological features associated with RCC, facilitating accurate diagnosis, grading, and prognosis prediction and enabling precise and reliable assessments. Nevertheless, implementing AI in renal cell carcinoma generates challenges concerning standardization, generalizability, benchmarking performance, and integration of data into clinical workflows. Developing methodologies that enable pathologists to interpret AI decisions accurately is imperative. Moreover, establishing more robust and standardized validation workflows is crucial to instill confidence in AI-powered systems' outcomes. These efforts are vital for advancing current state-of-the-art practices and enhancing patient care in the future.

Deep Learning for Image Analysis in Kidney Care

Analysis of medical images, such as radiological or tissue specimens, is an indispensable part of medical diagnostics. Conventionally done manually, the process may sometimes be time-consuming and prone to interobserver variability. Image classification and segmentation by deep learning strategies, predominantly convolutional neural networks, may provide a significant advance in the diagnostic process. In renal medicine, most evidence has been generated around the radiological assessment of renal abnormalities and histological analysis of renal biopsy specimens' segmentation. In this article, the basic principles of image analysis by convolutional neural networks, brief descriptions of convolutional neural networks, and their system architecture for image analysis are discussed, in combination with examples regarding their use in image analysis in nephrology.

Artificial intelligence to predict oncological outcome directly from hematoxylin and eosin-stained slides in urology

INTRODUCTION

Artificial intelligence (AI) has been successfully applied for automatic tumor detection and grading in histopathological image analysis in urologic oncology. The aim of this review was to assess the applicability of these approaches in image-based oncological outcome prediction.

EVIDENCE ACQUISITION

A systematic literature search was conducted using the databases MEDLINE through PubMed and Web of Science up to April 20, 2021. Studies investigating AI approaches to determine the risk of recurrence, metastasis, or survival directly from H&E-stained tissue sections in prostate, renal cell or urothelial carcinoma were included. Characteristics of the AI approach and performance metrics were extracted and summarized. Risk of bias (RoB) was assessed using the PROBAST tool.

EVIDENCE SYNTHESIS

16 studies yielding a total of 6658 patients and reporting on 17 outcome predictions were included. Six studies focused on renal cell, six on prostate and three on urothelial carcinoma while one study investigated renal cell and urothelial carcinoma. Handcrafted feature extraction was used in five, a convolutional neural network (CNN) in six and a deep feature extraction in four studies. One study compared a CNN with handcrafted feature extraction. In seven outcome predictions, a multivariable comparison with clinicopathological parameters was reported. Five of them showed statistically significant hazard ratios for the AI's model's-prediction. However, RoB was high in 15 outcome predictions and unclear in two.

CONCLUSIONS

The included studies are promising but predominantly early pilot studies, therefore primarily highlighting the potential of AI approaches. Additional well-designed studies are needed to assess the actual clinical applicability.

Deep learning can predict survival directly from histology in clear cell renal cell carcinoma

For clear cell renal cell carcinoma (ccRCC) risk-dependent diagnostic and therapeutic algorithms are routinely implemented in clinical practice. Artificial intelligence-based image analysis has the potential to improve outcome prediction and thereby risk stratification. Thus, we investigated whether a convolutional neural network (CNN) can extract relevant image features from a representative hematoxylin and eosin-stained slide to predict 5-year overall survival (5y-OS) in ccRCC. The CNN was trained to predict 5y-OS in a binary manner using slides from TCGA and validated using an independent in-house cohort. Multivariable logistic regression was used to combine of the CNNs prediction and clinicopathological parameters. A mean balanced accuracy of 72.0% (standard deviation [SD] = 7.9%), sensitivity of 72.4% (SD = 10.6%), specificity of 71.7% (SD = 11.9%) and area under receiver operating characteristics curve (AUROC) of 0.75 (SD = 0.07) was achieved on the TCGA training set (n = 254 patients / WSIs) using 10-fold cross-validation. On the external validation cohort (n = 99 patients / WSIs), mean accuracy, sensitivity, specificity and AUROC were 65.5% (95%-confidence interval [CI]: 62.9–68.1%), 86.2% (95%-CI: 81.8–90.5%), 44.9% (95%-CI: 40.2–49.6%), and 0.70 (95%-CI: 0.69–0.71). A multivariable model including age, tumor stage and metastasis yielded an AUROC of 0.75 on the TCGA cohort. The inclusion of the CNN-based classification (Odds ratio = 4.86, 95%-CI: 2.70–8.75, p < 0.01) raised the AUROC to 0.81. On the validation cohort, both models showed an AUROC of 0.88. In univariable Cox regression, the CNN showed a hazard ratio of 3.69 (95%-CI: 2.60–5.23, p < 0.01) on TCGA and 2.13 (95%-CI: 0.92–4.94, p = 0.08) on external validation. The results demonstrate that the CNN’s image-based prediction of survival is promising and thus this widely applicable technique should be further investigated with the aim of improving existing risk stratification in ccRCC.

Intratumoral Resolution of Driver Gene Mutation Heterogeneity in Renal Cancer Using Deep Learning

Intratumoral heterogeneity arising from tumor evolution poses significant challenges biologically and clinically. Dissecting this complexity may benefit from deep learning (DL) algorithms, which can infer molecular features from ubiquitous hematoxylin and eosin (H&E)-stained tissue sections. Although DL algorithms have been developed to predict some driver mutations from H&E images, the ability of these DL algorithms to resolve intratumoral mutation heterogeneity at subclonal spatial resolution is unexplored. Here, we apply DL to a paradigm of intratumoral heterogeneity, clear cell renal cell carcinoma (ccRCC), the most common type of kidney cancer. Matched IHC and H&E images were leveraged to develop DL models for predicting intratumoral genetic heterogeneity of the three most frequently mutated ccRCC genes, BAP1, PBRM1, and SETD2. DL models were generated on a large cohort (N = 1,282) and tested on several independent cohorts, including a TCGA cohort (N = 363 patients) and two tissue microarray (TMA) cohorts (N = 118 and 365 patients). These models were also expanded to a patient-derived xenograft (PDX) TMA, affording analysis of homotopic and heterotopic interactions of tumor and stroma. The status of all three genes could be inferred by DL, with BAP1 showing the highest sensitivity and performance within and across tissue samples (AUC = 0.87-0.89 on holdout). BAP1 results were validated on independent human (AUC = 0.77-0.84) and PDX (AUC = 0.80) cohorts. Finally, BAP1 predictions correlated with clinical outputs such as disease-specific survival. Overall, these data show that DL models can resolve intratumoral heterogeneity in cancer with potential diagnostic, prognostic, and biological implications.

SIGNIFICANCE

This work demonstrates the potential for deep learning analysis of histopathologic images to serve as a fast, low-cost method to assess genetic intratumoral heterogeneity. See related commentary by Song et al., p. 2672.

Multi-Omics-Based Autophagy-Related Untypical Subtypes in Patients with Cerebral Amyloid Pathology

Recent multi-omics analyses paved the way for a comprehensive understanding of pathological processes. However, only few studies have explored Alzheimer's disease (AD) despite the possibility of biological subtypes within these patients. For this study, unsupervised classification of four datasets (genetics, miRNA transcriptomics, proteomics, and blood-based biomarkers) using Multi-Omics Factor Analysis+ (MOFA+), along with systems-biological approaches following various downstream analyses are performed. New subgroups within 170 patients with cerebral amyloid pathology (Aβ+) are revealed and the features of them are identified based on the top-rated targets constructing multi-omics factors of both whole (M-TPAD) and immune-focused models (M-IPAD). The authors explored the characteristics of subtypes and possible key-drivers for AD pathogenesis. Further in-depth studies showed that these subtypes are associated with longitudinal brain changes and autophagy pathways are main contributors. The significance of autophagy or clustering tendency is validated in peripheral blood mononuclear cells (PBMCs; n = 120 including 30 Aβ- and 90 Aβ+), induced pluripotent stem cell-derived human brain organoids/microglia (n = 12 including 5 Aβ-, 5 Aβ+, and CRISPR-Cas9 apolipoprotein isogenic lines), and human brain transcriptome (n = 78). Collectively, this study provides a strategy for precision medicine therapy and drug development for AD using integrative multi-omics analysis and network modelling.

Deep learning features encode interpretable morphologies within histological images

Convolutional neural networks (CNNs) are revolutionizing digital pathology by enabling machine learning-based classification of a variety of phenotypes from hematoxylin and eosin (H&E) whole slide images (WSIs), but the interpretation of CNNs remains difficult. Most studies have considered interpretability in a post hoc fashion, e.g. by presenting example regions with strongly predicted class labels. However, such an approach does not explain the biological features that contribute to correct predictions. To address this problem, here we investigate the interpretability of H&E-derived CNN features (the feature weights in the final layer of a transfer-learning-based architecture). While many studies have incorporated CNN features into predictive models, there has been little empirical study of their properties. We show such features can be construed as abstract morphological genes (“mones”) with strong independent associations to biological phenotypes. Many mones are specific to individual cancer types, while others are found in multiple cancers especially from related tissue types. We also observe that mone-mone correlations are strong and robustly preserved across related cancers. Importantly, linear mone-based classifiers can very accurately separate 38 distinct classes (19 tumor types and their adjacent normals, AUC = \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$97.1\% \pm 2.8\%$$\end{document}97.1%±2.8% for each class prediction), and linear classifiers are also highly effective for universal tumor detection (AUC = \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$99.2\% \pm 0.12\%$$\end{document}99.2%±0.12%). This linearity provides evidence that individual mones or correlated mone clusters may be associated with interpretable histopathological features or other patient characteristics. In particular, the statistical similarity of mones to gene expression values allows integrative mone analysis via expression-based bioinformatics approaches. We observe strong correlations between individual mones and individual gene expression values, notably mones associated with collagen gene expression in ovarian cancer. Mone-expression comparisons also indicate that immunoglobulin expression can be identified using mones in colon adenocarcinoma and that immune activity can be identified across multiple cancer types, and we verify these findings by expert histopathological review. Our work demonstrates that mones provide a morphological H&E decomposition that can be effectively associated with diverse phenotypes, analogous to the interpretability of transcription via gene expression values. Our work also demonstrates mones can be interpreted without using a classifier as a proxy.

Integrating morphologic and molecular histopathological features through whole slide image registration and deep learning

Background

Modern molecular pathology workflows in neuro-oncology heavily rely on the integration of morphologic and immunohistochemical patterns for analysis, classification, and prognostication. However, despite the recent emergence of digital pathology platforms and artificial intelligence-driven computational image analysis tools, automating the integration of histomorphologic information found across these multiple studies is challenged by large files sizes of whole slide images (WSIs) and shifts/rotations in tissue sections introduced during slide preparation.

Methods

To address this, we develop a workflow that couples different computer vision tools including scale-invariant feature transform (SIFT) and deep learning to efficiently align and integrate histopathological information found across multiple independent studies. We highlight the utility and automation potential of this workflow in the molecular subclassification and discovery of previously unappreciated spatial patterns in diffuse gliomas.

Results

First, we show how a SIFT-driven computer vision workflow was effective at automated WSI alignment in a cohort of 107 randomly selected surgical neuropathology cases (97/107 (91%) showing appropriate matches, AUC = 0.96). This alignment allows our AI-driven diagnostic workflow to not only differentiate different brain tumor types, but also integrate and carry out molecular subclassification of diffuse gliomas using relevant immunohistochemical biomarkers (IDH1-R132H, ATRX). To highlight the discovery potential of this workflow, we also examined spatial distributions of tumors showing heterogenous expression of the proliferation marker MIB1 and Olig2. This analysis helped uncovered an interesting and unappreciated association of Olig2 positive and proliferative areas in some gliomas (r = 0.62).

Conclusion

This efficient neuropathologist-inspired workflow provides a generalizable approach to help automate a variety of advanced immunohistochemically compatible diagnostic and discovery exercises in surgical neuropathology and neuro-oncology.

Selection, Visualization, and Interpretation of Deep Features in Lung Adenocarcinoma and Squamous Cell Carcinoma

Although deep learning networks applied to digital images have shown impressive results for many pathology-related tasks, their black-box approach and limitation in terms of interpretability are significant obstacles for their widespread clinical utility. This study investigates the visualization of deep features (DFs) to characterize two lung cancer subtypes, adenocarcinoma and squamous cell carcinoma. This study demonstrates that a subset of DFs exist that can accurately distinguish these two cancer subtypes, prominent DFs. Visualization of such individual DFs allows us to understand better histopathologic patterns at both the whole-slide and patch levels, allowing discrimination of these cancer types. These DFs were visualized at the whole slide image level through DF-specific heatmaps and at tissue patch level through generating activation maps. In addition, we show that these prominent DFs contain information that can distinguish carcinomas of organs other than the lung. This framework may serve as a platform for evaluating the interpretability of any deep network for diagnostic decision making.

Searching Images for Consensus: Can AI Remove Observer Variability in Pathology?

One of the major obstacles in reaching diagnostic consensus is observer variability. With the recent success of artificial intelligence, particularly the deep networks, the question emerges as to whether the fundamental challenge of diagnostic imaging can now be resolved. This article briefly reviews the problem and how eventually both supervised and unsupervised AI technologies could help to overcome it.