Self-supervised attention-based deep learning for pan-cancer mutation prediction from histopathology

Explainable AI for computational pathology identifies model limitations and tissue biomarkers

Deep learning models have shown promise in histopathology image analysis, but their opaque decision-making process poses challenges in high-risk medical scenarios. Here we introduce HIPPO, an explainable AI method that interrogates attention-based multiple instance learning (ABMIL) models in computational pathology by generating counterfactual examples through tissue patch modifications in whole slide images. Applying HIPPO to ABMIL models trained to detect breast cancer metastasis reveals that they may overlook small tumors and can be misled by non-tumor tissue, while attention maps-widely used for interpretation-often highlight regions that do not directly influence predictions. By interpreting ABMIL models trained on a prognostic prediction task, HIPPO identified tissue areas with stronger prognostic effects than high-attention regions, which sometimes showed counterintuitive influences on risk scores. These findings demonstrate HIPPO's capacity for comprehensive model evaluation, bias detection, and quantitative hypothesis testing. HIPPO greatly expands the capabilities of explainable AI tools to assess the trustworthy and reliable development, deployment, and regulation of weakly-supervised models in computational pathology.

AI-driven Discovery of Morphomolecular Signatures in Toxicology

Early identification of drug toxicity is essential yet challenging in drug development. At the preclinical stage, toxicity is assessed with histopathological examination of tissue sections from animal models to detect morphological lesions. To complement this analysis, toxicogenomics is increasingly employed to understand the mechanism of action of the compound and ultimately identify lesion-specific safety biomarkers for which in vitro assays can be designed. However, existing works that aim to identify morphological correlates of expression changes rely on qualitative or semi-quantitative morphological characterization and remain limited in scale or morphological diversity. Artificial intelligence (AI) offers a promising approach for quantitatively modeling this relationship at an unprecedented scale. Here, we introduce GEESE, an AI model designed to impute morphomolecular signatures in toxicology data. Our model was trained to predict 1,536 gene targets on a cohort of 8,231 hematoxylin and eosin-stained liver sections from Rattus norvegicus across 127 preclinical toxicity studies. The model, evaluated on 2,002 tissue sections from 29 held-out studies, can yield pseudo-spatially resolved gene expression maps, which we correlate with six key drug-induced liver injuries (DILI). From the resulting 25 million lesion-expression pairs, we established quantitative relations between up and downregulated genes and lesions. Validation of these signatures against toxicogenomic databases, pathway enrichment analyses, and human hepatocyte cell lines asserted their relevance. Overall, our study introduces new methods for characterizing toxicity at an unprecedented scale and granularity, paving the way for AI-driven discovery of toxicity biomarkers.

Self-supervised learning for characterising histomorphological diversity and spatial RNA expression prediction across 23 human tissue types

As vast histological archives are digitised, there is a pressing need to be able to associate specific tissue substructures and incident pathology to disease outcomes without arduous annotation. Here, we learn self-supervised representations using a Vision Transformer, trained on 1.7 M histology images across 23 healthy tissues in 838 donors from the Genotype Tissue Expression consortium (GTEx). Using these representations, we can automatically segment tissues into their constituent tissue substructures and pathology proportions across thousands of whole slide images, outperforming other self-supervised methods (43% increase in silhouette score). Additionally, we can detect and quantify histological pathologies present, such as arterial calcification (AUROC = 0.93) and identify missing calcification diagnoses. Finally, to link gene expression to tissue morphology, we introduce RNAPath, a set of models trained on 23 tissue types that can predict and spatially localise individual RNA expression levels directly from H&E histology (mean genes significantly regressed = 5156, FDR 1%). We validate RNAPath spatial predictions with matched ground truth immunohistochemistry for several well characterised control genes, recapitulating their known spatial specificity. Together, these results demonstrate how self-supervised machine learning when applied to vast histological archives allows researchers to answer questions about tissue pathology, its spatial organisation and the interplay between morphological tissue variability and gene expression.

[A look into the neighboring discipline: eHealth in oncology]

Digitalization is dramatically changing the entire healthcare system. Keywords such as artificial intelligence, electronic patient files (ePA), electronic prescriptions (eRp), telemedicine, wearables, augmented reality and digital health applications (DiGA) represent the digital transformation that is already taking place. Digital becomes real! This article outlines the state of research and development, current plans and ongoing uses of digital tools in oncology in the first half of 2024. The possibilities for using artificial intelligence and the use of DiGAs in oncology are presented in more detail in this overview according to their stage of development as they already show a noticeable benefit in oncology.

Artificial intelligence to unlock real-world evidence in clinical oncology: A primer on recent advances

PURPOSE

Real world evidence is crucial to understanding the diffusion of new oncologic therapies, monitoring cancer outcomes, and detecting unexpected toxicities. In practice, real world evidence is challenging to collect rapidly and comprehensively, often requiring expensive and time-consuming manual case-finding and annotation of clinical text. In this Review, we summarise recent developments in the use of artificial intelligence to collect and analyze real world evidence in oncology.

METHODS

We performed a narrative review of the major current trends and recent literature in artificial intelligence applications in oncology.

RESULTS

Artificial intelligence (AI) approaches are increasingly used to efficiently phenotype patients and tumors at large scale. These tools also may provide novel biological insights and improve risk prediction through multimodal integration of radiographic, pathological, and genomic datasets. Custom language processing pipelines and large language models hold great promise for clinical prediction and phenotyping.

CONCLUSIONS

Despite rapid advances, continued progress in computation, generalizability, interpretability, and reliability as well as prospective validation are needed to integrate AI approaches into routine clinical care and real-time monitoring of novel therapies.

A review of big data technology and its application in cancer care

The development of modern medical devices and information technology has led to a rapid growth in the amount of data available for health protection information, with the concept of medical big data emerging globally, along with significant advances in cancer care relying on data-driven approaches. However, outstanding issues such as fragmented data governance, low-quality data specification, and data lock-in still make sharing challenging. Big data technology provides solutions for managing massive heterogeneous data while combining artificial intelligence (AI) techniques such as machine learning (ML) and deep learning (DL) to better mine the intrinsic connections between data. This paper surveys and organizes recent articles on big data technology and its applications in cancer, dividing them into three different types to outline their primary content and summarize their critical role in assisting cancer care. It then examines the latest research directions in big data technology in cancer and evaluates the current state of development of each type of application. Finally, current challenges and opportunities are discussed, and recommendations are made for the further integration of big data technology into the medical industry in the future.

Towards a general-purpose foundation model for computational pathology

Quantitative evaluation of tissue images is crucial for computational pathology (CPath) tasks, requiring the objective characterization of histopathological entities from whole-slide images (WSIs). The high resolution of WSIs and the variability of morphological features present significant challenges, complicating the large-scale annotation of data for high-performance applications. To address this challenge, current efforts have proposed the use of pretrained image encoders through transfer learning from natural image datasets or self-supervised learning on publicly available histopathology datasets, but have not been extensively developed and evaluated across diverse tissue types at scale. We introduce UNI, a general-purpose self-supervised model for pathology, pretrained using more than 100 million images from over 100,000 diagnostic H&E-stained WSIs (>77 TB of data) across 20 major tissue types. The model was evaluated on 34 representative CPath tasks of varying diagnostic difficulty. In addition to outperforming previous state-of-the-art models, we demonstrate new modeling capabilities in CPath such as resolution-agnostic tissue classification, slide classification using few-shot class prototypes, and disease subtyping generalization in classifying up to 108 cancer types in the OncoTree classification system. UNI advances unsupervised representation learning at scale in CPath in terms of both pretraining data and downstream evaluation, enabling data-efficient artificial intelligence models that can generalize and transfer to a wide range of diagnostically challenging tasks and clinical workflows in anatomic pathology.

A visual-language foundation model for computational pathology

The accelerated adoption of digital pathology and advances in deep learning have enabled the development of robust models for various pathology tasks across a diverse array of diseases and patient cohorts. However, model training is often difficult due to label scarcity in the medical domain, and a model's usage is limited by the specific task and disease for which it is trained. Additionally, most models in histopathology leverage only image data, a stark contrast to how humans teach each other and reason about histopathologic entities. We introduce CONtrastive learning from Captions for Histopathology (CONCH), a visual-language foundation model developed using diverse sources of histopathology images, biomedical text and, notably, over 1.17 million image-caption pairs through task-agnostic pretraining. Evaluated on a suite of 14 diverse benchmarks, CONCH can be transferred to a wide range of downstream tasks involving histopathology images and/or text, achieving state-of-the-art performance on histology image classification, segmentation, captioning, and text-to-image and image-to-text retrieval. CONCH represents a substantial leap over concurrent visual-language pretrained systems for histopathology, with the potential to directly facilitate a wide array of machine learning-based workflows requiring minimal or no further supervised fine-tuning.

Review of Molecular Technologies for Investigating Canine Cancer

Genetic molecular testing is starting to gain traction as part of standard clinical practice for dogs with cancer due to its multi-faceted benefits, such as potentially being able to provide diagnostic, prognostic and/or therapeutic information. However, the benefits and ultimate success of genomic analysis in the clinical setting are reliant on the robustness of the tools used to generate the results, which continually expand as new technologies are developed. To this end, we review the different materials from which tumour cells, DNA, RNA and the relevant proteins can be isolated and what methods are available for interrogating their molecular profile, including analysis of the genetic alterations (both somatic and germline), transcriptional changes and epigenetic modifications (including DNA methylation/acetylation and microRNAs). We also look to the future and the tools that are currently being developed, such as using artificial intelligence (AI) to identify genetic mutations from histomorphological criteria. In summary, we find that the molecular genetic characterisation of canine neoplasms has made a promising start. As we understand more of the genetics underlying these tumours and more targeted therapies become available, it will no doubt become a mainstay in the delivery of precision veterinary care to dogs with cancer.

Insight into deep learning for glioma IDH medical image analysis: A systematic review

BACKGROUND

Deep learning techniques explain the enormous potential of medical image analysis, particularly in digital pathology. Concurrently, molecular markers have gained increasing significance over the past decade in the context of glioma patients, providing novel insights into diagnosis and more personalized treatment options. Deep learning combined with imaging and molecular analysis enables more accurate prognostication of patients, more accurate treatment plan proposals, and accurate biomarker (IDH) prediction for gliomas. This systematic study examines the development of deep learning techniques for IDH prediction using histopathology images, spanning the period from 2019 to 2023.

METHOD

The study adhered to the PRISMA reporting requirements, and databases including PubMed, Google Scholar, Google Search, and preprint repositories (such as arXiv) were systematically queried for pertinent literature spanning the period from 2019 to the 30th of 2023. Search phrases related to deep learning, digital pathology, glioma, and IDH were collaboratively utilized.

RESULTS

Fifteen papers meeting the inclusion criteria were included in the analysis. These criteria specifically encompassed studies utilizing deep learning for the analysis of hematoxylin and eosin images to determine the IDH status in patients with gliomas.

CONCLUSIONS

When predicting the status of IDH, the classifier built on digital pathological images demonstrates exceptional performance. The study's predictive effectiveness is enhanced with the utilization of the appropriate deep learning model. However, external verification is necessary to showcase their resilience and universality. Larger sample sizes and multicenter samples are necessary for more comprehensive research to evaluate performance and confirm clinical advantages.

Regression-based Deep-Learning predicts molecular biomarkers from pathology slides

Deep Learning (DL) can predict biomarkers from cancer histopathology. Several clinically approved applications use this technology. Most approaches, however, predict categorical labels, whereas biomarkers are often continuous measurements. We hypothesize that regression-based DL outperforms classification-based DL. Therefore, we develop and evaluate a self-supervised attention-based weakly supervised regression method that predicts continuous biomarkers directly from 11,671 images of patients across nine cancer types. We test our method for multiple clinically and biologically relevant biomarkers: homologous recombination deficiency score, a clinically used pan-cancer biomarker, as well as markers of key biological processes in the tumor microenvironment. Using regression significantly enhances the accuracy of biomarker prediction, while also improving the predictions' correspondence to regions of known clinical relevance over classification. In a large cohort of colorectal cancer patients, regression-based prediction scores provide a higher prognostic value than classification-based scores. Our open-source regression approach offers a promising alternative for continuous biomarker analysis in computational pathology.

Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

Programmed death-ligand 1 (PD-L1) IHC is the most commonly used biomarker for immunotherapy response. However, quantification of PD-L1 status in pathology slides is challenging. Neither manual quantification nor a computer-based mimicking of manual readouts is perfectly reproducible, and the predictive performance of both approaches regarding immunotherapy response is limited. In this study, we developed a deep learning (DL) method to predict PD-L1 status directly from raw IHC image data, without explicit intermediary steps such as cell detection or pigment quantification. We trained the weakly supervised model on PD-L1-stained slides from the non-small cell lung cancer (NSCLC)-Memorial Sloan Kettering (MSK) cohort (N = 233) and validated it on the pan-cancer-Vall d'Hebron Institute of Oncology (VHIO) cohort (N = 108). We also investigated the performance of the model to predict response to immune checkpoint inhibitors (ICI) in terms of progression-free survival. In the pan-cancer-VHIO cohort, the performance was compared with tumor proportion score (TPS) and combined positive score (CPS). The DL model showed good performance in predicting PD-L1 expression (TPS ≥ 1%) in both NSCLC-MSK and pan-cancer-VHIO cohort (AUC 0.88 ± 0.06 and 0.80 ± 0.03, respectively). The predicted PD-L1 status showed an improved association with response to ICIs [HR: 1.5 (95% confidence interval: 1-2.3), P = 0.049] compared with TPS [HR: 1.4 (0.96-2.2), P = 0.082] and CPS [HR: 1.2 (0.79-1.9), P = 0.386]. Notably, our explainability analysis showed that the model does not just look at the amount of brown pigment in the IHC slides, but also considers morphologic factors such as lymphocyte conglomerates. Overall, end-to-end weakly supervised DL shows potential for improving patient stratification for cancer immunotherapy by analyzing PD-L1 IHC, holistically integrating morphology and PD-L1 staining intensity.

SIGNIFICANCE

The weakly supervised DL model to predict PD-L1 status from raw IHC data, integrating tumor staining intensity and morphology, enables enhanced patient stratification in cancer immunotherapy compared with traditional pathologist assessment.

End-to-end prognostication in colorectal cancer by deep learning: a retrospective, multicentre study

BACKGROUND

Precise prognosis prediction in patients with colorectal cancer (ie, forecasting survival) is pivotal for individualised treatment and care. Histopathological tissue slides of colorectal cancer specimens contain rich prognostically relevant information. However, existing studies do not have multicentre external validation with real-world sample processing protocols, and algorithms are not yet widely used in clinical routine.

METHODS

In this retrospective, multicentre study, we collected tissue samples from four groups of patients with resected colorectal cancer from Australia, Germany, and the USA. We developed and externally validated a deep learning-based prognostic-stratification system for automatic prediction of overall and cancer-specific survival in patients with resected colorectal cancer. We used the model-predicted risk scores to stratify patients into different risk groups and compared survival outcomes between these groups. Additionally, we evaluated the prognostic value of these risk groups after adjusting for established prognostic variables.

FINDINGS

We trained and validated our model on a total of 4428 patients. We found that patients could be divided into high-risk and low-risk groups on the basis of the deep learning-based risk score. On the internal test set, the group with a high-risk score had a worse prognosis than the group with a low-risk score, as reflected by a hazard ratio (HR) of 4·50 (95% CI 3·33-6·09) for overall survival and 8·35 (5·06-13·78) for disease-specific survival (DSS). We found consistent performance across three large external test sets. In a test set of 1395 patients, the high-risk group had a lower DSS than the low-risk group, with an HR of 3·08 (2·44-3·89). In two additional test sets, the HRs for DSS were 2·23 (1·23-4·04) and 3·07 (1·78-5·3). We showed that the prognostic value of the deep learning-based risk score is independent of established clinical risk factors.

INTERPRETATION

Our findings indicate that attention-based self-supervised deep learning can robustly offer a prognosis on clinical outcomes in patients with colorectal cancer, generalising across different populations and serving as a potentially new prognostic tool in clinical decision making for colorectal cancer management. We release all source codes and trained models under an open-source licence, allowing other researchers to reuse and build upon our work.

FUNDING

The German Federal Ministry of Health, the Max-Eder-Programme of German Cancer Aid, the German Federal Ministry of Education and Research, the German Academic Exchange Service, and the EU.

Deep learning-based phenotyping reclassifies combined hepatocellular-cholangiocarcinoma

Primary liver cancer arises either from hepatocytic or biliary lineage cells, giving rise to hepatocellular carcinoma (HCC) or intrahepatic cholangiocarcinoma (ICCA). Combined hepatocellular- cholangiocarcinomas (cHCC-CCA) exhibit equivocal or mixed features of both, causing diagnostic uncertainty and difficulty in determining proper management. Here, we perform a comprehensive deep learning-based phenotyping of multiple cohorts of patients. We show that deep learning can reproduce the diagnosis of HCC vs. CCA with a high performance. We analyze a series of 405 cHCC-CCA patients and demonstrate that the model can reclassify the tumors as HCC or ICCA, and that the predictions are consistent with clinical outcomes, genetic alterations and in situ spatial gene expression profiling. This type of approach could improve treatment decisions and ultimately clinical outcome for patients with rare and biphenotypic cancers such as cHCC-CCA.

One label is all you need: Interpretable AI-enhanced histopathology for oncology

Artificial Intelligence (AI)-enhanced histopathology presents unprecedented opportunities to benefit oncology through interpretable methods that require only one overall label per hematoxylin and eosin (H&E) slide with no tissue-level annotations. We present a structured review of these methods organized by their degree of verifiability and by commonly recurring application areas in oncological characterization. First, we discuss morphological markers (tumor presence/absence, metastases, subtypes, grades) in which AI-identified regions of interest (ROIs) within whole slide images (WSIs) verifiably overlap with pathologist-identified ROIs. Second, we discuss molecular markers (gene expression, molecular subtyping) that are not verified via H&E but rather based on overlap with positive regions on adjacent tissue. Third, we discuss genetic markers (mutations, mutational burden, microsatellite instability, chromosomal instability) that current technologies cannot verify if AI methods spatially resolve specific genetic alterations. Fourth, we discuss the direct prediction of survival to which AI-identified histopathological features quantitatively correlate but are nonetheless not mechanistically verifiable. Finally, we discuss in detail several opportunities and challenges for these one-label-per-slide methods within oncology. Opportunities include reducing the cost of research and clinical care, reducing the workload of clinicians, personalized medicine, and unlocking the full potential of histopathology through new imaging-based biomarkers. Current challenges include explainability and interpretability, validation via adjacent tissue sections, reproducibility, data availability, computational needs, data requirements, domain adaptability, external validation, dataset imbalances, and finally commercialization and clinical potential. Ultimately, the relative ease and minimum upfront cost with which relevant data can be collected in addition to the plethora of available AI methods for outcome-driven analysis will surmount these current limitations and achieve the innumerable opportunities associated with AI-driven histopathology for the benefit of oncology.

Prediction models for hormone receptor status in female breast cancer do not extend to males: further evidence of sex-based disparity in breast cancer

Breast cancer prognosis and management for both men and women are reliant upon estrogen receptor alpha (ERα) and progesterone receptor (PR) expression to inform therapy. Previous studies have shown that there are sex-specific binding characteristics of ERα and PR in breast cancer and, counterintuitively, ERα expression is more common in male than female breast cancer. We hypothesized that these differences could have morphological manifestations that are undetectable to human observers but could be elucidated computationally. To investigate this, we trained attention-based multiple instance learning prediction models for ERα and PR using H&E-stained images of female breast cancer from the Cancer Genome Atlas (TCGA) (n = 1085) and deployed them on external female (n = 192) and male breast cancer images (n = 245). Both targets were predicted in the internal (AUROC for ERα prediction: 0.86 ± 0.02, p < 0.001; AUROC for PR prediction = 0.76 ± 0.03, p < 0.001) and external female cohorts (AUROC for ERα prediction: 0.78 ± 0.03, p < 0.001; AUROC for PR prediction = 0.80 ± 0.04, p < 0.001) but not the male cohort (AUROC for ERα prediction: 0.66 ± 0.14, p = 0.43; AUROC for PR prediction = 0.63 ± 0.04, p = 0.05). This suggests that subtle morphological differences invisible upon visual inspection may exist between the sexes, supporting previous immunohistochemical, genomic, and transcriptomic analyses.

Image-Based Deep Learning Detection of High-Grade B-Cell Lymphomas Directly from Hematoxylin and Eosin Images

Deep learning applications are emerging as promising new tools that can support the diagnosis and classification of different cancer types. While such solutions hold great potential for hematological malignancies, there have been limited studies describing the use of such applications in this field. The rapid diagnosis of double/triple-hit lymphomas (DHLs/THLs) involving MYC, BCL2 and/or BCL6 rearrangements is obligatory for optimal patient care. Here, we present a novel deep learning tool for diagnosing DHLs/THLs directly from scanned images of biopsy slides. A total of 57 biopsies, including 32 in a training set (including five DH lymphoma cases) and 25 in a validation set (including 10 DH/TH cases), were included. The DHL-classifier demonstrated a sensitivity of 100%, a specificity of 87% and an AUC of 0.95, with only two false positive cases, compared to FISH. The DHL-classifier showed a 92% predictive value as a screening tool for performing conventional FISH analysis, over-performing currently used criteria. The work presented here provides the proof of concept for the potential use of an AI tool for the identification of DH/TH events. However, more extensive follow-up studies are required to assess the robustness of this tool and achieve high performances in a diverse population.

Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study

Deep learning (DL) can accelerate the prediction of prognostic biomarkers from routine pathology slides in colorectal cancer (CRC). However, current approaches rely on convolutional neural networks (CNNs) and have mostly been validated on small patient cohorts. Here, we develop a new transformer-based pipeline for end-to-end biomarker prediction from pathology slides by combining a pre-trained transformer encoder with a transformer network for patch aggregation. Our transformer-based approach substantially improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training and evaluating on a large multicenter cohort of over 13,000 patients from 16 colorectal cancer cohorts, we achieve a sensitivity of 0.99 with a negative predictive value of over 0.99 for prediction of microsatellite instability (MSI) on surgical resection specimens. We demonstrate that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem.

Artificial Intelligence in the Pathology of Gastric Cancer

Recent advances in artificial intelligence (AI) have provided novel tools for rapid and precise pathologic diagnosis. The introduction of digital pathology has enabled the acquisition of scanned slide images that are essential for the application of AI. The application of AI for improved pathologic diagnosis includes the error-free detection of potentially negligible lesions, such as a minute focus of metastatic tumor cells in lymph nodes, the accurate diagnosis of potentially controversial histologic findings, such as very well-differentiated carcinomas mimicking normal epithelial tissues, and the pathological subtyping of the cancers. Additionally, the utilization of AI algorithms enables the precise decision of the score of immunohistochemical markers for targeted therapies, such as human epidermal growth factor receptor 2 and programmed death-ligand 1. Studies have revealed that AI assistance can reduce the discordance of interpretation between pathologists and more accurately predict clinical outcomes. Several approaches have been employed to develop novel biomarkers from histologic images using AI. Moreover, AI-assisted analysis of the cancer microenvironment showed that the distribution of tumor-infiltrating lymphocytes was related to the response to the immune checkpoint inhibitor therapy, emphasizing its value as a biomarker. As numerous studies have demonstrated the significance of AI-assisted interpretation and biomarker development, the AI-based approach will advance diagnostic pathology.

Artificial Intelligence in Clinical Oncology: From Data to Digital Pathology and Treatment

Recently, a wide spectrum of artificial intelligence (AI)-based applications in the broader categories of digital pathology, biomarker development, and treatment have been explored. In the domain of digital pathology, these have included novel analytical strategies for realizing new information derived from standard histology to guide treatment selection and biomarker development to predict treatment selection and response. In therapeutics, these have included AI-driven drug target discovery, drug design and repurposing, combination regimen optimization, modulated dosing, and beyond. Given the continued advances that are emerging, it is important to develop workflows that seamlessly combine the various segments of AI innovation to comprehensively augment the diagnostic and interventional arsenal of the clinical oncology community. To overcome challenges that remain with regard to the ideation, validation, and deployment of AI in clinical oncology, recommendations toward bringing this workflow to fruition are also provided from clinical, engineering, implementation, and health care economics considerations. Ultimately, this work proposes frameworks that can potentially integrate these domains toward the sustainable adoption of practice-changing AI by the clinical oncology community to drive improved patient outcomes.

Direct image to subtype prediction for brain tumors using deep learning

Background

Deep Learning (DL) can predict molecular alterations of solid tumors directly from routine histopathology slides. Since the 2021 update of the World Health Organization (WHO) diagnostic criteria, the classification of brain tumors integrates both histopathological and molecular information. We hypothesize that DL can predict molecular alterations as well as WHO subtyping of brain tumors from hematoxylin and eosin-stained histopathology slides.

Methods

We used weakly supervised DL and applied it to three large cohorts of brain tumor samples, comprising N = 2845 patients.

Results

We found that the key molecular alterations for subtyping, IDH and ATRX, as well as 1p19q codeletion, were predictable from histology with an area under the receiver operating characteristic curve (AUROC) of 0.95, 0.90, and 0.80 in the training cohort, respectively. These findings were upheld in external validation cohorts with AUROCs of 0.90, 0.79, and 0.87 for prediction of IDH, ATRX, and 1p19q codeletion, respectively.

Conclusions

In the future, such DL-based implementations could ease diagnostic workflows, particularly for situations in which advanced molecular testing is not readily available.