Derivation of prognostic contextual histopathological features from whole-slide images of tumours via graph deep learning

Clinical evaluation of deep learning-based risk profiling in breast cancer histopathology and comparison to an established multigene assay

PURPOSE

To evaluate the Stratipath Breast tool for image-based risk profiling and compare it with an established prognostic multigene assay for risk profiling in a real-world case series of estrogen receptor (ER)-positive and human epidermal growth factor receptor 2 (HER2)-negative early breast cancer patients categorized as intermediate risk based on classic clinicopathological variables and eligible for chemotherapy.

METHODS

In a case series comprising 234 invasive ER-positive/HER2-negative tumors, clinicopathological data including Prosigna results and corresponding HE-stained tissue slides were retrieved. The digitized HE slides were analysed by Stratipath Breast.

RESULTS

Our findings showed that the Stratipath Breast analysis identified 49.6% of the clinically intermediate tumors as low risk and 50.4% as high risk. The Prosigna assay classified 32.5%, 47.0% and 20.5% tumors as low, intermediate and high risk, respectively. Among Prosigna intermediate-risk tumors, 47.3% were stratified as Stratipath low risk and 52.7% as high risk. In addition, 89.7% of Stratipath low-risk cases were classified as Prosigna low/intermediate risk. The overall agreement between the two tests for low-risk and high-risk groups (N = 124) was 71.0%, with a Cohen's kappa of 0.42. For both risk profiling tests, grade and Ki67 differed significantly between risk groups.

CONCLUSION

The results from this clinical evaluation of image-based risk stratification shows a considerable agreement to an established gene expression assay in routine breast pathology.

Prediction of recurrence risk in endometrial cancer with multimodal deep learning

Predicting distant recurrence of endometrial cancer (EC) is crucial for personalized adjuvant treatment. The current gold standard of combined pathological and molecular profiling is costly, hampering implementation. Here we developed HECTOR (histopathology-based endometrial cancer tailored outcome risk), a multimodal deep learning prognostic model using hematoxylin and eosin-stained, whole-slide images and tumor stage as input, on 2,072 patients from eight EC cohorts including the PORTEC-1/-2/-3 randomized trials. HECTOR demonstrated C-indices in internal (n = 353) and two external (n = 160 and n = 151) test sets of 0.789, 0.828 and 0.815, respectively, outperforming the current gold standard, and identified patients with markedly different outcomes (10-year distant recurrence-free probabilities of 97.0%, 77.7% and 58.1% for HECTOR low-, intermediate- and high-risk groups, respectively, by Kaplan-Meier analysis). HECTOR also predicted adjuvant chemotherapy benefit better than current methods. Morphological and genomic feature extraction identified correlates of HECTOR risk groups, some with therapeutic potential. HECTOR improves on the current gold standard and may help delivery of personalized treatment in EC.

AI-based histopathology image analysis reveals a distinct subset of endometrial cancers

Endometrial cancer (EC) has four molecular subtypes with strong prognostic value and therapeutic implications. The most common subtype (NSMP; No Specific Molecular Profile) is assigned after exclusion of the defining features of the other three molecular subtypes and includes patients with heterogeneous clinical outcomes. In this study, we employ artificial intelligence (AI)-powered histopathology image analysis to differentiate between p53abn and NSMP EC subtypes and consequently identify a sub-group of NSMP EC patients that has markedly inferior progression-free and disease-specific survival (termed 'p53abn-like NSMP'), in a discovery cohort of 368 patients and two independent validation cohorts of 290 and 614 from other centers. Shallow whole genome sequencing reveals a higher burden of copy number abnormalities in the 'p53abn-like NSMP' group compared to NSMP, suggesting that this group is biologically distinct compared to other NSMP ECs. Our work demonstrates the power of AI to detect prognostically different and otherwise unrecognizable subsets of EC where conventional and standard molecular or pathologic criteria fall short, refining image-based tumor classification. This study's findings are applicable exclusively to females.

Analysis of 3D pathology samples using weakly supervised AI

Human tissue, which is inherently three-dimensional (3D), is traditionally examined through standard-of-care histopathology as limited two-dimensional (2D) cross-sections that can insufficiently represent the tissue due to sampling bias. To holistically characterize histomorphology, 3D imaging modalities have been developed, but clinical translation is hampered by complex manual evaluation and lack of computational platforms to distill clinical insights from large, high-resolution datasets. We present TriPath, a deep-learning platform for processing tissue volumes and efficiently predicting clinical outcomes based on 3D morphological features. Recurrence risk-stratification models were trained on prostate cancer specimens imaged with open-top light-sheet microscopy or microcomputed tomography. By comprehensively capturing 3D morphologies, 3D volume-based prognostication achieves superior performance to traditional 2D slice-based approaches, including clinical/histopathological baselines from six certified genitourinary pathologists. Incorporating greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, further emphasizing the value of capturing larger extents of heterogeneous morphology.

Bi-level Graph Learning Unveils Prognosis-Relevant Tumor Microenvironment Patterns from Breast Multiplexed Digital Pathology

The tumor microenvironment is widely recognized for its central role in driving cancer progression and influencing prognostic outcomes. Despite extensive research efforts dedicated to characterizing this complex and heterogeneous environment, considerable challenges persist. In this study, we introduce a data-driven approach for identifying patterns of cell organizations in the tumor microenvironment that are associated with patient prognoses. Our methodology relies on the construction of a bi-level graph model: (i) a cellular graph, which models the intricate tumor microenvironment, and (ii) a population graph that captures inter-patient similarities, given their respective cellular graphs, by means of a soft Weisfeiler-Lehman subtree kernel. This systematic integration of information across different scales enables us to identify patient subgroups exhibiting unique prognoses while unveiling tumor microenvironment patterns that characterize them. We demonstrate our approach in a cohort of breast cancer patients, where the identified tumor microenvironment patterns result in a risk stratification system that provides complementary, new information with respect to alternative standards. Our results, which are validated in a completely independent cohort, allow for new insights into the prognostic implications of the breast tumor microenvironment, and this methodology could be applied to other cancer types more generally.

Deep learning on tertiary lymphoid structures in hematoxylin-eosin predicts cancer prognosis and immunotherapy response

Tertiary lymphoid structures (TLSs) have been associated with favorable immunotherapy responses and prognosis in various cancers. Despite their significance, their quantification using multiplex immunohistochemistry (mIHC) staining of T and B lymphocytes remains labor-intensive, limiting its clinical utility. To address this challenge, we curated a dataset from matched mIHC and H&E whole-slide images (WSIs) and developed a deep learning model for automated segmentation of TLSs. The model achieved Dice coefficients of 0.91 on the internal test set and 0.866 on the external validation set, along with intersection over union (IoU) scores of 0.819 and 0.787, respectively. The TLS ratio, defined as the segmented TLS area over the total tissue area, correlated with B lymphocyte levels and the expression of CXCL13, a chemokine associated with TLS formation, in 6140 patients spanning 16 tumor types from The Cancer Genome Atlas (TCGA). The prognostic models for overall survival indicated that the inclusion of the TLS ratio with TNM staging significantly enhanced the models' discriminative ability, outperforming the traditional models that solely incorporated TNM staging, in 10 out of 15 TCGA tumor types. Furthermore, when applied to biopsied treatment-naïve tumor samples, higher TLS ratios predicted a positive immunotherapy response across multiple cohorts, including specific therapies for esophageal squamous cell carcinoma, non-small cell lung cancer, and stomach adenocarcinoma. In conclusion, our deep learning-based approach offers an automated and reproducible method for TLS segmentation and quantification, highlighting its potential in predicting immunotherapy response and informing cancer prognosis.

Mapping cancer biology in space: applications and perspectives on spatial omics for oncology

Technologies to decipher cellular biology, such as bulk sequencing technologies and single-cell sequencing technologies, have greatly assisted novel findings in tumor biology. Recent findings in tumor biology suggest that tumors construct architectures that influence the underlying cancerous mechanisms. Increasing research has reported novel techniques to map the tissue in a spatial context or targeted sampling-based characterization and has introduced such technologies to solve oncology regarding tumor heterogeneity, tumor microenvironment, and spatially located biomarkers. In this study, we address spatial technologies that can delineate the omics profile in a spatial context, novel findings discovered via spatial technologies in oncology, and suggest perspectives regarding therapeutic approaches and further technological developments.

Clinical decision support tool for breast cancer recurrence prediction using SHAP value in cooperative game theory

Background

Recurrence remains the primary cause of death in patients with breast cancer. Although machine learning can efficiently predict the prognosis of breast cancer patients, the black-box nature of the model may result in a lack of evidence for clinicians when making critical decisions.

Methods

In this study, our main objective was twofold: (1) to develop a clinical decision support tool for predicting the prognosis of breast cancer and (2) to identify and explore the key factors that influence breast cancer recurrence. To achieve this, we employed an explainable ensemble learning method called Shapley additive explanation (SHAP), which leverages cooperative game theory. Using real-world data from 1629 breast cancer patients, we analyzed and uncovered the key factors associated with breast cancer recurrence. Subsequently, we used these identified factors to create a recurrence prediction model and establish a decision mechanism for the tool. The proposed method not only provides accurate recurrence predictions but also offers transparent explanations for these predictions.

Results

By utilizing four key factors, namely, tumor size, clinical stage III, number of lymph node metastases, and age, our decision support tool for predicting breast cancer recurrence achieved significant improvements. The extra-tree model exhibited an increased area under the receiver operating characteristic curve (AUC) of 0.97, while the Random Forest model demonstrated an improved AUC of 0.96. We also offer a decision mechanism for a recurrence prediction model based on the identified key factors. This transparent and interpretable decision-making process facilitated by our explainable ensemble learning model enhances trust and promotes its applicability in clinical settings.

Conclusions

The proposed explainable ensemble learning method shows promising results in predicting breast cancer recurrence, outperforming existing methods with high accuracy and transparency. This advancement has the potential to significantly improve clinical decision-making and patient outcomes in breast cancer treatment.

Graph Neural Networks in Cancer and Oncology Research: Emerging and Future Trends

Next-generation cancer and oncology research needs to take full advantage of the multimodal structured, or graph, information, with the graph data types ranging from molecular structures to spatially resolved imaging and digital pathology, biological networks, and knowledge graphs. Graph Neural Networks (GNNs) efficiently combine the graph structure representations with the high predictive performance of deep learning, especially on large multimodal datasets. In this review article, we survey the landscape of recent (2020-present) GNN applications in the context of cancer and oncology research, and delineate six currently predominant research areas. We then identify the most promising directions for future research. We compare GNNs with graphical models and "non-structured" deep learning, and devise guidelines for cancer and oncology researchers or physician-scientists, asking the question of whether they should adopt the GNN methodology in their research pipelines.

Artificial intelligence-based risk stratification, accurate diagnosis and treatment prediction in gynecologic oncology

As data-driven science, artificial intelligence (AI) has paved a promising path toward an evolving health system teeming with thrilling opportunities for precision oncology. Notwithstanding the tremendous success of oncological AI in such fields as lung carcinoma, breast tumor and brain malignancy, less attention has been devoted to investigating the influence of AI on gynecologic oncology. Hereby, this review sheds light on the ever-increasing contribution of state-of-the-art AI techniques to the refined risk stratification and whole-course management of patients with gynecologic tumors, in particular, cervical, ovarian and endometrial cancer, centering on information and features extracted from clinical data (electronic health records), cancer imaging including radiological imaging, colposcopic images, cytological and histopathological digital images, and molecular profiling (genomics, transcriptomics, metabolomics and so forth). However, there are still noteworthy challenges beyond performance validation. Thus, this work further describes the limitations and challenges faced in the real-word implementation of AI models, as well as potential solutions to address these issues.

Unleashing the potential of AI for pathology: challenges and recommendations

Computational pathology is currently witnessing a surge in the development of AI techniques, offering promise for achieving breakthroughs and significantly impacting the practices of pathology and oncology. These AI methods bring with them the potential to revolutionize diagnostic pipelines as well as treatment planning and overall patient care. Numerous peer-reviewed studies reporting remarkable performance across diverse tasks serve as a testimony to the potential of AI in the field. However, widespread adoption of these methods in clinical and pre-clinical settings still remains a challenge. In this review article, we present a detailed analysis of the major obstacles encountered during the development of effective models and their deployment in practice. We aim to provide readers with an overview of the latest developments, assist them with insights into identifying some specific challenges that may require resolution, and suggest recommendations and potential future research directions. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Artificial intelligence-quantified tumour-lymphocyte spatial interaction predicts disease-free survival in resected lung adenocarcinoma: A graph-based, multicentre study

BACKGROUND AND OBJECTIVE

A high degree of lymphocyte infiltration is related to superior outcomes amongst patients with lung adenocarcinoma. Recent evidence indicates that the spatial interactions between tumours and lymphocytes also influence the anti-tumour immune responses, but the spatial analysis at the cellular level remains insufficient.

METHODS

We proposed an artificial intelligence-quantified Tumour-Lymphocyte Spatial Interaction score (TLSI-score) by calculating the ratio between the number of spatial adjacent tumour-lymphocyte and the number of tumour cells based on topology cell graph constructed using H&E-stained whole-slide images. The association of TLSI-score with disease-free survival (DFS) was explored in 529 patients with lung adenocarcinoma across three independent cohorts (D1, 275; V1, 139; V2, 115).

RESULTS

After adjusting for pTNM stage and other clinicopathologic risk factors, a higher TLSI-score was independently associated with longer DFS than a low TLSI-score in the three cohorts [D1, adjusted hazard ratio (HR), 0.674; 95% confidence interval (CI) 0.463-0.983; p = 0.040; V1, adjusted HR, 0.408; 95% CI 0.223-0.746; p = 0.004; V2, adjusted HR, 0.294; 95% CI 0.130-0.666; p = 0.003]. By integrating the TLSI-score with clinicopathologic risk factors, the integrated model (full model) improves the prediction of DFS in three independent cohorts (C-index, D1, 0.716 vs. 0.701; V1, 0.666 vs. 0.645; V2, 0.708 vs. 0.662) CONCLUSIONS: TLSI-score shows the second highest relative contribution to the prognostic prediction model, next to the pTNM stage. TLSI-score can assist in the characterising of tumour microenvironment and is expected to promote individualized treatment and follow-up decision-making in clinical practice.