Deep learning with biopsy whole slide images for pretreatment prediction of pathological complete response to neoadjuvant chemotherapy in breast cancer：A multicenter study

DIFLF: A domain-invariant features learning framework for single-source domain generalization in mammogram classification

BACKGROUND AND OBJECTIVE

Single-source domain generalization (SSDG) aims to generalize a deep learning (DL) model trained on one source dataset to multiple unseen datasets. This is important for the clinical applications of DL-based models to breast cancer screening, wherein a DL-based model is commonly developed in an institute and then tested in other institutes. One challenge of SSDG is to alleviate the domain shifts using only one domain dataset.

METHODS

The present study proposed a domain-invariant features learning framework (DIFLF) for single-source domain. Specifically, a style-augmentation module (SAM) and a content-style disentanglement module (CSDM) are proposed in DIFLF. SAM includes two different color jitter transforms, which transforms each mammogram in the source domain into two synthesized mammograms with new styles. Thus, it can greatly increase the feature diversity of the source domain, reducing the overfitting of the trained model. CSDM includes three feature disentanglement units, which extracts domain-invariant content (DIC) features by disentangling them from domain-specific style (DSS) features, reducing the influence of the domain shifts resulting from different feature distributions. Our code is available for open access on Github (https://github.com/85675/DIFLF).

RESULTS

DIFLF is trained in a private dataset (PRI1), and tested first in another private dataset (PRI2) with similar feature distribution to PRI1 and then tested in two public datasets (INbreast and MIAS) with greatly different feature distributions from PRI1. As revealed by the experiment results, DIFLF presents excellent performance for classifying mammograms in the unseen target datasets of PRI2, INbreast, and MIAS. The accuracy and AUC of DIFLF are 0.917 and 0.928 in PRI2, 0.882 and 0.893 in INbreast, 0.767 and 0.710 in MIAS, respectively.

CONCLUSIONS

DIFLF can alleviate the influence of domain shifts only using one source dataset. Moreover, DIFLF can achieve an excellent mammogram classification performance even in the unseen datasets with great feature distribution differences from the training dataset.

Tumor infiltrating lymphocytes (TILs) - Pathologia, quo vadis? - A global survey

Tumor-infiltrating lymphocytes (TILs) and the tumor microenvironment have become increasingly important in cancer research, and immunotherapy has achieved major breakthroughs in improving patient outcomes. Despite the significant role of the pathologist in identifying, subtyping and reporting TILs, the implementation and assessment of TILs in pathology routine remains vague. To assess the actual use of TILs in routine clinical practice, a formal standardized questionnaire was disseminated on two social media platforms ("X" and LinkedIn) and by email in June 2024. Based on the results, we conducted a literature review on TILs via Medline/Pubmed in the two most scored and reported entities, namely malignant melanoma and colorectal cancer (CRC). 77 participants from 24 different countries around the world, mostly pathologists (n = 63, 82.0 %), completed the survey. More than half of the participants do not assess or report TILs in their daily (clinical) practice, a trend consistent across the countries included in the study. A variety of methods are used to report TILs, ranging from Artificial Intelligence (AI)-based scoring algorithms to quantification by eyeballing. Despite recognizing the importance of TIL assessment in clinical routine, many participants find it time-consuming and express a strong preference for AI-based quantification. Our survey reflects the perspective of mostly early career pathologists who recognize the importance of TILs in cancer but face challenges in implementation. The development of AI tools and consensus guidelines could alleviate these barriers. In addition, increasing the visibility and understanding of the role of pathologists within the medical community remains critical.

Current status and prospects of artificial intelligence in breast cancer pathology: convolutional neural networks to prospective Vision Transformers

Breast cancer is the most prevalent cancer among women, and its diagnosis requires the accurate identification and classification of histological features for effective patient management. Artificial intelligence, particularly through deep learning, represents the next frontier in cancer diagnosis and management. Notably, the use of convolutional neural networks and emerging Vision Transformers (ViT) has been reported to automate pathologists' tasks, including tumor detection and classification, in addition to improving the efficiency of pathology services. Deep learning applications have also been extended to the prediction of protein expression, molecular subtype, mutation status, therapeutic efficacy, and outcome prediction directly from hematoxylin and eosin-stained slides, bypassing the need for immunohistochemistry or genetic testing. This review explores the current status and prospects of deep learning in breast cancer diagnosis with a focus on whole-slide image analysis. Artificial intelligence applications are increasingly applied to many tasks in breast pathology ranging from disease diagnosis to outcome prediction, thus serving as valuable tools for assisting pathologists and supporting breast cancer management.

Advancing personalized oncology: a systematic review on the integration of artificial intelligence in monitoring neoadjuvant treatment for breast cancer patients

PURPOSE

Despite suffering from the same disease, each patient exhibits a distinct microbiological profile and variable reactivity to prescribed treatments. Most doctors typically use a standardized treatment approach for all patients suffering from a specific disease. Consequently, the challenge lies in the effectiveness of this standardized treatment and in adapting it to each individual patient. Personalized medicine is an emerging field in which doctors use diagnostic tests to identify the most effective medical treatments for each patient. Prognosis, disease monitoring, and treatment planning rely on manual, error-prone methods. Artificial intelligence (AI) uses predictive techniques capable of automating prognostic and monitoring processes, thus reducing the error rate associated with conventional methods.

METHODS

This paper conducts an analysis of current literature, encompassing the period from January 2015 to 2023, based on Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA).

RESULTS

In assessing 25 pertinent studies concerning predicting neoadjuvant treatment (NAT) response in breast cancer (BC) patients, the studies explored various imaging modalities (Magnetic Resonance Imaging, Ultrasound, etc.), evaluating results based on accuracy, sensitivity, and area under the curve. Additionally, the technologies employed, such as machine learning (ML), deep learning (DL), statistics, and hybrid models, were scrutinized. The presentation of datasets used for predicting complete pathological response (PCR) was also considered.

CONCLUSION

This paper seeks to unveil crucial insights into the application of AI techniques in personalized oncology, particularly in the monitoring and prediction of responses to NAT for BC patients. Finally, the authors suggest avenues for future research into AI-based monitoring systems.

Cross-modal deep learning model for predicting pathologic complete response to neoadjuvant chemotherapy in breast cancer

Pathological complete response (pCR) serves as a critical measure of the success of neoadjuvant chemotherapy (NAC) in breast cancer, directly influencing subsequent therapeutic decisions. With the continuous advancement of artificial intelligence, methods for early and accurate prediction of pCR are being extensively explored. In this study, we propose a cross-modal multi-pathway automated prediction model that integrates temporal and spatial information. This model fuses digital pathology images from biopsy specimens and multi-temporal ultrasound (US) images to predict pCR status early in NAC. The model demonstrates exceptional predictive efficacy. Our findings lay the foundation for developing personalized treatment paradigms based on individual responses. This approach has the potential to become a critical auxiliary tool for the early prediction of NAC response in breast cancer patients.

Deep learning-based risk stratification of preoperative breast biopsies using digital whole slide images

BACKGROUND

Nottingham histological grade (NHG) is a well established prognostic factor in breast cancer histopathology but has a high inter-assessor variability with many tumours being classified as intermediate grade, NHG2. Here, we evaluate if DeepGrade, a previously developed model for risk stratification of resected tumour specimens, could be applied to risk-stratify tumour biopsy specimens.

METHODS

A total of 11,955,755 tiles from 1169 whole slide images of preoperative biopsies from 896 patients diagnosed with breast cancer in Stockholm, Sweden, were included. DeepGrade, a deep convolutional neural network model, was applied for the prediction of low- and high-risk tumours. It was evaluated against clinically assigned grades NHG1 and NHG3 on the biopsy specimen but also against the grades assigned to the corresponding resection specimen using area under the operating curve (AUC). The prognostic value of the DeepGrade model in the biopsy setting was evaluated using time-to-event analysis.

RESULTS

Based on preoperative biopsy images, the DeepGrade model predicted resected tumour cases of clinical grades NHG1 and NHG3 with an AUC of 0.908 (95% CI: 0.88; 0.93). Furthermore, out of the 432 resected clinically-assigned NHG2 tumours, 281 (65%) were classified as DeepGrade-low and 151 (35%) as DeepGrade-high. Using a multivariable Cox proportional hazards model the hazard ratio between DeepGrade low- and high-risk groups was estimated as 2.01 (95% CI: 1.06; 3.79).

CONCLUSIONS

DeepGrade provided prediction of tumour grades NHG1 and NHG3 on the resection specimen using only the biopsy specimen. The results demonstrate that the DeepGrade model can provide decision support to identify high-risk tumours based on preoperative biopsies, thus improving early treatment decisions.

Artificial intelligence in digital histopathology for predicting patient prognosis and treatment efficacy in breast cancer

INTRODUCTION

Histological images contain phenotypic information predictive of patient outcomes. Due to the heavy workload of pathologists, the time-consuming nature of quantitatively assessing histological features, and human eye limitations to recognize spatial patterns, manually extracting prognostic information in routine pathological workflows remains challenging. Digital pathology has facilitated the mining and quantification of these features utilizing whole-slide image (WSI) scanners and artificial intelligence (AI) algorithms. AI algorithms to identify image-based biomarkers from the tumor microenvironment (TME) have the potential to revolutionize the field of oncology, reducing delays between diagnosis and prognosis determination, allowing for rapid stratification of patients and prescription of optimal treatment regimes, thereby improving patient outcomes.

AREAS COVERED

In this review, the authors discuss how AI algorithms and digital pathology can predict breast cancer patient prognosis and treatment outcomes using image-based biomarkers, along with the challenges of adopting this technology in clinical settings.

EXPERT OPINION

The integration of AI and digital pathology presents significant potential for analyzing the TME and its diagnostic, prognostic, and predictive value in breast cancer patients. Widespread clinical adoption of AI faces ethical, regulatory, and technical challenges, although prospective trials may offer reassurance and promote uptake, ultimately improving patient outcomes by reducing diagnosis-to-prognosis delivery delays.

Deep learning-based predictive model for pathological complete response to neoadjuvant chemotherapy in breast cancer from biopsy pathological images: a multicenter study

Introduction: Early predictive pathological complete response (pCR) is beneficial for optimizing neoadjuvant chemotherapy (NAC) strategies for breast cancer. The hematoxylin and eosin (HE)-stained slices of biopsy tissues contain a large amount of information on tumor epithelial cells and stromal. The fusion of pathological image features and clinicopathological features is expected to build a model to predict pCR of NAC in breast cancer. Methods: We retrospectively collected a total of 440 breast cancer patients from three hospitals who underwent NAC. HE-stained slices of biopsy tissues were scanned to form whole-slide images (WSIs), and pathological images of representative regions of interest (ROI) of each WSI were selected at different magnifications. Based on several different deep learning models, we propose a novel feature extraction method on pathological images with different magnifications. Further, fused with clinicopathological features, a multimodal breast cancer NAC pCR prediction model based on a support vector machine (SVM) classifier was developed and validated with two additional validation cohorts (VCs). Results: Through experimental validation of several different deep learning models, we found that the breast cancer pCR prediction model based on the SVM classifier, which uses the VGG16 model for feature extraction of pathological images at ×20 magnification, has the best prediction efficacy. The area under the curve (AUC) of deep learning pathological model (DPM) were 0.79, 0.73, and 0.71 for TC, VC1, and VC2, respectively, all of which exceeded 0.70. The AUCs of clinical model (CM), a clinical prediction model established by using clinicopathological features, were 0.79 for TC, 0.73 for VC1, and 0.71 for VC2, respectively. The multimodal deep learning clinicopathological model (DPCM) established by fusing pathological images and clinicopathological features improved the AUC of TC from 0.79 to 0.84. The AUC of VC2 improved from 0.71 to 0.78. Conclusion: Our study reveals that pathological images of HE-stained slices of pre-NAC biopsy tissues can be used to build a pCR prediction model. Combining pathological images and clinicopathological features can further enhance the predictive efficacy of the model.

Artificial intelligence-based pathology as a biomarker of sensitivity to atezolizumab-bevacizumab in patients with hepatocellular carcinoma: a multicentre retrospective study

BACKGROUND

Clinical benefits of atezolizumab plus bevacizumab (atezolizumab-bevacizumab) are observed only in a subset of patients with hepatocellular carcinoma and the development of biomarkers is needed to improve therapeutic strategies. The atezolizumab-bevacizumab response signature (ABRS), assessed by molecular biology profiling techniques, has been shown to be associated with progression-free survival after treatment initiation. The primary objective of our study was to develop an artificial intelligence (AI) model able to estimate ABRS expression directly from histological slides, and to evaluate if model predictions were associated with progression-free survival.

METHODS

In this multicentre retrospective study, we developed a model (ABRS-prediction; ABRS-P), which was derived from the previously published clustering-constrained attention multiple instance learning (or CLAM) pipeline. We trained the model fit for regression analysis using a multicentre dataset from The Cancer Genome Atlas (patients treated by surgical resection, n=336). The ABRS-P model was externally validated on two independent series of samples from patients with hepatocellular carcinoma (a surgical resection series, n=225; and a biopsy series, n=157). The predictive value of the model was further tested in a series of biopsy samples from a multicentre cohort of patients with hepatocellular carcinoma treated with atezolizumab-bevacizumab (n=122). All samples in the study were from adults (aged ≥18 years). The validation sets were sampled between Jan 1, 2008, to Jan 1, 2023. For the multicentre validation set, the primary objective was to assess the association of high versus low ABRS-P values, defined relative to cross-validation median split thresholds in the first biopsy series, with progression-free survival after treatment initiation. Finally, we performed spatial transcriptomics and matched prediction heatmaps with in situ expression profiles.

FINDINGS

Of the 840 patients sampled, 641 (76%) were male and 199 (24%) were female. Across the development and validation datasets, hepatocellular carcinoma risk factors included alcohol intake, hepatitis B and C virus infections, and non-alcoholic steatohepatitis. Using cross-validation in the development series, the mean Pearson's correlation between ABRS-P values and ABRS score (mean expression of ABRS genes) was r=0·62 (SD 0·09; mean p<0·0001, SD<0·0001). The ABRS-P generalised well on the external validation series (surgical resection series, r=0·60 [95% CI 0·51-0·68], p<0·0001; biopsy series, r=0·53 [0·40-0·63], p<0·0001). In the 122 patients treated with atezolizumab-bevacizumab, those with ABRS-P-high tumours (n=74) showed significantly longer median progression-free survival than those with ABRS-P-low tumours (n=48) after treatment initiation (12 months [95% CI 7-not reached] vs 7 months [4-9]; p=0·014). Spatial transcriptomics showed significantly higher ABRS score, along with upregulation of various other immune effectors, in tumour areas with high ABRS-P values versus areas with low ABRS-P values.

INTERPRETATION

Our study indicates that AI applied on hepatocellular carcinoma digital slides is able to serve as a biomarker for progression-free survival in patients treated with atezolizumab-bevacizumab. This approach could be used in the development of inexpensive and fast biomarkers for targeted therapies. The combination of AI heatmaps with spatial transcriptomics provides insight on the molecular features associated with predictions. This methodology could be applied to other cancers or diseases and improve understanding of the biological mechanisms that drive responses to treatments.

FUNDING

Institut National du Cancer, Fondation ARC, China Scholarship Council, Ligue Contre le Cancer du Val de Marne, Fondation de l'Avenir, Ipsen, and Fondation Bristol Myers Squibb Pour la Recherche en Immuno-Oncologie.

Tumor Mutation Burden-Related Histopathologic Features for Predicting Overall Survival in Gliomas Using Graph Deep Learning

Tumor mutation burden (TMB) is a potential biomarker for evaluating the prognosis and response to immune checkpoint inhibitors, but its costly and time-consuming method of measurement limits its widespread application. This study aimed to identify the TMB-related histopathologic features from hematoxylin and eosin slides and explore their prognostic value in gliomas. TMB-related features were detected using a graph convolutional neural network from whole-slide images of patients from The Cancer Genome Atlas data set (619 patients), and the correlation between features and TMB was evaluated in an external validation set (237 patients). TMB-related features were used for predicting overall survival (OS) of patients to investigate whether these features have potential for prognostic prediction. Moreover, biological pathways underlying the prognostic value of the features were further explored. Histopathologic features derived from whole-slide images were significantly associated with patient TMB (P = 0.007 in the external validation set). TMB-related features showed excellent performance for OS prediction, and patients with lower-grade gliomas could be further stratified into different risk groups according to the features (P = 0.00013; hazard ratio, 4.004). Pathways involved in the cell cycle and execution of immune response were enriched in patients with higher OS risk. The TMB-related features could be used to estimate TMB and aid in prognostic risk stratification of patients with glioma with dysregulated biological pathways.

Development and Assessment of a Novel Core Biopsy-Based Prediction Model for Pathological Complete Response to Neoadjuvant Chemotherapy in Women with Breast Cancer

Purpose: Pathological complete response (pCR), the goal of NAC, is considered a surrogate for favorable outcomes in breast cancer (BC) patients administrated neoadjuvant chemotherapy (NAC). This study aimed to develop and assess a novel nomogram model for predicting the probability of pCR based on the core biopsy. Methods: This was a retrospective study involving 920 BC patients administered NAC between January 2012 and December 2018. The patients were divided into a primary cohort (769 patients from January 2012 to December 2017) and a validation cohort (151 patients from January 2017 to December 2018). After converting continuous variables to categorical variables, variables entering the model were sequentially identified via univariate analysis, a multicollinearity test, and binary logistic regression analysis, and then, a nomogram model was developed. The performance of the model was assessed concerning its discrimination, accuracy, and clinical utility. Results: The optimal predictive threshold for estrogen receptor (ER), Ki67, and p53 were 22.5%, 32.5%, and 37.5%, respectively (all p < 0.001). Five variables were selected to develop the model: clinical T staging (cT), clinical nodal (cN) status, ER status, Ki67 status, and p53 status (all p ≤ 0.001). The nomogram showed good discrimination with the area under the curve (AUC) of 0.804 and 0.774 for the primary and validation cohorts, respectively, and good calibration. Decision curve analysis (DCA) showed that the model had practical clinical value. Conclusions: This study constructed a novel nomogram model based on cT, cN, ER status, Ki67 status, and p53 status, which could be applied to personalize the prediction of pCR in BC patients treated with NAC.