Progress on deep learning in digital pathology of breast cancer: a narrative review

Artificial intelligence in digital histopathology for predicting patient prognosis and treatment efficacy in breast cancer

INTRODUCTION

Histological images contain phenotypic information predictive of patient outcomes. Due to the heavy workload of pathologists, the time-consuming nature of quantitatively assessing histological features, and human eye limitations to recognize spatial patterns, manually extracting prognostic information in routine pathological workflows remains challenging. Digital pathology has facilitated the mining and quantification of these features utilizing whole-slide image (WSI) scanners and artificial intelligence (AI) algorithms. AI algorithms to identify image-based biomarkers from the tumor microenvironment (TME) have the potential to revolutionize the field of oncology, reducing delays between diagnosis and prognosis determination, allowing for rapid stratification of patients and prescription of optimal treatment regimes, thereby improving patient outcomes.

AREAS COVERED

In this review, the authors discuss how AI algorithms and digital pathology can predict breast cancer patient prognosis and treatment outcomes using image-based biomarkers, along with the challenges of adopting this technology in clinical settings.

EXPERT OPINION

The integration of AI and digital pathology presents significant potential for analyzing the TME and its diagnostic, prognostic, and predictive value in breast cancer patients. Widespread clinical adoption of AI faces ethical, regulatory, and technical challenges, although prospective trials may offer reassurance and promote uptake, ultimately improving patient outcomes by reducing diagnosis-to-prognosis delivery delays.

Artificial Intelligence-Powered Assessment of Pathologic Response to Neoadjuvant Atezolizumab in Patients With NSCLC: Results From the LCMC3 Study

INTRODUCTION

Pathologic response (PathR) by histopathologic assessment of resected specimens may be an early clinical end point associated with long-term outcomes with neoadjuvant therapy. Digital pathology may improve the efficiency and precision of PathR assessment. LCMC3 (NCT02927301) evaluated neoadjuvant atezolizumab in patients with resectable NSCLC and reported a 20% major PathR rate.

METHODS

We determined PathR in primary tumor resection specimens using guidelines-based visual techniques and developed a convolutional neural network model using the same criteria to digitally measure the percent viable tumor on whole-slide images. Concordance was evaluated between visual determination of percent viable tumor (n = 151) performed by one of the 47 local pathologists and three central pathologists.

RESULTS

For concordance among visual determination of percent viable tumor, the interclass correlation coefficient was 0.87 (95% confidence interval [CI]: 0.84-0.90). Agreement for visually assessed 10% or less viable tumor (major PathR [MPR]) in the primary tumor was 92.1% (Fleiss kappa = 0.83). Digitally assessed percent viable tumor (n = 136) correlated with visual assessment (Pearson r = 0.73; digital/visual slope = 0.28). Digitally assessed MPR predicted visually assessed MPR with outstanding discrimination (area under receiver operating characteristic curve, 0.98) and was associated with longer disease-free survival (hazard ratio [HR] = 0.30; 95% CI: 0.09-0.97, p = 0.033) and overall survival (HR = 0.14, 95% CI: 0.02-1.06, p = 0.027) versus no MPR. Digitally assessed PathR strongly correlated with visual measurements.

CONCLUSIONS

Artificial intelligence-powered digital pathology exhibits promise in assisting pathologic assessments in neoadjuvant NSCLC clinical trials. The development of artificial intelligence-powered approaches in clinical settings may aid pathologists in clinical operations, including routine PathR assessments, and subsequently support improved patient care and long-term outcomes.

Ultrasound deep learning radiomics and clinical machine learning models to predict low nuclear grade, ER, PR, and HER2 receptor status in pure ductal carcinoma in situ

Background

Low nuclear grade ductal carcinoma in situ (DCIS) patients can adopt proactive management strategies to avoid unnecessary surgical resection. Different personalized treatment modalities may be selected based on the expression status of molecular markers, which is also predictive of different outcomes and risks of recurrence. DCIS ultrasound findings are mostly non mass lesions, making it difficult to determine boundaries. Currently, studies have shown that models based on deep learning radiomics (DLR) have advantages in automatic recognition of tumor contours. Machine learning models based on clinical imaging features can explain the importance of imaging features.

Methods

The available ultrasound data of 349 patients with pure DCIS confirmed by surgical pathology [54 low nuclear grade, 175 positive estrogen receptor (ER+), 163 positive progesterone receptor (PR+), and 81 positive human epidermal growth factor receptor 2 (HER2+)] were collected. Radiologists extracted ultrasonographic features of DCIS lesions based on the 5th Edition of Breast Imaging Reporting and Data System (BI-RADS). Patient age and BI-RADS characteristics were used to construct clinical machine learning (CML) models. The RadImageNet pretrained network was used for extracting radiomics features and as an input for DLR modeling. For training and validation datasets, 80% and 20% of the data, respectively, were used. Logistic regression (LR), support vector machine (SVM), random forest (RF), and eXtreme Gradient Boosting (XGBoost) algorithms were performed and compared for the final classification modeling. Each task used the area under the receiver operating characteristic curve (AUC) to evaluate the effectiveness of DLR and CML models.

Results

In the training dataset, low nuclear grade, ER+, PR+, and HER2+ DCIS lesions accounted for 19.20%, 65.12%, 61.21%, and 30.19%, respectively; the validation set, they consisted of 19.30%, 62.50%, 57.14%, and 30.91%, respectively. In the DLR models we developed, the best AUC values for identifying features were 0.633 for identifying low nuclear grade, completed by the XGBoost Classifier of ResNet50; 0.618 for identifying ER, completed by the RF Classifier of InceptionV3; 0.755 for identifying PR, completed by the XGBoost Classifier of InceptionV3; and 0.713 for identifying HER2, completed by the LR Classifier of ResNet50. The CML models had better performance than DLR in predicting low nuclear grade, ER+, PR+, and HER2+ DCIS lesions. The best AUC values by classification were as follows: for low nuclear grade by RF classification, AUC: 0.719; for ER+ by XGBoost classification, AUC: 0.761; for PR+ by XGBoost classification, AUC: 0.780; and for HER2+ by RF classification, AUC: 0.723.

Conclusions

Based on small-scale datasets, our study showed that the DLR models developed using RadImageNet pretrained network and CML models may help predict low nuclear grade, ER+, PR+, and HER2+ DCIS lesions so that patients benefit from hierarchical and personalized treatment.

Current status and prospects of artificial intelligence in breast cancer pathology: convolutional neural networks to prospective Vision Transformers

Breast cancer is the most prevalent cancer among women, and its diagnosis requires the accurate identification and classification of histological features for effective patient management. Artificial intelligence, particularly through deep learning, represents the next frontier in cancer diagnosis and management. Notably, the use of convolutional neural networks and emerging Vision Transformers (ViT) has been reported to automate pathologists' tasks, including tumor detection and classification, in addition to improving the efficiency of pathology services. Deep learning applications have also been extended to the prediction of protein expression, molecular subtype, mutation status, therapeutic efficacy, and outcome prediction directly from hematoxylin and eosin-stained slides, bypassing the need for immunohistochemistry or genetic testing. This review explores the current status and prospects of deep learning in breast cancer diagnosis with a focus on whole-slide image analysis. Artificial intelligence applications are increasingly applied to many tasks in breast pathology ranging from disease diagnosis to outcome prediction, thus serving as valuable tools for assisting pathologists and supporting breast cancer management.

Application of automated machine learning for histological evaluation of feline endoscopic samples

Differentiating intestinal T-cell lymphoma from chronic enteropathy (CE) in endoscopic samples is often challenging. In the present study, automated machine learning systems were developed to distinguish between the two diseases, predict clonality, and detect prognostic factors of intestinal lymphoma in cats. Four models were created for four experimental conditions: experiment 1 to distinguish between intestinal T-cell lymphoma and CE; experiment 2 to distinguish large cell lymphoma, small cell lymphoma, and CE; experiment 3 to distinguish granzyme B+ lymphoma, granzyme B- lymphoma, and CE; and experiment 4 to distinguish between T-cell receptor (TCR) clonal population and TCR polyclonal population. After each experiment, a pathologist reviewed the test images and scored for lymphocytic infiltration, epitheliotropism, and epithelial injury. The models of experiments 1-4 achieved area under the receiver operating characteristic curve scores of 0.943 (precision, 87.59%; recall, 87.59%), 0.962 (precision, 86.30%; recall, 86.30%), 0.904 (precision, 82.86%; recall, 80%), and 0.904 (precision, 81.25%; recall, 81.25%), respectively. The images predicted as intestinal T-cell lymphoma showed significant infiltration of lymphocytes and epitheliotropism than CE. These models can provide evaluation tools to assist pathologists with differentiating between intestinal T-cell lymphoma and CE.

Domain and Histopathology Adaptations-Based Classification for Malignancy Grading System

Accurate proliferation rate quantification can be used to devise an appropriate treatment for breast cancer. Pathologists use breast tissue biopsy glass slides stained with hematoxylin and eosin to obtain grading information. However, this manual evaluation may lead to high costs and be ineffective because diagnosis depends on the facility and the pathologists' insights and experiences. Convolutional neural network acts as a computer-based observer to improve clinicians' capacity in grading breast cancer. Therefore, this study proposes a novel scheme for automatic breast cancer malignancy grading from invasive ductal carcinoma. The proposed classifiers implement multistage transfer learning incorporating domain and histopathologic transformations. Domain adaptation using pretrained models, such as InceptionResNetV2, InceptionV3, NASNet-Large, ResNet50, ResNet101, VGG19, and Xception, was applied to classify the ×40 magnification BreaKHis data set into eight classes. Subsequently, InceptionV3 and Xception, which contain the domain and histopathology pretrained weights, were determined to be the best for this study and used to categorize the Databiox database into grades 1, 2, or 3. To provide a comprehensive report, this study offered a patchless automated grading system for magnification-dependent and magnification-independent classifications. With an overall accuracy (means ± SD) of 90.17% ± 3.08% to 97.67% ± 1.09% and an F1 score of 0.9013 to 0.9760 for magnification-dependent classification, the classifiers in this work achieved outstanding performance. The proposed approach could be used for breast cancer grading systems in clinical settings.

The development and validation of pathological sections based U-Net deep learning segmentation model for the detection of esophageal mucosa and squamous cell neoplasm

Background

Deep learning methods have demonstrated great potential for processing high-resolution images. The U-Net model, in particular, has shown proficiency in the segmentation of biomedical images. However, limited research has examined the application of deep learning to esophageal squamous cell carcinoma (ESCC) segmentation. Therefore, this study aimed to develop deep learning segmentation systems specifically for ESCC.

Methods

A Visual Geometry Group (VGG)-based U-Net neural network architecture was utilized to develop the segmentation models. A pathological image cohort of surgical specimens was used for model training and internal validation, with two additional endoscopic biopsy section cohort for external validation. Model efficacy was evaluated across several metrics including Intersection over Union (IOU), accuracy, positive predict value (PPV), true positive rate (TPR), specificity, dice similarity coefficient (DSC), area under the receiver operating characteristic curve (AUC), and F1-Score.

Results

Surgical samples from ten patients were analyzed retrospectively, with each biopsy section cohort encompassing five patients. Transfer learning models based on U-Net weights yielded optimal results. For mucosa segmentation, the in internal validation achieved 93.81% IOU, with other parameters exceeding 96% (96.96% accuracy, 96.45% PPV, 96.65% TPR, 98.41% specificity, 96.81% DSC, 96.11% AUC, and 96.55% F1-Score). The tumor segmentation model attained an IOU of 91.95%, along with other parameters surpassing 95% (95.90% accuracy, 95.62% PPV, 95.71% TPR, 97.88% specificity, 95.81% DSC, 94.92% AUC, and 95.67% F1-Score). In the external validation for tumor segmentation model, IOU was 59.86% for validation database 1 (72.74% for accuracy, 76.03% for PPV, 77.17% for TPR, 83.80% for specificity, 74.89% for DSC, 71.83% for AUC, and 76.60% for F1-Score), and 50.88% for validation cohort 2 (68.03% for accuracy, 59.02% for PPV, 66.87% for TPR, 78.48% for specificity, 67.44% for DSC, 64.68% for AUC, and 62.70% for F1-Score).

Conclusions

The models exhibited satisfactory results, paving the way for their potential deployment on standard computers and integration with other artificial intelligence models in clinical practice in the future. However, limited to the size of study, the generalizability of models is impaired in the external validation, larger pathological section cohort would be needed in future development to ensure robustness and generalization.

Research for accurate auxiliary diagnosis of lung cancer based on intracellular fluorescent fingerprint information

The distinctions in pathological types and genetic subtypes of lung cancer have a direct impact on the choice of treatment choices and clinical prognosis in clinical practice. This study used pathological histological sections of surgically removed or biopsied tumor tissue from 36 patients. Based on a small sample size, millions of spectral data points were extracted to investigate the feasibility of employing intracellular fluorescent fingerprint information to diagnose the pathological types and mutational status of lung cancer. The intracellular fluorescent fingerprint information revealed the EGFR gene mutation characteristics in lung cancer, and the area under the curve (AUC) value for the optimal model was 0.98. For the classification of lung cancer pathological types, the macro average AUC value for the ensemble-learning model was 0.97. Our research contributes new idea for pathological diagnosis of lung cancer and offers a quick, easy, and accurate auxiliary diagnostic approach.

Advances in the application of computational pathology in diagnosis, immunomicroenvironment recognition, and immunotherapy evaluation of breast cancer: a narrative review

BACKGROUND

Breast cancer (BC) is a prevalent and highly lethal malignancy affecting women worldwide. Immunotherapy has emerged as a promising therapeutic strategy for BC, offering potential improvements in patient survival. Neoadjuvant therapy (NAT) has also gained significant clinical traction. With the advancement of computer technology, Artificial Intelligence (AI) has been increasingly applied in pathology research, expanding and redefining the scope of the field. This narrative review aims to provide a comprehensive overview of the current literature on the application of computational pathology in BC, specifically focusing on diagnosis, immune microenvironment recognition, and the evaluation of immunotherapy and NAT response.

METHODS

A thorough examination of relevant literature was conducted, focusing on studies investigating the role of computational pathology in BC diagnosis, immune microenvironment recognition, and immunotherapy and NAT assessment.

RESULTS

The application of computational pathology has shown significant potential in BC management. AI-based techniques enable improved diagnosis and classification of BC subtypes, enhance the identification and characterization of the immune microenvironment, and facilitate the evaluation of immunotherapy and NAT response. However, challenges related to data quality, standardization, and algorithm development still need to be addressed.

CONCLUSION

The integration of computational pathology and AI has transformative implications for BC patient care. By leveraging AI-based technologies, clinicians can make more informed decisions in diagnosis, treatment planning, and therapeutic response assessment. Future research should focus on refining AI algorithms, addressing technical challenges, and conducting large-scale clinical validation studies to facilitate the translation of computational pathology into routine clinical practice for BC patients.