Quantitative nuclear histomorphometry predicts oncotype DX risk categories for early stage ER+ breast cancer

Role of artificial intelligence in digital pathology for gynecological cancers

The diagnosis of cancer is typically based on histopathological sections or biopsies on glass slides. Artificial intelligence (AI) approaches have greatly enhanced our ability to extract quantitative information from digital histopathology images as a rapid growth in oncology data. Gynecological cancers are major diseases affecting women's health worldwide. They are characterized by high mortality and poor prognosis, underscoring the critical importance of early detection, treatment, and identification of prognostic factors. This review highlights the various clinical applications of AI in gynecological cancers using digitized histopathology slides. Particularly, deep learning models have shown promise in accurately diagnosing, classifying histopathological subtypes, and predicting treatment response and prognosis. Furthermore, the integration with transcriptomics, proteomics, and other multi-omics techniques can provide valuable insights into the molecular features of diseases. Despite the considerable potential of AI, substantial challenges remain. Further improvements in data acquisition and model optimization are required, and the exploration of broader clinical applications, such as the biomarker discovery, need to be explored.

Decoding pathology: the role of computational pathology in research and diagnostics

Traditional histopathology, characterized by manual quantifications and assessments, faces challenges such as low-throughput and inter-observer variability that hinder the introduction of precision medicine in pathology diagnostics and research. The advent of digital pathology allowed the introduction of computational pathology, a discipline that leverages computational methods, especially based on deep learning (DL) techniques, to analyze histopathology specimens. A growing body of research shows impressive performances of DL-based models in pathology for a multitude of tasks, such as mutation prediction, large-scale pathomics analyses, or prognosis prediction. New approaches integrate multimodal data sources and increasingly rely on multi-purpose foundation models. This review provides an introductory overview of advancements in computational pathology and discusses their implications for the future of histopathology in research and diagnostics.

PatchSorter: a high throughput deep learning digital pathology tool for object labeling

The discovery of patterns associated with diagnosis, prognosis, and therapy response in digital pathology images often requires intractable labeling of large quantities of histological objects. Here we release an open-source labeling tool, PatchSorter, which integrates deep learning with an intuitive web interface. Using >100,000 objects, we demonstrate a >7x improvement in labels per second over unaided labeling, with minimal impact on labeling accuracy, thus enabling high-throughput labeling of large datasets.

AI powered quantification of nuclear morphology in cancers enables prediction of genome instability and prognosis

While alterations in nucleus size, shape, and color are ubiquitous in cancer, comprehensive quantification of nuclear morphology across a whole-slide histologic image remains a challenge. Here, we describe the development of a pan-tissue, deep learning-based digital pathology pipeline for exhaustive nucleus detection, segmentation, and classification and the utility of this pipeline for nuclear morphologic biomarker discovery. Manually-collected nucleus annotations were used to train an object detection and segmentation model for identifying nuclei, which was deployed to segment nuclei in H&E-stained slides from the BRCA, LUAD, and PRAD TCGA cohorts. Interpretable features describing the shape, size, color, and texture of each nucleus were extracted from segmented nuclei and compared to measurements of genomic instability, gene expression, and prognosis. The nuclear segmentation and classification model trained herein performed comparably to previously reported models. Features extracted from the model revealed differences sufficient to distinguish between BRCA, LUAD, and PRAD. Furthermore, cancer cell nuclear area was associated with increased aneuploidy score and homologous recombination deficiency. In BRCA, increased fibroblast nuclear area was indicative of poor progression-free and overall survival and was associated with gene expression signatures related to extracellular matrix remodeling and anti-tumor immunity. Thus, we developed a powerful pan-tissue approach for nucleus segmentation and featurization, enabling the construction of predictive models and the identification of features linking nuclear morphology with clinically-relevant prognostic biomarkers across multiple cancer types.

Artificial intelligence in digital histopathology for predicting patient prognosis and treatment efficacy in breast cancer

INTRODUCTION

Histological images contain phenotypic information predictive of patient outcomes. Due to the heavy workload of pathologists, the time-consuming nature of quantitatively assessing histological features, and human eye limitations to recognize spatial patterns, manually extracting prognostic information in routine pathological workflows remains challenging. Digital pathology has facilitated the mining and quantification of these features utilizing whole-slide image (WSI) scanners and artificial intelligence (AI) algorithms. AI algorithms to identify image-based biomarkers from the tumor microenvironment (TME) have the potential to revolutionize the field of oncology, reducing delays between diagnosis and prognosis determination, allowing for rapid stratification of patients and prescription of optimal treatment regimes, thereby improving patient outcomes.

AREAS COVERED

In this review, the authors discuss how AI algorithms and digital pathology can predict breast cancer patient prognosis and treatment outcomes using image-based biomarkers, along with the challenges of adopting this technology in clinical settings.

EXPERT OPINION

The integration of AI and digital pathology presents significant potential for analyzing the TME and its diagnostic, prognostic, and predictive value in breast cancer patients. Widespread clinical adoption of AI faces ethical, regulatory, and technical challenges, although prospective trials may offer reassurance and promote uptake, ultimately improving patient outcomes by reducing diagnosis-to-prognosis delivery delays.

Prediction of immunochemotherapy response for diffuse large B-cell lymphoma using artificial intelligence digital pathology

Diffuse large B-cell lymphoma (DLBCL) is a heterogeneous and prevalent subtype of aggressive non-Hodgkin lymphoma that poses diagnostic and prognostic challenges, particularly in predicting drug responsiveness. In this study, we used digital pathology and deep learning to predict responses to immunochemotherapy in patients with DLBCL. We retrospectively collected 251 slide images from 216 DLBCL patients treated with rituximab, cyclophosphamide, doxorubicin, vincristine, and prednisone (R-CHOP), with their immunochemotherapy response labels. The digital pathology images were processed using contrastive learning for feature extraction. A multi-modal prediction model was developed by integrating clinical data and pathology image features. Knowledge distillation was employed to mitigate overfitting on gigapixel histopathology images to create a model that predicts responses based solely on pathology images. Based on the importance derived from the attention mechanism of the model, we extracted histological features that were considered key textures associated with drug responsiveness. The multi-modal prediction model achieved an impressive area under the ROC curve of 0.856, demonstrating significant associations with clinical variables such as Ann Arbor stage, International Prognostic Index, and bulky disease. Survival analyses indicated their effectiveness in predicting relapse-free survival. External validation using TCGA datasets supported the model's ability to predict survival differences. Additionally, pathology-based predictions show promise as independent prognostic indicators. Histopathological analysis identified centroblastic and immunoblastic features to be associated with treatment response, aligning with previous morphological classifications and highlighting the objectivity and reproducibility of artificial intelligence-based diagnosis. This study introduces a novel approach that combines digital pathology and clinical data to predict the response to immunochemotherapy in patients with DLBCL. This model shows great promise as a diagnostic and prognostic tool for clinical management of DLBCL. Further research and genomic data integration hold the potential to enhance its impact on clinical practice, ultimately improving patient outcomes.

Development of an Artificial-Intelligence-Based Tool for Automated Assessment of Cellularity in Bone Marrow Biopsies in Ph-Negative Myeloproliferative Neoplasms

The cellularity assessment in bone marrow biopsies (BMBs) for the diagnosis of Philadelphia chromosome (Ph)-negative myeloproliferative neoplasms (MPNs) is a key diagnostic feature and is usually performed by the human eyes through an optical microscope with consequent inter-observer and intra-observer variability. Thus, the use of an automated tool may reduce variability, improving the uniformity of the evaluation. The aim of this work is to develop an accurate AI-based tool for the automated quantification of cellularity in BMB histology. A total of 55 BMB histological slides, diagnosed as Ph- MPN between January 2018 and June 2023 from the archives of the Pathology Unit of University "Luigi Vanvitelli" in Naples (Italy), were scanned on Ventana DP200 or Epredia P1000 and exported as whole-slide images (WSIs). Fifteen BMBs were randomly selected to obtain a training set of AI-based tools. An expert pathologist and a trained resident performed annotations of hematopoietic tissue and adipose tissue, and annotations were exported as .tiff images and .png labels with two colors (black for hematopoietic tissue and yellow for adipose tissue). Subsequently, we developed a semantic segmentation model for hematopoietic tissue and adipose tissue. The remaining 40 BMBs were used for model verification. The performance of our model was compared with an evaluation of the cellularity of five expert hematopathologists and three trainees; we obtained an optimal concordance between our model and the expert pathologists' evaluation, with poorer concordance for trainees. There were no significant differences in cellularity assessments between two different scanners.

A nomogram prediction model of pseudomyxoma peritonei established based on new prognostic factors of HE stained pathological images analysis

BACKGROUND

Pseudomyxoma peritonei (PMP) is a rare clinical malignant syndrome, and its rarity causes a lack of pathology research. This study aims to quantitatively analyze HE-stained pathological images (PIs), and develop a new predictive model integrating digital pathological parameters with clinical information.

METHODS

Ninety-two PMP patients with complete clinic-pathological information, were included. QuPath was used for PIs quantitative feature analysis at tissue-, cell-, and nucleus-level. The correlations between overall survival (OS) and general clinicopathological characteristics, and PIs features were analyzed. A nomogram was established based on independent prognostic factors and evaluated.

RESULTS

Among the 92 PMP patients, there were 34 (37.0%) females and 58 (63.0%) males, with a median age of 57 (range: 31-76). A total of 449 HE stained images were obtained for QuPath analysis, which extracted 40 pathological parameters at three levels. Kaplan-Meier survival analysis revealed eight clinicopathological characteristics and 20 PIs features significantly associated with OS (p < 0.05). Partial least squares regression was used to screen the multicollinearity features and synthesize four new features. Multivariate survival analysis identified the following five independent prognostic factors: preoperative CA199, completeness of cytoreduction, histopathological type, component one at tissue-level, and tumor nuclei circularity variance. A nomogram was established with internal validation C-index 0.795 and calibration plots indicating improved prediction performance.

CONCLUSIONS

The quantitative analysis of HE-stained PIs could extract the new prognostic information on PMP. A nomogram established by five independent prognosticators is the first model integrating digital pathological information with clinical data for improved clinical outcome prediction.

Artificial intelligence's impact on breast cancer pathology: a literature review

This review discusses the profound impact of artificial intelligence (AI) on breast cancer (BC) diagnosis and management within the field of pathology. It examines the various applications of AI across diverse aspects of BC pathology, highlighting key findings from multiple studies. Integrating AI into routine pathology practice stands to improve diagnostic accuracy, thereby contributing to reducing avoidable errors. Additionally, AI has excelled in identifying invasive breast tumors and lymph node metastasis through its capacity to process large whole-slide images adeptly. Adaptive sampling techniques and powerful convolutional neural networks mark these achievements. The evaluation of hormonal status, which is imperative for BC treatment choices, has also been enhanced by AI quantitative analysis, aiding interobserver concordance and reliability. Breast cancer grading and mitotic count evaluation also benefit from AI intervention. AI-based frameworks effectively classify breast carcinomas, even for moderately graded cases that traditional methods struggle with. Moreover, AI-assisted mitotic figures quantification surpasses manual counting in precision and sensitivity, fostering improved prognosis. The assessment of tumor-infiltrating lymphocytes in triple-negative breast cancer using AI yields insights into patient survival prognosis. Furthermore, AI-powered predictions of neoadjuvant chemotherapy response demonstrate potential for streamlining treatment strategies. Addressing limitations, such as preanalytical variables, annotation demands, and differentiation challenges, is pivotal for realizing AI's full potential in BC pathology. Despite the existing hurdles, AI's multifaceted contributions to BC pathology hold great promise, providing enhanced accuracy, efficiency, and standardization. Continued research and innovation are crucial for overcoming obstacles and fully harnessing AI's transformative capabilities in breast cancer diagnosis and assessment.

Machine learning-based models for the prediction of breast cancer recurrence risk

Breast cancer is the most common malignancy diagnosed in women worldwide. The prevalence and incidence of breast cancer is increasing every year; therefore, early diagnosis along with suitable relapse detection is an important strategy for prognosis improvement. This study aimed to compare different machine algorithms to select the best model for predicting breast cancer recurrence. The prediction model was developed by using eleven different machine learning (ML) algorithms, including logistic regression (LR), random forest (RF), support vector classification (SVC), extreme gradient boosting (XGBoost), gradient boosting decision tree (GBDT), decision tree, multilayer perceptron (MLP), linear discriminant analysis (LDA), adaptive boosting (AdaBoost), Gaussian naive Bayes (GaussianNB), and light gradient boosting machine (LightGBM), to predict breast cancer recurrence. The area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and F1 score were used to evaluate the performance of the prognostic model. Based on performance, the optimal ML was selected, and feature importance was ranked by Shapley Additive Explanation (SHAP) values. Compared to the other 10 algorithms, the results showed that the AdaBoost algorithm had the best prediction performance for successfully predicting breast cancer recurrence and was adopted in the establishment of the prediction model. Moreover, CA125, CEA, Fbg, and tumor diameter were found to be the most important features in our dataset to predict breast cancer recurrence. More importantly, our study is the first to use the SHAP method to improve the interpretability of clinicians to predict the recurrence model of breast cancer based on the AdaBoost algorithm. The AdaBoost algorithm offers a clinical decision support model and successfully identifies the recurrence of breast cancer.

AI-enabled routine H&E image based prognostic marker for early-stage luminal breast cancer

Breast cancer (BC) grade is a well-established subjective prognostic indicator of tumour aggressiveness. Tumour heterogeneity and subjective assessment result in high degree of variability among observers in BC grading. Here we propose an objective Haematoxylin & Eosin (H&E) image-based prognostic marker for early-stage luminal/Her2-negative BReAst CancEr that we term as the BRACE marker. The proposed BRACE marker is derived from AI based assessment of heterogeneity in BC at a detailed level using the power of deep learning. The prognostic ability of the marker is validated in two well-annotated cohorts (Cohort-A/Nottingham: n = 2122 and Cohort-B/Coventry: n = 311) on early-stage luminal/HER2-negative BC patients treated with endocrine therapy and with long-term follow-up. The BRACE marker is able to stratify patients for both distant metastasis free survival (p = 0.001, C-index: 0.73) and BC specific survival (p < 0.0001, C-index: 0.84) showing comparable prediction accuracy to Nottingham Prognostic Index and Magee scores, which are both derived from manual histopathological assessment, to identify luminal BC patients that may be likely to benefit from adjuvant chemotherapy.

Predicting early breast cancer recurrence from histopathological images in the Carolina Breast Cancer Study

Approaches for rapidly identifying patients at high risk of early breast cancer recurrence are needed. Image-based methods for prescreening hematoxylin and eosin (H&E) stained tumor slides could offer temporal and financial efficiency. We evaluated a data set of 704 1-mm tumor core H&E images (2-4 cores per case), corresponding to 202 participants (101 who recurred; 101 non-recurrent matched on age and follow-up time) from breast cancers diagnosed between 2008-2012 in the Carolina Breast Cancer Study. We leveraged deep learning to extract image information and trained a model to identify recurrence. Cross-validation accuracy for predicting recurrence was 62.4% [95% CI: 55.7, 69.1], similar to grade (65.8% [95% CI: 59.3, 72.3]) and ER status (66.3% [95% CI: 59.8, 72.8]). Interestingly, 70% (19/27) of early-recurrent low-intermediate grade tumors were identified by our image model. Relative to existing markers, image-based analyses provide complementary information for predicting early recurrence.

Future Practices of Breast Pathology Using Digital and Computational Pathology

Pathology clinical practice has evolved by adopting technological advancements initially regarded as potentially disruptive, such as electron microscopy, immunohistochemistry, and genomic sequencing. Breast pathology has a critical role as a medical domain, where the patient's pathology diagnosis has significant implications for prognostication and treatment of diseases. The advent of digital and computational pathology has brought about significant advancements in the field, offering new possibilities for enhancing diagnostic accuracy and improving patient care. Digital slide scanning enables to conversion of glass slides into high-fidelity digital images, supporting the review of cases in a digital workflow. Digitization offers the capability to render specimen diagnoses, digital archival of patient specimens, collaboration, and telepathology. Integration of image analysis and machine learning-based systems layered atop the high-resolution digital images offers novel workflows to assist breast pathologists in their clinical, educational, and research endeavors. Decision support tools may improve the detection and classification of breast lesions and the quantification of immunohistochemical studies. Computational biomarkers may help to contribute to patient management or outcomes. Furthermore, using digital and computational pathology may increase standardization and quality assurance, especially in areas with high interobserver variability. This review explores the current landscape and possible future applications of digital and computational techniques in the field of breast pathology.

Artificial intelligence in diagnostic pathology

Digital pathology (DP) is being increasingly employed in cancer diagnostics, providing additional tools for faster, higher-quality, accurate diagnosis. The practice of diagnostic pathology has gone through a staggering transformation wherein new tools such as digital imaging, advanced artificial intelligence (AI) algorithms, and computer-aided diagnostic techniques are being used for assisting, augmenting and empowering the computational histopathology and AI-enabled diagnostics. This is paving the way for advancement in precision medicine in cancer. Automated whole slide imaging (WSI) scanners are now rendering diagnostic quality, high-resolution images of entire glass slides and combining these images with innovative digital pathology tools is making it possible to integrate imaging into all aspects of pathology reporting including anatomical, clinical, and molecular pathology. The recent approvals of WSI scanners for primary diagnosis by the FDA as well as the approval of prostate AI algorithm has paved the way for starting to incorporate this exciting technology for use in primary diagnosis. AI tools can provide a unique platform for innovations and advances in anatomical and clinical pathology workflows. In this review, we describe the milestones and landmark trials in the use of AI in clinical pathology with emphasis on future directions.

Artificial Intelligence in Breast Cancer Diagnosis and Personalized Medicine

Breast cancer is a significant cause of cancer-related mortality in women worldwide. Early and precise diagnosis is crucial, and clinical outcomes can be markedly enhanced. The rise of artificial intelligence (AI) has ushered in a new era, notably in image analysis, paving the way for major advancements in breast cancer diagnosis and individualized treatment regimens. In the diagnostic workflow for patients with breast cancer, the role of AI encompasses screening, diagnosis, staging, biomarker evaluation, prognostication, and therapeutic response prediction. Although its potential is immense, its complete integration into clinical practice is challenging. Particularly, these challenges include the imperatives for extensive clinical validation, model generalizability, navigating the "black-box" conundrum, and pragmatic considerations of embedding AI into everyday clinical environments. In this review, we comprehensively explored the diverse applications of AI in breast cancer care, underlining its transformative promise and existing impediments. In radiology, we specifically address AI in mammography, tomosynthesis, risk prediction models, and supplementary imaging methods, including magnetic resonance imaging and ultrasound. In pathology, our focus is on AI applications for pathologic diagnosis, evaluation of biomarkers, and predictions related to genetic alterations, treatment response, and prognosis in the context of breast cancer diagnosis and treatment. Our discussion underscores the transformative potential of AI in breast cancer management and emphasizes the importance of focused research to realize the full spectrum of benefits of AI in patient care.

Applications of Artificial Intelligence in Breast Pathology

CONTEXT.—

Increasing implementation of whole slide imaging together with digital workflow and advances in computing capacity enable the use of artificial intelligence (AI) in pathology, including breast pathology. Breast pathologists often face a significant workload, with diagnosis complexity, tedious repetitive tasks, and semiquantitative evaluation of biomarkers. Recent advances in developing AI algorithms have provided promising approaches to meet the demand in breast pathology.

OBJECTIVE.—

To provide an updated review of AI in breast pathology. We examined the success and challenges of current and potential AI applications in diagnosing and grading breast carcinomas and other pathologic changes, detecting lymph node metastasis, quantifying breast cancer biomarkers, predicting prognosis and therapy response, and predicting potential molecular changes.

DATA SOURCES.—

We obtained data and information by searching and reviewing literature on AI in breast pathology from PubMed and based our own experience.

CONCLUSIONS.—

With the increasing application in breast pathology, AI not only assists in pathology diagnosis to improve accuracy and reduce pathologists' workload, but also provides new information in predicting prognosis and therapy response.

Deep Learning-Based Computational Cytopathologic Diagnosis of Metastatic Breast Carcinoma in Pleural Fluid

A Pleural effusion cytology is vital for treating metastatic breast cancer; however, concerns have arisen regarding the low accuracy and inter-observer variability in cytologic diagnosis. Although artificial intelligence-based image analysis has shown promise in cytopathology research, its application in diagnosing breast cancer in pleural fluid remains unexplored. To overcome these limitations, we evaluate the diagnostic accuracy of an artificial intelligence-based model using a large collection of cytopathological slides, to detect the malignant pleural effusion cytology associated with breast cancer. This study includes a total of 569 cytological slides of malignant pleural effusion of metastatic breast cancer from various institutions. We extracted 34,221 augmented image patches from whole-slide images and trained and validated a deep convolutional neural network model (DCNN) (Inception-ResNet-V2) with the images. Using this model, we classified 845 randomly selected patches, which were reviewed by three pathologists to compare their accuracy. The DCNN model outperforms the pathologists by demonstrating higher accuracy, sensitivity, and specificity compared to the pathologists (81.1% vs. 68.7%, 95.0% vs. 72.5%, and 98.6% vs. 88.9%, respectively). The pathologists reviewed the discordant cases of DCNN. After re-examination, the average accuracy, sensitivity, and specificity of the pathologists improved to 87.9, 80.2, and 95.7%, respectively. This study shows that DCNN can accurately diagnose malignant pleural effusion cytology in breast cancer and has the potential to support pathologists.

Computational pathology improves risk stratification of a multi-gene assay for early stage ER+ breast cancer

Prognostic markers currently utilized in clinical practice for estrogen receptor-positive (ER+) and lymph node-negative (LN-) invasive breast cancer (IBC) patients include the Nottingham grading system and Oncotype Dx (ODx). However, these biomarkers are not always optimal and remain subject to inter-/intra-observer variability and high cost. In this study, we evaluated the association between computationally derived image features from H&E images and disease-free survival (DFS) in ER+ and LN- IBC. H&E images from a total of n = 321 patients with ER+ and LN- IBC from three cohorts were employed for this study (Training set: D1 (n = 116), Validation sets: D2 (n = 121) and D3 (n = 84)). A total of 343 features relating to nuclear morphology, mitotic activity, and tubule formation were computationally extracted from each slide image. A Cox regression model (IbRiS) was trained to identify significant predictors of DFS and predict a high/low-risk category using D1 and was validated on independent testing sets D2 and D3 as well as within each ODx risk category. IbRiS was significantly prognostic of DFS with a hazard ratio (HR) of 2.33 (95% confidence interval (95% CI) = 1.02-5.32, p = 0.045) on D2 and a HR of 2.94 (95% CI = 1.18-7.35, p = 0.0208) on D3. In addition, IbRiS yielded significant risk stratification within high ODx risk categories (D1 + D2: HR = 10.35, 95% CI = 1.20-89.18, p = 0.0106; D1: p = 0.0238; D2: p = 0.0389), potentially providing more granular risk stratification than offered by ODx alone.

Integration of clinical features and deep learning on pathology for the prediction of breast cancer recurrence assays and risk of recurrence

Gene expression-based recurrence assays are strongly recommended to guide the use of chemotherapy in hormone receptor-positive, HER2-negative breast cancer, but such testing is expensive, can contribute to delays in care, and may not be available in low-resource settings. Here, we describe the training and independent validation of a deep learning model that predicts recurrence assay result and risk of recurrence using both digital histology and clinical risk factors. We demonstrate that this approach outperforms an established clinical nomogram (area under the receiver operating characteristic curve of 0.83 versus 0.76 in an external validation cohort, p = 0.0005) and can identify a subset of patients with excellent prognoses who may not need further genomic testing.

Biological insights and novel biomarker discovery through deep learning approaches in breast cancer histopathology

Breast cancer remains a highly prevalent disease with considerable inter- and intra-tumoral heterogeneity complicating prognostication and treatment decisions. The utilization and depth of genomic, transcriptomic and proteomic data for cancer has exploded over recent times and the addition of spatial context to this information, by understanding the correlating morphologic and spatial patterns of cells in tissue samples, has created an exciting frontier of research, histo-genomics. At the same time, deep learning (DL), a class of machine learning algorithms employing artificial neural networks, has rapidly progressed in the last decade with a confluence of technical developments - including the advent of modern graphic processing units (GPU), allowing efficient implementation of increasingly complex architectures at scale; advances in the theoretical and practical design of network architectures; and access to larger datasets for training - all leading to sweeping advances in image classification and object detection. In this review, we examine recent developments in the application of DL in breast cancer histology with particular emphasis of those producing biologic insights or novel biomarkers, spanning the extraction of genomic information to the use of stroma to predict cancer recurrence, with the aim of suggesting avenues for further advancing this exciting field.

Artificial intelligence in breast cancer histopathology

This is a review on the use of artificial intelligence for digital breast pathology. A systematic search on PubMed was conducted, identifying 17,324 research papers related to breast cancer pathology. Following a semimanual screening, 664 papers were retrieved and pursued. The papers are grouped into six major tasks performed by pathologists-namely, molecular and hormonal analysis, grading, mitotic figure counting, ki-67 indexing, tumour-infiltrating lymphocyte assessment, and lymph node metastases identification. Under each task, open-source datasets for research to build artificial intelligence (AI) tools are also listed. Many AI tools showed promise and demonstrated feasibility in the automation of routine pathology investigations. We expect continued growth of AI in this field as new algorithms mature.

Validation and real-world clinical application of an artificial intelligence algorithm for breast cancer detection in biopsies

Breast cancer is the most common malignant disease worldwide, with over 2.26 million new cases in 2020. Its diagnosis is determined by a histological review of breast biopsy specimens, which can be labor-intensive, subjective, and error-prone. Artificial Intelligence (AI)-based tools can support cancer detection and classification in breast biopsies ensuring rapid, accurate, and objective diagnosis. We present here the development, external clinical validation, and deployment in routine use of an AI-based quality control solution for breast biopsy review. The underlying AI algorithm is trained to identify 51 different types of clinical and morphological features, and it achieves very high accuracy in a large, multi-site validation study. Specifically, the area under the receiver operating characteristic curves (AUC) for the detection of invasive carcinoma and of ductal carcinoma in situ (DCIS) are 0.99 (specificity and sensitivity of 93.57 and 95.51%, respectively) and 0.98 (specificity and sensitivity of 93.79 and 93.20% respectively), respectively. The AI algorithm differentiates well between subtypes of invasive and different grades of in situ carcinomas with an AUC of 0.97 for invasive ductal carcinoma (IDC) vs. invasive lobular carcinoma (ILC) and AUC of 0.92 for DCIS high grade vs. low grade/atypical ductal hyperplasia, respectively, as well as accurately identifies stromal tumor-infiltrating lymphocytes (TILs) with an AUC of 0.965. Deployment of this AI solution as a real-time quality control solution in clinical routine leads to the identification of cancers initially missed by the reviewing pathologist, demonstrating both clinical utility and accuracy in real-world clinical application.

Automatic multi-tissue segmentation in pancreatic pathological images with selected multi-scale attention network

The morphology of tissues in pathological images has been used routinely by pathologists to assess the degree of malignancy of pancreatic ductal adenocarcinoma (PDAC). Automatic and accurate segmentation of tumor cells and their surrounding tissues is often a crucial step to obtain reliable morphological statistics. Nonetheless, it is still a challenge due to the great variation of appearance and morphology. In this paper, a selected multi-scale attention network (SMANet) is proposed to segment tumor cells, blood vessels, nerves, islets and ducts in pancreatic pathological images. The selected multi-scale attention module is proposed to enhance effective information, supplement useful information and suppress redundant information at different scales from the encoder and decoder. It includes selection unit (SU) module and multi-scale attention (MA) module. The selection unit module can effectively filter features. The multi-scale attention module enhances effective information through spatial attention and channel attention, and combines different level features to supplement useful information. This helps learn the information of different receptive fields to improve the segmentation of tumor cells, blood vessels and nerves. An original-feature fusion unit is also proposed to supplement the original image information to reduce the under-segmentation of small tissues such as islets and ducts. The proposed method outperforms state-of-the-arts deep learning algorithms on our PDAC pathological images and achieves competitive results on the GlaS challenge dataset. The mDice and mIoU have reached 0.769 and 0.665 in our PDAC dataset.

Machine Learning Techniques in Predicting BRAF Mutation Status in Cutaneous Melanoma From Clinical and Histopathologic Features

Melanoma is the cutaneous neoplasm responsible for more patient deaths in all countries. BRAF mutations are the most common driver mutation and with the development of molecular targeted therapy, the precise knowledge of BRAF status has become increasingly important. Evaluation of BRAF mutation status has routinely been performed by polymerase chain reaction, a time consuming and expensive technique. Immunohistochemistry has been suggested as a cheaper alternative, but it has not gained general acceptance. A retrospective observational study in a cohort of 106 patients with invasive melanoma was conducted in order to develop and evaluate a machine learning approach to predict BRAF status using clinical and histologic variables. We compared the performance of different common machine learning algorithms and use SHapley Additive exPlanations (SHAP) to explain individual predictions and extract medical insights to define a heuristic model to estimate BRAF mutation probability. The Extreme Gradient Boosting algorithms obtained the best performance. Interpretability of models shows that the most important variables to estimate BRAF mutation probability are: age, Breslow thickness, and Breslow density. Based in this interpretation and medical knowledge, a simplify heuristic model is proposed to predict BRAF status using only 7 variables and obtain a performance of 0.878 of area under the curve. We propose a heuristic model that could be used by clinicians to obtain a good estimator of BRAF mutation probability.

Application of Artificial Intelligence Techniques to Predict Risk of Recurrence of Breast Cancer: A Systematic Review

Breast cancer is the most common disease among women, with over 2.1 million new diagnoses each year worldwide. About 30% of patients initially presenting with early stage disease have a recurrence of cancer within 10 years. Predicting who will have a recurrence and who will not remains challenging, with consequent implications for associated treatment. Artificial intelligence strategies that can predict the risk of recurrence of breast cancer could help breast cancer clinicians avoid ineffective overtreatment. Despite its significance, most breast cancer recurrence datasets are insufficiently large, not publicly available, or imbalanced, making these studies more difficult. This systematic review investigates the role of artificial intelligence in the prediction of breast cancer recurrence. We summarise common techniques, features, training and testing methodologies, metrics, and discuss current challenges relating to implementation in clinical practice. We systematically reviewed works published between 1 January 2011 and 1 November 2021 using the methodology of Kitchenham and Charter. We leveraged Springer, Google Scholar, PubMed, and IEEE search engines. This review found three areas that require further work. First, there is no agreement on artificial intelligence methodologies, feature predictors, or assessment metrics. Second, issues such as sampling strategies, missing data, and class imbalance problems are rarely addressed or discussed. Third, representative datasets for breast cancer recurrence are scarce, which hinders model validation and deployment. We conclude that predicting breast cancer recurrence remains an open problem despite the use of artificial intelligence.

MiRNA expression deregulation correlates with the Oncotype DX® DCIS score

Background

Current clinical criteria do not discriminate well between women who will or those who will not develop ipsilateral invasive breast cancer (IBC), or a DCIS recurrence after a ductal carcinoma in situ (DCIS) diagnosis. The 12-gene Oncotype DX® DCIS assay (RT qPCR gene-based scoring system) was established and shown to predict the risk of subsequent ipsilateral IBC or DCIS recurrence. Recent studies have shown that microRNA (miRNA) expression deregulation can contribute to the development of IBC, but very few have evaluated miRNA deregulation in DCIS lesions. In this study, we sought to determine whether specific miRNA expression changes may correlate with Oncotype DX® DCIS scores.

Methods

For this study, we used archived formalin-fixed, paraffin-embedded (FFPE) specimens from 41 women diagnosed with DCIS between 2012 and 2018. The DCIS lesions were stratified into low (n = 26), intermediate (n = 10), and high (n = 5) risk score groups using the Oncotype DX® DCIS assay. Total RNA was extracted from DCIS lesions by macro-dissection of unstained FFPE sections, and next-generation small-RNA sequencing was performed. We evaluated the correlation between miRNA expression data and Oncotype score, as well as patient age. RT-qPCR validations were performed to validate the topmost differentially expressed miRNAs identified between the different risk score groups.

Results

MiRNA sequencing of 32 FFPE DCIS specimens from the three different risk group scores identified a correlation between expression deregulation of 17 miRNAs and Oncotype scores. Our analyses also revealed a correlation between the expression deregulation of 9 miRNAs and the patient’s age. Based on these results, a total of 15 miRNAs were selected for RT-qPCR validation. Of these, miR-190b (p = 0.043), miR-135a (p = 0.05), miR-205 (p = 0.00056), miR-30c (p = 0.011), and miR-744 (p = 0.038) showed a decreased expression in the intermediate/high Oncotype group when compared to the low-risk score group. A composite risk score was established using these 5 miRNAs and indicated a significant association between miRNA expression deregulation and the Oncotype DX® DCIS Score (p < 0.0021), between high/intermediate and low risk groups.

Conclusions

Our analyses identified a subset of 5 miRNAs able to discriminate between Oncotype DX® DCIS score subgroups. Together, our data suggest that miRNA expression analysis may add value to the predictive and prognostic evaluation of DCIS lesions.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13058-022-01558-4.

Current Developments of Artificial Intelligence in Digital Pathology and Its Future Clinical Applications in Gastrointestinal Cancers

The implementation of DP will revolutionize current practice by providing pathologists with additional tools and algorithms to improve workflow. Furthermore, DP will open up opportunities for development of AI-based tools for more precise and reproducible diagnosis through computational pathology. One of the key features of AI is its capability to generate perceptions and recognize patterns beyond the human senses. Thus, the incorporation of AI into DP can reveal additional morphological features and information. At the current rate of AI development and adoption of DP, the interest in computational pathology is expected to rise in tandem. There have already been promising developments related to AI-based solutions in prostate cancer detection; however, in the GI tract, development of more sophisticated algorithms is required to facilitate histological assessment of GI specimens for early and accurate diagnosis. In this review, we aim to provide an overview of the current histological practices in AP laboratories with respect to challenges faced in image preprocessing, present the existing AI-based algorithms, discuss their limitations and present clinical insight with respect to the application of AI in early detection and diagnosis of GI cancer.

HEROHE Challenge: Predicting HER2 Status in Breast Cancer from Hematoxylin–Eosin Whole-Slide Imaging

Breast cancer is the most common malignancy in women worldwide, and is responsible for more than half a million deaths each year. The appropriate therapy depends on the evaluation of the expression of various biomarkers, such as the human epidermal growth factor receptor 2 (HER2) transmembrane protein, through specialized techniques, such as immunohistochemistry or in situ hybridization. In this work, we present the HER2 on hematoxylin and eosin (HEROHE) challenge, a parallel event of the 16th European Congress on Digital Pathology, which aimed to predict the HER2 status in breast cancer based only on hematoxylin–eosin-stained tissue samples, thus avoiding specialized techniques. The challenge consisted of a large, annotated, whole-slide images dataset (509), specifically collected for the challenge. Models for predicting HER2 status were presented by 21 teams worldwide. The best-performing models are presented by detailing the network architectures and key parameters. Methods are compared and approaches, core methodologies, and software choices contrasted. Different evaluation metrics are discussed, as well as the performance of the presented models for each of these metrics. Potential differences in ranking that would result from different choices of evaluation metrics highlight the need for careful consideration at the time of their selection, as the results show that some metrics may misrepresent the true potential of a model to solve the problem for which it was developed. The HEROHE dataset remains publicly available to promote advances in the field of computational pathology.

Unsupervised Learning Based on Multiple Descriptors for WSIs Diagnosis

An automatic pathological diagnosis is a challenging task because histopathological images with different cellular heterogeneity representations are sometimes limited. To overcome this, we investigated how the holistic and local appearance features with limited information can be fused to enhance the analysis performance. We propose an unsupervised deep learning model for whole-slide image diagnosis, which uses stacked autoencoders simultaneously feeding multiple-image descriptors such as the histogram of oriented gradients and local binary patterns along with the original image to fuse the heterogeneous features. The pre-trained latent vectors are extracted from each autoencoder, and these fused feature representations are utilized for classification. We observed that training with additional descriptors helps the model to overcome the limitations of multiple variants and the intricate cellular structure of histopathology data by various experiments. Our model outperforms existing state-of-the-art approaches by achieving the highest accuracies of 87.2 for ICIAR2018, 94.6 for Dartmouth, and other significant metrics for public benchmark datasets. Our model does not rely on a specific set of pre-trained features based on classifiers to achieve high performance. Unsupervised spaces are learned from the number of independent multiple descriptors and can be used with different variants of classifiers to classify cancer diseases from whole-slide images. Furthermore, we found that the proposed model classifies the types of breast and lung cancer similar to the viewpoint of pathologists by visualization. We also designed our whole-slide image processing toolbox to extract and process the patches from whole-slide images.

Global Increase in Breast Cancer Incidence: Risk Factors and Preventive Measures

Breast cancer is a global cause for concern owing to its high incidence around the world. The alarming increase in breast cancer cases emphasizes the management of disease at multiple levels. The management should start from the beginning that includes stringent cancer screening or cancer registry to effective diagnostic and treatment strategies. Breast cancer is highly heterogeneous at morphology as well as molecular levels and needs different therapeutic regimens based on the molecular subtype. Breast cancer patients with respective subtype have different clinical outcome prognoses. Breast cancer heterogeneity emphasizes the advanced molecular testing that will help on-time diagnosis and improved survival. Emerging fields such as liquid biopsy and artificial intelligence would help to under the complexity of breast cancer disease and decide the therapeutic regimen that helps in breast cancer management. In this review, we have discussed various risk factors and advanced technology available for breast cancer diagnosis to combat the worst breast cancer status and areas that need to be focused for the better management of breast cancer.

Progress on deep learning in digital pathology of breast cancer: a narrative review

BACKGROUND AND OBJECTIVE

Pathology is the gold standard criteria for breast cancer diagnosis and has important guiding value in formulating the clinical treatment plan and predicting the prognosis. However, traditional microscopic examinations of tissue sections are time consuming and labor intensive, with unavoidable subjective variations. Deep learning (DL) can evaluate and extract the most important information from images with less need for human instruction, providing a promising approach to assist in the pathological diagnosis of breast cancer. To provide an informative and up-to-date summary on the topic of DL-based diagnostic systems for breast cancer pathology image analysis and discuss the advantages and challenges to the routine clinical application of digital pathology.

METHODS

A PubMed search with keywords ("breast neoplasm" or "breast cancer") and ("pathology" or "histopathology") and ("artificial intelligence" or "deep learning") was conducted. Relevant publications in English published from January 2000 to October 2021 were screened manually for their title, abstract, and even full text to determine their true relevance. References from the searched articles and other supplementary articles were also studied.

KEY CONTENT AND FINDINGS

DL-based computerized image analysis has obtained impressive achievements in breast cancer pathology diagnosis, classification, grading, staging, and prognostic prediction, providing powerful methods for faster, more reproducible, and more precise diagnoses. However, all artificial intelligence (AI)-assisted pathology diagnostic models are still in the experimental stage. Improving their economic efficiency and clinical adaptability are still required to be developed as the focus of further researches.

CONCLUSIONS

Having searched PubMed and other databases and summarized the application of DL-based AI models in breast cancer pathology, we conclude that DL is undoubtedly a promising tool for assisting pathologists in routines, but further studies are needed to realize the digitization and automation of clinical pathology.

Assessment of deep learning algorithms to predict histopathological diagnosis of breast cancer: first Moroccan prospective study on a private dataset

Objective

Breast cancer is a critical public health issue and a leading cause of cancer-related deaths among women worldwide. Its early diagnosis and detection can effectively help in increasing the chances of survival rate. For this reason, the diagnosis and classification of breast cancer using Deep learning algorithms have attracted a lot of attention. Therefore, our study aimed to design a computational approach based on deep convolutional neural networks for an efficient classification of breast cancer histopathological images by using our own created dataset. We collected overall 328 digital slides, from 116 of surgical breast specimens diagnosed with invasive breast carcinoma of non-specific type, and referred to the histopathology department of the National Institute of Oncology in Rabat, Morocco. We used two models of deep neural network architectures in order to accurately classify the images into one of three categories: normal tissue-benign lesions, in situ carcinoma or invasive carcinoma.

Results

Both Resnet50 and Xception models achieved comparable results, with a small advantage to Xception extracted features. We reported high degrees of overall correct classification accuracy (88%), and sensitivity (95%) for detection of carcinoma cases, which is important for diagnostic pathology workflow in order to assist pathologists for diagnosing breast cancer with precision. The results of the present study showed that the designed classification model has a good generalization performance in predicting diagnosis of breast cancer, in spite of the limited size of the data. To our knowledge, this approach can be highly compared with other common methods in the automated analysis of breast cancer images reported in literature.

Computer extracted features of nuclear morphology in hematoxylin and eosin images distinguish Stage II and IV colon tumors

We assessed the utility of quantitative features of colon cancer nuclei, extracted from digitized hematoxylin and eosin-stained whole slide images (WSIs), to distinguish between Stage II from Stage IV colon cancers. Our discovery cohort comprised 100 Stage II and Stage IV colon cancer cases sourced from the University Hospitals Cleveland Medical Center (UHCMC). We performed initial (independent) model validation on 51 (143) Stage II and 79 (54) Stage IV colon cancer cases from UHCMC (The Cancer Genome Atlas's Colon Adenocarcinoma, TCGA-COAD, cohort). Our approach comprised the following steps, (1) a fully convolutional deep neural network with VGG-18 architecture was trained to locate cancer on WSIs, (2) another deep-learning model based on Mask-RCNN with Resnet-50 architecture was used to segment all nuclei from within the identified cancer region, (3) a total of 26,641 quantitative morphometric features pertaining to nuclear shape, size, and texture were extracted from within and outside tumor nuclei, (4) a random forest classifier was trained to distinguish between Stage II and Stage IV colon cancers using the 5 most discriminatory features selected by the Wilcoxon rank-sum test. Our trained classifier using these top 5 features yielded an AUC of 0.81 and 0.78, respectively, on the held-out cases in UHCMC and TCGA validation sets. For 197 TCGA-COAD cases, the Cox-proportional hazards model yielded a hazard ratio of 2.20 (95% CI: 1.24-3.88) with a concordance index of 0.71 using only top-five features for risk stratification of overall survival. The Kaplan-Meier estimate also showed statistically significant separation between the low-risk and high-risk patients with a log-rank p-value of 0.0097. Finally, unsupervised clustering of the top-five features revealed that Stage IV colon cancers with peritoneal spread were morphologically more similar to Stage II colon cancers with no long-term metastases than Stage IV colon cancers with hematogenous spread. This article is protected by copyright. All rights reserved.

Artificial intelligence applied to breast pathology

The convergence of digital pathology and computer vision is increasingly enabling computers to perform tasks performed by humans. As a result, artificial intelligence (AI) is having an astoundingly positive effect on the field of pathology, including breast pathology. Research using machine learning and the development of algorithms that learn patterns from labeled digital data based on "deep learning" neural networks and feature-engineered approaches to analyze histology images have recently provided promising results. Thus far, image analysis and more complex AI-based tools have demonstrated excellent success performing tasks such as the quantification of breast biomarkers and Ki67, mitosis detection, lymph node metastasis recognition, tissue segmentation for diagnosing breast carcinoma, prognostication, computational assessment of tumor-infiltrating lymphocytes, and prediction of molecular expression as well as treatment response and benefit of therapy from routine H&E images. This review critically examines the literature regarding these applications of AI in the area of breast pathology.

Predicting Axillary Lymph Node Metastasis in Early Breast Cancer Using Deep Learning on Primary Tumor Biopsy Slides

Objectives

To develop and validate a deep learning (DL)-based primary tumor biopsy signature for predicting axillary lymph node (ALN) metastasis preoperatively in early breast cancer (EBC) patients with clinically negative ALN.

Methods

A total of 1,058 EBC patients with pathologically confirmed ALN status were enrolled from May 2010 to August 2020. A DL core-needle biopsy (DL-CNB) model was built on the attention-based multiple instance-learning (AMIL) framework to predict ALN status utilizing the DL features, which were extracted from the cancer areas of digitized whole-slide images (WSIs) of breast CNB specimens annotated by two pathologists. Accuracy, sensitivity, specificity, receiver operating characteristic (ROC) curves, and areas under the ROC curve (AUCs) were analyzed to evaluate our model.

Results

The best-performing DL-CNB model with VGG16_BN as the feature extractor achieved an AUC of 0.816 (95% confidence interval (CI): 0.758, 0.865) in predicting positive ALN metastasis in the independent test cohort. Furthermore, our model incorporating the clinical data, which was called DL-CNB+C, yielded the best accuracy of 0.831 (95%CI: 0.775, 0.878), especially for patients younger than 50 years (AUC: 0.918, 95%CI: 0.825, 0.971). The interpretation of DL-CNB model showed that the top signatures most predictive of ALN metastasis were characterized by the nucleus features including density (p = 0.015), circumference (p = 0.009), circularity (p = 0.010), and orientation (p = 0.012).

Conclusion

Our study provides a novel DL-based biomarker on primary tumor CNB slides to predict the metastatic status of ALN preoperatively for patients with EBC.

Quick Annotator: an open-source digital pathology based rapid image annotation tool

Image-based biomarker discovery typically requires accurate segmentation of histologic structures (e.g. cell nuclei, tubules, and epithelial regions) in digital pathology whole slide images (WSIs). Unfortunately, annotating each structure of interest is laborious and often intractable even in moderately sized cohorts. Here, we present an open-source tool, Quick Annotator (QA), designed to improve annotation efficiency of histologic structures by orders of magnitude. While the user annotates regions of interest (ROIs) via an intuitive web interface, a deep learning (DL) model is concurrently optimized using these annotations and applied to the ROI. The user iteratively reviews DL results to either (1) accept accurately annotated regions or (2) correct erroneously segmented structures to improve subsequent model suggestions, before transitioning to other ROIs. We demonstrate the effectiveness of QA over comparable manual efforts via three use cases. These include annotating (1) 337,386 nuclei in 5 pancreatic WSIs, (2) 5,692 tubules in 10 colorectal WSIs, and (3) 14,187 regions of epithelium in 10 breast WSIs. Efficiency gains in terms of annotations per second of 102×, 9×, and 39× were, respectively, witnessed while retaining f-scores >0.95, suggesting that QA may be a valuable tool for efficiently fully annotating WSIs employed in downstream biomarker studies.

Integrating pathomics with radiomics and genomics for cancer prognosis: A brief review

In the last decade, the focus of computational pathology research community has shifted from replicating the pathological examination for diagnosis done by pathologists to unlocking and discovering "sub-visual" prognostic image cues from the histopathological image. While we are getting more knowledge and experience in digital pathology, the emerging goal is to integrate other-omics or modalities that will contribute for building a better prognostic assay. In this paper, we provide a brief review of representative works that focus on integrating pathomics with radiomics and genomics for cancer prognosis. It includes: correlation of pathomics and genomics; fusion of pathomics and genomics; fusion of pathomics and radiomics. We also present challenges, potential opportunities, and avenues for future work.

Visual Analytics for Hypothesis-Driven Exploration in Computational Pathology

Recent advances in computational and algorithmic power are evolving the field of medical imaging rapidly. In cancer research, many new directions are sought to characterize patients with additional imaging features derived from radiology and pathology images. The emerging field of Computational Pathology targets the high-throughput extraction and analysis of the spatial distribution of cells from digital histopathology images. The associated morphological and architectural features allow researchers to quantify and characterize new imaging biomarkers for cancer diagnosis, prognosis, and treatment decisions. However, while the image feature space grows, exploration and analysis become more difficult and ineffective. There is a need for dedicated interfaces for interactive data manipulation and visual analysis of computational pathology and clinical data. For this purpose, we present IIComPath, a visual analytics approach that enables clinical researchers to formulate hypotheses and create computational pathology pipelines involving cohort construction, spatial analysis of image-derived features, and cohort analysis. We demonstrate our approach through use cases that investigate the prognostic value of current diagnostic features and new computational pathology biomarkers.

Collagen fiber orientation disorder from H&E images is prognostic for early stage breast cancer: clinical trial validation

Collagen fiber organization has been found to be implicated in breast cancer prognosis. In this study, we evaluated whether computerized features of Collagen Fiber Orientation Disorder in Tumor-associated Stroma (CFOD-TS) on Hematoxylin & Eosin (H&E) slide images were prognostic of Disease Free Survival (DFS) in early stage Estrogen Receptor Positive (ER+) Invasive Breast Cancers (IBC). A Cox regression model named M_CFOD-TS, was constructed using cohort S_t (N = 78) to predict DFS based on CFOD-TS features. The prognostic performance of M_CFOD-TS was validated on cohort S_v (N = 219), a prospective clinical trial dataset (ECOG 2197). M_CFOD-TS was prognostic of DFS in both S_t and S_v, independent of clinicopathological variables. Additionally, the molecular pathways regarding cell cycle regulation were identified as being significantly associated with M_CFOD-TS derived risk scores. Our results also found that collagen fiber organization was more ordered in patients with short DFS. Our study provided a H&E image-based pipeline to derive a potential prognostic biomarker for early stage ER+ IBC without the need of special collagen staining or advanced microscopy techniques.

Computerized spermatogenesis staging (CSS) of mouse testis sections via quantitative histomorphological analysis

Spermatogenesis in mammals is a cyclic process of spermatogenic cell development in the seminiferous epithelium that can be subdivided into 12 subsequent stages. Histological staging analysis of testis sections, specifically of seminiferous tubule cross-sections, is the only effective method to evaluate the quality of the spermatogenic process and to determine developmental defects leading to infertility. Such staging analysis, however, is tedious and time-consuming, and it may take a long time to become proficient. We now have developed a Computerized Staging system of Spermatogenesis (CSS) for mouse testis sections through learning of an expert with decades of experience in mouse testis staging. The development of the CSS system comprised three major parts: 1) Developing computational image analysis models for mouse testis sections; 2) Automated classification of each seminiferous tubule cross-section into three stage groups: Early Stages (ES: stages I-V), Middle Stages (MS: stages VI-VIII), and Late Stages (LS: stages IV-XII); 3) Automated classification of MS into distinct stages VI, VII-mVIII, and late VIII based on newly developed histomorphological features. A cohort of 40 H&E stained normal mouse testis sections was built according to three modules where 28 cross-sections were leveraged for developing tubule region segmentation, spermatogenic cells types and multi-concentric-layers segmentation models. The rest of 12 testis cross-sections, approximately 2314 tubules whose stages were manually annotated by two expert testis histologists, served as the basis for developing the CSS system. The CSS system's accuracy of mean and standard deviation (MSD) in identifying ES, MS, and LS were 0.93 ± 0.03, 0.94 ± 0.11, and 0.89 ± 0.05 and 0.85 ± 0.12, 0.88 ± 0.07, and 0.96 ± 0.04 for one with 5 years of experience, respectively. The CSS system's accuracy of MSD in identifying stages VI, VII-mVIII, and late VIII are 0.74 ± 0.03, 0.85 ± 0.04, and 0.78 ± 0.06 and 0.34 ± 0.18, 0.78 ± 0.16, and 0.44 ± 0.25 for one with 5 years of experience, respectively. In terms of time it takes to collect these data, it takes on average 3 hours for a histologist and 1.87 hours for the CSS system to finish evaluating an entire testis section (computed with a PC (I7-6800k 4.0 GHzwith 32GB of RAM & 256G SSD) and a Titan 1080Ti GPU). Therefore, the CSS system is more accurate and faster compared to a human histologist in staging, and further optimization and development will not only lead to a complete staging of all 12 stages of mouse spermatogenesis but also could aid in the future diagnosis of human infertility. Moreover, the top-ranking histomorphological features identified by the CSS classifier are consistent with the primary features used by histologists in discriminating stages VI, VII-mVIII, and late VIII.

Artificial Intelligence & Tissue Biomarkers: Advantages, Risks and Perspectives for Pathology

Tissue Biomarkers are information written in the tissue and used in Pathology to recognize specific subsets of patients with diagnostic, prognostic or predictive purposes, thus representing the key elements of Personalized Medicine. The advent of Artificial Intelligence (AI) promises to further reinforce the role of Pathology in the scenario of Personalized Medicine: AI-based devices are expected to standardize the evaluation of tissue biomarkers and also to discover novel information, which would otherwise be ignored by human review, and use them to make specific predictions. In this review we will present how AI has been used to support Tissue Biomarkers evaluation in the specific field of Pathology, give an insight to the intriguing field of AI-based biomarkers and discuss possible advantages, risk and perspectives for Pathology.

Deriving Polarimetry Feature Parameters to Characterize Microstructural Features in Histological Sections of Breast Tissues

OBJECTIVE

Mueller matrix polarimetry technique has been regarded as a powerful tool for probing the microstructural information of tissues. The multiplying of cells and remodeling of collagen fibers in breast carcinoma tissues have been reported to be related to patient survival and prognosis, and they give rise to observable patterns in hematoxylin and eosin (H&E) sections of typical breast tissues (TBTs) that the pathologist can label as three distinctive pathological features (DPFs)-cell nuclei, aligned collagen, and disorganized collagen. The aim of this paper is to propose a pixel-based extraction approach of polarimetry feature parameters (PFPs) using a linear discriminant analysis (LDA) classifier. These parameters provide quantitative characterization of the three DPFs in four types of TBTs.

METHODS

The LDA-based training method learns to find the most simplified linear combination from polarimetry basis parameters (PBPs) constrained under the accuracy remains constant to characterize the specific microstructural feature quantitatively in TBTs.

RESULTS

We present results from a cohort of 32 clinical patients with analysis of 224 regions-of-interest. The characterization accuracy for PFPs ranges from 0.82 to 0.91.

CONCLUSION

This work demonstrates the ability of PFPs to quantitatively characterize the DPFs in the H&E pathological sections of TBTs.

SIGNIFICANCE

This technique paves the way for automatic and quantitative evaluation of specific microstructural features in histopathological digitalization and computer-aided diagnosis.

Analysis of tumor nuclear features using artificial intelligence to predict response to neoadjuvant chemotherapy in high-risk breast cancer patients

PURPOSE

Neoadjuvant chemotherapy (NAC) is used to treat patients with high-risk breast cancer. The tumor response to NAC can be classified as either a pathological partial response (pPR) or pathological complete response (pCR), defined as complete eradication of invasive tumor cells, with a pCR conferring a significantly lower risk of recurrence. Predicting the response to NAC, however, remains a significant clinical challenge. The objective of this study was to determine if analysis of nuclear features on core biopsies using artificial intelligence (AI) can predict response to NAC.

METHODS

Fifty-eight HER2-positive or triple-negative breast cancer patients were included in this study (pCR n = 37, pPR n = 21). Multiple deep convolutional neural networks were developed to automate tumor detection and nuclear segmentation. Nuclear count, area, and circularity, as well as image-based first- and second-order features including mean pixel intensity and correlation of the gray-level co-occurrence matrix (GLCM-COR) were determined.

RESULTS

In univariate analysis, the pCR group had fewer multifocal/multicentric tumors, higher nuclear intensity, and lower GLCM-COR compared to the pPR group. In multivariate binary logistic regression, tumor multifocality/multicentricity (OR = 0.14, p = 0.012), nuclear intensity (OR = 1.23, p = 0.018), and GLCM-COR (OR = 0.96, p = 0.043) were each independently associated with likelihood of achieving a pCR, and the model was able to successful classify 79% of cases (62% for pPR and 89% for pCR).

CONCLUSION

Analysis of tumor nuclear features using digital pathology/AI can significantly improve models to predict pathological response to NAC.

Peri-tumoural stroma collagen organization of invasive ductal carcinoma assessed by polarized light microscopy differs between OncotypeDX risk group

A commercially available genomic test, OncotypeDX has emerged as a useful postsurgical treatment guide for early stage breast cancer. Despite widespread clinical adoption, there remain logistical issues with its implementation. Collagenous stromal architecture has been shown to hold prognostic value that may complement OncotypeDX. Polarimetric analysis of breast cancer surgical samples allows for the quantification of collagenous stroma abundance and organization. We examine intratumoural collagen abundance and alignment along the tumor-host interface for 45 human samples of invasive ductal carcinoma categorized as low or higher risk by OncotypeDX. Furthermore, we probe the separatory power of collagen alignment patterns to classify unlabeled samples as low or higher OncotypeDX risk group using a linear discriminant (LD) model. No significant difference in mean collagen abundance was found between the two risk groups. However, collagen alignment along the tumor boundary was found to be significantly lower in higher risk samples. The LD model achieved a 71% total accuracy and 81% sensitivity to higher risk samples. Prognostic information extracted from the stromal morphology has potential to complement OncotypeDX as an easy-to-implement prescreening methodology.

Development and evaluation of deep learning-based segmentation of histologic structures in the kidney cortex with multiple histologic stains

The application of deep learning for automated segmentation (delineation of boundaries) of histologic primitives (structures) from whole slide images can facilitate the establishment of novel protocols for kidney biopsy assessment. Here, we developed and validated deep learning networks for the segmentation of histologic structures on kidney biopsies and nephrectomies. For development, we examined 125 biopsies for Minimal Change Disease collected across 29 NEPTUNE enrolling centers along with 459 whole slide images stained with Hematoxylin & Eosin (125), Periodic Acid Schiff (125), Silver (102), and Trichrome (107) divided into training, validation and testing sets (ratio 6:1:3). Histologic structures were manually segmented (30048 total annotations) by five nephropathologists. Twenty deep learning models were trained with optimal digital magnification across the structures and stains. Periodic Acid Schiff-stained whole slide images yielded the best concordance between pathologists and deep learning segmentation across all structures (F-scores: 0.93 for glomerular tufts, 0.94 for glomerular tuft plus Bowman’s capsule, 0.91 for proximal tubules, 0.93 for distal tubular segments, 0.81 for peritubular capillaries, and 0.85 for arteries and afferent arterioles). Optimal digital magnifications were 5X for glomerular tuft/tuft plus Bowman’s capsule, 10X for proximal/distal tubule, arteries and afferent arterioles, and 40X for peritubular capillaries. Silver stained whole slide images yielded the worst deep learning performance. Thus, this largest study to date adapted deep learning for the segmentation of kidney histologic structures across multiple stains and pathology laboratories. All data used for training and testing and a detailed online tutorial will be publicly available.

A deep learning model to predict RNA-Seq expression of tumours from whole slide images

Deep learning methods for digital pathology analysis are an effective way to address multiple clinical questions, from diagnosis to prediction of treatment outcomes. These methods have also been used to predict gene mutations from pathology images, but no comprehensive evaluation of their potential for extracting molecular features from histology slides has yet been performed. We show that HE2RNA, a model based on the integration of multiple data modes, can be trained to systematically predict RNA-Seq profiles from whole-slide images alone, without expert annotation. Through its interpretable design, HE2RNA provides virtual spatialization of gene expression, as validated by CD3- and CD20-staining on an independent dataset. The transcriptomic representation learned by HE2RNA can also be transferred on other datasets, even of small size, to increase prediction performance for specific molecular phenotypes. We illustrate the use of this approach in clinical diagnosis purposes such as the identification of tumors with microsatellite instability.

RNA-sequencing of tumour tissue can provide important diagnostic and prognostic information but this is costly and not routinely performed in all clinical settings. Here, the authors show that whole slide histology slides—part of routine care—can be used to predict RNA-sequencing data and thus reduce the need for additional analyses.

Replication Fork Remodeling and Therapy Escape in DNA Damage Response-Deficient Cancers

Most cancers have lost a critical DNA damage response (DDR) pathway during tumor evolution. These alterations provide a useful explanation for the initial sensitivity of tumors to DNA-targeting chemotherapy. A striking example is dysfunctional homology-directed repair (HDR), e.g., due to inactivating mutations in BRCA1 and BRCA2 genes. Extensive efforts are being made to develop novel targeted therapies exploiting such an HDR defect. Inhibitors of poly(ADP-ribose) polymerase (PARP) are an instructive example of this approach. Despite the success of PARP inhibitors, the presence of primary or acquired therapy resistance remains a major challenge in clinical oncology. To move the field of precision medicine forward, we need to understand the precise mechanisms causing therapy resistance. Using preclinical models, various mechanisms underlying chemotherapy resistance have been identified. Restoration of HDR seems to be a prevalent mechanism but this does not explain resistance in all cases. Interestingly, some factors involved in DNA damage response (DDR) have independent functions in replication fork (RF) biology and their loss causes RF instability and therapy sensitivity. However, in BRCA-deficient tumors, loss of these factors leads to restored stability of RFs and acquired drug resistance. In this review we discuss the recent advances in the field of RF biology and its potential implications for chemotherapy response in DDR-defective cancers. Additionally, we review the role of DNA damage tolerance (DDT) pathways in maintenance of genome integrity and their alterations in cancer. Furthermore, we refer to novel tools that, combined with a better understanding of drug resistance mechanisms, may constitute a great advance in personalized diagnosis and therapeutic strategies for patients with HDR-deficient tumors.

Quantitative Assessment of the Effects of Compression on Deep Learning in Digital Pathology Image Analysis

PURPOSE

Deep learning (DL), a class of approaches involving self-learned discriminative features, is increasingly being applied to digital pathology (DP) images for tasks such as disease identification and segmentation of tissue primitives (eg, nuclei, glands, lymphocytes). One application of DP is in telepathology, which involves digitally transmitting DP slides over the Internet for secondary diagnosis by an expert at a remote location. Unfortunately, the places benefiting most from telepathology often have poor Internet quality, resulting in prohibitive transmission times of DP images. Image compression may help, but the degree to which image compression affects performance of DL algorithms has been largely unexplored.

METHODS

We investigated the effects of image compression on the performance of DL strategies in the context of 3 representative use cases involving segmentation of nuclei (n = 137), segmentation of lymph node metastasis (n = 380), and lymphocyte detection (n = 100). For each use case, test images at various levels of compression (JPEG compression quality score ranging from 1-100 and JPEG2000 compression peak signal-to-noise ratio ranging from 18-100 dB) were evaluated by a DL classifier. Performance metrics including F1 score and area under the receiver operating characteristic curve were computed at the various compression levels.

RESULTS

Our results suggest that DP images can be compressed by 85% while still maintaining the performance of the DL algorithms at 95% of what is achievable without any compression. Interestingly, the maximum compression level sustainable by DL algorithms is similar to where pathologists also reported difficulties in providing accurate interpretations.

CONCLUSION

Our findings seem to suggest that in low-resource settings, DP images can be significantly compressed before transmission for DL-based telepathology applications.