1
|
Vahed SZ, Khatibi SMH, Saadat YR, Emdadi M, Khodaei B, Alishani MM, Boostani F, Dizaj SM, Pirmoradi S. Introducing effective genes in lymph node metastasis of breast cancer patients using SHAP values based on the mRNA expression data. PLoS One 2024; 19:e0308531. [PMID: 39150915 PMCID: PMC11329117 DOI: 10.1371/journal.pone.0308531] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2024] [Accepted: 07/24/2024] [Indexed: 08/18/2024] Open
Abstract
OBJECTIVE Breast cancer, a global concern predominantly impacting women, poses a significant threat when not identified early. While survival rates for breast cancer patients are typically favorable, the emergence of regional metastases markedly diminishes survival prospects. Detecting metastases and comprehending their molecular underpinnings are crucial for tailoring effective treatments and improving patient survival outcomes. METHODS Various artificial intelligence methods and techniques were employed in this study to achieve accurate outcomes. Initially, the data was organized and underwent hold-out cross-validation, data cleaning, and normalization. Subsequently, feature selection was conducted using ANOVA and binary Particle Swarm Optimization (PSO). During the analysis phase, the discriminative power of the selected features was evaluated using machine learning classification algorithms. Finally, the selected features were considered, and the SHAP algorithm was utilized to identify the most significant features for enhancing the decoding of dominant molecular mechanisms in lymph node metastases. RESULTS In this study, five main steps were followed for the analysis of mRNA expression data: reading, preprocessing, feature selection, classification, and SHAP algorithm. The RF classifier utilized the candidate mRNAs to differentiate between negative and positive categories with an accuracy of 61% and an AUC of 0.6. During the SHAP process, intriguing relationships between the selected mRNAs and positive/negative lymph node status were discovered. The results indicate that GDF5, BAHCC1, LCN2, FGF14-AS2, and IDH2 are among the top five most impactful mRNAs based on their SHAP values. CONCLUSION The prominent identified mRNAs including GDF5, BAHCC1, LCN2, FGF14-AS2, and IDH2, are implicated in lymph node metastasis. This study holds promise in elucidating a thorough insight into key candidate genes that could significantly impact the early detection and tailored therapeutic strategies for lymph node metastasis in patients with breast cancer.
Collapse
Affiliation(s)
| | - Seyed Mahdi Hosseiniyan Khatibi
- Kidney Research Center, Tabriz University of Medical Sciences, Tabriz, Iran
- Rahat Breath and Sleep Research Center, Tabriz University of Medical Science, Tabriz, Iran
| | | | - Manijeh Emdadi
- Department of Computer Engineering, Abadan Branch, Islamic Azad University, Abadan, Iran
| | - Bahareh Khodaei
- Clinical Research Development Unit of Tabriz Valiasr Hospital, Tabriz University of Medical Sciences, Tabriz, Iran
| | - Mohammad Matin Alishani
- Department of Computer Science, Faculty of Information Technology, University of Shahid Madani of Tabriz, Tabriz, Iran
| | - Farnaz Boostani
- Clinical Research Development Unit of Tabriz Valiasr Hospital, Tabriz University of Medical Sciences, Tabriz, Iran
| | - Solmaz Maleki Dizaj
- Dental and Periodontal Research Center, Tabriz University of Medical Sciences, Tabriz, Iran
| | - Saeed Pirmoradi
- Clinical Research Development Unit of Tabriz Valiasr Hospital, Tabriz University of Medical Sciences, Tabriz, Iran
| |
Collapse
|
2
|
Retamero JA, Gulturk E, Bozkurt A, Liu S, Gorgan M, Moral L, Horton M, Parke A, Malfroid K, Sue J, Rothrock B, Oakley G, DeMuth G, Millar E, Fuchs TJ, Klimstra DS. Artificial Intelligence Helps Pathologists Increase Diagnostic Accuracy and Efficiency in the Detection of Breast Cancer Lymph Node Metastases. Am J Surg Pathol 2024; 48:846-854. [PMID: 38809272 PMCID: PMC11191045 DOI: 10.1097/pas.0000000000002248] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/30/2024]
Abstract
The detection of lymph node metastases is essential for breast cancer staging, although it is a tedious and time-consuming task where the sensitivity of pathologists is suboptimal. Artificial intelligence (AI) can help pathologists detect lymph node metastases, which could help alleviate workload issues. We studied how pathologists' performance varied when aided by AI. An AI algorithm was trained using more than 32 000 breast sentinel lymph node whole slide images (WSIs) matched with their corresponding pathology reports from more than 8000 patients. The algorithm highlighted areas suspicious of harboring metastasis. Three pathologists were asked to review a dataset comprising 167 breast sentinel lymph node WSIs, of which 69 harbored cancer metastases of different sizes, enriched for challenging cases. Ninety-eight slides were benign. The pathologists read the dataset twice, both digitally, with and without AI assistance, randomized for slide and reading orders to reduce bias, separated by a 3-week washout period. Their slide-level diagnosis was recorded, and they were timed during their reads. The average reading time per slide was 129 seconds during the unassisted phase versus 58 seconds during the AI-assisted phase, resulting in an overall efficiency gain of 55% ( P <0.001). These efficiency gains are applied to both benign and malignant WSIs. Two of the 3 reading pathologists experienced significant sensitivity improvements, from 74.5% to 93.5% ( P ≤0.006). This study highlights that AI can help pathologists shorten their reading times by more than half and also improve their metastasis detection rate.
Collapse
Affiliation(s)
| | | | | | - Sandy Liu
- New England Pathology Associates, Springfield, MA
| | - Maria Gorgan
- New England Pathology Associates, Springfield, MA
| | - Luis Moral
- New England Pathology Associates, Springfield, MA
| | | | | | | | - Jill Sue
- Paige.AI. 11 Times Square, New York, NY
| | | | | | | | - Ewan Millar
- Paige.AI. 11 Times Square, New York, NY
- Department of Anatomical Pathology, NSW Health Pathology, St George Hospital, Sydney, NSW, Australia
| | - Thomas J. Fuchs
- Paige.AI. 11 Times Square, New York, NY
- Windreich Department of Artificial Intelligence and Human Health, Icahn School of Medicine at Mount Sinai, New York, NY
- Hasso Platner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai, New York, NY
| | | |
Collapse
|
3
|
van Dooijeweert C, Flach RN, Ter Hoeve ND, Vreuls CPH, Goldschmeding R, Freund JE, Pham P, Nguyen TQ, van der Wall E, Frederix GWJ, Stathonikos N, van Diest PJ. Clinical implementation of artificial-intelligence-assisted detection of breast cancer metastases in sentinel lymph nodes: the CONFIDENT-B single-center, non-randomized clinical trial. NATURE CANCER 2024:10.1038/s43018-024-00788-z. [PMID: 38937624 DOI: 10.1038/s43018-024-00788-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Accepted: 05/29/2024] [Indexed: 06/29/2024]
Abstract
Pathologists' assessment of sentinel lymph nodes (SNs) for breast cancer (BC) metastases is a treatment-guiding yet labor-intensive and costly task because of the performance of immunohistochemistry (IHC) in morphologically negative cases. This non-randomized, single-center clinical trial (International Standard Randomized Controlled Trial Number:14323711) assessed the efficacy of an artificial intelligence (AI)-assisted workflow for detecting BC metastases in SNs while maintaining diagnostic safety standards. From September 2022 to May 2023, 190 SN specimens were consecutively enrolled and allocated biweekly to the intervention arm (n = 100) or control arm (n = 90). In both arms, digital whole-slide images of hematoxylin-eosin sections of SN specimens were assessed by an expert pathologist, who was assisted by the 'Metastasis Detection' app (Visiopharm) in the intervention arm. Our primary endpoint showed a significantly reduced adjusted relative risk of IHC use (0.680, 95% confidence interval: 0.347-0.878) for AI-assisted pathologists, with subsequent cost savings of ~3,000 €. Secondary endpoints showed significant time reductions and up to 30% improved sensitivity for AI-assisted pathologists. This trial demonstrates the safety and potential for cost and time savings of AI assistance.
Collapse
Affiliation(s)
- C van Dooijeweert
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands.
| | - R N Flach
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - N D Ter Hoeve
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - C P H Vreuls
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - R Goldschmeding
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - J E Freund
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - P Pham
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - T Q Nguyen
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - E van der Wall
- Department of Medical Oncology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - G W J Frederix
- Department of Epidemiology and Health Economics, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
| | - N Stathonikos
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - P J van Diest
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands.
| |
Collapse
|
4
|
van Diest PJ, Flach RN, van Dooijeweert C, Makineli S, Breimer GE, Stathonikos N, Pham P, Nguyen TQ, Veta M. Pros and cons of artificial intelligence implementation in diagnostic pathology. Histopathology 2024; 84:924-934. [PMID: 38433288 DOI: 10.1111/his.15153] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Revised: 12/29/2023] [Accepted: 01/19/2024] [Indexed: 03/05/2024]
Abstract
The rapid introduction of digital pathology has greatly facilitated development of artificial intelligence (AI) models in pathology that have shown great promise in assisting morphological diagnostics and quantitation of therapeutic targets. We are now at a tipping point where companies have started to bring algorithms to the market, and questions arise whether the pathology community is ready to implement AI in routine workflow. However, concerns also arise about the use of AI in pathology. This article reviews the pros and cons of introducing AI in diagnostic pathology.
Collapse
Affiliation(s)
- Paul J van Diest
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Rachel N Flach
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
- Department of Oncological Urology, University Medical Center Utrecht, Utrecht, the Netherlands
| | | | - Seher Makineli
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
- Department of Surgical Oncology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Gerben E Breimer
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Nikolas Stathonikos
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Paul Pham
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Tri Q Nguyen
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Mitko Veta
- Department of Oncological Urology, University Medical Center Utrecht, Utrecht, the Netherlands
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands
| |
Collapse
|
5
|
Johnston L, Yu Z. A novel cost function for nuclei segmentation and classification in imbalanced histopathology data-sets. Comput Med Imaging Graph 2023; 109:102296. [PMID: 37797534 DOI: 10.1016/j.compmedimag.2023.102296] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Revised: 09/01/2023] [Accepted: 09/02/2023] [Indexed: 10/07/2023]
Abstract
Cancer is a major global health problem, causing millions of deaths yearly. Histopathological analysis plays a crucial role in detecting and diagnosing various types of cancer, enabling an accurate diagnosis to inform targeted treatment planning, allowing for better cancer staging, and ultimately improving prognosis. We aim to detect cancer earlier, which can ultimately help reduce mortality rates and enhance patients' quality of life. However, detecting and classifying rare cells is a key challenge for pathologists and researchers. Many histopathological data-sets contain imbalanced data, with only a few instances of rare cells whose unique morphological structures can impede early diagnosis efforts. Our model, SPNet, a spatially aware convolutional neural network, addresses this problem by employing a spatial data balancing technique, enhancing the classification of rare nuclei by 21.8 %. Since nuclei often cluster and exhibit patterns of the same class, SPNet's novel cost function targets spatial regions, resulting in a 1.9 % increase in the F1 classification of rare class types within the CoNSeP dataset. When integrated with a ResNet50-SE encoder, SPNet increases the mean F1 score for classifying all nuclei in the CoNSeP dataset by 4.3 %, compared to the benchmark set by the state-of-the-art HoVer-Net model. The potential integration of SPNet into existing medical devices could allow us to streamline diagnostic processes and minimise false negatives.
Collapse
Affiliation(s)
- Luke Johnston
- Department of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Zhangsheng Yu
- Department of Bioinformatics and Biostatistics, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai 200240, China; SJTU-Yale Joint Center for Biostatistics and Data Science Organization, Shanghai Jiao Tong University, Shanghai 200240, China; Clinical Research Institute, Shanghai Jiao Tong University School of Medicine, Shanghai 200025, China; Center for Biomedical Data Science, Translational Science Institute, Shanghai Jiao Tong University, Shanghai 200025, China.
| |
Collapse
|
6
|
Wu S, Hong G, Xu A, Zeng H, Chen X, Wang Y, Luo Y, Wu P, Liu C, Jiang N, Dang Q, Yang C, Liu B, Shen R, Chen Z, Liao C, Lin Z, Wang J, Lin T. Artificial intelligence-based model for lymph node metastases detection on whole slide images in bladder cancer: a retrospective, multicentre, diagnostic study. Lancet Oncol 2023; 24:360-370. [PMID: 36893772 DOI: 10.1016/s1470-2045(23)00061-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2022] [Revised: 02/01/2023] [Accepted: 02/02/2023] [Indexed: 03/08/2023]
Abstract
BACKGROUND Accurate lymph node staging is important for the diagnosis and treatment of patients with bladder cancer. We aimed to develop a lymph node metastases diagnostic model (LNMDM) on whole slide images and to assess the clinical effect of an artificial intelligence-assisted (AI) workflow. METHODS In this retrospective, multicentre, diagnostic study in China, we included consecutive patients with bladder cancer who had radical cystectomy and pelvic lymph node dissection, and from whom whole slide images of lymph node sections were available, for model development. We excluded patients with non-bladder cancer and concurrent surgery, or low-quality images. Patients from two hospitals (Sun Yat-sen Memorial Hospital of Sun Yat-sen University and Zhujiang Hospital of Southern Medical University, Guangzhou, Guangdong, China) were assigned before a cutoff date to a training set and after the date to internal validation sets for each hospital. Patients from three other hospitals (the Third Affiliated Hospital of Sun Yat-sen University, Nanfang Hospital of Southern Medical University, and the Third Affiliated Hospital of Southern Medical University, Guangzhou, Guangdong, China) were included as external validation sets. A validation subset of challenging cases from the five validation sets was used to compare performance between the LNMDM and pathologists, and two other datasets (breast cancer from the CAMELYON16 dataset and prostate cancer from the Sun Yat-sen Memorial Hospital of Sun Yat-sen University) were collected for a multi-cancer test. The primary endpoint was diagnostic sensitivity in the four prespecified groups (ie, the five validation sets, a single-lymph-node test set, the multi-cancer test set, and the subset for a performance comparison between the LNMDM and pathologists). FINDINGS Between Jan 1, 2013 and Dec 31, 2021, 1012 patients with bladder cancer had radical cystectomy and pelvic lymph node dissection and were included (8177 images and 20 954 lymph nodes). We excluded 14 patients (165 images) with concurrent non-bladder cancer and also excluded 21 low-quality images. We included 998 patients and 7991 images (881 [88%] men; 117 [12%] women; median age 64 years [IQR 56-72]; ethnicity data not available; 268 [27%] with lymph node metastases) to develop the LNMDM. The area under the curve (AUC) for accurate diagnosis of the LNMDM ranged from 0·978 (95% CI 0·960-0·996) to 0·998 (0·996-1·000) in the five validation sets. Performance comparisons between the LNMDM and pathologists showed that the diagnostic sensitivity of the model (0·983 [95% CI 0·941-0·998]) substantially exceeded that of both junior pathologists (0·906 [0·871-0·934]) and senior pathologists (0·947 [0·919-0·968]), and that AI assistance improved sensitivity for both junior (from 0·906 without AI to 0·953 with AI) and senior (from 0·947 to 0·986) pathologists. In the multi-cancer test, the LNMDM maintained an AUC of 0·943 (95% CI 0·918-0·969) in breast cancer images and 0·922 (0·884-0·960) in prostate cancer images. In 13 patients, the LNMDM detected tumour micrometastases that had been missed by pathologists who had previously classified these patients' results as negative. Receiver operating characteristic curves showed that the LNMDM would enable pathologists to exclude 80-92% of negative slides while maintaining 100% sensitivity in clinical application. INTERPRETATION We developed an AI-based diagnostic model that did well in detecting lymph node metastases, particularly micrometastases. The LNMDM showed substantial potential for clinical applications in improving the accuracy and efficiency of pathologists' work. FUNDING National Natural Science Foundation of China, the Science and Technology Planning Project of Guangdong Province, the National Key Research and Development Programme of China, and the Guangdong Provincial Clinical Research Centre for Urological Diseases.
Collapse
Affiliation(s)
- Shaoxu Wu
- Department of Urology, Sun Yat-sen University, Guangzhou, Guangdong, China; Guangdong Provincial Key Laboratory of Malignant Tumour Epigenetics and Gene Regulation, Sun Yat-sen University, Guangzhou, Guangdong, China; Guangdong Provincial Clinical Research Centre for Urological Diseases, Guangzhou, Guangdong, China
| | - Guibin Hong
- Department of Urology, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Abai Xu
- Department of Urology, Zhujiang Hospital, Southern Medical University, Guangzhou, Guangdong, China
| | - Hong Zeng
- Department of Pathology, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Xulin Chen
- Cells Vision Medical Technology, Guangzhou, Guangdong, China
| | - Yun Wang
- Department of Urology, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Yun Luo
- Sun Yat-sen Memorial Hospital and Department of Urology, The Third Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Peng Wu
- Department of Urology, Nanfang Hospital, Southern Medical University, Guangzhou, Guangdong, China
| | - Cundong Liu
- Department of Urology, The Third Affiliated Hospital, Southern Medical University, Guangzhou, Guangdong, China
| | - Ning Jiang
- Department of Urology, Zhujiang Hospital, Southern Medical University, Guangzhou, Guangdong, China
| | - Qiang Dang
- Department of Urology, Nanfang Hospital, Southern Medical University, Guangzhou, Guangdong, China
| | - Cheng Yang
- Department of Urology, The Third Affiliated Hospital, Southern Medical University, Guangzhou, Guangdong, China
| | - Bohao Liu
- Sun Yat-sen Memorial Hospital and Department of Urology, The Third Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Runnan Shen
- Department of Urology, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Zeshi Chen
- Department of Urology, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Chengxiao Liao
- Department of Urology, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Zhen Lin
- Cells Vision Medical Technology, Guangzhou, Guangdong, China
| | - Jin Wang
- Cells Vision Medical Technology, Guangzhou, Guangdong, China
| | - Tianxin Lin
- Department of Urology, Sun Yat-sen University, Guangzhou, Guangdong, China; Guangdong Provincial Key Laboratory of Malignant Tumour Epigenetics and Gene Regulation, Sun Yat-sen University, Guangzhou, Guangdong, China; Guangdong Provincial Clinical Research Centre for Urological Diseases, Guangzhou, Guangdong, China.
| |
Collapse
|
7
|
Meroueh C, Chen ZE. Artificial intelligence in anatomical pathology: building a strong foundation for precision medicine. Hum Pathol 2023; 132:31-38. [PMID: 35870567 DOI: 10.1016/j.humpath.2022.07.008] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/06/2022] [Accepted: 07/08/2022] [Indexed: 02/07/2023]
Abstract
With the convergence of digital pathology (DP) and artificial intelligence (AI), anatomic pathology practice has been experiencing an exciting paradigm shifting. Pathologists will be provided with an augmented ability to improve diagnostic accuracy, efficiency, and consistency. There will be subvisual morphometric features discovered and multiomics data integrated to provide better prognostic and theragnostic information to guide individual patients' management. The perspective for future precision medicine is promising. However, there are many challenges before AI-assisted DP diagnostic workflows can be successfully implemented. Herein, we briefly review some examples of AI application in anatomic pathology with an emphasis on the subspecialty of gastrointestinal pathology and discuss potential challenges for clinical implementation.
Collapse
Affiliation(s)
- Chady Meroueh
- Division of Anatomic Pathology, Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, 55905, USA
| | - Zongming Eric Chen
- Division of Anatomic Pathology, Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, 55905, USA.
| |
Collapse
|
8
|
Zhu J, Liu M, Li X. Progress on deep learning in digital pathology of breast cancer: a narrative review. Gland Surg 2022; 11:751-766. [PMID: 35531111 PMCID: PMC9068546 DOI: 10.21037/gs-22-11] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2022] [Accepted: 03/04/2022] [Indexed: 01/26/2024]
Abstract
BACKGROUND AND OBJECTIVE Pathology is the gold standard criteria for breast cancer diagnosis and has important guiding value in formulating the clinical treatment plan and predicting the prognosis. However, traditional microscopic examinations of tissue sections are time consuming and labor intensive, with unavoidable subjective variations. Deep learning (DL) can evaluate and extract the most important information from images with less need for human instruction, providing a promising approach to assist in the pathological diagnosis of breast cancer. To provide an informative and up-to-date summary on the topic of DL-based diagnostic systems for breast cancer pathology image analysis and discuss the advantages and challenges to the routine clinical application of digital pathology. METHODS A PubMed search with keywords ("breast neoplasm" or "breast cancer") and ("pathology" or "histopathology") and ("artificial intelligence" or "deep learning") was conducted. Relevant publications in English published from January 2000 to October 2021 were screened manually for their title, abstract, and even full text to determine their true relevance. References from the searched articles and other supplementary articles were also studied. KEY CONTENT AND FINDINGS DL-based computerized image analysis has obtained impressive achievements in breast cancer pathology diagnosis, classification, grading, staging, and prognostic prediction, providing powerful methods for faster, more reproducible, and more precise diagnoses. However, all artificial intelligence (AI)-assisted pathology diagnostic models are still in the experimental stage. Improving their economic efficiency and clinical adaptability are still required to be developed as the focus of further researches. CONCLUSIONS Having searched PubMed and other databases and summarized the application of DL-based AI models in breast cancer pathology, we conclude that DL is undoubtedly a promising tool for assisting pathologists in routines, but further studies are needed to realize the digitization and automation of clinical pathology.
Collapse
Affiliation(s)
- Jingjin Zhu
- School of Medicine, Nankai University, Tianjin, China
| | - Mei Liu
- Department of Pathology, Chinese People’s Liberation Army General Hospital, Beijing, China
| | - Xiru Li
- Department of General Surgery, Chinese People’s Liberation Army General Hospital, Beijing, China
| |
Collapse
|
9
|
Yousif M, van Diest PJ, Laurinavicius A, Rimm D, van der Laak J, Madabhushi A, Schnitt S, Pantanowitz L. Artificial intelligence applied to breast pathology. Virchows Arch 2021; 480:191-209. [PMID: 34791536 DOI: 10.1007/s00428-021-03213-3] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Revised: 09/12/2021] [Accepted: 09/27/2021] [Indexed: 12/12/2022]
Abstract
The convergence of digital pathology and computer vision is increasingly enabling computers to perform tasks performed by humans. As a result, artificial intelligence (AI) is having an astoundingly positive effect on the field of pathology, including breast pathology. Research using machine learning and the development of algorithms that learn patterns from labeled digital data based on "deep learning" neural networks and feature-engineered approaches to analyze histology images have recently provided promising results. Thus far, image analysis and more complex AI-based tools have demonstrated excellent success performing tasks such as the quantification of breast biomarkers and Ki67, mitosis detection, lymph node metastasis recognition, tissue segmentation for diagnosing breast carcinoma, prognostication, computational assessment of tumor-infiltrating lymphocytes, and prediction of molecular expression as well as treatment response and benefit of therapy from routine H&E images. This review critically examines the literature regarding these applications of AI in the area of breast pathology.
Collapse
Affiliation(s)
- Mustafa Yousif
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA.
- Department of Pathology, Vanderbilt University Medical Center, Nashville, TN, USA.
| | - Paul J van Diest
- Department of Pathology, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Arvydas Laurinavicius
- Department of Pathology, Pharmacology and Forensic Medicine, Faculty of Medicine, Vilnius University, and National Center of Pathology, Affiliate of Vilnius University Hospital Santaros Klinikos, Vilnius, Lithuania
| | - David Rimm
- Department of Pathology, Yale University School of Medicine, New Haven, CT, USA
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Anant Madabhushi
- Department of Biomedical Engineering, Case Western Reserve University, and Louis Stokes Cleveland Veterans Administration Medical Center, Cleveland, OH, USA
- Louis Stokes Cleveland Veterans Administration Medical Center, Cleveland, OH, USA
| | - Stuart Schnitt
- Department of Pathology, Brigham and Women's Hospital and Harvard Medical School, Breast Oncology Program, Dana-Farber/Brigham and Women's Cancer Center, Boston, MA, USA
| | | |
Collapse
|
10
|
Diao JA, Wang JK, Chui WF, Mountain V, Gullapally SC, Srinivasan R, Mitchell RN, Glass B, Hoffman S, Rao SK, Maheshwari C, Lahiri A, Prakash A, McLoughlin R, Kerner JK, Resnick MB, Montalto MC, Khosla A, Wapinski IN, Beck AH, Elliott HL, Taylor-Weiner A. Human-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes. Nat Commun 2021; 12:1613. [PMID: 33712588 PMCID: PMC7955068 DOI: 10.1038/s41467-021-21896-9] [Citation(s) in RCA: 88] [Impact Index Per Article: 29.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2020] [Accepted: 02/15/2021] [Indexed: 02/06/2023] Open
Abstract
Computational methods have made substantial progress in improving the accuracy and throughput of pathology workflows for diagnostic, prognostic, and genomic prediction. Still, lack of interpretability remains a significant barrier to clinical integration. We present an approach for predicting clinically-relevant molecular phenotypes from whole-slide histopathology images using human-interpretable image features (HIFs). Our method leverages >1.6 million annotations from board-certified pathologists across >5700 samples to train deep learning models for cell and tissue classification that can exhaustively map whole-slide images at two and four micron-resolution. Cell- and tissue-type model outputs are combined into 607 HIFs that quantify specific and biologically-relevant characteristics across five cancer types. We demonstrate that these HIFs correlate with well-known markers of the tumor microenvironment and can predict diverse molecular signatures (AUROC 0.601-0.864), including expression of four immune checkpoint proteins and homologous recombination deficiency, with performance comparable to 'black-box' methods. Our HIF-based approach provides a comprehensive, quantitative, and interpretable window into the composition and spatial architecture of the tumor microenvironment.
Collapse
Affiliation(s)
- James A Diao
- PathAI, Inc., Boston, MA, USA
- Program in Health Sciences and Technology, Harvard Medical School, Boston, MA, USA
| | - Jason K Wang
- PathAI, Inc., Boston, MA, USA
- Program in Health Sciences and Technology, Harvard Medical School, Boston, MA, USA
| | - Wan Fung Chui
- PathAI, Inc., Boston, MA, USA
- Program in Health Sciences and Technology, Harvard Medical School, Boston, MA, USA
| | | | | | | | - Richard N Mitchell
- Program in Health Sciences and Technology, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Brigham & Women's Hospital, Harvard Medical School, Boston, MA, USA
| | | | | | | | | | | | | | | | | | - Murray B Resnick
- PathAI, Inc., Boston, MA, USA
- Department of Pathology, Warren Alpert Medical School, Providence, RI, USA
| | | | | | | | | | | | | |
Collapse
|
11
|
Wang X, Chen Y, Gao Y, Zhang H, Guan Z, Dong Z, Zheng Y, Jiang J, Yang H, Wang L, Huang X, Ai L, Yu W, Li H, Dong C, Zhou Z, Liu X, Yu G. Predicting gastric cancer outcome from resected lymph node histopathology images using deep learning. Nat Commun 2021; 12:1637. [PMID: 33712598 PMCID: PMC7954798 DOI: 10.1038/s41467-021-21674-7] [Citation(s) in RCA: 58] [Impact Index Per Article: 19.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2020] [Accepted: 01/28/2021] [Indexed: 02/06/2023] Open
Abstract
N-staging is a determining factor for prognostic assessment and decision-making for stage-based cancer therapeutic strategies. Visual inspection of whole-slides of intact lymph nodes is currently the main method used by pathologists to calculate the number of metastatic lymph nodes (MLNs). Moreover, even at the same N stage, the outcome of patients varies dramatically. Here, we propose a deep-learning framework for analyzing lymph node whole-slide images (WSIs) to identify lymph nodes and tumor regions, and then to uncover tumor-area-to-MLN-area ratio (T/MLN). After training, our model’s tumor detection performance was comparable to that of experienced pathologists and achieved similar performance on two independent gastric cancer validation cohorts. Further, we demonstrate that T/MLN is an interpretable independent prognostic factor. These findings indicate that deep-learning models could assist not only pathologists in detecting lymph nodes with metastases but also oncologists in exploring new prognostic factors, especially those that are difficult to calculate manually. The ratio of tumour area to metastatic lymph node area (T/MLN) is a clinical indicator that can improve prognosis prediction of gastric cancer. Here, the authors use machine learning on whole slide images to generate a method that can predict metastatic lymph nodes and obtain T/MLN.
Collapse
Affiliation(s)
- Xiaodong Wang
- School of Computer Science and Technology, Xidian University, Xi'an, China
| | - Ying Chen
- Department of Pathology Center of Gastroenterology, Changhai Hospital, Shanghai, China
| | - Yunshu Gao
- Department of Oncology, General Hospital of PLA, Beijing, China
| | - Huiqing Zhang
- Department of Gastrointestinal Medical Oncology, Jiangxi Provincial Cancer Hospital, Nangchang, China
| | - Zehui Guan
- School of Computer Science, Northwestern Polytechnical University, Xi'an, China
| | - Zhou Dong
- School of Computer Science, Northwestern Polytechnical University, Xi'an, China
| | - Yuxuan Zheng
- School of Computer Science and Technology, Xidian University, Xi'an, China
| | - Jiarui Jiang
- School of Computer Science and Technology, Xidian University, Xi'an, China
| | - Haoqing Yang
- School of Computer Science and Technology, Xidian University, Xi'an, China
| | - Liming Wang
- School of Computer Science and Technology, Xidian University, Xi'an, China
| | - Xianming Huang
- Department of Gastrointestinal Medical Oncology, Jiangxi Provincial Cancer Hospital, Nangchang, China
| | - Lirong Ai
- School of Computer Science, Northwestern Polytechnical University, Xi'an, China
| | - Wenlong Yu
- Department of Surgery Oncology, Eastern Hepatobiliary Surgery Hospital, Shanghai, China
| | - Hongwei Li
- Department of Oncology, Longhua Hospital Affiliated to Shanghai University of Traditional Chinese Medicine, Shanghai, China
| | - Changsheng Dong
- Department of Oncology, Longhua Hospital Affiliated to Shanghai University of Traditional Chinese Medicine, Shanghai, China
| | - Zhou Zhou
- Department of Oncology, Longhua Hospital Affiliated to Shanghai University of Traditional Chinese Medicine, Shanghai, China
| | - Xiyang Liu
- School of Computer Science and Technology, Xidian University, Xi'an, China.
| | - Guanzhen Yu
- Department of Oncology, Longhua Hospital Affiliated to Shanghai University of Traditional Chinese Medicine, Shanghai, China. .,Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, Shanghai, China.
| |
Collapse
|
12
|
Murtaza G, Abdul Wahab AW, Raza G, Shuib L. A tree-based multiclassification of breast tumor histopathology images through deep learning. Comput Med Imaging Graph 2021; 89:101870. [PMID: 33545489 DOI: 10.1016/j.compmedimag.2021.101870] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2020] [Revised: 12/28/2020] [Accepted: 01/21/2021] [Indexed: 10/22/2022]
Abstract
Worldwide, the burden of cancer is drastically increasing over the past few years. Among all types of cancers in women, breast cancer (BrC) is the main cause of unnatural deaths. For early diagnosis, histopathology (Hp) imaging is a gold standard for positive and detailed (at tissue level) diagnosis of breast tumor (BrT) compared to mammogram images. A large number of studies used BrT Hp images to solve binary or multiclassification problems using high computational resources. However, classification models' performance may be compromised due to the high correlation among various types of BrT in Hp images, which raises the misclassification rate. Thus, this paper aims to develop a tree-based BrT multiclassification model via deep learning (DL) to extract discriminative features to solve the multiclassification problem with better performance using less computational resources. The main contributions of this work are to create an ensemble, tree-based DL model that is pre-trained on the BreakHis dataset, and implementation of a misclassification reduction algorithm. The ensemble, tree-based DL model, extracts discriminative BrT features from Hp images. The target dataset (i.e., Bioimaging challenge 2015 breast histology) is small in size; thus, to avoid overfitting of the proposed model, pretraining is performed on the BreakHis dataset. Whereas, misclassification reduction algorithm is implemented to enhance the performance of the classification model. The experimental results show that the proposed model outperformed the existing state-of-the-art baseline studies. The achieved classification accuracy is ranging from 87.50 % to 100 % for four subtypes of BrT. Thus, the proposed model can assist doctors as the second opinion in any healthcare centre.
Collapse
Affiliation(s)
- Ghulam Murtaza
- Faculty of Computer Science and Information Technology, University of Malaya, 50603, Kuala Lumpur, Malaysia; Department of Computer Science, Sukkur IBA University, Sukkur, Pakistan.
| | - Ainuddin Wahid Abdul Wahab
- Faculty of Computer Science and Information Technology, University of Malaya, 50603, Kuala Lumpur, Malaysia.
| | - Ghulam Raza
- Our Lady of Lourdes Hospital Drogheda Ireland, Ireland.
| | - Liyana Shuib
- Faculty of Computer Science and Information Technology, University of Malaya, 50603, Kuala Lumpur, Malaysia.
| |
Collapse
|
13
|
Pan C, Schoppe O, Parra-Damas A, Cai R, Todorov MI, Gondi G, von Neubeck B, Böğürcü-Seidel N, Seidel S, Sleiman K, Veltkamp C, Förstera B, Mai H, Rong Z, Trompak O, Ghasemigharagoz A, Reimer MA, Cuesta AM, Coronel J, Jeremias I, Saur D, Acker-Palmer A, Acker T, Garvalov BK, Menze B, Zeidler R, Ertürk A. Deep Learning Reveals Cancer Metastasis and Therapeutic Antibody Targeting in the Entire Body. Cell 2020; 179:1661-1676.e19. [PMID: 31835038 DOI: 10.1016/j.cell.2019.11.013] [Citation(s) in RCA: 112] [Impact Index Per Article: 28.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2019] [Revised: 10/02/2019] [Accepted: 11/12/2019] [Indexed: 12/20/2022]
Abstract
Reliable detection of disseminated tumor cells and of the biodistribution of tumor-targeting therapeutic antibodies within the entire body has long been needed to better understand and treat cancer metastasis. Here, we developed an integrated pipeline for automated quantification of cancer metastases and therapeutic antibody targeting, named DeepMACT. First, we enhanced the fluorescent signal of cancer cells more than 100-fold by applying the vDISCO method to image metastasis in transparent mice. Second, we developed deep learning algorithms for automated quantification of metastases with an accuracy matching human expert manual annotation. Deep learning-based quantification in 5 different metastatic cancer models including breast, lung, and pancreatic cancer with distinct organotropisms allowed us to systematically analyze features such as size, shape, spatial distribution, and the degree to which metastases are targeted by a therapeutic monoclonal antibody in entire mice. DeepMACT can thus considerably improve the discovery of effective antibody-based therapeutics at the pre-clinical stage. VIDEO ABSTRACT.
Collapse
Affiliation(s)
- Chenchen Pan
- Institute for Tissue Engineering and Regenerative Medicine (iTERM), Helmholtz Zentrum München, 85764 Neuherberg, Germany; Institute for Stroke and Dementia Research, Klinikum der Universität München, Ludwig Maximilian University of Munich (LMU), 81377 Munich, Germany
| | - Oliver Schoppe
- Department of Informatics, Technical University of Munich, 85748 Munich, Germany; Center for Translational Cancer Research (TranslaTUM), Klinikum rechts der Isar, Technical University of Munich, 81675 Munich, Germany
| | - Arnaldo Parra-Damas
- Institute for Stroke and Dementia Research, Klinikum der Universität München, Ludwig Maximilian University of Munich (LMU), 81377 Munich, Germany
| | - Ruiyao Cai
- Institute for Tissue Engineering and Regenerative Medicine (iTERM), Helmholtz Zentrum München, 85764 Neuherberg, Germany; Institute for Stroke and Dementia Research, Klinikum der Universität München, Ludwig Maximilian University of Munich (LMU), 81377 Munich, Germany
| | - Mihail Ivilinov Todorov
- Institute for Tissue Engineering and Regenerative Medicine (iTERM), Helmholtz Zentrum München, 85764 Neuherberg, Germany; Institute for Stroke and Dementia Research, Klinikum der Universität München, Ludwig Maximilian University of Munich (LMU), 81377 Munich, Germany; Graduate School of Systemic Neurosciences (GSN), 82152 Munich, Germany
| | - Gabor Gondi
- Research Unit Gene Vectors, Helmholtz Zentrum München, 81377 Munich, Germany
| | - Bettina von Neubeck
- Research Unit Gene Vectors, Helmholtz Zentrum München, 81377 Munich, Germany
| | | | - Sascha Seidel
- Institute of Cell Biology and Neuroscience and Buchmann Institute for Molecular Life Sciences (BMLS), University of Frankfurt, 60323 Frankfurt, Germany
| | - Katia Sleiman
- Center for Translational Cancer Research (TranslaTUM), Klinikum rechts der Isar, Technical University of Munich, 81675 Munich, Germany; Division of Translational Cancer Research, German Cancer Research Center (DKFZ), 69120 Heidelberg, Germany; German Cancer Consortium (DKTK), Partner Site Munich, Klinikum rechts der Isar, Technische Universität München, 81675 Munich, Germany
| | - Christian Veltkamp
- Center for Translational Cancer Research (TranslaTUM), Klinikum rechts der Isar, Technical University of Munich, 81675 Munich, Germany; Division of Translational Cancer Research, German Cancer Research Center (DKFZ), 69120 Heidelberg, Germany; German Cancer Consortium (DKTK), Partner Site Munich, Klinikum rechts der Isar, Technische Universität München, 81675 Munich, Germany
| | - Benjamin Förstera
- Institute for Tissue Engineering and Regenerative Medicine (iTERM), Helmholtz Zentrum München, 85764 Neuherberg, Germany; Institute for Stroke and Dementia Research, Klinikum der Universität München, Ludwig Maximilian University of Munich (LMU), 81377 Munich, Germany
| | - Hongcheng Mai
- Institute for Tissue Engineering and Regenerative Medicine (iTERM), Helmholtz Zentrum München, 85764 Neuherberg, Germany; Institute for Stroke and Dementia Research, Klinikum der Universität München, Ludwig Maximilian University of Munich (LMU), 81377 Munich, Germany
| | - Zhouyi Rong
- Institute for Tissue Engineering and Regenerative Medicine (iTERM), Helmholtz Zentrum München, 85764 Neuherberg, Germany; Institute for Stroke and Dementia Research, Klinikum der Universität München, Ludwig Maximilian University of Munich (LMU), 81377 Munich, Germany
| | - Omelyan Trompak
- Institute of Neuropathology, University of Giessen, 35390 Giessen, Germany
| | - Alireza Ghasemigharagoz
- Institute for Stroke and Dementia Research, Klinikum der Universität München, Ludwig Maximilian University of Munich (LMU), 81377 Munich, Germany
| | - Madita Alice Reimer
- Institute for Stroke and Dementia Research, Klinikum der Universität München, Ludwig Maximilian University of Munich (LMU), 81377 Munich, Germany
| | - Angel M Cuesta
- Institute of Cell Biology and Neuroscience and Buchmann Institute for Molecular Life Sciences (BMLS), University of Frankfurt, 60323 Frankfurt, Germany
| | - Javier Coronel
- Department of Informatics, Technical University of Munich, 85748 Munich, Germany
| | - Irmela Jeremias
- Research Unit Apoptosis in Hematopoietic Stem Cells, Helmholtz Zentrum München, German Center for Environmental Health (HMGU), 81377 Munich, Germany; Department of Pediatrics, Dr. von Hauner Childrens Hospital, Ludwig Maximilian University of Munich (LMU), 81377 Munich, Germany; German Consortium for Translational Cancer Research (DKTK), Partnering Site Munich, 80336 Munich, Germany
| | - Dieter Saur
- Center for Translational Cancer Research (TranslaTUM), Klinikum rechts der Isar, Technical University of Munich, 81675 Munich, Germany; Division of Translational Cancer Research, German Cancer Research Center (DKFZ), 69120 Heidelberg, Germany; German Cancer Consortium (DKTK), Partner Site Munich, Klinikum rechts der Isar, Technische Universität München, 81675 Munich, Germany
| | - Amparo Acker-Palmer
- Institute of Cell Biology and Neuroscience and Buchmann Institute for Molecular Life Sciences (BMLS), University of Frankfurt, 60323 Frankfurt, Germany
| | - Till Acker
- Institute of Neuropathology, University of Giessen, 35390 Giessen, Germany
| | - Boyan K Garvalov
- Institute of Neuropathology, University of Giessen, 35390 Giessen, Germany; Department of Microvascular Biology and Pathobiology, European Center for Angioscience (ECAS), Medical Faculty Mannheim, University of Heidelberg, 68167 Mannheim, Germany
| | - Bjoern Menze
- Department of Informatics, Technical University of Munich, 85748 Munich, Germany; Center for Translational Cancer Research (TranslaTUM), Klinikum rechts der Isar, Technical University of Munich, 81675 Munich, Germany; Munich School of Bioengineering, Technical University of Munich, 85748 Munich, Germany
| | - Reinhard Zeidler
- Research Unit Gene Vectors, Helmholtz Zentrum München, 81377 Munich, Germany; Department for Otorhinolaryngology, Klinikum der Universität München, Ludwig Maximilian University of Munich (LMU), 81377 Munich, Germany
| | - Ali Ertürk
- Institute for Tissue Engineering and Regenerative Medicine (iTERM), Helmholtz Zentrum München, 85764 Neuherberg, Germany; Institute for Stroke and Dementia Research, Klinikum der Universität München, Ludwig Maximilian University of Munich (LMU), 81377 Munich, Germany; Munich Cluster for Systems Neurology (SyNergy), 81377 Munich, Germany.
| |
Collapse
|
14
|
Deep-UV excitation fluorescence microscopy for detection of lymph node metastasis using deep neural network. Sci Rep 2019; 9:16912. [PMID: 31729459 PMCID: PMC6858352 DOI: 10.1038/s41598-019-53405-w] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2019] [Accepted: 10/31/2019] [Indexed: 02/07/2023] Open
Abstract
Deep-UV (DUV) excitation fluorescence microscopy has potential to provide rapid diagnosis with simple technique comparing to conventional histopathology based on hematoxylin and eosin (H&E) staining. We established a fluorescent staining protocol for DUV excitation fluorescence imaging that has enabled clear discrimination of nucleoplasm, nucleolus, and cytoplasm. Fluorescence images of metastasis-positive/-negative lymph nodes of gastric cancer patients were used for patch-based training with a deep neural network (DNN) based on Inception-v3 architecture. The performance on small patches of the fluorescence images was comparable with that of H&E images. Gradient-weighted class activation mapping analysis revealed the areas where the trained model identified metastatic lesions in the images containing cancer cells. We extended the method to large-size image analysis enabling accurate detection of metastatic lesions. We discuss usefulness of DUV excitation fluorescence imaging with the aid of DNN analysis, which is promising for assisting pathologists in assessment of lymph node metastasis.
Collapse
|
15
|
Serag A, Ion-Margineanu A, Qureshi H, McMillan R, Saint Martin MJ, Diamond J, O'Reilly P, Hamilton P. Translational AI and Deep Learning in Diagnostic Pathology. Front Med (Lausanne) 2019; 6:185. [PMID: 31632973 PMCID: PMC6779702 DOI: 10.3389/fmed.2019.00185] [Citation(s) in RCA: 123] [Impact Index Per Article: 24.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2019] [Accepted: 07/30/2019] [Indexed: 12/15/2022] Open
Abstract
There has been an exponential growth in the application of AI in health and in pathology. This is resulting in the innovation of deep learning technologies that are specifically aimed at cellular imaging and practical applications that could transform diagnostic pathology. This paper reviews the different approaches to deep learning in pathology, the public grand challenges that have driven this innovation and a range of emerging applications in pathology. The translation of AI into clinical practice will require applications to be embedded seamlessly within digital pathology workflows, driving an integrated approach to diagnostics and providing pathologists with new tools that accelerate workflow and improve diagnostic consistency and reduce errors. The clearance of digital pathology for primary diagnosis in the US by some manufacturers provides the platform on which to deliver practical AI. AI and computational pathology will continue to mature as researchers, clinicians, industry, regulatory organizations and patient advocacy groups work together to innovate and deliver new technologies to health care providers: technologies which are better, faster, cheaper, more precise, and safe.
Collapse
|
16
|
Murtaza G, Shuib L, Abdul Wahab AW, Mujtaba G, Mujtaba G, Nweke HF, Al-garadi MA, Zulfiqar F, Raza G, Azmi NA. Deep learning-based breast cancer classification through medical imaging modalities: state of the art and research challenges. Artif Intell Rev 2019. [DOI: 10.1007/s10462-019-09716-5] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
17
|
Litjens G, Bandi P, Ehteshami Bejnordi B, Geessink O, Balkenhol M, Bult P, Halilovic A, Hermsen M, van de Loo R, Vogels R, Manson QF, Stathonikos N, Baidoshvili A, van Diest P, Wauters C, van Dijk M, van der Laak J. 1399 H&E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset. Gigascience 2018; 7:5026175. [PMID: 29860392 PMCID: PMC6007545 DOI: 10.1093/gigascience/giy065] [Citation(s) in RCA: 138] [Impact Index Per Article: 23.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2017] [Accepted: 05/22/2018] [Indexed: 12/27/2022] Open
Abstract
Background The presence of lymph node metastases is one of the most important factors in breast cancer prognosis. The most common way to assess regional lymph node status is the sentinel lymph node procedure. The sentinel lymph node is the most likely lymph node to contain metastasized cancer cells and is excised, histopathologically processed, and examined by a pathologist. This tedious examination process is time-consuming and can lead to small metastases being missed. However, recent advances in whole-slide imaging and machine learning have opened an avenue for analysis of digitized lymph node sections with computer algorithms. For example, convolutional neural networks, a type of machine-learning algorithm, can be used to automatically detect cancer metastases in lymph nodes with high accuracy. To train machine-learning models, large, well-curated datasets are needed. Results We released a dataset of 1,399 annotated whole-slide images (WSIs) of lymph nodes, both with and without metastases, in 3 terabytes of data in the context of the CAMELYON16 and CAMELYON17 Grand Challenges. Slides were collected from five medical centers to cover a broad range of image appearance and staining variations. Each WSI has a slide-level label indicating whether it contains no metastases, macro-metastases, micro-metastases, or isolated tumor cells. Furthermore, for 209 WSIs, detailed hand-drawn contours for all metastases are provided. Last, open-source software tools to visualize and interact with the data have been made available. Conclusions A unique dataset of annotated, whole-slide digital histopathology images has been provided with high potential for re-use.
Collapse
Affiliation(s)
- Geert Litjens
- Diagnostic Image Analysis Group, Department of Pathology, Radboud University Medical Center, Huispost 824, Geert Grootteplein-Zuid 10, 6525GA Nijmegen, The Netherlands
| | - Peter Bandi
- Department of Pathology, University Medical Center Huispost H04.312, Heidelberglaan 100, 3584CX, Utrecht, The Netherlands
| | - Babak Ehteshami Bejnordi
- Diagnostic Image Analysis Group, Department of Pathology, Radboud University Medical Center, Huispost 824, Geert Grootteplein-Zuid 10, 6525GA Nijmegen, The Netherlands
| | - Oscar Geessink
- Diagnostic Image Analysis Group, Department of Pathology, Radboud University Medical Center, Huispost 824, Geert Grootteplein-Zuid 10, 6525GA Nijmegen, The Netherlands
| | - Maschenka Balkenhol
- Diagnostic Image Analysis Group, Department of Pathology, Radboud University Medical Center, Huispost 824, Geert Grootteplein-Zuid 10, 6525GA Nijmegen, The Netherlands
| | - Peter Bult
- Diagnostic Image Analysis Group, Department of Pathology, Radboud University Medical Center, Huispost 824, Geert Grootteplein-Zuid 10, 6525GA Nijmegen, The Netherlands
| | - Altuna Halilovic
- Diagnostic Image Analysis Group, Department of Pathology, Radboud University Medical Center, Huispost 824, Geert Grootteplein-Zuid 10, 6525GA Nijmegen, The Netherlands
| | - Meyke Hermsen
- Diagnostic Image Analysis Group, Department of Pathology, Radboud University Medical Center, Huispost 824, Geert Grootteplein-Zuid 10, 6525GA Nijmegen, The Netherlands
| | - Rob van de Loo
- Diagnostic Image Analysis Group, Department of Pathology, Radboud University Medical Center, Huispost 824, Geert Grootteplein-Zuid 10, 6525GA Nijmegen, The Netherlands
| | - Rob Vogels
- Diagnostic Image Analysis Group, Department of Pathology, Radboud University Medical Center, Huispost 824, Geert Grootteplein-Zuid 10, 6525GA Nijmegen, The Netherlands
| | - Quirine F Manson
- Department of Pathology, University Medical Center Huispost H04.312, Heidelberglaan 100, 3584CX, Utrecht, The Netherlands
| | - Nikolas Stathonikos
- Department of Pathology, University Medical Center Huispost H04.312, Heidelberglaan 100, 3584CX, Utrecht, The Netherlands
| | - Alexi Baidoshvili
- Laboratory for Pathology East Netherlands (LabPON), Postbus 516, 7550AM Hengelo, The Netherlands
| | - Paul van Diest
- Department of Pathology, University Medical Center Huispost H04.312, Heidelberglaan 100, 3584CX, Utrecht, The Netherlands
| | - Carla Wauters
- Department of Pathology, Canisius-Wilhelmina Hospital, Postbus 9015, 6500GS Nijmegen, The Netherlands
| | - Marcory van Dijk
- Department of Pathology, Rijnstate Hospital, Pathology-DNA, Postbus 9555, 6800TA Arnhem, The Netherlands
| | - Jeroen van der Laak
- Diagnostic Image Analysis Group, Department of Pathology, Radboud University Medical Center, Huispost 824, Geert Grootteplein-Zuid 10, 6525GA Nijmegen, The Netherlands
| |
Collapse
|
18
|
Ehteshami Bejnordi B, Veta M, Johannes van Diest P, van Ginneken B, Karssemeijer N, Litjens G, van der Laak JAWM. Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer. JAMA 2017; 318:2199-2210. [PMID: 29234806 PMCID: PMC5820737 DOI: 10.1001/jama.2017.14585] [Citation(s) in RCA: 1365] [Impact Index Per Article: 195.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/22/2017] [Accepted: 10/26/2017] [Indexed: 02/06/2023]
Abstract
Importance Application of deep learning algorithms to whole-slide pathology images can potentially improve diagnostic accuracy and efficiency. Objective Assess the performance of automated deep learning algorithms at detecting metastases in hematoxylin and eosin-stained tissue sections of lymph nodes of women with breast cancer and compare it with pathologists' diagnoses in a diagnostic setting. Design, Setting, and Participants Researcher challenge competition (CAMELYON16) to develop automated solutions for detecting lymph node metastases (November 2015-November 2016). A training data set of whole-slide images from 2 centers in the Netherlands with (n = 110) and without (n = 160) nodal metastases verified by immunohistochemical staining were provided to challenge participants to build algorithms. Algorithm performance was evaluated in an independent test set of 129 whole-slide images (49 with and 80 without metastases). The same test set of corresponding glass slides was also evaluated by a panel of 11 pathologists with time constraint (WTC) from the Netherlands to ascertain likelihood of nodal metastases for each slide in a flexible 2-hour session, simulating routine pathology workflow, and by 1 pathologist without time constraint (WOTC). Exposures Deep learning algorithms submitted as part of a challenge competition or pathologist interpretation. Main Outcomes and Measures The presence of specific metastatic foci and the absence vs presence of lymph node metastasis in a slide or image using receiver operating characteristic curve analysis. The 11 pathologists participating in the simulation exercise rated their diagnostic confidence as definitely normal, probably normal, equivocal, probably tumor, or definitely tumor. Results The area under the receiver operating characteristic curve (AUC) for the algorithms ranged from 0.556 to 0.994. The top-performing algorithm achieved a lesion-level, true-positive fraction comparable with that of the pathologist WOTC (72.4% [95% CI, 64.3%-80.4%]) at a mean of 0.0125 false-positives per normal whole-slide image. For the whole-slide image classification task, the best algorithm (AUC, 0.994 [95% CI, 0.983-0.999]) performed significantly better than the pathologists WTC in a diagnostic simulation (mean AUC, 0.810 [range, 0.738-0.884]; P < .001). The top 5 algorithms had a mean AUC that was comparable with the pathologist interpreting the slides in the absence of time constraints (mean AUC, 0.960 [range, 0.923-0.994] for the top 5 algorithms vs 0.966 [95% CI, 0.927-0.998] for the pathologist WOTC). Conclusions and Relevance In the setting of a challenge competition, some deep learning algorithms achieved better diagnostic performance than a panel of 11 pathologists participating in a simulation exercise designed to mimic routine pathology workflow; algorithm performance was comparable with an expert pathologist interpreting whole-slide images without time constraints. Whether this approach has clinical utility will require evaluation in a clinical setting.
Collapse
Affiliation(s)
- Babak Ehteshami Bejnordi
- Diagnostic Image Analysis Group, Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Mitko Veta
- Medical Image Analysis Group, Eindhoven University of Technology, Eindhoven, the Netherlands
| | | | - Bram van Ginneken
- Diagnostic Image Analysis Group, Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Nico Karssemeijer
- Diagnostic Image Analysis Group, Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Geert Litjens
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | | | | |
Collapse
|
19
|
Truin W, Roumen RM, Siesling S, van der Heiden-van der Loo M, Lobbezoo DJ, Tjan-Heijnen VC, Voogd AC. Sentinel Lymph Node Biopsy and Isolated Tumor Cells in Invasive Lobular Versus Ductal Breast Cancer. Clin Breast Cancer 2016; 16:e75-82. [DOI: 10.1016/j.clbc.2016.03.007] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2015] [Accepted: 03/22/2016] [Indexed: 11/17/2022]
|
20
|
Apple SK. Sentinel Lymph Node in Breast Cancer: Review Article from a Pathologist's Point of View. J Pathol Transl Med 2016; 50:83-95. [PMID: 26757203 PMCID: PMC4804148 DOI: 10.4132/jptm.2015.11.23] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2015] [Accepted: 11/23/2015] [Indexed: 11/17/2022] Open
Abstract
Breast cancer staging, in particular N-stage changed most significantly due to the advanced technique of sentinel lymph node biopsy two decades ago. Pathologists have more thoroughly examined and scrutinized sentinel lymph node and found increased number of small volume metastases. While pathologists use the strict criteria from the Tumor Lymph Node Metastasis (TNM) Classification, studies have shown poor reproducibility in the application of American Joint Committee on Cancer and International Union Against Cancer/TNM guidelines for sentinel lymph node classification in breast cancer. In this review article, a brief history of TNM with a focus on N-stage is described, followed by innate problems with the guidelines, and why pathologists may have difficulties in assessing lymph node metastases uniformly. Finally, clinical significance of isolated tumor cells, micrometastasis, and macrometastasis is described by reviewing historical retrospective data and significant prospective clinical trials.
Collapse
Affiliation(s)
- Sophia K Apple
- Department of Pathology and Laboratory Medicine, University of California at Los Angeles (UCLA), Los Angeles, CA, USA
| |
Collapse
|
21
|
Tsuda H. Histological examination of sentinel lymph nodes: significance of macrometastasis, micrometastasis, and isolated tumor cells. Breast Cancer 2015; 22:221-9. [PMID: 25663030 DOI: 10.1007/s12282-015-0588-9] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2015] [Accepted: 01/22/2015] [Indexed: 02/06/2023]
Abstract
Sentinel lymph node biopsy has been started in 1990s and has become one of the standard diagnostic procedures used to treat patients with early breast cancer in this century. In Japan, for the microscopic diagnosis of metastasis to sentinel lymph nodes, intraoperative frozen section diagnosis is widely used in combination with subsequent permanent section diagnosis of the residual specimens. Metastatic foci to sentinel lymph nodes have been classified into macrometastasis, micrometastasis, and isolated tumor cells in 2002, and the definition of isolated tumor cells was modified in 2010. Clinical significance of occult sentinel lymph node metastases, being mostly composed of micrometastasis and isolated tumor cells, has been clarified in terms of predictive factors for non-sentinel lymph node metastasis and patient prognosis by large-scale retrospective studies and prospective randomized clinical trials. In the present review, clinical implications of micrometastases and isolated tumor cells in sentinel lymph nodes and the methods for pathological examination of SLN metastases employed in these studies were overviewed.
Collapse
Affiliation(s)
- Hitoshi Tsuda
- Department of Basic Pathology, National Defense Medical College, 3-2 Namiki, Tokorozawa, Saitama, 359-8513, Japan,
| |
Collapse
|
22
|
Houvenaeghel G, Classe JM, Barranger E. L’exploration et le traitement de la région axillaire des tumeurs infiltrantes du sein (RPC 2013). ONCOLOGIE 2013. [DOI: 10.1007/s10269-013-2338-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
23
|
Heilmann T, Mathiak M, Hofmann J, Mundhenke C, van Mackelenbergh M, Alkatout I, Wenners A, Eckmann-Scholz C, Schem C. Intra-operative use of one-step nucleic acid amplification (OSNA) for detection of the tumor load of sentinel lymph nodes in breast cancer patients. J Cancer Res Clin Oncol 2013; 139:1649-55. [DOI: 10.1007/s00432-013-1481-6] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2013] [Accepted: 07/17/2013] [Indexed: 12/21/2022]
|
24
|
Scharl A, Thomssen C, Harbeck N, Müller V. AGO Recommendations for Diagnosis and Treatment of Patients with Early Breast Cancer: Update 2013. Breast Care (Basel) 2013; 8:174-80. [PMID: 24415966 PMCID: PMC3728627 DOI: 10.1159/000353617] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023] Open
Affiliation(s)
- Anton Scharl
- Frauenklinik, Martin-Luther Universität Halle/Saale, Hamburg, Germany
| | | | - Nadia Harbeck
- Brustzentrum, Frauenklinik, Universität München, Hamburg, Germany
| | - Volkmar Müller
- Klinik für Gynäkologie, Universitätsklinikum Hamburg-Eppendorf, Hamburg, Germany
| |
Collapse
|
25
|
Houvenaeghel G, Cohen M, Chereau Ewald E, Bannier M, Buttarelli M, Lambaudie E. Indication du curage axillaire en cas de ganglion sentinelle envahi — essais cliniques. ONCOLOGIE 2013. [DOI: 10.1007/s10269-013-2293-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
26
|
Postma EL, Verkooijen HM, van Diest PJ, Willems SM, van den Bosch MAAJ, van Hillegersberg R. Discrepancy between routine and expert pathologists' assessment of non-palpable breast cancer and its impact on locoregional and systemic treatment. Eur J Pharmacol 2013; 717:31-5. [PMID: 23545360 DOI: 10.1016/j.ejphar.2012.12.033] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2012] [Revised: 11/26/2012] [Accepted: 12/18/2012] [Indexed: 10/27/2022]
Abstract
Histopathological parameters are essential for deciding on adjuvant treatment following breast cancer surgery. We assessed the impact of inter-observer variability on treatment strategy in patients operated for clinically node negative, non-palpable breast carcinomas. In the context of a multicenter randomised controlled trial, clinical and histological data of 310 patients with clinically node negative non-palpable invasive breast cancer were prospectively collected. Histological assessment of the primary tumour and sentinel nodes was first performed in a routine setting, subsequently central review took place. In case of discordance between local en central assessments, we determined the impact on locoregional and systemic treatment strategy. Discordance between local and central review was observed in 13% of the patients for type (kappa 0.60, 95% CI 0.50-0.71), in 12% for grade (k=0.796, 95% CI 0.73-0.86), in 1% for ER status (k=0.898, 95% CI 0.80-1.0), in 2% for PR status (k=0.940 95% CI 0.89-0.99). Discrepancy in the assessment of the sentinel node(s) was seen in 2% of the patients (k=0.954, 95% CI 0.92-0.98). Applying current Dutch Guidelines, central review would have affected locoregional treatment in 2% (7/310), systemic treatment in 5% (16/310) and both in 1% (2/310) of the patients. For the 9 (3%) patients in whom central review would have led to additional systemic treatment, Adjuvant! predicted 10 years mortality and recurrence rate would have decreased with a median of 4.6% and 15%, respectively. Discordance between routine histological assessment and central review of non-palpable breast carcinoma specimens and sentinel nodes was observed in 24% of patients. This inter-observer variation would have impacted locoregional and/or systemic treatment strategies in 8% of the patients.
Collapse
Affiliation(s)
- Emily L Postma
- Department of Surgery, University Medical Centre Utrecht, PO box 85500 3508 GA Utrecht, the Netherlands.
| | | | | | | | | | | |
Collapse
|