1
|
Budginaite E, Magee DR, Kloft M, Woodruff HC, Grabsch HI. Computational methods for metastasis detection in lymph nodes and characterization of the metastasis-free lymph node microarchitecture: A systematic-narrative hybrid review. J Pathol Inform 2024; 15:100367. [PMID: 38455864 PMCID: PMC10918266 DOI: 10.1016/j.jpi.2024.100367] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Revised: 01/31/2024] [Accepted: 01/31/2024] [Indexed: 03/09/2024] Open
Abstract
Background Histological examination of tumor draining lymph nodes (LNs) plays a vital role in cancer staging and prognostication. However, as soon as a LN is classed as metastasis-free, no further investigation will be performed and thus, potentially clinically relevant information detectable in tumor-free LNs is currently not captured. Objective To systematically study and critically assess methods for the analysis of digitized histological LN images described in published research. Methods A systematic search was conducted in several public databases up to December 2023 using relevant search terms. Studies using brightfield light microscopy images of hematoxylin and eosin or immunohistochemically stained LN tissue sections aiming to detect and/or segment LNs, their compartments or metastatic tumor using artificial intelligence (AI) were included. Dataset, AI methodology, cancer type, and study objective were compared between articles. Results A total of 7201 articles were collected and 73 articles remained for detailed analyses after article screening. Of the remaining articles, 86% aimed at LN metastasis identification, 8% aimed at LN compartment segmentation, and remaining focused on LN contouring. Furthermore, 78% of articles used patch classification and 22% used pixel segmentation models for analyses. Five out of six studies (83%) of metastasis-free LNs were performed on publicly unavailable datasets, making quantitative article comparison impossible. Conclusions Multi-scale models mimicking multiple microscopy zooms show promise for computational LN analysis. Large-scale datasets are needed to establish the clinical relevance of analyzing metastasis-free LN in detail. Further research is needed to identify clinically interpretable metrics for LN compartment characterization.
Collapse
Affiliation(s)
- Elzbieta Budginaite
- Department of Pathology, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
- Department of Precision Medicine, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
| | | | - Maximilian Kloft
- Department of Pathology, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
- Department of Internal Medicine, Justus-Liebig-University, Giessen, Germany
| | - Henry C. Woodruff
- Department of Precision Medicine, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
| | - Heike I. Grabsch
- Department of Pathology, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
- Pathology and Data Analytics, Leeds Institute of Medical Research at St James’s, University of Leeds, Leeds, UK
| |
Collapse
|
2
|
Zhang X, Zhao Z, Wang R, Chen H, Zheng X, Liu L, Lan L, Li P, Wu S, Cao Q, Luo R, Hu W, Lyu S, Zhang Z, Xie D, Ye Y, Wang Y, Cai M. A multicenter proof-of-concept study on deep learning-based intraoperative discrimination of primary central nervous system lymphoma. Nat Commun 2024; 15:3768. [PMID: 38704409 PMCID: PMC11069536 DOI: 10.1038/s41467-024-48171-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2023] [Accepted: 04/18/2024] [Indexed: 05/06/2024] Open
Abstract
Accurate intraoperative differentiation of primary central nervous system lymphoma (PCNSL) remains pivotal in guiding neurosurgical decisions. However, distinguishing PCNSL from other lesions, notably glioma, through frozen sections challenges pathologists. Here we sought to develop and validate a deep learning model capable of precisely distinguishing PCNSL from non-PCNSL lesions, especially glioma, using hematoxylin and eosin (H&E)-stained frozen whole-slide images. Also, we compared its performance against pathologists of varying expertise. Additionally, a human-machine fusion approach integrated both model and pathologic diagnostics. In external cohorts, LGNet achieved AUROCs of 0.965 and 0.972 in distinguishing PCNSL from glioma and AUROCs of 0.981 and 0.993 in differentiating PCNSL from non-PCNSL lesions. Outperforming several pathologists, LGNet significantly improved diagnostic performance, further augmented to some extent by fusion approach. LGNet's proficiency in frozen section analysis and its synergy with pathologists indicate its valuable role in intraoperative diagnosis, particularly in discriminating PCNSL from glioma, alongside other lesions.
Collapse
Affiliation(s)
- Xinke Zhang
- Department of Pathology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou, 510060, China
| | - Zihan Zhao
- Department of Pathology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou, 510060, China
| | - Ruixuan Wang
- School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, 510006, China
| | - Haohua Chen
- Department of Pathology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou, 510060, China
| | - Xueyi Zheng
- Department of Pathology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou, 510060, China
| | - Lili Liu
- Department of Pathology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou, 510060, China
| | - Lilong Lan
- Department of Pathology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou, 510060, China
| | - Peng Li
- Department of Pathology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou, 510060, China
| | - Shuyang Wu
- Department of Pathology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou, 510060, China
| | - Qinghua Cao
- Department of Pathology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, 510080, China
| | - Rongzhen Luo
- Department of Pathology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou, 510060, China
| | - Wanming Hu
- Department of Pathology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou, 510060, China
| | - Shanshan Lyu
- Department of Pathology, Guangdong Provincial People's Hospital, Guangzhou, 510080, China
| | - Zhengyu Zhang
- Department of Pathology, Nanfang Hospital, Soutern Medical University, Guangzhou, 510515, China
| | - Dan Xie
- Department of Pathology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou, 510060, China.
| | - Yaping Ye
- Department of Pathology, Nanfang Hospital, Soutern Medical University, Guangzhou, 510515, China.
| | - Yu Wang
- Department of Pathology, Zhujiang Hospital, Soutern Medical University, Guangzhou, 510280, China.
| | - Muyan Cai
- Department of Pathology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou, 510060, China.
| |
Collapse
|
3
|
Katayama A, Aoki Y, Watanabe Y, Horiguchi J, Rakha EA, Oyama T. Current status and prospects of artificial intelligence in breast cancer pathology: convolutional neural networks to prospective Vision Transformers. Int J Clin Oncol 2024:10.1007/s10147-024-02513-3. [PMID: 38619651 DOI: 10.1007/s10147-024-02513-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Accepted: 03/12/2024] [Indexed: 04/16/2024]
Abstract
Breast cancer is the most prevalent cancer among women, and its diagnosis requires the accurate identification and classification of histological features for effective patient management. Artificial intelligence, particularly through deep learning, represents the next frontier in cancer diagnosis and management. Notably, the use of convolutional neural networks and emerging Vision Transformers (ViT) has been reported to automate pathologists' tasks, including tumor detection and classification, in addition to improving the efficiency of pathology services. Deep learning applications have also been extended to the prediction of protein expression, molecular subtype, mutation status, therapeutic efficacy, and outcome prediction directly from hematoxylin and eosin-stained slides, bypassing the need for immunohistochemistry or genetic testing. This review explores the current status and prospects of deep learning in breast cancer diagnosis with a focus on whole-slide image analysis. Artificial intelligence applications are increasingly applied to many tasks in breast pathology ranging from disease diagnosis to outcome prediction, thus serving as valuable tools for assisting pathologists and supporting breast cancer management.
Collapse
Affiliation(s)
- Ayaka Katayama
- Diagnostic Pathology, Gunma University Graduate School of Medicine, 3-39-22 Showamachi, Maebashi, Gunma, 371-8511, Japan.
| | - Yuki Aoki
- Center for Mathematics and Data Science, Gunma University, Maebashi, Japan
| | - Yukako Watanabe
- Clinical Training Center, Gunma University Hospital, Maebashi, Japan
| | - Jun Horiguchi
- Department of Breast Surgery, International University of Health and Welfare, Narita, Japan
| | - Emad A Rakha
- Department of Histopathology School of Medicine, University of Nottingham, University Park, Nottingham, UK
- Department of Pathology, Hamad Medical Corporation, Doha, Qatar
| | - Tetsunari Oyama
- Diagnostic Pathology, Gunma University Graduate School of Medicine, 3-39-22 Showamachi, Maebashi, Gunma, 371-8511, Japan
| |
Collapse
|
4
|
Zhao Z, Chen C, Guan H, Guo L, Tian W, Liu X, Zhang H, Li J, Qiu T, Du J, Guo Q, Sun F, Zheng S, Ma J. Analysis of false reasons based on the artificial intelligence RRCART model to identify frozen sections of lymph nodes in breast cancer. Diagn Pathol 2024; 19:18. [PMID: 38254204 PMCID: PMC10802064 DOI: 10.1186/s13000-023-01432-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2023] [Accepted: 12/17/2023] [Indexed: 01/24/2024] Open
Abstract
BACKGROUND Breast cancer is the most common malignant tumor in the world. Intraoperative frozen section of sentinel lymph nodes is an important basis for determining whether axillary lymph node dissection is required for breast cancer surgery. We propose an RRCART model based on a deep-learning network to identify metastases in 2362 frozen sections and count the wrongly identified sections and the associated reasons. The purpose is to summarize the factors that affect the accuracy of the artificial intelligence model and propose corresponding solutions. METHODS We took the pathological diagnosis of senior pathologists as the gold standard and identified errors. The pathologists and artificial intelligence engineers jointly read the images and heatmaps to determine the locations of the identified errors on sections, and the pathologists found the reasons (false reasons) for the errors. Through NVivo 12 Plus, qualitative analysis of word frequency analysis and nodal analysis was performed on the error reasons, and the top-down error reason framework of "artificial intelligence RRCART model to identify frozen sections of breast cancer lymph nodes" was constructed based on the importance of false reasons. RESULTS There were 101 incorrectly identified sections in 2362 slides, including 42 false negatives and 59 false positives. Through NVivo 12 Plus software, the error causes were node-coded, and finally, 2 parent nodes (high-frequency error, low-frequency error) and 5 child nodes (section quality, normal lymph node structure, secondary reaction of lymph nodes, micrometastasis, and special growth pattern of tumor) were obtained; among them, the error of highest frequency was that caused by normal lymph node structure, with a total of 45 cases (44.55%), followed by micrometastasis, which occurred in 30 cases (29.70%). CONCLUSIONS The causes of identification errors in examination of sentinel lymph node frozen sections by artificial intelligence are, in descending order of influence, normal lymph node structure, micrometastases, section quality, special tumor growth patterns and secondary lymph node reactions. In this study, by constructing an artificial intelligence model to identify the error causes of frozen sections of lymph nodes in breast cancer and by analyzing the model in detail, we found that poor quality of slices was the preproblem of many identification errors, which can lead to other errors, such as unclear recognition of lymph node structure by computer. Therefore, we believe that the process of artificial intelligence pathological diagnosis should be optimized, and the quality control of the pathological sections included in the artificial intelligence reading should be carried out first to exclude the influence of poor section quality on the computer model. For cases of micrometastasis, we suggest that by differentiating slices into high- and low-confidence groups, low-confidence micrometastatic slices can be separated for manual identification. The normal lymph node structure can be improved by adding samples and training the model in a targeted manner.
Collapse
Affiliation(s)
- Zuxuan Zhao
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinses Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Cancan Chen
- Digital Health China Technologies Corporation Limited, Beijing, 100080, China
- Infervision Medical Technology Co., Ltd, Beijing, 100025, China
| | - Hanwen Guan
- School of Health Management, Harbin Medical University, Harbin, 150081, Heilongjiang Province, China
| | - Lei Guo
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinses Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Wanxin Tian
- Department of Medical Affairs, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Xiaoqi Liu
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinses Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Huijuan Zhang
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinses Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital/ Shenzhen Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Shenzhen, 528116, China
| | - Jiangtao Li
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinses Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Tinglin Qiu
- Department of Medical Affairs, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Jun Du
- Administration Office, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Qiang Guo
- Department of Big Data, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Fenglong Sun
- Digital Health China Technologies Corporation Limited, Beijing, 100080, China.
- Healthcare IT Department, Genertec Universal Medical Group Co., Ltd, Beijing, 100062, China.
| | - Shan Zheng
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinses Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China.
| | - Jianhui Ma
- Department of Urology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| |
Collapse
|
5
|
Gorman BG, Lifson MA, Vidal NY. Artificial intelligence and frozen section histopathology: A systematic review. J Cutan Pathol 2023; 50:852-859. [PMID: 37394789 DOI: 10.1111/cup.14481] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2022] [Revised: 05/14/2023] [Accepted: 05/29/2023] [Indexed: 07/04/2023]
Abstract
Frozen sections are a useful pathologic tool, but variable image quality may impede the use of artificial intelligence and machine learning in their interpretation. We aimed to identify the current research on machine learning models trained or tested on frozen section images. We searched PubMed and Web of Science for articles presenting new machine learning models published in any year. Eighteen papers met all inclusion criteria. All papers presented at least one novel model trained or tested on frozen section images. Overall, convolutional neural networks tended to have the best performance. When physicians were able to view the output of the model, they tended to perform better than either the model or physicians alone at the tested task. Models trained on frozen sections performed well when tested on other slide preparations, but models trained on only formalin-fixed tissue performed significantly worse across other modalities. This suggests not only that machine learning can be applied to frozen section image processing, but also use of frozen section images may increase model generalizability. Additionally, expert physicians working in concert with artificial intelligence may be the future of frozen section histopathology.
Collapse
Affiliation(s)
- Benjamin G Gorman
- Mayo Clinic Alix School of Medicine, Rochester, Minnesota, USA
- Mayo Clinic Graduate School of Biomedical Sciences, Rochester, Minnesota, USA
| | - Mark A Lifson
- Center for Digital Health, Mayo Clinic, Rochester, Minnesota, USA
| | - Nahid Y Vidal
- Department of Dermatology, Mayo Clinic, Rochester, Minnesota, USA
- Division of Dermatologic Surgery, Mayo Clinic, Rochester, Minnesota, USA
| |
Collapse
|
6
|
Kim YG, Kim S, Cho CE, Song IH, Lee HJ, Ahn S, Park SY, Gong G, Kim N. Effectiveness of transfer learning for enhancing tumor classification with a convolutional neural network on frozen sections. Sci Rep 2020; 10:21899. [PMID: 33318495 PMCID: PMC7736325 DOI: 10.1038/s41598-020-78129-0] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2020] [Accepted: 11/18/2020] [Indexed: 12/20/2022] Open
Abstract
Fast and accurate confirmation of metastasis on the frozen tissue section of intraoperative sentinel lymph node biopsy is an essential tool for critical surgical decisions. However, accurate diagnosis by pathologists is difficult within the time limitations. Training a robust and accurate deep learning model is also difficult owing to the limited number of frozen datasets with high quality labels. To overcome these issues, we validated the effectiveness of transfer learning from CAMELYON16 to improve performance of the convolutional neural network (CNN)-based classification model on our frozen dataset (N = 297) from Asan Medical Center (AMC). Among the 297 whole slide images (WSIs), 157 and 40 WSIs were used to train deep learning models with different dataset ratios at 2, 4, 8, 20, 40, and 100%. The remaining, i.e., 100 WSIs, were used to validate model performance in terms of patch- and slide-level classification. An additional 228 WSIs from Seoul National University Bundang Hospital (SNUBH) were used as an external validation. Three initial weights, i.e., scratch-based (random initialization), ImageNet-based, and CAMELYON16-based models were used to validate their effectiveness in external validation. In the patch-level classification results on the AMC dataset, CAMELYON16-based models trained with a small dataset (up to 40%, i.e., 62 WSIs) showed a significantly higher area under the curve (AUC) of 0.929 than those of the scratch- and ImageNet-based models at 0.897 and 0.919, respectively, while CAMELYON16-based and ImageNet-based models trained with 100% of the training dataset showed comparable AUCs at 0.944 and 0.943, respectively. For the external validation, CAMELYON16-based models showed higher AUCs than those of the scratch- and ImageNet-based models. Model performance for slide feasibility of the transfer learning to enhance model performance was validated in the case of frozen section datasets with limited numbers.
Collapse
Affiliation(s)
- Young-Gon Kim
- Transdisciplinary Department of Medicine & Advanced Technology, Seoul National University Hospital, Seoul, 03080, Korea
| | - Sungchul Kim
- Department of Convergence Medicine, Asan Institute of Life Science, University of Ulsan College of Medicine, Asan Medical Center, Seoul, 05505, Korea
| | - Cristina Eunbee Cho
- Department of Convergence Medicine, Asan Institute of Life Science, University of Ulsan College of Medicine, Asan Medical Center, Seoul, 05505, Korea
| | - In Hye Song
- Department of Hospital Pathology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, 06591, Korea
| | - Hee Jin Lee
- Department of Pathology, University of Ulsan, College of Medicine, Asan Medical Center, Seoul, 05505, Korea
| | - Soomin Ahn
- Department of Pathology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Gyeonggi, 13620, Korea
| | - So Yeon Park
- Department of Pathology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Gyeonggi, 13620, Korea
| | - Gyungyub Gong
- Department of Pathology, University of Ulsan, College of Medicine, Asan Medical Center, Seoul, 05505, Korea
| | - Namkug Kim
- Department of Convergence Medicine, Asan Institute of Life Science, University of Ulsan College of Medicine, Asan Medical Center, Seoul, 05505, Korea.
| |
Collapse
|