1
|
Varnava Y, Jakate K, Garnett R, Androutsos D, Tyrrell PN, Khademi A. Out-of-distribution generalization for segmentation of lymph node metastasis in breast cancer. Sci Rep 2025; 15:1127. [PMID: 39775089 PMCID: PMC11707152 DOI: 10.1038/s41598-024-80495-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2024] [Accepted: 11/19/2024] [Indexed: 01/11/2025] Open
Abstract
Pathology provides the definitive diagnosis, and Artificial Intelligence (AI) tools are poised to improve accuracy, inter-rater agreement, and turn-around time (TAT) of pathologists, leading to improved quality of care. A high value clinical application is the grading of Lymph Node Metastasis (LNM) which is used for breast cancer staging and guides treatment decisions. A challenge of implementing AI tools widely for LNM classification is domain shift, where Out-of-Distribution (OOD) data has a different distribution than the In-Distribution (ID) data used to train the model, resulting in a drop in performance in OOD data. This work proposes a novel clustering and sampling method to automatically curate training datasets in an unsupervised manner with the aim of improving model generalization abilities. To evaluate the generalization performance of the proposed models, we applied a novel use of the Two One-sided Tests (TOST) method. This method examines whether the performance on ID and OOD data is equivalent, serving as a proxy for generalization. We provide the first evidence for computing equivalence margins that are data-dependent, which reduces subjectivity. The proposed framework shows the ensembled models constructed from models that generalized across both tumor and normal patches enhanced performance, achieving an F1 score of 0.81 for LNM classification on unseen ID and OOD samples. Interactive viewing of slide-level segmentations can be accessed on PathcoreFlow™ through https://web.pathcore.com/folder/18555?s=QTJVHJuhrfe5 . Segmentation models are available at https://github.com/IAMLAB-Ryerson/OOD-Generalization-LNM .
Collapse
Affiliation(s)
- Yiannis Varnava
- Department of Electrical, Computer, and Biomedical Engineering, Toronto Metropolitan University, Toronto, ON, Canada.
| | - Kiran Jakate
- Department of Pathology, Unity Health Toronto, Toronto, ON, Canada
| | - Richard Garnett
- Department of Electrical, Computer, and Biomedical Engineering, Toronto Metropolitan University, Toronto, ON, Canada
| | - Dimitrios Androutsos
- Department of Electrical, Computer, and Biomedical Engineering, Toronto Metropolitan University, Toronto, ON, Canada
| | - Pascal N Tyrrell
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Department of Statistical Sciences, University of Toronto, Toronto, ON, Canada
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
| | - April Khademi
- Department of Electrical, Computer, and Biomedical Engineering, Toronto Metropolitan University, Toronto, ON, Canada
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- Keenan Research Centre for Biomedical Science, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Institute for Biomedical Engineering, Science Tech (iBEST), A Partnership Between St. Michael's Hospital and Toronto Metropolitan University, Toronto, ON, Canada
- Vector Institute for Artificial Intelligence, Toronto, ON, Canada
| |
Collapse
|
2
|
Liu H, Zou L, Xu N, Shen H, Zhang Y, Wan P, Wen B, Zhang X, He Y, Gui L, Kong W. Deep learning radiomics based prediction of axillary lymph node metastasis in breast cancer. NPJ Breast Cancer 2024; 10:22. [PMID: 38472210 PMCID: PMC10933422 DOI: 10.1038/s41523-024-00628-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Accepted: 02/28/2024] [Indexed: 03/14/2024] Open
Abstract
This study aimed to develop and validate a deep learning radiomics nomogram (DLRN) for the preoperative evaluation of axillary lymph node (ALN) metastasis status in patients with a newly diagnosed unifocal breast cancer. A total of 883 eligible patients with breast cancer who underwent preoperative breast and axillary ultrasound were retrospectively enrolled between April 1, 2016, and June 30, 2022. The training cohort comprised 621 patients from Hospital I; the external validation cohorts comprised 112, 87, and 63 patients from Hospitals II, III, and IV, respectively. A DLR signature was created based on the deep learning and handcrafted features, and the DLRN was then developed based on the signature and four independent clinical parameters. The DLRN exhibited good performance, yielding areas under the receiver operating characteristic curve (AUC) of 0.914, 0.929, and 0.952 in the three external validation cohorts, respectively. Decision curve and calibration curve analyses demonstrated the favorable clinical value and calibration of the nomogram. In addition, the DLRN outperformed five experienced radiologists in all cohorts. This has the potential to guide appropriate management of the axilla in patients with breast cancer, including avoiding overtreatment.
Collapse
Affiliation(s)
- Han Liu
- Department of Ultrasound, Nanjing Drum Tower Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, 210002, China
| | - Liwen Zou
- Department of Mathematics, Nanjing University, Nanjing, 210008, China
| | - Nan Xu
- Department of Ultrasound, Jinling Hospital, Medical School of Nanjing University/General Hospital of Eastern Theater Command, Nanjing, 210002, China
| | - Haiyun Shen
- Department of Ultrasound, Nanjing Drum Tower Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, 210002, China
| | - Yu Zhang
- Department of Mathematics, Nanjing University, Nanjing, 210008, China
| | - Peng Wan
- College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing, 211106, China
| | - Baojie Wen
- Department of Ultrasound, Nanjing Drum Tower Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, 210002, China
| | - Xiaojing Zhang
- Department of Ultrasound, Taizhou Hospital Affiliated to Nanjing University of Chinese Medicine, Taizhou, 225300, China
| | - Yuhong He
- Department of Ultrasound, Nanjing Drum Tower Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, 210002, China
| | - Luying Gui
- School of Mathematics and Statistics, Nanjing University of Science and Technology, Nanjing, 210094, China.
| | - Wentao Kong
- Department of Ultrasound, Nanjing Drum Tower Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, 210002, China.
| |
Collapse
|