1
|
Tafavvoghi M, Bongo LA, Shvetsov N, Busund LTR, Møllersen K. Publicly available datasets of breast histopathology H&E whole-slide images: A scoping review. J Pathol Inform 2024; 15:100363. [PMID: 38405160 PMCID: PMC10884505 DOI: 10.1016/j.jpi.2024.100363] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 11/24/2023] [Accepted: 01/23/2024] [Indexed: 02/27/2024] Open
Abstract
Advancements in digital pathology and computing resources have made a significant impact in the field of computational pathology for breast cancer diagnosis and treatment. However, access to high-quality labeled histopathological images of breast cancer is a big challenge that limits the development of accurate and robust deep learning models. In this scoping review, we identified the publicly available datasets of breast H&E-stained whole-slide images (WSIs) that can be used to develop deep learning algorithms. We systematically searched 9 scientific literature databases and 9 research data repositories and found 17 publicly available datasets containing 10 385 H&E WSIs of breast cancer. Moreover, we reported image metadata and characteristics for each dataset to assist researchers in selecting proper datasets for specific tasks in breast cancer computational pathology. In addition, we compiled 2 lists of breast H&E patches and private datasets as supplementary resources for researchers. Notably, only 28% of the included articles utilized multiple datasets, and only 14% used an external validation set, suggesting that the performance of other developed models may be susceptible to overestimation. The TCGA-BRCA was used in 52% of the selected studies. This dataset has a considerable selection bias that can impact the robustness and generalizability of the trained algorithms. There is also a lack of consistent metadata reporting of breast WSI datasets that can be an issue in developing accurate deep learning models, indicating the necessity of establishing explicit guidelines for documenting breast WSI dataset characteristics and metadata.
Collapse
Affiliation(s)
- Masoud Tafavvoghi
- Department of Community Medicine, Uit The Arctic University of Norway, Tromsø, Norway
| | - Lars Ailo Bongo
- Department of Computer Science, Uit The Arctic University of Norway, Tromsø, Norway
| | - Nikita Shvetsov
- Department of Computer Science, Uit The Arctic University of Norway, Tromsø, Norway
| | | | - Kajsa Møllersen
- Department of Community Medicine, Uit The Arctic University of Norway, Tromsø, Norway
| |
Collapse
|
2
|
McCaffrey C, Jahangir C, Murphy C, Burke C, Gallagher WM, Rahman A. Artificial intelligence in digital histopathology for predicting patient prognosis and treatment efficacy in breast cancer. Expert Rev Mol Diagn 2024; 24:363-377. [PMID: 38655907 DOI: 10.1080/14737159.2024.2346545] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Accepted: 04/19/2024] [Indexed: 04/26/2024]
Abstract
INTRODUCTION Histological images contain phenotypic information predictive of patient outcomes. Due to the heavy workload of pathologists, the time-consuming nature of quantitatively assessing histological features, and human eye limitations to recognize spatial patterns, manually extracting prognostic information in routine pathological workflows remains challenging. Digital pathology has facilitated the mining and quantification of these features utilizing whole-slide image (WSI) scanners and artificial intelligence (AI) algorithms. AI algorithms to identify image-based biomarkers from the tumor microenvironment (TME) have the potential to revolutionize the field of oncology, reducing delays between diagnosis and prognosis determination, allowing for rapid stratification of patients and prescription of optimal treatment regimes, thereby improving patient outcomes. AREAS COVERED In this review, the authors discuss how AI algorithms and digital pathology can predict breast cancer patient prognosis and treatment outcomes using image-based biomarkers, along with the challenges of adopting this technology in clinical settings. EXPERT OPINION The integration of AI and digital pathology presents significant potential for analyzing the TME and its diagnostic, prognostic, and predictive value in breast cancer patients. Widespread clinical adoption of AI faces ethical, regulatory, and technical challenges, although prospective trials may offer reassurance and promote uptake, ultimately improving patient outcomes by reducing diagnosis-to-prognosis delivery delays.
Collapse
Affiliation(s)
- Christine McCaffrey
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Chowdhury Jahangir
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Clodagh Murphy
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Caoimbhe Burke
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - William M Gallagher
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Arman Rahman
- UCD School of Medicine, UCD Conway Institute, University College Dublin, Dublin, Ireland
| |
Collapse
|
3
|
Fisher TB, Saini G, Rekha TS, Krishnamurthy J, Bhattarai S, Callagy G, Webber M, Janssen EAM, Kong J, Aneja R. Digital image analysis and machine learning-assisted prediction of neoadjuvant chemotherapy response in triple-negative breast cancer. Breast Cancer Res 2024; 26:12. [PMID: 38238771 PMCID: PMC10797728 DOI: 10.1186/s13058-023-01752-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Accepted: 12/11/2023] [Indexed: 01/22/2024] Open
Abstract
BACKGROUND Pathological complete response (pCR) is associated with favorable prognosis in patients with triple-negative breast cancer (TNBC). However, only 30-40% of TNBC patients treated with neoadjuvant chemotherapy (NAC) show pCR, while the remaining 60-70% show residual disease (RD). The role of the tumor microenvironment in NAC response in patients with TNBC remains unclear. In this study, we developed a machine learning-based two-step pipeline to distinguish between various histological components in hematoxylin and eosin (H&E)-stained whole slide images (WSIs) of TNBC tissue biopsies and to identify histological features that can predict NAC response. METHODS H&E-stained WSIs of treatment-naïve biopsies from 85 patients (51 with pCR and 34 with RD) of the model development cohort and 79 patients (41 with pCR and 38 with RD) of the validation cohort were separated through a stratified eightfold cross-validation strategy for the first step and leave-one-out cross-validation strategy for the second step. A tile-level histology label prediction pipeline and four machine-learning classifiers were used to analyze 468,043 tiles of WSIs. The best-trained classifier used 55 texture features from each tile to produce a probability profile during testing. The predicted histology classes were used to generate a histology classification map of the spatial distributions of different tissue regions. A patient-level NAC response prediction pipeline was trained with features derived from paired histology classification maps. The top graph-based features capturing the relevant spatial information across the different histological classes were provided to the radial basis function kernel support vector machine (rbfSVM) classifier for NAC treatment response prediction. RESULTS The tile-level prediction pipeline achieved 86.72% accuracy for histology class classification, while the patient-level pipeline achieved 83.53% NAC response (pCR vs. RD) prediction accuracy of the model development cohort. The model was validated with an independent cohort with tile histology validation accuracy of 83.59% and NAC prediction accuracy of 81.01%. The histological class pairs with the strongest NAC response predictive ability were tumor and tumor tumor-infiltrating lymphocytes for pCR and microvessel density and polyploid giant cancer cells for RD. CONCLUSION Our machine learning pipeline can robustly identify clinically relevant histological classes that predict NAC response in TNBC patients and may help guide patient selection for NAC treatment.
Collapse
Affiliation(s)
- Timothy B Fisher
- Department of Biology, Georgia State University, Atlanta, GA, 30302, USA
| | - Geetanjali Saini
- School of Health Professions, University of Alabama at Birmingham, Birmingham, AL, 35294, USA
| | - T S Rekha
- JSSAHER (JSS Academy of Higher Education and Research) Medical College, Mysuru, Karnataka, India
| | - Jayashree Krishnamurthy
- JSSAHER (JSS Academy of Higher Education and Research) Medical College, Mysuru, Karnataka, India
| | - Shristi Bhattarai
- School of Health Professions, University of Alabama at Birmingham, Birmingham, AL, 35294, USA
| | - Grace Callagy
- Discipline of Pathology, University of Galway, Galway, Ireland
| | - Mark Webber
- Discipline of Pathology, University of Galway, Galway, Ireland
| | - Emiel A M Janssen
- Department of Pathology, Stavanger University Hospital, Stavanger, Norway
- Department of Chemistry, Bioscience and Environmental Engineering, University of Stavanger, Stavanger, Norway
| | - Jun Kong
- Department of Mathematics and Statistics, Georgia State University, Atlanta, GA, 30303, USA.
| | - Ritu Aneja
- Department of Biology, Georgia State University, Atlanta, GA, 30302, USA.
- School of Health Professions, University of Alabama at Birmingham, Birmingham, AL, 35294, USA.
| |
Collapse
|
4
|
Bhattarai S, Saini G, Li H, Seth G, Fisher TB, Janssen EAM, Kiraz U, Kong J, Aneja R. Predicting Neoadjuvant Treatment Response in Triple-Negative Breast Cancer Using Machine Learning. Diagnostics (Basel) 2023; 14:74. [PMID: 38201383 PMCID: PMC10871101 DOI: 10.3390/diagnostics14010074] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Revised: 12/19/2023] [Accepted: 12/21/2023] [Indexed: 01/12/2024] Open
Abstract
BACKGROUND Neoadjuvant chemotherapy (NAC) is the standard treatment for early-stage triple negative breast cancer (TNBC). The primary endpoint of NAC is a pathological complete response (pCR). NAC results in pCR in only 30-40% of TNBC patients. Tumor-infiltrating lymphocytes (TILs), Ki67 and phosphohistone H3 (pH3) are a few known biomarkers to predict NAC response. Currently, systematic evaluation of the combined value of these biomarkers in predicting NAC response is lacking. In this study, the predictive value of markers derived from H&E and IHC stained biopsy tissue was comprehensively evaluated using a supervised machine learning (ML)-based approach. Identifying predictive biomarkers could help guide therapeutic decisions by enabling precise stratification of TNBC patients into responders and partial or non-responders. METHODS Serial sections from core needle biopsies (n = 76) were stained with H&E and immunohistochemically for the Ki67 and pH3 markers, followed by whole-slide image (WSI) generation. The serial section stains in H&E stain, Ki67 and pH3 markers formed WSI triplets for each patient. The resulting WSI triplets were co-registered with H&E WSIs serving as the reference. Separate mask region-based CNN (MRCNN) models were trained with annotated H&E, Ki67 and pH3 images for detecting tumor cells, stromal and intratumoral TILs (sTILs and tTILs), Ki67+, and pH3+ cells. Top image patches with a high density of cells of interest were identified as hotspots. Best classifiers for NAC response prediction were identified by training multiple ML models and evaluating their performance by accuracy, area under curve, and confusion matrix analyses. RESULTS Highest prediction accuracy was achieved when hotspot regions were identified by tTIL counts and each hotspot was represented by measures of tTILs, sTILs, tumor cells, Ki67+, and pH3+ features. Regardless of the hotspot selection metric, a complementary use of multiple histological features (tTILs, sTILs) and molecular biomarkers (Ki67 and pH3) resulted in top ranked performance at the patient level. CONCLUSIONS Overall, our results emphasize that prediction models for NAC response should be based on biomarkers in combination rather than in isolation. Our study provides compelling evidence to support the use of ML-based models to predict NAC response in patients with TNBC.
Collapse
Affiliation(s)
- Shristi Bhattarai
- Department of Clinical and Diagnostic Sciences, School of Health Professions, University of Alabama at Birmingham, Birmingham, AL 35294, USA; (S.B.); (G.S.); (G.S.)
| | - Geetanjali Saini
- Department of Clinical and Diagnostic Sciences, School of Health Professions, University of Alabama at Birmingham, Birmingham, AL 35294, USA; (S.B.); (G.S.); (G.S.)
| | - Hongxiao Li
- Department of Mathematics and Statistics, Georgia State University, Atlanta, GA 30302, USA;
| | - Gaurav Seth
- Department of Clinical and Diagnostic Sciences, School of Health Professions, University of Alabama at Birmingham, Birmingham, AL 35294, USA; (S.B.); (G.S.); (G.S.)
| | - Timothy B. Fisher
- Department of Biology, Georgia State University, Atlanta, GA 30302, USA;
| | - Emiel A. M. Janssen
- Department of Pathology, Stavanger University Hospital, 4011 Stavanger, Norway; (E.A.M.J.); (U.K.)
- Department of Chemistry, Bioscience and Environmental Engineering, Stavanger University, 4021 Stavanger, Norway
| | - Umay Kiraz
- Department of Pathology, Stavanger University Hospital, 4011 Stavanger, Norway; (E.A.M.J.); (U.K.)
- Department of Chemistry, Bioscience and Environmental Engineering, Stavanger University, 4021 Stavanger, Norway
| | - Jun Kong
- Department of Mathematics and Statistics, Georgia State University, Atlanta, GA 30302, USA;
| | - Ritu Aneja
- Department of Clinical and Diagnostic Sciences, School of Health Professions, University of Alabama at Birmingham, Birmingham, AL 35294, USA; (S.B.); (G.S.); (G.S.)
| |
Collapse
|
5
|
Aswolinskiy W, Munari E, Horlings HM, Mulder L, Bogina G, Sanders J, Liu YH, van den Belt-Dusebout AW, Tessier L, Balkenhol M, Stegeman M, Hoven J, Wesseling J, van der Laak J, Lips EH, Ciompi F. PROACTING: predicting pathological complete response to neoadjuvant chemotherapy in breast cancer from routine diagnostic histopathology biopsies with deep learning. Breast Cancer Res 2023; 25:142. [PMID: 37957667 PMCID: PMC10644597 DOI: 10.1186/s13058-023-01726-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 10/02/2023] [Indexed: 11/15/2023] Open
Abstract
BACKGROUND Invasive breast cancer patients are increasingly being treated with neoadjuvant chemotherapy; however, only a fraction of the patients respond to it completely. To prevent overtreatment, there is an urgent need for biomarkers to predict treatment response before administering the therapy. METHODS In this retrospective study, we developed hypothesis-driven interpretable biomarkers based on deep learning, to predict the pathological complete response (pCR, i.e., the absence of tumor cells in the surgical resection specimens) to neoadjuvant chemotherapy solely using digital pathology H&E images of pre-treatment breast biopsies. Our approach consists of two steps: First, we use deep learning to characterize aspects of the tumor micro-environment by detecting mitoses and segmenting tissue into several morphology compartments including tumor, lymphocytes and stroma. Second, we derive computational biomarkers from the segmentation and detection output to encode slide-level relationships of components of the tumor microenvironment, such as tumor and mitoses, stroma, and tumor infiltrating lymphocytes (TILs). RESULTS We developed and evaluated our method on slides from n = 721 patients from three European medical centers with triple-negative and Luminal B breast cancers and performed external independent validation on n = 126 patients from a public dataset. We report the predictive value of the investigated biomarkers for predicting pCR with areas under the receiver operating characteristic curve between 0.66 and 0.88 across the tested cohorts. CONCLUSION The proposed computational biomarkers predict pCR, but will require more evaluation and finetuning for clinical application. Our results further corroborate the potential role of deep learning to automate TILs quantification, and their predictive value in breast cancer neoadjuvant treatment planning, along with automated mitoses quantification. We made our method publicly available to extract segmentation-based biomarkers for research purposes.
Collapse
Affiliation(s)
- Witali Aswolinskiy
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Enrico Munari
- Department of Molecular and Translational Medicine, University of Brescia, Brescia, Italy
| | - Hugo M Horlings
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | - Lennart Mulder
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | - Giuseppe Bogina
- Pathology Unit, IRCCS Sacro Cuore Don Calabria Hospital, Negrar di Valpolicella, Verona, Italy
| | - Joyce Sanders
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | - Yat-Hee Liu
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | | | - Leslie Tessier
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
- Center for Integrated Oncology (Institut du cancer de l'Ouest), Angers, France
| | - Maschenka Balkenhol
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Michelle Stegeman
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jeffrey Hoven
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jelle Wesseling
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
- Leiden University Medical Center, Leiden, The Netherlands
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Esther H Lips
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands.
| |
Collapse
|
6
|
Gonzàlez-Farré M, Gibert J, Santiago-Díaz P, Santos J, García P, Massó J, Bellosillo B, Lloveras B, Albanell J, Vázquez I, Comerma L. Automated quantification of stromal tumour infiltrating lymphocytes is associated with prognosis in breast cancer. Virchows Arch 2023; 483:655-663. [PMID: 37500796 DOI: 10.1007/s00428-023-03608-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2023] [Revised: 07/10/2023] [Accepted: 07/17/2023] [Indexed: 07/29/2023]
Abstract
Stromal tumour infiltrating lymphocytes (sTIL) in haematoxylin and eosin (H&E) stained sections has been linked to better outcomes and better responses to neoadjuvant therapy in triple-negative and HER2-positive breast cancer (TNBC and HER2 +). However, the infiltrate includes different cell populations that have specific roles in the tumour immune microenvironment. Various studies have found high concordance between sTIL visual quantification and computational assessment, but specific data on the individual prognostic impact of plasma cells or lymphocytes within sTIL on patient prognosis is still unknown. In this study, we validated a deep-learning breast cancer sTIL scoring model (smsTIL) based on the segmentation of tumour cells, benign ductal cells, lymphocytes, plasma cells, necrosis, and 'other' cells in whole slide images (WSI). Focusing on HER2 + and TNBC patient samples, we assessed the concordance between sTIL visual scoring and the smsTIL in 130 WSI. Furthermore, we analysed 175 WSI to correlate smsTIL with clinical data and patient outcomes. We found a high correlation between sTIL values scored visually and semi-automatically (R = 0.76; P = 2.2e-16). Patients with higher smsTIL had better overall survival (OS) in TNBC (P = 0.0021). In the TNBC cohort, smsTIL was as an independent prognostic factor for OS. As part of this work, we introduce a new segmentation dataset of H&E-stained WSI.
Collapse
Affiliation(s)
- Mònica Gonzàlez-Farré
- Department of Pathology, Hospital del Mar, Passeig Marítim de la Barceloneta 25-29, 08003, Barcelona, Spain.
- Cancer Research Program, IMIM (Hospital del Mar Medical Research Institute), 08003, Barcelona, Spain.
| | - Joan Gibert
- Department of Pathology, Hospital del Mar, Passeig Marítim de la Barceloneta 25-29, 08003, Barcelona, Spain
- Cancer Research Program, IMIM (Hospital del Mar Medical Research Institute), 08003, Barcelona, Spain
| | - Pablo Santiago-Díaz
- Department of Pathology, Hospital del Mar, Passeig Marítim de la Barceloneta 25-29, 08003, Barcelona, Spain
| | - Jordina Santos
- Department of Pathology, Hospital del Mar, Passeig Marítim de la Barceloneta 25-29, 08003, Barcelona, Spain
| | - Pilar García
- Department of Pathology, Hospital del Mar, Passeig Marítim de la Barceloneta 25-29, 08003, Barcelona, Spain
| | - Jordi Massó
- Department of Pathology, Hospital del Mar, Passeig Marítim de la Barceloneta 25-29, 08003, Barcelona, Spain
| | - Beatriz Bellosillo
- Department of Pathology, Hospital del Mar, Passeig Marítim de la Barceloneta 25-29, 08003, Barcelona, Spain
- Cancer Research Program, IMIM (Hospital del Mar Medical Research Institute), 08003, Barcelona, Spain
- Department of Medicine and Life Sciences (MELIS), University Pompeu Fabra, Doctor Aiguader 88, 08003, Barcelona, Spain
| | - Belén Lloveras
- Department of Pathology, Hospital del Mar, Passeig Marítim de la Barceloneta 25-29, 08003, Barcelona, Spain
- Cancer Research Program, IMIM (Hospital del Mar Medical Research Institute), 08003, Barcelona, Spain
- Department of Medicine and Life Sciences (MELIS), University Pompeu Fabra, Doctor Aiguader 88, 08003, Barcelona, Spain
| | - Joan Albanell
- Cancer Research Program, IMIM (Hospital del Mar Medical Research Institute), 08003, Barcelona, Spain
- Department of Medicine and Life Sciences (MELIS), University Pompeu Fabra, Doctor Aiguader 88, 08003, Barcelona, Spain
- Department of Medical Oncology, Hospital del Mar, 08003, Barcelona, Spain
- Center for Biomedical Network Research On Cancer (CIBERONC), 28029, Madrid, Spain
| | - Ivonne Vázquez
- Department of Pathology, Hospital del Mar, Passeig Marítim de la Barceloneta 25-29, 08003, Barcelona, Spain
| | - Laura Comerma
- Department of Pathology, Hospital del Mar, Passeig Marítim de la Barceloneta 25-29, 08003, Barcelona, Spain
- Cancer Research Program, IMIM (Hospital del Mar Medical Research Institute), 08003, Barcelona, Spain
| |
Collapse
|
7
|
Choi S, Cho SI, Jung W, Lee T, Choi SJ, Song S, Park G, Park S, Ma M, Pereira S, Yoo D, Shin S, Ock CY, Kim S. Deep learning model improves tumor-infiltrating lymphocyte evaluation and therapeutic response prediction in breast cancer. NPJ Breast Cancer 2023; 9:71. [PMID: 37648694 PMCID: PMC10469174 DOI: 10.1038/s41523-023-00577-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Accepted: 08/17/2023] [Indexed: 09/01/2023] Open
Abstract
Tumor-infiltrating lymphocytes (TILs) have been recognized as key players in the tumor microenvironment of breast cancer, but substantial interobserver variability among pathologists has impeded its utility as a biomarker. We developed a deep learning (DL)-based TIL analyzer to evaluate stromal TILs (sTILs) in breast cancer. Three pathologists evaluated 402 whole slide images of breast cancer and interpreted the sTIL scores. A standalone performance of the DL model was evaluated in the 210 cases (52.2%) exhibiting sTIL score differences of less than 10 percentage points, yielding a concordance correlation coefficient of 0.755 (95% confidence interval [CI], 0.693-0.805) in comparison to the pathologists' scores. For the 226 slides (56.2%) showing a 10 percentage points or greater variance between pathologists and the DL model, revisions were made. The number of discordant cases was reduced to 116 (28.9%) with the DL assistance (p < 0.001). The DL assistance also increased the concordance correlation coefficient of the sTIL score among every two pathologists. In triple-negative and human epidermal growth factor receptor 2 (HER2)-positive breast cancer patients who underwent the neoadjuvant chemotherapy, the DL-assisted revision notably accentuated higher sTIL scores in responders (26.8 ± 19.6 vs. 19.0 ± 16.4, p = 0.003). Furthermore, the DL-assistant revision disclosed the correlation of sTIL-high tumors (sTIL ≥ 50) with the chemotherapeutic response (odd ratio 1.28 [95% confidence interval, 1.01-1.63], p = 0.039). Through enhancing inter-pathologist concordance in sTIL interpretation and predicting neoadjuvant chemotherapy response, here we report the utility of the DL-based tool as a reference for sTIL scoring in breast cancer assessment.
Collapse
Affiliation(s)
- Sangjoon Choi
- Department of Pathology and Translational Genomics, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | | | | | | | - Su Jin Choi
- Department of Pathology, Ajou University School of Medicine, Suwon, Republic of Korea
- Department of Biomedical Sciences, Ajou University Graduate School of Medicine, Suwon, Republic of Korea
| | | | | | | | - Minuk Ma
- Lunit Inc, Seoul, Republic of Korea
| | | | | | | | | | - Seokhwi Kim
- Department of Pathology, Ajou University School of Medicine, Suwon, Republic of Korea.
- Department of Biomedical Sciences, Ajou University Graduate School of Medicine, Suwon, Republic of Korea.
| |
Collapse
|
8
|
Fisher TB, Saini G, Ts R, Krishnamurthy J, Bhattarai S, Callagy G, Webber M, Janssen EAM, Kong J, Aneja R. Digital image analysis and machine learning-assisted prediction of neoadjuvant chemotherapy response in triple-negative breast cancer. RESEARCH SQUARE 2023:rs.3.rs-3243195. [PMID: 37645881 PMCID: PMC10462230 DOI: 10.21203/rs.3.rs-3243195/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/31/2023]
Abstract
Background Pathological complete response (pCR) is associated with favorable prognosis in patients with triple-negative breast cancer (TNBC). However, only 30-40% of TNBC patients treated with neoadjuvant chemotherapy (NAC) show pCR, while the remaining 60-70% show residual disease (RD). The role of the tumor microenvironment (TME) in NAC response in patients with TNBC remains unclear. In this study, we developed a machine learning-based two-step pipeline to distinguish between various histological components in hematoxylin and eosin (H&E)-stained whole slide images (WSIs) of TNBC tissue biopsies and to identify histological features that can predict NAC response. Methods H&E-stained WSIs of treatment-naïve biopsies from 85 patients (51 with pCR and 34 with RD) were separated through a stratified 8-fold cross validation strategy for the first step and leave one out cross validation strategy for the second step. A tile-level histology label prediction pipeline and four machine learning classifiers were used to analyze 468,043 tiles of WSIs. The best-trained classifier used 55 texture features from each tile to produce a probability profile during testing. The predicted histology classes were used to generate a histology classification map of the spatial distributions of different tissue regions. A patient-level NAC response prediction pipeline was trained with features derived from paired histology classification maps. The top graph-based features capturing the relevant spatial information across the different histological classes were provided to the radial basis function kernel support vector machine (rbfSVM) classifier for NAC treatment response prediction. Results The tile-level prediction pipeline achieved 86.72% accuracy for histology class classification, while the patient-level pipeline achieved 83.53% NAC response (pCR vs. RD) prediction accuracy. The histological class pairs with the strongest NAC response predictive ability were tumor and tumor tumor-infiltrating lymphocytes for pCR and microvessel density and polyploid giant cancer cells for RD. Conclusion Our machine learning pipeline can robustly identify clinically relevant histological classes that predict NAC response in TNBC patients and may help guide patient selection for NAC treatment.
Collapse
Affiliation(s)
| | | | - Rekha Ts
- JSSAHER (JSS Academy of Higher Education and Research) Medical College
| | | | | | | | | | | | | | | |
Collapse
|
9
|
Thagaard J, Broeckx G, Page DB, Jahangir CA, Verbandt S, Kos Z, Gupta R, Khiroya R, Abduljabbar K, Acosta Haab G, Acs B, Akturk G, Almeida JS, Alvarado‐Cabrero I, Amgad M, Azmoudeh‐Ardalan F, Badve S, Baharun NB, Balslev E, Bellolio ER, Bheemaraju V, Blenman KRM, Botinelly Mendonça Fujimoto L, Bouchmaa N, Burgues O, Chardas A, Chon U Cheang M, Ciompi F, Cooper LAD, Coosemans A, Corredor G, Dahl AB, Dantas Portela FL, Deman F, Demaria S, Doré Hansen J, Dudgeon SN, Ebstrup T, Elghazawy M, Fernandez‐Martín C, Fox SB, Gallagher WM, Giltnane JM, Gnjatic S, Gonzalez‐Ericsson PI, Grigoriadis A, Halama N, Hanna MG, Harbhajanka A, Hart SN, Hartman J, Hauberg S, Hewitt S, Hida AI, Horlings HM, Husain Z, Hytopoulos E, Irshad S, Janssen EAM, Kahila M, Kataoka TR, Kawaguchi K, Kharidehal D, Khramtsov AI, Kiraz U, Kirtani P, Kodach LL, Korski K, Kovács A, Laenkholm A, Lang‐Schwarz C, Larsimont D, Lennerz JK, Lerousseau M, Li X, Ly A, Madabhushi A, Maley SK, Manur Narasimhamurthy V, Marks DK, McDonald ES, Mehrotra R, Michiels S, Minhas FUAA, Mittal S, Moore DA, Mushtaq S, Nighat H, Papathomas T, Penault‐Llorca F, Perera RD, Pinard CJ, Pinto‐Cardenas JC, Pruneri G, Pusztai L, Rahman A, Rajpoot NM, Rapoport BL, Rau TT, Reis‐Filho JS, Ribeiro JM, Rimm D, Roslind A, Vincent‐Salomon A, Salto‐Tellez M, Saltz J, Sayed S, Scott E, Siziopikou KP, Sotiriou C, Stenzinger A, Sughayer MA, Sur D, Fineberg S, Symmans F, Tanaka S, Taxter T, Tejpar S, Teuwen J, Thompson EA, Tramm T, Tran WT, van der Laak J, van Diest PJ, Verghese GE, Viale G, Vieth M, Wahab N, Walter T, Waumans Y, Wen HY, Yang W, Yuan Y, Zin RM, Adams S, Bartlett J, Loibl S, Denkert C, Savas P, Loi S, Salgado R, Specht Stovgaard E. Pitfalls in machine learning-based assessment of tumor-infiltrating lymphocytes in breast cancer: A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer. J Pathol 2023; 260:498-513. [PMID: 37608772 PMCID: PMC10518802 DOI: 10.1002/path.6155] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Accepted: 06/07/2023] [Indexed: 08/24/2023]
Abstract
The clinical significance of the tumor-immune interaction in breast cancer is now established, and tumor-infiltrating lymphocytes (TILs) have emerged as predictive and prognostic biomarkers for patients with triple-negative (estrogen receptor, progesterone receptor, and HER2-negative) breast cancer and HER2-positive breast cancer. How computational assessments of TILs might complement manual TIL assessment in trial and daily practices is currently debated. Recent efforts to use machine learning (ML) to automatically evaluate TILs have shown promising results. We review state-of-the-art approaches and identify pitfalls and challenges of automated TIL evaluation by studying the root cause of ML discordances in comparison to manual TIL quantification. We categorize our findings into four main topics: (1) technical slide issues, (2) ML and image analysis aspects, (3) data challenges, and (4) validation issues. The main reason for discordant assessments is the inclusion of false-positive areas or cells identified by performance on certain tissue patterns or design choices in the computational implementation. To aid the adoption of ML for TIL assessment, we provide an in-depth discussion of ML and image analysis, including validation issues that need to be considered before reliable computational reporting of TILs can be incorporated into the trial and routine clinical management of patients with triple-negative breast cancer. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Jeppe Thagaard
- Technical University of DenmarkKongens LyngbyDenmark
- Visiopharm A/SHørsholmDenmark
| | - Glenn Broeckx
- Department of PathologyGZA‐ZNA HospitalsAntwerpBelgium
- Centre for Oncological Research (CORE), MIPPRO, Faculty of MedicineAntwerp UniversityAntwerpBelgium
| | - David B Page
- Earle A Chiles Research InstituteProvidence Cancer InstitutePortlandORUSA
| | - Chowdhury Arif Jahangir
- UCD School of Biomolecular and Biomedical Science, UCD Conway InstituteUniversity College DublinDublinIreland
| | - Sara Verbandt
- Digestive Oncology, Department of OncologyKU LeuvenLeuvenBelgium
| | - Zuzana Kos
- Department of Pathology and Laboratory MedicineBC Cancer Vancouver Centre, University of British ColumbiaVancouverBritish ColumbiaCanada
| | - Rajarsi Gupta
- Department of Biomedical InformaticsStony Brook UniversityStony BrookNYUSA
| | - Reena Khiroya
- Department of Cellular PathologyUniversity College Hospital LondonLondonUK
| | | | | | - Balazs Acs
- Department of Oncology and PathologyKarolinska InstitutetStockholmSweden
- Department of Clinical Pathology and Cancer DiagnosticsKarolinska University HospitalStockholmSweden
| | - Guray Akturk
- Translational Molecular Biomarkers, Merck & Co IncRahwayNJUSA
| | - Jonas S Almeida
- Division of Cancer Epidemiology and Genetics (DCEG)National Cancer Institute (NCI)Rockville, MDUSA
| | | | - Mohamed Amgad
- Department of PathologyNorthwestern University Feinberg School of MedicineChicagoILUSA
| | | | - Sunil Badve
- Department of Pathology and Laboratory Medicine, Emory University School of MedicineEmory University Winship Cancer InstituteAtlantaGAUSA
| | | | - Eva Balslev
- Department of PathologyHerlev and Gentofte HospitalHerlevDenmark
| | - Enrique R Bellolio
- Departamento de Anatomía Patológica, Facultad de MedicinaUniversidad de La FronteraTemucoChile
| | | | - Kim RM Blenman
- Department of Internal Medicine Section of Medical Oncology and Yale Cancer CenterYale School of MedicineNew HavenCTUSA
- Department of Computer ScienceYale School of Engineering and Applied ScienceNew HavenCTUSA
| | | | - Najat Bouchmaa
- Institute of Biological Sciences, Faculty of Medical SciencesMohammed VI Polytechnic University (UM6P)Ben‐GuerirMorocco
| | - Octavio Burgues
- Pathology DepartmentHospital Cliníco Universitario de Valencia/InclivaValenciaSpain
| | - Alexandros Chardas
- Department of Pathobiology & Population SciencesThe Royal Veterinary CollegeLondonUK
| | - Maggie Chon U Cheang
- Head of Integrative Genomics Analysis in Clinical Trials, ICR‐CTSU, Division of Clinical StudiesThe Institute of Cancer ResearchLondonUK
| | - Francesco Ciompi
- Radboud University Medical CenterDepartment of PathologyNijmegenThe Netherlands
| | - Lee AD Cooper
- Department of PathologyNorthwestern Feinberg School of MedicineChicagoILUSA
| | - An Coosemans
- Department of Oncology, Laboratory of Tumor Immunology and ImmunotherapyKU LeuvenLeuvenBelgium
| | - Germán Corredor
- Biomedical Engineering DepartmentEmory UniversityAtlantaGAUSA
| | - Anders B Dahl
- Technical University of DenmarkKongens LyngbyDenmark
| | | | | | - Sandra Demaria
- Department of Radiation OncologyWeill Cornell MedicineNew YorkNYUSA
- Department of Pathology and Laboratory MedicineWeill Cornell MedicineNew YorkNYUSA
| | | | - Sarah N Dudgeon
- Conputational Biology and BioinformaticsYale UniversityNew HavenCTUSA
| | | | | | - Claudio Fernandez‐Martín
- Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, HUMAN‐techUniversitat Politècnica de ValènciaValenciaSpain
| | - Stephen B Fox
- Pathology, Peter MacCallum Cancer Centre and Sir Peter MacCallum Department of OncologyUniversity of MelbourneMelbourneVictoriaAustralia
| | - William M Gallagher
- UCD School of Biomolecular and Biomedical Science, UCD Conway InstituteUniversity College DublinDublinIreland
| | | | - Sacha Gnjatic
- Department of Oncological Sciences, Medicine Hem/Onc, and Pathology, Tisch Cancer Institute – Precision Immunology InstituteIcahn School of Medicine at Mount SinaiNew YorkNYUSA
| | | | - Anita Grigoriadis
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- The Breast Cancer Now Research Unit, School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Niels Halama
- Department of Translational ImmunotherapyGerman Cancer Research CenterHeidelbergGermany
| | - Matthew G Hanna
- Department of PathologyMemorial Sloan Kettering Cancer CenterNew YorkUSA
| | | | - Steven N Hart
- Department of Laboratory Medicine and PathologyMayo ClinicRochester, MNUSA
| | - Johan Hartman
- Department of Oncology and PathologyKarolinska InstitutetStockholmSweden
- Department of Clinical Pathology and Cancer DiagnosticsKarolinska University HospitalStockholmSweden
| | - Søren Hauberg
- Technical University of DenmarkKongens LyngbyDenmark
| | - Stephen Hewitt
- Laboratory of Pathology, Center for Cancer Research, National Cancer InstituteNational Institutes of HealthBethesdaMDUSA
| | - Akira I Hida
- Department of PathologyMatsuyama Shimin HospitalMatsuyamaJapan
| | - Hugo M Horlings
- Division of PathologyNetherlands Cancer Institute (NKI)AmsterdamThe Netherlands
| | | | | | - Sheeba Irshad
- King's College London & Guy's & St Thomas’ NHS TrustLondonUK
| | - Emiel AM Janssen
- Department of PathologyStavanger University HospitalStavangerNorway
- Department of Chemistry, Bioscience and Environmental TechnologyUniversity of StavangerStavangerNorway
| | | | | | - Kosuke Kawaguchi
- Department of Breast SurgeryKyoto University Graduate School of MedicineKyotoJapan
| | | | - Andrey I Khramtsov
- Department of Pathology and Laboratory MedicineAnn & Robert H. Lurie Children's Hospital of ChicagoChicagoILUSA
| | - Umay Kiraz
- Department of PathologyStavanger University HospitalStavangerNorway
- Department of Chemistry, Bioscience and Environmental TechnologyUniversity of StavangerStavangerNorway
| | - Pawan Kirtani
- Department of HistopathologyAakash Healthcare Super Speciality HospitalNew DelhiIndia
| | - Liudmila L Kodach
- Department of PathologyNetherlands Cancer Institute – Antoni van Leeuwenhoek HospitalAmsterdamThe Netherlands
| | - Konstanty Korski
- Data, Analytics and Imaging, Product DevelopmentF. Hoffmann‐La Roche AGBaselSwitzerland
| | - Anikó Kovács
- Department of Clinical PathologySahlgrenska University HospitalGothenburgSweden
- Institute of Biomedicine, Sahlgrenska AcademyUniversity of GothenburgGothenburgSweden
| | - Anne‐Vibeke Laenkholm
- Department of Surgical PathologyZealand University HospitalRoskildeDenmark
- Department of Surgical PathologyUniversity of CopenhagenCopenhagenDenmark
| | - Corinna Lang‐Schwarz
- Institute of Pathology, Klinikum Bayreuth GmbHFriedrich‐Alexander‐University Erlangen‐NurembergBayreuthGermany
| | - Denis Larsimont
- Institut Jules BordetUniversité Libre de BruxellesBrusselsBelgium
| | - Jochen K Lennerz
- Center for Integrated DiagnosticsMassachusetts General Hospital/Harvard Medical SchoolBostonMAUSA
| | - Marvin Lerousseau
- Centre for Computational Biology (CBIO)Mines Paris, PSL UniversityParisFrance
- Institut CuriePSL UniversityParisFrance
- INSERMParisFrance
| | - Xiaoxian Li
- Department of Pathology and Laboratory MedicineEmory UniversityAtlantaGAUSA
| | - Amy Ly
- Department of PathologyMassachusetts General HospitalBostonMAUSA
| | - Anant Madabhushi
- Department of Biomedical Engineering, Radiology and Imaging Sciences, Biomedical Informatics, PathologyGeorgia Institute of Technology and Emory UniversityAtlantaGAUSA
| | - Sai K Maley
- NRG Oncology/NSABP FoundationPittsburghPAUSA
| | | | | | - Elizabeth S McDonald
- Breast Cancer Translational Research GroupUniversity of PennsylvaniaPhiladelphiaPAUSA
| | - Ravi Mehrotra
- Indian Cancer Genomic AtlasPuneIndia
- Centre for Health, Innovation and Policy FoundationNoidaIndia
| | - Stefan Michiels
- Office of Biostatistics and Epidemiology, Gustave Roussy, Oncostat U1018, InsermUniversity Paris‐Saclay, Ligue Contre le Cancer labeled TeamVillejuifFrance
| | - Fayyaz ul Amir Afsar Minhas
- Tissue Image Analytics Centre, Warwick Cancer Research Centre, PathLAKE Consortium, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Shachi Mittal
- Department of Chemical Engineering, Department of Laboratory Medicine and PathologyUniversity of WashingtonSeattle, WAUSA
| | - David A Moore
- CRUK Lung Cancer Centre of Excellence, UCL and Cellular Pathology DepartmentUCLHLondonUK
| | - Shamim Mushtaq
- Department of BiochemistryZiauddin UniversityKarachiPakistan
| | - Hussain Nighat
- Pathology and Laboratory MedicineAll India Institute of Medical sciencesRaipurIndia
| | - Thomas Papathomas
- Institute of Metabolism and Systems ResearchUniversity of BirminghamBirminghamUK
- Department of Clinical PathologyDrammen Sykehus, Vestre Viken HFDrammenNorway
| | - Frederique Penault‐Llorca
- Centre Jean Perrin, Université Clermont Auvergne, INSERM, U1240 Imagerie Moléculaire et Stratégies ThéranostiquesClermont FerrandFrance
| | - Rashindrie D Perera
- School of Electrical, Mechanical and Infrastructure EngineeringUniversity of MelbourneMelbourneVictoriaAustralia
- Division of Cancer ResearchPeter MacCallum Cancer CentreMelbourneVictoriaAustralia
| | - Christopher J Pinard
- Radiogenomics LaboratorySunnybrook Health Sciences CentreTorontoOntarioCanada
- Department of Clinical Studies, Ontario Veterinary CollegeUniversity of GuelphGuelphOntarioCanada
- Department of OncologyLakeshore Animal Health PartnersMississaugaOntarioCanada
- Centre for Advancing Responsible and Ethical Artificial Intelligence (CARE‐AI)University of GuelphGuelphOntarioCanada
| | | | - Giancarlo Pruneri
- Department of Pathology and Laboratory MedicineFondazione IRCCS Istituto Nazionale dei TumoriMilanItaly
- Faculty of Medicine and SurgeryUniversity of MilanMilanItaly
| | - Lajos Pusztai
- Yale Cancer CenterYale UniversityNew HavenCTUSA
- Department of Medical Oncology, Yale School of MedicineYale UniversityNew HavenCTUSA
| | - Arman Rahman
- UCD School of Biomolecular and Biomedical Science, UCD Conway InstituteUniversity College DublinDublinIreland
| | | | - Bernardo Leon Rapoport
- The Medical Oncology Centre of RosebankJohannesburgSouth Africa
- Department of Immunology, Faculty of Health SciencesUniversity of PretoriaPretoriaSouth Africa
| | - Tilman T Rau
- Institute of PathologyUniversity Hospital Düsseldorf and Heinrich‐Heine‐University DüsseldorfDüsseldorfGermany
| | - Jorge S Reis‐Filho
- Department of Pathology and Laboratory MedicineMemorial Sloan Kettering Cancer CenterNew YorkNYUSA
| | - Joana M Ribeiro
- Département de Médecine OncologiqueGustave RoussyVillejuifFrance
| | - David Rimm
- Department of PathologyYale University School of MedicineNew HavenCTUSA
- Department of MedicineYale University School of MedicineNew HavenCTUSA
| | - Anne Roslind
- Department of PathologyHerlev and Gentofte HospitalHerlevDenmark
| | - Anne Vincent‐Salomon
- Department of Diagnostic and Theranostic Medicine, Institut CurieUniversity Paris‐Sciences et LettresParisFrance
| | - Manuel Salto‐Tellez
- Integrated Pathology UnitThe Institute of Cancer ResearchLondonUK
- Precision Medicine CentreQueen's University BelfastBelfastUK
| | - Joel Saltz
- Department of Biomedical InformaticsStony Brook UniversityStony BrookNYUSA
| | - Shahin Sayed
- Department of PathologyAga Khan UniversityNairobiKenya
| | - Ely Scott
- Translational PathologyTranslational Sciences and Diagnostics/Translational Medicine/R&D, Bristol Myers SquibbPrincetonNJUSA
| | - Kalliopi P Siziopikou
- Department of Pathology, Section of Breast PathologyNorthwestern University Feinberg School of MedicineChicagoILUSA
| | - Christos Sotiriou
- Breast Cancer Translational Research Laboratory J.‐C. Heuson, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (HUB)Université Libre de Bruxelles (ULB)BrusselsBelgium
- Medical Oncology Department, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (HUB)Université Libre de Bruxelles (ULB)BrusselsBelgium
| | - Albrecht Stenzinger
- Institute of PathologyUniversity Hospital HeidelbergHeidelbergGermany
- Centers for Personalized Medicine (ZPM)HeidelbergGermany
| | | | - Daniel Sur
- Department of Medical OncologyUniversity of Medicine and Pharmacy “Iuliu Hatieganu”Cluj‐NapocaRomania
| | - Susan Fineberg
- Montefiore Medical CenterBronxNYUSA
- Albert Einstein College of MedicineBronxNYUSA
| | - Fraser Symmans
- University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | | | | | - Sabine Tejpar
- Digestive Oncology, Department of OncologyKU LeuvenLeuvenBelgium
| | - Jonas Teuwen
- AI for Oncology Lab, The Netherlands Cancer InstituteAmsterdamThe Netherlands
| | | | - Trine Tramm
- Department of PathologyAarhus University HospitalAarhusDenmark
- Institute of Clinical MedicineAarhus UniversityAarhusDenmark
| | - William T Tran
- Department of Radiation OncologyUniversity of Toronto and Sunnybrook Health Sciences CentreTorontoOntarioCanada
| | - Jeroen van der Laak
- Department of PathologyRadboud University Medical CenterNijmegenThe Netherlands
| | - Paul J van Diest
- Department of PathologyUniversity Medical Center UtrechtThe Netherlands
- Johns Hopkins Oncology CenterBaltimoreMDUSA
| | - Gregory E Verghese
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- The Breast Cancer Now Research Unit, School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Giuseppe Viale
- Department of PathologyEuropean Institute of OncologyMilanItaly
- Department of PathologyUniversity of MilanMilanItaly
| | - Michael Vieth
- Institute of Pathology, Klinikum Bayreuth GmbHFriedrich‐Alexander‐University Erlangen‐NurembergBayreuthGermany
| | - Noorul Wahab
- Tissue Image Analytics Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Thomas Walter
- Centre for Computational Biology (CBIO)Mines Paris, PSL UniversityParisFrance
- Institut CuriePSL UniversityParisFrance
- INSERMParisFrance
| | | | - Hannah Y Wen
- Department of PathologyMemorial Sloan Kettering Cancer CenterNew YorkUSA
| | - Wentao Yang
- Fudan Medical University Shanghai Cancer CenterShanghaiPR China
| | - Yinyin Yuan
- Department of Translational Molecular Pathology, Division of Pathology and Laboratory MedicineThe University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | - Reena Md Zin
- Department of Pathology, Faculty of MedicineUniversiti Kebangsaan MalaysiaKuala LumpurMalaysia
| | - Sylvia Adams
- Perlmutter Cancer CenterNYU Langone HealthNew YorkNYUSA
- Department of MedicineNYU Grossman School of MedicineManhattanNYUSA
| | | | - Sibylle Loibl
- Department of Medicine and ResearchGerman Breast GroupNeu‐IsenburgGermany
| | - Carsten Denkert
- Institut für PathologiePhilipps‐Universität Marburg und Universitätsklinikum MarburgMarburgGermany
| | - Peter Savas
- Division of Cancer ResearchPeter MacCallum Cancer CentreMelbourneVictoriaAustralia
- The Sir Peter MacCallum Department of Medical OncologyUniversity of MelbourneMelbourneVictoriaAustralia
| | - Sherene Loi
- Division of Cancer ResearchPeter MacCallum Cancer CentreMelbourneVictoriaAustralia
- The Sir Peter MacCallum Department of Medical OncologyUniversity of MelbourneMelbourneVictoriaAustralia
| | - Roberto Salgado
- Department of PathologyGZA‐ZNA HospitalsAntwerpBelgium
- Division of Cancer ResearchPeter MacCallum Cancer CentreMelbourneVictoriaAustralia
| | - Elisabeth Specht Stovgaard
- Department of PathologyHerlev and Gentofte HospitalHerlevDenmark
- Department of Clinical MedicineUniversity of CopenhagenCopenhagenDenmark
| |
Collapse
|
10
|
Wang G, Yao Y, Huang H, Zhou J, Ni C. Multiomics technologies for comprehensive tumor microenvironment analysis in triple-negative breast cancer under neoadjuvant chemotherapy. Front Oncol 2023; 13:1131259. [PMID: 37284197 PMCID: PMC10239824 DOI: 10.3389/fonc.2023.1131259] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2022] [Accepted: 05/08/2023] [Indexed: 06/08/2023] Open
Abstract
Triple-negative breast cancer (TNBC) is one of the most aggressive breast cancer subtypes and is characterized by abundant infiltrating immune cells within the microenvironment. As standard care, chemotherapy remains the fundamental neoadjuvant treatment in TNBC, and there is increasing evidence that supplementation with immune checkpoint inhibitors may potentiate the therapeutic efficiency of neoadjuvant chemotherapy (NAC). However, 20-60% of TNBC patients still have residual tumor burden after NAC and require additional chemotherapy; therefore, it is critical to understand the dynamic change in the tumor microenvironment (TME) during treatment to help improve the rate of complete pathological response and long-term prognosis. Traditional methods, including immunohistochemistry, bulk tumor sequencing, and flow cytometry, have been applied to elucidate the TME of breast cancer, but the low resolution and throughput may overlook key information. With the development of diverse high-throughput technologies, recent reports have provided new insights into TME alterations during NAC in four fields, including tissue imaging, cytometry, next-generation sequencing, and spatial omics. In this review, we discuss the traditional methods and the latest advances in high-throughput techniques to decipher the TME of TNBC and the prospect of translating these techniques to clinical practice.
Collapse
Affiliation(s)
- Gang Wang
- Department of Surgical Oncology, The First Affiliated Hospital of Bengbu Medical College, Bengbu, Anhui, China
| | - Yao Yao
- Department of Breast Surgery, Second Affiliated Hospital, Zhejiang University, Hangzhou, Zhejiang, China
- Key Laboratory of Tumor Microenvironment and Immune Therapy of Zhejiang Province, Second Affiliated Hospital, Zhejiang University, Hangzhou, Zhejiang, China
- Cancer Center, Zhejiang University, Hangzhou, China
| | - Huanhuan Huang
- Department of Breast Surgery, Second Affiliated Hospital, Zhejiang University, Hangzhou, Zhejiang, China
- Key Laboratory of Tumor Microenvironment and Immune Therapy of Zhejiang Province, Second Affiliated Hospital, Zhejiang University, Hangzhou, Zhejiang, China
- Cancer Center, Zhejiang University, Hangzhou, China
| | - Jun Zhou
- Department of Breast Surgery, Affiliated Hangzhou First People’s Hospital, Zhejiang University, Hangzhou, Zhejiang, China
| | - Chao Ni
- Department of Breast Surgery, Second Affiliated Hospital, Zhejiang University, Hangzhou, Zhejiang, China
- Key Laboratory of Tumor Microenvironment and Immune Therapy of Zhejiang Province, Second Affiliated Hospital, Zhejiang University, Hangzhou, Zhejiang, China
- Cancer Center, Zhejiang University, Hangzhou, China
| |
Collapse
|
11
|
Bhattarai S, Saini G, Li H, Duanmu H, Seth G, Fisher TB, Janssen EAM, Kiraz U, Kong J, Aneja R. Predicting neoadjuvant treatment response in triple-negative breast cancer using machine learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.04.17.536459. [PMID: 37131688 PMCID: PMC10153161 DOI: 10.1101/2023.04.17.536459] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Background Neoadjuvant chemotherapy (NAC) is the standard treatment for early-stage triple negative breast cancer (TNBC). The primary endpoint of NAC is a pathological complete response (pCR). NAC results in pCR in only 30%â€"40% of TNBC patients. Tumor-infiltrating lymphocytes (TILs), Ki67 and phosphohistone H3 (pH3) are a few known biomarkers to predict NAC response. Currently, systematic evaluation of the combined value of these biomarkers in predicting NAC response is lacking. In this study, the predictive value of markers derived from H&E and IHC stained biopsy tissue was comprehensively evaluated using a supervised machine learning (ML)-based approach. Identifying predictive biomarkers could help guide therapeutic decisions by enabling precise stratification of TNBC patients into responders and partial or non-responders. Methods Serial sections from core needle biopsies (n=76) were stained with H&E, and immunohistochemically for the Ki67 and pH3 markers, followed by whole slide image (WSI) generation. The resulting WSI triplets were co-registered with H&E WSIs serving as the reference. Separate mask region-based CNN (MRCNN) models were trained with annotated H&E, Ki67 and pH3 images for detecting tumor cells, stromal and intratumoral TILs (sTILs and tTILs), Ki67 + , and pH3 + cells. Top image patches with a high density of cells of interest were identified as hotspots. Best classifiers for NAC response prediction were identified by training multiple ML models, and evaluating their performance by accuracy, area under curve, and confusion matrix analyses. Results Highest prediction accuracy was achieved when hotspot regions were identified by tTIL counts and each hotspot was represented by measures of tTILs, sTILs, tumor cells, Ki67 + , and pH3 + features. Regardless of the hotspot selection metric, a complementary use of multiple histological features (tTILs, sTILs) and molecular biomarkers (Ki67 and pH3) resulted in top ranked performance at the patient level. Conclusions Overall, our results emphasize that prediction models for NAC response should be based on biomarkers in combination rather than in isolation. Our study provides compelling evidence to support the use of ML-based models to predict NAC response in patients with TNBC.
Collapse
|
12
|
Verdicchio M, Brancato V, Cavaliere C, Isgrò F, Salvatore M, Aiello M. A pathomic approach for tumor-infiltrating lymphocytes classification on breast cancer digital pathology images. Heliyon 2023; 9:e14371. [PMID: 36950640 PMCID: PMC10025040 DOI: 10.1016/j.heliyon.2023.e14371] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Revised: 03/03/2023] [Accepted: 03/03/2023] [Indexed: 03/11/2023] Open
Abstract
Background and objectives The detection of tumor-infiltrating lymphocytes (TILs) could aid in the development of objective measures of the infiltration grade and can support decision-making in breast cancer (BC). However, manual quantification of TILs in BC histopathological whole slide images (WSI) is currently based on a visual assessment, thus resulting not standardized, not reproducible, and time-consuming for pathologists. In this work, a novel pathomic approach, aimed to apply high-throughput image feature extraction techniques to analyze the microscopic patterns in WSI, is proposed. In fact, pathomic features provide additional information concerning the underlying biological processes compared to the WSI visual interpretation, thus providing more easily interpretable and explainable results than the most frequently investigated Deep Learning based methods in the literature. Methods A dataset containing 1037 regions of interest with tissue compartments and TILs annotated on 195 TNBC and HER2+ BC hematoxylin and eosin (H&E)-stained WSI was used. After segmenting nuclei within tumor-associated stroma using a watershed-based approach, 71 pathomic features were extracted from each nucleus and reduced using a Spearman's correlation filter followed by a nonparametric Wilcoxon rank-sum test and least absolute shrinkage and selection operator. The relevant features were used to classify each candidate nucleus as either TILs or non-TILs using 5 multivariable machine learning classification models trained using 5-fold cross-validation (1) without resampling, (2) with the synthetic minority over-sampling technique and (3) with downsampling. The prediction performance of the models was assessed using ROC curves. Results 21 features were selected, with most of them related to the well-known TILs properties of having regular shape, clearer margins, high peak intensity, more homogeneous enhancement and different textural pattern than other cells. The best performance was obtained by Random-Forest with ROC AUC of 0.86, regardless of resampling technique. Conclusions The presented approach holds promise for the classification of TILs in BC H&E-stained WSI and could provide support to pathologists for a reliable, rapid and interpretable clinical assessment of TILs in BC.
Collapse
Affiliation(s)
| | - Valentina Brancato
- IRCCS SYNLAB SDN, Via E. Gianturco 113, Naples, 80143, Italy
- Corresponding author.
| | - Carlo Cavaliere
- IRCCS SYNLAB SDN, Via E. Gianturco 113, Naples, 80143, Italy
| | - Francesco Isgrò
- Department of Electrical Engineering and Information Technologies, University of Naples Federico II, Claudio 21, Naples, 80125, Italy
| | - Marco Salvatore
- IRCCS SYNLAB SDN, Via E. Gianturco 113, Naples, 80143, Italy
| | - Marco Aiello
- IRCCS SYNLAB SDN, Via E. Gianturco 113, Naples, 80143, Italy
| |
Collapse
|
13
|
Liu A, Li X, Wu H, Guo B, Jonnagaddala J, Zhang H, Xu S. Prognostic Significance of Tumor-Infiltrating Lymphocytes Determined Using LinkNet on Colorectal Cancer Pathology Images. JCO Precis Oncol 2023; 7:e2200522. [PMID: 36848612 DOI: 10.1200/po.22.00522] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/01/2023] Open
Abstract
PURPOSE Tumor-infiltrating lymphocytes (TILs) have a significant prognostic value in cancers. However, very few automated, deep learning-based TIL scoring algorithms have been developed for colorectal cancer (CRC). MATERIALS AND METHODS We developed an automated, multiscale LinkNet workflow for quantifying TILs at the cellular level in CRC tumors using H&E-stained images from the Lizard data set with annotations of lymphocytes. The predictive performance of the automatic TIL scores (TILsLink) for disease progression and overall survival (OS) was evaluated using two international data sets, including 554 patients with CRC from The Cancer Genome Atlas (TCGA) and 1,130 patients with CRC from Molecular and Cellular Oncology (MCO). RESULTS The LinkNet model provided outstanding precision (0.9508), recall (0.9185), and overall F1 score (0.9347). Clear continuous TIL-hazard relationships were observed between TILsLink and the risk of disease progression or death in both TCGA and MCO cohorts. Both univariate and multivariate Cox regression analyses for the TCGA data demonstrated that patients with high TIL abundance had a significant (approximately 75%) reduction in risk for disease progression. In both the MCO and TCGA cohorts, the TIL-high group was significantly associated with improved OS in univariate analysis (30% and 54% reduction in risk, respectively). The favorable effects of high TIL levels were consistently observed in different subgroups (classified according to known risk factors). CONCLUSION The proposed deep-learning workflow for automatic TIL quantification on the basis of LinkNet can be a useful tool for CRC. TILsLink is likely an independent risk factor for disease progression and carries predictive information of disease progression beyond the current clinical risk factors and biomarkers. The prognostic significance of TILsLink for OS is also evident.
Collapse
Affiliation(s)
- Anran Liu
- Department of Statistics and Finance, School of Management, University of Science and Technology of China, Hefei, Anhui, China
| | - Xingyu Li
- Department of Statistics and Finance, School of Management, University of Science and Technology of China, Hefei, Anhui, China
| | - Hongyi Wu
- Department of Statistics and Finance, School of Management, University of Science and Technology of China, Hefei, Anhui, China
| | - Bangwei Guo
- School of Data Science, University of Science and Technology of China, Hefei, Anhui, China
| | | | - Hong Zhang
- Department of Statistics and Finance, School of Management, University of Science and Technology of China, Hefei, Anhui, China
| | - Steven Xu
- Clinical Pharmacology and Quantitative Science, Genmab Inc, Princeton, NJ
| |
Collapse
|
14
|
Deep-learning based breast cancer detection for cross-staining histopathology images. Heliyon 2023; 9:e13171. [PMID: 36755605 PMCID: PMC9900267 DOI: 10.1016/j.heliyon.2023.e13171] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Revised: 01/13/2023] [Accepted: 01/19/2023] [Indexed: 01/23/2023] Open
Abstract
Hematoxylin and eosin (H&E) staining is the gold standard for tissue characterization in routine pathological diagnoses. However, these visible light dyes do not exclusively label the nuclei and cytoplasm, making clear-cut segmentation of staining signals challenging. Currently, fluorescent staining technology is much more common in clinical research for analyzing tissue morphology and protein distribution owing to its advantages of channel independence, multiplex labeling, and the possibility of enabling 3D tissue labeling. Although both H&E and fluorescent dyes can stain the nucleus and cytoplasm for representative tissue morphology, color variation between these two staining technologies makes cross-analysis difficult, especially with computer-assisted artificial intelligence (AI) algorithms. In this study, we applied color normalization and nucleus extraction methods to overcome the variation between staining technologies. We also developed an available workflow for using an H&E-stained segmentation AI model in the analysis of fluorescent nucleic acid staining images in breast cancer tumor recognition, resulting in 89.6% and 80.5% accuracy in recognizing specific tumor features in H&E- and fluorescent-stained pathological images, respectively. The results show that the cross-staining inference maintained the same precision level as the proposed workflow, providing an opportunity for an expansion of the application of current pathology AI models.
Collapse
|
15
|
Association between CD8+ Tumor Infiltrating Lymphocytes and the Clinical Outcome of Patients with Operable Breast Cancer Treated with Adjuvant Dose-Dense Chemotherapy-A 10 Year Follow-Up Report of a Hellenic Cooperative Oncology Group Observational Study. Cancers (Basel) 2022; 14:cancers14225635. [PMID: 36428728 PMCID: PMC9688913 DOI: 10.3390/cancers14225635] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2022] [Revised: 11/01/2022] [Accepted: 11/11/2022] [Indexed: 11/19/2022] Open
Abstract
Tumor-infiltrating lymphocytes (TILs) contribute to breast cancer (BC) prognosis. We investigated the prognostic impact of CD8+ TILs in patients with early breast cancer treated with adjuvant chemotherapy in a large observational clinical trial. Along with a 10 year follow-up, considering the efficacy and safety, we report the results of the translational part of our study. We examined the patients' tumors for total (t), stromal (s), and intratumoral (i) CD8 lymphocyte density (counts/mm2) on tissue-microarray cores. The impact of CD8+ TILs counts on DFS and OS, and its correlation with breast cancer subtypes and standard clinicopathological parameters, were investigated, along with efficacy and safety data. Among the 928 eligible patients, 627 had available CD8+ data. Of which, 24.9% had a high expression of sCD8, iCD8, and total CD8, which were correlated with higher Ki67, TILs density, ER/PgR negativity, and higher histological grade. The 5year DFS and OS rates were 86.1% and 91.4%, respectively. Patients with high iCD8 and tCD8 had longer DFS and OS compared to those with low counts/mm2 (DFS: HR = 0.58, p = 0.011 and HR = 0.65, p = 0.034 and OS: HR = 0.63, p = 0.043 and HR = 0.58, p = 0.020, respectively). Upon adjustment for clinicopathological parameters, iCD8 and tCD8 retained their favorable prognostic significance for DFS and OS, whereas high sCD8 was only prognostic for DFS. Menopausal status, tumor size, and nodal status retained their prognostic significance in all examined multivariate models. CD8+ TILs, and especially their intratumoral subset, represent a potential favorable prognostic factor in early BC.
Collapse
|
16
|
Qiao Y, Zhao L, Luo C, Luo Y, Wu Y, Li S, Bu D, Zhao Y. Multi-modality artificial intelligence in digital pathology. Brief Bioinform 2022; 23:6702380. [PMID: 36124675 PMCID: PMC9677480 DOI: 10.1093/bib/bbac367] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2022] [Revised: 07/27/2022] [Accepted: 08/05/2022] [Indexed: 12/14/2022] Open
Abstract
In common medical procedures, the time-consuming and expensive nature of obtaining test results plagues doctors and patients. Digital pathology research allows using computational technologies to manage data, presenting an opportunity to improve the efficiency of diagnosis and treatment. Artificial intelligence (AI) has a great advantage in the data analytics phase. Extensive research has shown that AI algorithms can produce more up-to-date and standardized conclusions for whole slide images. In conjunction with the development of high-throughput sequencing technologies, algorithms can integrate and analyze data from multiple modalities to explore the correspondence between morphological features and gene expression. This review investigates using the most popular image data, hematoxylin-eosin stained tissue slide images, to find a strategic solution for the imbalance of healthcare resources. The article focuses on the role that the development of deep learning technology has in assisting doctors' work and discusses the opportunities and challenges of AI.
Collapse
Affiliation(s)
- Yixuan Qiao
- Research Center for Ubiquitous Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China,University of Chinese Academy of Sciences, Beijing 100049, China
| | - Lianhe Zhao
- Corresponding authors: Yi Zhao, Research Center for Ubiquitous Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China; University of Chinese Academy of Sciences; Shandong First Medical University & Shandong Academy of Medical Sciences. Tel.: +86 10 6260 0822; Fax: +86 10 6260 1356; E-mail: ; Lianhe Zhao, Research Center for Ubiquitous Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China; University of Chinese Academy of Sciences. Tel.: +86 18513983324; E-mail:
| | - Chunlong Luo
- Research Center for Ubiquitous Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China,University of Chinese Academy of Sciences, Beijing 100049, China
| | - Yufan Luo
- Research Center for Ubiquitous Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China,University of Chinese Academy of Sciences, Beijing 100049, China
| | - Yang Wu
- Research Center for Ubiquitous Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China
| | - Shengtong Li
- Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Dechao Bu
- Research Center for Ubiquitous Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China
| | - Yi Zhao
- Corresponding authors: Yi Zhao, Research Center for Ubiquitous Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China; University of Chinese Academy of Sciences; Shandong First Medical University & Shandong Academy of Medical Sciences. Tel.: +86 10 6260 0822; Fax: +86 10 6260 1356; E-mail: ; Lianhe Zhao, Research Center for Ubiquitous Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China; University of Chinese Academy of Sciences. Tel.: +86 18513983324; E-mail:
| |
Collapse
|
17
|
Sandarenu P, Millar EKA, Song Y, Browne L, Beretov J, Lynch J, Graham PH, Jonnagaddala J, Hawkins N, Huang J, Meijering E. Survival prediction in triple negative breast cancer using multiple instance learning of histopathological images. Sci Rep 2022; 12:14527. [PMID: 36008541 PMCID: PMC9411153 DOI: 10.1038/s41598-022-18647-1] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 08/17/2022] [Indexed: 11/09/2022] Open
Abstract
Computational pathology is a rapidly expanding area for research due to the current global transformation of histopathology through the adoption of digital workflows. Survival prediction of breast cancer patients is an important task that currently depends on histopathology assessment of cancer morphological features, immunohistochemical biomarker expression and patient clinical findings. To facilitate the manual process of survival risk prediction, we developed a computational pathology framework for survival prediction using digitally scanned haematoxylin and eosin-stained tissue microarray images of clinically aggressive triple negative breast cancer. Our results show that the model can produce an average concordance index of 0.616. Our model predictions are analysed for independent prognostic significance in univariate analysis (hazard ratio = 3.12, 95% confidence interval [1.69,5.75], p < 0.005) and multivariate analysis using clinicopathological data (hazard ratio = 2.68, 95% confidence interval [1.44,4.99], p < 0.005). Through qualitative analysis of heatmaps generated from our model, an expert pathologist is able to associate tissue features highlighted in the attention heatmaps of high-risk predictions with morphological features associated with more aggressive behaviour such as low levels of tumour infiltrating lymphocytes, stroma rich tissues and high-grade invasive carcinoma, providing explainability of our method for triple negative breast cancer.
Collapse
Affiliation(s)
- Piumi Sandarenu
- School of Computer Science and Engineering, UNSW Sydney, Kensington, NSW, 2052, Australia
| | - Ewan K A Millar
- Department of Anatomical Pathology, NSW Health Pathology, St. George Hospital, Kogarah, NSW, 2217, Australia.,St. George and Sutherland Clinical School, UNSW Sydney, Kensington, NSW, 2052, Australia.,Faculty of Medicine and Health Sciences, Sydney Western University, Campbelltown, NSW, 2560, Australia.,University of Technology Sydney, Ultimo, NSW, 2007, Australia
| | - Yang Song
- School of Computer Science and Engineering, UNSW Sydney, Kensington, NSW, 2052, Australia
| | - Lois Browne
- Cancer Care Centre, St. George Hospital, Kogarah, NSW, 2217, Australia
| | - Julia Beretov
- Department of Anatomical Pathology, NSW Health Pathology, St. George Hospital, Kogarah, NSW, 2217, Australia.,St. George and Sutherland Clinical School, UNSW Sydney, Kensington, NSW, 2052, Australia.,Cancer Care Centre, St. George Hospital, Kogarah, NSW, 2217, Australia
| | - Jodi Lynch
- St. George and Sutherland Clinical School, UNSW Sydney, Kensington, NSW, 2052, Australia.,Cancer Care Centre, St. George Hospital, Kogarah, NSW, 2217, Australia
| | - Peter H Graham
- St. George and Sutherland Clinical School, UNSW Sydney, Kensington, NSW, 2052, Australia.,Cancer Care Centre, St. George Hospital, Kogarah, NSW, 2217, Australia
| | | | - Nicholas Hawkins
- School of Medical Sciences, UNSW Sydney, Kensington, NSW, 2052, Australia
| | - Junzhou Huang
- University of Texas at Arlington, Arlington, TX, 76019, USA
| | - Erik Meijering
- School of Computer Science and Engineering, UNSW Sydney, Kensington, NSW, 2052, Australia.
| |
Collapse
|
18
|
Development of Training Materials for Pathologists to Provide Machine Learning Validation Data of Tumor-Infiltrating Lymphocytes in Breast Cancer. Cancers (Basel) 2022; 14:cancers14102467. [PMID: 35626070 PMCID: PMC9139395 DOI: 10.3390/cancers14102467] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Revised: 05/07/2022] [Accepted: 05/08/2022] [Indexed: 02/05/2023] Open
Abstract
Simple Summary The High Throughput Truthing project aims to develop a dataset of stromal tumor-infiltrating lymphocytes (sTILs) density evaluations in hematoxylin and eosin-stained invasive breast cancer specimens fit for a regulatory purpose. After completion of the pilot study, the analysis demonstrated inconsistencies and gaps in the provided training to pathologists. Select regions of interest (ROIs) were reviewed by an expert panel, who provided annotations and commentary on the challenges of the sTILs assessment. We used these annotations to develop a training document and reference standard for new training materials. These materials will train crowd-sourced pathologists to help create an algorithm validation dataset and contribute to sTILs evaluations in clinical practice. Abstract The High Throughput Truthing project aims to develop a dataset for validating artificial intelligence and machine learning models (AI/ML) fit for regulatory purposes. The context of this AI/ML validation dataset is the reporting of stromal tumor-infiltrating lymphocytes (sTILs) density evaluations in hematoxylin and eosin-stained invasive breast cancer biopsy specimens. After completing the pilot study, we found notable variability in the sTILs estimates as well as inconsistencies and gaps in the provided training to pathologists. Using the pilot study data and an expert panel, we created custom training materials to improve pathologist annotation quality for the pivotal study. We categorized regions of interest (ROIs) based on their mean sTILs density and selected ROIs with the highest and lowest sTILs variability. In a series of eight one-hour sessions, the expert panel reviewed each ROI and provided verbal density estimates and comments on features that confounded the sTILs evaluation. We aggregated and shaped the comments to identify pitfalls and instructions to improve our training materials. From these selected ROIs, we created a training set and proficiency test set to improve pathologist training with the goal to improve data collection for the pivotal study. We are not exploring AI/ML performance in this paper. Instead, we are creating materials that will train crowd-sourced pathologists to be the reference standard in a pivotal study to create an AI/ML model validation dataset. The issues discussed here are also important for clinicians to understand about the evaluation of sTILs in clinical practice and can provide insight to developers of AI/ML models.
Collapse
|
19
|
Fassler DJ, Torre-Healy LA, Gupta R, Hamilton AM, Kobayashi S, Van Alsten SC, Zhang Y, Kurc T, Moffitt RA, Troester MA, Hoadley KA, Saltz J. Spatial Characterization of Tumor-Infiltrating Lymphocytes and Breast Cancer Progression. Cancers (Basel) 2022; 14:2148. [PMID: 35565277 PMCID: PMC9105398 DOI: 10.3390/cancers14092148] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2022] [Revised: 04/09/2022] [Accepted: 04/15/2022] [Indexed: 12/15/2022] Open
Abstract
Tumor-infiltrating lymphocytes (TILs) have been established as a robust prognostic biomarker in breast cancer, with emerging utility in predicting treatment response in the adjuvant and neoadjuvant settings. In this study, the role of TILs in predicting overall survival and progression-free interval was evaluated in two independent cohorts of breast cancer from the Cancer Genome Atlas (TCGA BRCA) and the Carolina Breast Cancer Study (UNC CBCS). We utilized machine learning and computer vision algorithms to characterize TIL infiltrates in digital whole-slide images (WSIs) of breast cancer stained with hematoxylin and eosin (H&E). Multiple parameters were used to characterize the global abundance and spatial features of TIL infiltrates. Univariate and multivariate analyses show that large aggregates of peritumoral and intratumoral TILs (forests) were associated with longer survival, whereas the absence of intratumoral TILs (deserts) is associated with increased risk of recurrence. Patients with two or more high-risk spatial features were associated with significantly shorter progression-free interval (PFI). This study demonstrates the practical utility of Pathomics in evaluating the clinical significance of the abundance and spatial patterns of distribution of TIL infiltrates as important biomarkers in breast cancer.
Collapse
Affiliation(s)
- Danielle J. Fassler
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY 11790, USA; (D.J.F.); (L.A.T.-H.); (R.G.); (S.K.); (Y.Z.); (T.K.); (R.A.M.)
| | - Luke A. Torre-Healy
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY 11790, USA; (D.J.F.); (L.A.T.-H.); (R.G.); (S.K.); (Y.Z.); (T.K.); (R.A.M.)
| | - Rajarsi Gupta
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY 11790, USA; (D.J.F.); (L.A.T.-H.); (R.G.); (S.K.); (Y.Z.); (T.K.); (R.A.M.)
| | - Alina M. Hamilton
- Department of Pathology and Laboratory Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA; (A.M.H.); (S.C.V.A.); (M.A.T.)
| | - Soma Kobayashi
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY 11790, USA; (D.J.F.); (L.A.T.-H.); (R.G.); (S.K.); (Y.Z.); (T.K.); (R.A.M.)
| | - Sarah C. Van Alsten
- Department of Pathology and Laboratory Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA; (A.M.H.); (S.C.V.A.); (M.A.T.)
| | - Yuwei Zhang
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY 11790, USA; (D.J.F.); (L.A.T.-H.); (R.G.); (S.K.); (Y.Z.); (T.K.); (R.A.M.)
| | - Tahsin Kurc
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY 11790, USA; (D.J.F.); (L.A.T.-H.); (R.G.); (S.K.); (Y.Z.); (T.K.); (R.A.M.)
| | - Richard A. Moffitt
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY 11790, USA; (D.J.F.); (L.A.T.-H.); (R.G.); (S.K.); (Y.Z.); (T.K.); (R.A.M.)
| | - Melissa A. Troester
- Department of Pathology and Laboratory Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA; (A.M.H.); (S.C.V.A.); (M.A.T.)
| | - Katherine A. Hoadley
- Department of Genetics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA;
| | - Joel Saltz
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY 11790, USA; (D.J.F.); (L.A.T.-H.); (R.G.); (S.K.); (Y.Z.); (T.K.); (R.A.M.)
| |
Collapse
|