1
|
Leon-Ferre RA, Carter JM, Zahrieh D, Sinnwell JP, Salgado R, Suman VJ, Hillman DW, Boughey JC, Kalari KR, Couch FJ, Ingle JN, Balkenhol M, Ciompi F, van der Laak J, Goetz MP. Automated mitotic spindle hotspot counts are highly associated with clinical outcomes in systemically untreated early-stage triple-negative breast cancer. NPJ Breast Cancer 2024; 10:25. [PMID: 38553444 PMCID: PMC10980681 DOI: 10.1038/s41523-024-00629-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2023] [Accepted: 03/08/2024] [Indexed: 04/02/2024] Open
Abstract
Operable triple-negative breast cancer (TNBC) has a higher risk of recurrence and death compared to other subtypes. Tumor size and nodal status are the primary clinical factors used to guide systemic treatment, while biomarkers of proliferation have not demonstrated value. Recent studies suggest that subsets of TNBC have a favorable prognosis, even without systemic therapy. We evaluated the association of fully automated mitotic spindle hotspot (AMSH) counts with recurrence-free (RFS) and overall survival (OS) in two separate cohorts of patients with early-stage TNBC who did not receive systemic therapy. AMSH counts were obtained from areas with the highest mitotic density in digitized whole slide images processed with a convolutional neural network trained to detect mitoses. In 140 patients from the Mayo Clinic TNBC cohort, AMSH counts were significantly associated with RFS and OS in a multivariable model controlling for nodal status, tumor size, and tumor-infiltrating lymphocytes (TILs) (p < 0.0001). For every 10-point increase in AMSH counts, there was a 16% increase in the risk of an RFS event (HR 1.16, 95% CI 1.08-1.25), and a 7% increase in the risk of death (HR 1.07, 95% CI 1.00-1.14). We corroborated these findings in a separate cohort of systemically untreated TNBC patients from Radboud UMC in the Netherlands. Our findings suggest that AMSH counts offer valuable prognostic information in patients with early-stage TNBC who did not receive systemic therapy, independent of tumor size, nodal status, and TILs. If further validated, AMSH counts could help inform future systemic therapy de-escalation strategies.
Collapse
Affiliation(s)
| | | | | | | | - Roberto Salgado
- GZA-ZNA-Hospitals, Antwerp, Belgium
- Peter Mac Callum Cancer Centre, Melbourne, Australia
| | | | | | | | | | | | | | | | | | - Jeroen van der Laak
- Radboud University Medical Center, Nijmegen, Netherlands
- Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden
| | | |
Collapse
|
2
|
Aswolinskiy W, Munari E, Horlings HM, Mulder L, Bogina G, Sanders J, Liu YH, van den Belt-Dusebout AW, Tessier L, Balkenhol M, Stegeman M, Hoven J, Wesseling J, van der Laak J, Lips EH, Ciompi F. PROACTING: predicting pathological complete response to neoadjuvant chemotherapy in breast cancer from routine diagnostic histopathology biopsies with deep learning. Breast Cancer Res 2023; 25:142. [PMID: 37957667 PMCID: PMC10644597 DOI: 10.1186/s13058-023-01726-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 10/02/2023] [Indexed: 11/15/2023] Open
Abstract
BACKGROUND Invasive breast cancer patients are increasingly being treated with neoadjuvant chemotherapy; however, only a fraction of the patients respond to it completely. To prevent overtreatment, there is an urgent need for biomarkers to predict treatment response before administering the therapy. METHODS In this retrospective study, we developed hypothesis-driven interpretable biomarkers based on deep learning, to predict the pathological complete response (pCR, i.e., the absence of tumor cells in the surgical resection specimens) to neoadjuvant chemotherapy solely using digital pathology H&E images of pre-treatment breast biopsies. Our approach consists of two steps: First, we use deep learning to characterize aspects of the tumor micro-environment by detecting mitoses and segmenting tissue into several morphology compartments including tumor, lymphocytes and stroma. Second, we derive computational biomarkers from the segmentation and detection output to encode slide-level relationships of components of the tumor microenvironment, such as tumor and mitoses, stroma, and tumor infiltrating lymphocytes (TILs). RESULTS We developed and evaluated our method on slides from n = 721 patients from three European medical centers with triple-negative and Luminal B breast cancers and performed external independent validation on n = 126 patients from a public dataset. We report the predictive value of the investigated biomarkers for predicting pCR with areas under the receiver operating characteristic curve between 0.66 and 0.88 across the tested cohorts. CONCLUSION The proposed computational biomarkers predict pCR, but will require more evaluation and finetuning for clinical application. Our results further corroborate the potential role of deep learning to automate TILs quantification, and their predictive value in breast cancer neoadjuvant treatment planning, along with automated mitoses quantification. We made our method publicly available to extract segmentation-based biomarkers for research purposes.
Collapse
Affiliation(s)
- Witali Aswolinskiy
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Enrico Munari
- Department of Molecular and Translational Medicine, University of Brescia, Brescia, Italy
| | - Hugo M Horlings
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | - Lennart Mulder
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | - Giuseppe Bogina
- Pathology Unit, IRCCS Sacro Cuore Don Calabria Hospital, Negrar di Valpolicella, Verona, Italy
| | - Joyce Sanders
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | - Yat-Hee Liu
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | | | - Leslie Tessier
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
- Center for Integrated Oncology (Institut du cancer de l'Ouest), Angers, France
| | - Maschenka Balkenhol
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Michelle Stegeman
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jeffrey Hoven
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jelle Wesseling
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
- Leiden University Medical Center, Leiden, The Netherlands
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Esther H Lips
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands.
| |
Collapse
|
3
|
Guo J, Hu J, Zheng Y, Zhao S, Ma J. Artificial intelligence: opportunities and challenges in the clinical applications of triple-negative breast cancer. Br J Cancer 2023; 128:2141-2149. [PMID: 36871044 PMCID: PMC10241896 DOI: 10.1038/s41416-023-02215-z] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Revised: 02/08/2023] [Accepted: 02/21/2023] [Indexed: 03/06/2023] Open
Abstract
Triple-negative breast cancer (TNBC) accounts for 15-20% of all invasive breast cancer subtypes. Owing to its clinical characteristics, such as the lack of effective therapeutic targets, high invasiveness, and high recurrence rate, TNBC is difficult to treat and has a poor prognosis. Currently, with the accumulation of large amounts of medical data and the development of computing technology, artificial intelligence (AI), particularly machine learning, has been applied to various aspects of TNBC research, including early screening, diagnosis, identification of molecular subtypes, personalised treatment, and prediction of prognosis and treatment response. In this review, we discussed the general principles of artificial intelligence, summarised its main applications in the diagnosis and treatment of TNBC, and provided new ideas and theoretical basis for the clinical diagnosis and treatment of TNBC.
Collapse
Affiliation(s)
- Jiamin Guo
- Department of Medical Oncology, West China Hospital, Sichuan University, 610041, Chengdu, Sichuan Province, P. R. China
| | - Junjie Hu
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, 610065, Chengdu, Sichuan Province, P. R. China
| | - Yichen Zheng
- Department of Medical Oncology, West China Hospital, Sichuan University, 610041, Chengdu, Sichuan Province, P. R. China
| | - Shuang Zhao
- Department of Radiology, West China Hospital of Sichuan University, 610041, Chengdu, Sichuan Province, P. R. China.
| | - Ji Ma
- Department of Medical Oncology, West China Hospital, Sichuan University, 610041, Chengdu, Sichuan Province, P. R. China.
| |
Collapse
|
4
|
Mercan C, Balkenhol M, Salgado R, Sherman M, Vielh P, Vreuls W, Polónia A, Horlings HM, Weichert W, Carter JM, Bult P, Christgen M, Denkert C, van de Vijver K, Bokhorst JM, van der Laak J, Ciompi F. Deep learning for fully-automated nuclear pleomorphism scoring in breast cancer. NPJ Breast Cancer 2022; 8:120. [DOI: 10.1038/s41523-022-00488-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2021] [Accepted: 10/21/2022] [Indexed: 11/11/2022] Open
Abstract
AbstractTo guide the choice of treatment, every new breast cancer is assessed for aggressiveness (i.e., graded) by an experienced histopathologist. Typically, this tumor grade consists of three components, one of which is the nuclear pleomorphism score (the extent of abnormalities in the overall appearance of tumor nuclei). The degree of nuclear pleomorphism is subjectively classified from 1 to 3, where a score of 1 most closely resembles epithelial cells of normal breast epithelium and 3 shows the greatest abnormalities. Establishing numerical criteria for grading nuclear pleomorphism is challenging, and inter-observer agreement is poor. Therefore, we studied the use of deep learning to develop fully automated nuclear pleomorphism scoring in breast cancer. The reference standard used for training the algorithm consisted of the collective knowledge of an international panel of 10 pathologists on a curated set of regions of interest covering the entire spectrum of tumor morphology in breast cancer. To fully exploit the information provided by the pathologists, a first-of-its-kind deep regression model was trained to yield a continuous scoring rather than limiting the pleomorphism scoring to the standard three-tiered system. Our approach preserves the continuum of nuclear pleomorphism without necessitating a large data set with explicit annotations of tumor nuclei. Once translated to the traditional system, our approach achieves top pathologist-level performance in multiple experiments on regions of interest and whole-slide images, compared to a panel of 10 and 4 pathologists, respectively.
Collapse
|
5
|
Finkelman BS, Meindl A, LaBoy C, Griffin B, Narayan S, Brancamp R, Siziopikou KP, Pincus JL, Blanco LZ. Correlation of manual semi-quantitative and automated quantitative Ki-67 proliferative index with OncotypeDXTM recurrence score in invasive breast carcinoma. Breast Dis 2021; 41:55-65. [PMID: 34397396 DOI: 10.3233/bd-201011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
BACKGROUND Ki-67 immunohistochemistry (IHC) staining is a widely used cancer proliferation assay; however, its limitations could be improved with automated scoring. The OncotypeDXTM Recurrence Score (ORS), which primarily evaluates cancer proliferation genes, is a prognostic indicator for breast cancer chemotherapy response; however, it is more expensive and slower than Ki-67. OBJECTIVE To compare manual Ki-67 (mKi-67) with automated Ki-67 (aKi-67) algorithm results based on manually selected Ki-67 "hot spots" in breast cancer, and correlate both with ORS. METHODS 105 invasive breast carcinoma cases from 100 patients at our institution (2011-2013) with available ORS were evaluated. Concordance was assessed via Cohen's Kappa (κ). RESULTS 57/105 cases showed agreement between mKi-67 and aKi-67 (κ 0.31, 95% CI 0.18-0.45), with 41 cases overestimated by aKi-67. Concordance was higher when estimated on the same image (κ 0.53, 95% CI 0.37-0.69). Concordance between mKi-67 score and ORS was fair (κ 0.27, 95% CI 0.11-0.42), and concordance between aKi-67 and ORS was poor (κ 0.10, 95% CI -0.03-0.23). CONCLUSIONS These results highlight the limits of Ki-67 algorithms that use manual "hot spot" selection. Due to suboptimal concordance, Ki-67 is likely most useful as a complement to, rather than a surrogate for ORS, regardless of scoring method.
Collapse
Affiliation(s)
- Brian S Finkelman
- Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Amanda Meindl
- Department of Pathology, Great Lakes Pathologists, West Allis, WI, USA
| | - Carissa LaBoy
- Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Brannan Griffin
- Department of Pathology, University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Suguna Narayan
- Department of Pathology, University of Colorado Denver School of Medicine, Aurora, CO, USA
| | - Ryan Brancamp
- Department of Pathology, Microbiology, and Immunology, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Kalliopi P Siziopikou
- Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Jennifer L Pincus
- Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Luis Z Blanco
- Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| |
Collapse
|
6
|
Balkenhol MC, Ciompi F, Świderska-Chadaj Ż, van de Loo R, Intezar M, Otte-Höller I, Geijs D, Lotz J, Weiss N, de Bel T, Litjens G, Bult P, van der Laak JA. Optimized tumour infiltrating lymphocyte assessment for triple negative breast cancer prognostics. Breast 2021; 56:78-87. [PMID: 33640523 PMCID: PMC7933536 DOI: 10.1016/j.breast.2021.02.007] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2020] [Revised: 02/05/2021] [Accepted: 02/08/2021] [Indexed: 12/29/2022] Open
Abstract
The tumour microenvironment has been shown to be a valuable source of prognostic information for different cancer types. This holds in particular for triple negative breast cancer (TNBC), a breast cancer subtype for which currently no prognostic biomarkers are established. Although different methods to assess tumour infiltrating lymphocytes (TILs) have been published, it remains unclear which method (marker, region) yields the most optimal prognostic information. In addition, to date, no objective TILs assessment methods are available. For this proof of concept study, a subset of our previously described TNBC cohort (n = 94) was stained for CD3, CD8 and FOXP3 using multiplex immunohistochemistry and subsequently imaged by a multispectral imaging system. Advanced whole-slide image analysis algorithms, including convolutional neural networks (CNN) were used to register unmixed multispectral images and corresponding H&E sections, to segment the different tissue compartments (tumour, stroma) and to detect all individual positive lymphocytes. Densities of positive lymphocytes were analysed in different regions within the tumour and its neighbouring environment and correlated to relapse free survival (RFS) and overall survival (OS). We found that for all TILs markers the presence of a high density of positive cells correlated with an improved survival. None of the TILs markers was superior to the others. The results of TILs assessment in the various regions did not show marked differences between each other. The negative correlation between TILs and survival in our cohort are in line with previous studies. Our results provide directions for optimizing TILs assessment methodology.
Collapse
Affiliation(s)
- Maschenka Ca Balkenhol
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands.
| | - Francesco Ciompi
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Żaneta Świderska-Chadaj
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands; Warsaw University of Technology, Faculty of Electrical Engineering, Warsaw, Poland
| | - Rob van de Loo
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Milad Intezar
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Irene Otte-Höller
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Daan Geijs
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Johannes Lotz
- Fraunhofer Institute for Image Computing MEVIS, Lübeck, Germany
| | - Nick Weiss
- Fraunhofer Institute for Image Computing MEVIS, Lübeck, Germany
| | - Thomas de Bel
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Geert Litjens
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Peter Bult
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Jeroen Awm van der Laak
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands; Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden
| |
Collapse
|
7
|
Representation of Differential Learning Method for Mitosis Detection. JOURNAL OF HEALTHCARE ENGINEERING 2021. [DOI: 10.1155/2021/6688477] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
The breast cancer microscopy images acquire information about the patient’s ailment, and the automated mitotic cell detection outcomes have generally been utilized to ease the massive amount of pathologist’s work and help the pathologists make clinical decisions quickly. Several previous methods were introduced to solve automated mitotic cell count problems. However, they failed to differentiate between mitotic and nonmitotic cells and come up with an imbalance problem, which affects the performance. This paper proposes a Representation Differential Learning Method (RDLM) for mitosis detection through deep learning to detect the accurate mitotic cell area on pathological images. Our proposed method has been divided into two parts: Global bank Feature Pyramid Network (GLB-FPN) and focal loss (FL). The GLB feature fusion method with FPN essentially makes the encoder-decoder pay attention, to further extract the region of interest (ROIs) for mitotic cells. On this basis, we extend the GLB-FPN with a focal loss to mitigate the data imbalance problem during the training stage. Extensive experiments have shown that RDLM significantly outperforms on visualization view and achieves the best performance in quantitative matrices than other proposed approaches on the MITOS-ATYPIA-14 contest dataset. Our framework reaches a 0.692 F1-score. Additionally, RDLM achieves 5% improvements than GLB with FPN in F1-score on the mitosis detection task.
Collapse
|
8
|
Moxley-Wyles B, Colling R, Verrill C. Artificial intelligence in pathology: an overview. ACTA ACUST UNITED AC 2020. [DOI: 10.1016/j.mpdhp.2020.08.004] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
|
9
|
Pantanowitz L, Hartman D, Qi Y, Cho EY, Suh B, Paeng K, Dhir R, Michelow P, Hazelhurst S, Song SY, Cho SY. Accuracy and efficiency of an artificial intelligence tool when counting breast mitoses. Diagn Pathol 2020; 15:80. [PMID: 32622359 PMCID: PMC7335442 DOI: 10.1186/s13000-020-00995-z] [Citation(s) in RCA: 42] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2020] [Accepted: 06/25/2020] [Indexed: 12/17/2022] Open
Abstract
BACKGROUND The mitotic count in breast carcinoma is an important prognostic marker. Unfortunately substantial inter- and intra-laboratory variation exists when pathologists manually count mitotic figures. Artificial intelligence (AI) coupled with whole slide imaging offers a potential solution to this problem. The aim of this study was to accordingly critique an AI tool developed to quantify mitotic figures in whole slide images of invasive breast ductal carcinoma. METHODS A representative H&E slide from 320 breast invasive ductal carcinoma cases was scanned at 40x magnification. Ten expert pathologists from two academic medical centers labeled mitotic figures in whole slide images to train and validate an AI algorithm to detect and count mitoses. Thereafter, 24 readers of varying expertise were asked to count mitotic figures with and without AI support in 140 high-power fields derived from a separate dataset. Their accuracy and efficiency of performing these tasks were calculated and statistical comparisons performed. RESULTS For each experience level the accuracy, precision and sensitivity of counting mitoses by users improved with AI support. There were 21 readers (87.5%) that identified more mitoses using AI support and 13 reviewers (54.2%) that decreased the quantity of falsely flagged mitoses with AI. More time was spent on this task for most participants when not provided with AI support. AI assistance resulted in an overall time savings of 27.8%. CONCLUSIONS This study demonstrates that pathology end-users were more accurate and efficient at quantifying mitotic figures in digital images of invasive breast carcinoma with the aid of AI. Higher inter-pathologist agreement with AI assistance suggests that such algorithms can also help standardize practice. Not surprisingly, there is much enthusiasm in pathology regarding the prospect of using AI in routine practice to perform mundane tasks such as counting mitoses.
Collapse
Affiliation(s)
- Liron Pantanowitz
- Department of Pathology, University of Pittsburgh Medical Center Cancer Pavilion, Suite 201, 5150 Centre Ave, Pittsburgh, PA, 15232, USA.
- Department of Anatomical Pathology, University of the Witwatersrand and National Health Laboratory Services, Johannesburg, South Africa.
| | - Douglas Hartman
- Department of Pathology, University of Pittsburgh Medical Center Cancer Pavilion, Suite 201, 5150 Centre Ave, Pittsburgh, PA, 15232, USA
| | - Yan Qi
- School of Medicine, University of Pittsburgh, Pittsburgh, PA, USA
| | - Eun Yoon Cho
- Department of Pathology, Samsung Medical Center, Seoul, South Korea
| | | | | | - Rajiv Dhir
- Department of Pathology, University of Pittsburgh Medical Center Cancer Pavilion, Suite 201, 5150 Centre Ave, Pittsburgh, PA, 15232, USA
| | - Pamela Michelow
- Department of Anatomical Pathology, University of the Witwatersrand and National Health Laboratory Services, Johannesburg, South Africa
| | - Scott Hazelhurst
- School of Electrical & Information Engineering and Sydney Brenner Institute for Molecular Bioscience, University of the Witwatersrand, Johannesburg, South Africa
| | - Sang Yong Song
- Department of Pathology, Samsung Medical Center, Seoul, South Korea
| | - Soo Youn Cho
- Department of Pathology, Samsung Medical Center, Seoul, South Korea
| |
Collapse
|
10
|
Cherezov D, Paul R, Fetisov N, Gillies RJ, Schabath MB, Goldgof DB, Hall LO. Lung Nodule Sizes Are Encoded When Scaling CT Image for CNN's. ACTA ACUST UNITED AC 2020; 6:209-215. [PMID: 32548298 PMCID: PMC7289250 DOI: 10.18383/j.tom.2019.00024] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
Noninvasive diagnosis of lung cancer in early stages is one task where radiomics helps. Clinical practice shows that the size of a nodule has high predictive power for malignancy. In the literature, convolutional neural networks (CNNs) have become widely used in medical image analysis. We study the ability of a CNN to capture nodule size in computed tomography images after images are resized for CNN input. For our experiments, we used the National Lung Screening Trial data set. Nodules were labeled into 2 categories (small/large) based on the original size of a nodule. After all extracted patches were re-sampled into 100-by-100-pixel images, a CNN was able to successfully classify test nodules into small- and large-size groups with high accuracy. To show the generality of our discovery, we repeated size classification experiments using Common Objects in Context (COCO) data set. From the data set, we selected 3 categories of images, namely, bears, cats, and dogs. For all 3 categories a 5- × 2-fold cross-validation was performed to put them into small and large classes. The average area under receiver operating curve is 0.954, 0.952, and 0.979 for the bear, cat, and dog categories, respectively. Thus, camera image rescaling also enables a CNN to discover the size of an object. The source code for experiments with the COCO data set is publicly available in Github (https://github.com/VisionAI-USF/COCO_Size_Decoding/).
Collapse
Affiliation(s)
- Dmitry Cherezov
- Department of Computer Sciences and Engineering, University of South Florida, Tampa, FL
| | - Rahul Paul
- Department of Computer Sciences and Engineering, University of South Florida, Tampa, FL
| | - Nikolai Fetisov
- Department of Computer Sciences and Engineering, University of South Florida, Tampa, FL
| | | | - Matthew B Schabath
- Cancer Epidemiology, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL
| | - Dmitry B Goldgof
- Department of Computer Sciences and Engineering, University of South Florida, Tampa, FL
| | - Lawrence O Hall
- Department of Computer Sciences and Engineering, University of South Florida, Tampa, FL
| |
Collapse
|
11
|
Shiraishi T, Shinto E, Nearchou IP, Tsuda H, Kajiwara Y, Einama T, Caie PD, Kishi Y, Ueno H. Prognostic significance of mesothelin expression in colorectal cancer disclosed by area-specific four-point tissue microarrays. Virchows Arch 2020; 477:409-420. [PMID: 32107600 DOI: 10.1007/s00428-020-02775-y] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2019] [Revised: 01/17/2020] [Accepted: 02/13/2020] [Indexed: 02/06/2023]
Abstract
Mesothelin (MSLN) is a cell surface glycoprotein present in many cancer types. Its expression is generally associated with an unfavorable prognosis. This study examined the prognostic significance of MSLN expression in different areas of individual colorectal cancers (CRCs) using tissue microarrays (TMAs) by enrolling 314 patients with stage II (T3-T4, N0, M0) CRCs. Using formalin-fixed paraffin-embedded tissue blocks from patients, TMA blocks were constructed. Tissue core specimens were obtained from submucosal invasive front (Fr-sm), subserosal invasive front (Fr-ss), central area (Ce), and rolled edge (Ro) of each tumor. Using these four-point TMA sets, MSLN expression was immunohistochemically surveyed. The area-specific prognostic significance of MSLN expression was evaluated. A deep learning convolutional neural network algorithm was used for imaging analysis and evaluating our judgment's objectivity. MSLN staining ratio was positively correlated between the manual and machine-learning analyses (r = 0.71). The correlation coefficient between Ro and Ce, Ro and Fr-sm, and Ro and Fr-ss was r = 0.63, r = 0.54, and r = 0.61, respectively. Disease-specific survival curves for the MSLN-positive and MSLN-negative groups in Fr-sm, Fr-ss, and Ro were significantly different (five-year survival rates 88.1% and 95.5% (P = 0.024), 85.0 and 96.2% (P = 0.0087), 87.8 and 95.5% (P = 0.051), and 77.9 and 95.8% (P = 0.046) for Fr-sm, Fr-ss, Ce, and Ro, respectively). The analysis performed using area-specific four-point TMAs clearly demonstrated that MSLN expression in stage II CRC was relatively homogeneous within tumors. Additionally, high MSLN expression showed or tended to show unfavorable prognostic significance regardless of the tumor area.
Collapse
Affiliation(s)
- Takehiro Shiraishi
- Department of Surgery, National Defense Medical College, 3-2 Namiki, Tokorozawa, Saitama, 359-0042, Japan
| | - Eiji Shinto
- Department of Surgery, National Defense Medical College, 3-2 Namiki, Tokorozawa, Saitama, 359-0042, Japan.
| | - Ines P Nearchou
- Quantitative and Digital Pathology, School of Medicine, University of St. Andrews, St. Andrews, KY16 9TF, UK
| | - Hitoshi Tsuda
- Department of Basic Pathology, National Defense Medical College, 3-2 Namiki, Tokorozawa, Saitama, 359-0042, Japan
| | - Yoshiki Kajiwara
- Department of Surgery, National Defense Medical College, 3-2 Namiki, Tokorozawa, Saitama, 359-0042, Japan
| | - Takahiro Einama
- Department of Surgery, National Defense Medical College, 3-2 Namiki, Tokorozawa, Saitama, 359-0042, Japan
| | - Peter D Caie
- Quantitative and Digital Pathology, School of Medicine, University of St. Andrews, St. Andrews, KY16 9TF, UK
| | - Yoji Kishi
- Department of Surgery, National Defense Medical College, 3-2 Namiki, Tokorozawa, Saitama, 359-0042, Japan
| | - Hideki Ueno
- Department of Surgery, National Defense Medical College, 3-2 Namiki, Tokorozawa, Saitama, 359-0042, Japan
| |
Collapse
|
12
|
Bhattacharjee S, Kim CH, Park HG, Prakash D, Madusanka N, Cho NH, Choi HK. Multi-Features Classification of Prostate Carcinoma Observed in Histological Sections: Analysis of Wavelet-Based Texture and Colour Features. Cancers (Basel) 2019; 11:E1937. [PMID: 31817111 PMCID: PMC6966617 DOI: 10.3390/cancers11121937] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2019] [Revised: 11/14/2019] [Accepted: 11/28/2019] [Indexed: 11/16/2022] Open
Abstract
Microscopic biopsy images are coloured in nature because pathologists use the haematoxylin and eosin chemical colour dyes for biopsy examinations. In this study, biopsy images are used for histological grading and the analysis of benign and malignant prostate tissues. The following PCa grades are analysed in the present study: benign, grade 3, grade 4, and grade 5. Biopsy imaging has become increasingly important for the clinical assessment of PCa. In order to analyse and classify the histological grades of prostate carcinomas, pixel-based colour moment descriptor (PCMD) and gray-level co-occurrence matrix (GLCM) methods were used to extract the most significant features for multilayer perceptron (MLP) neural network classification. Haar wavelet transformation was carried out to extract GLCM texture features, and colour features were extracted from RGB (red/green/blue) colour images of prostate tissues. The MANOVA statistical test was performed to select significant features based on F-values and P-values using the R programming language. We obtained an average highest accuracy of 92.7% using level-1 wavelet texture and colour features. The MLP classifier performed well, and our study shows promising results based on multi-feature classification of histological sections of prostate carcinomas.
Collapse
Affiliation(s)
- Subrata Bhattacharjee
- Department of Computer Engineering, u-AHRC, Inje University, Gimhae 50834, Korea; (S.B.); (H.-G.P.); (D.P.); (N.M.)
| | - Cho-Hee Kim
- Department of Digital Anti-Aging Healthcare, Inje University, Gimhae 50834, Korea;
| | - Hyeon-Gyun Park
- Department of Computer Engineering, u-AHRC, Inje University, Gimhae 50834, Korea; (S.B.); (H.-G.P.); (D.P.); (N.M.)
| | - Deekshitha Prakash
- Department of Computer Engineering, u-AHRC, Inje University, Gimhae 50834, Korea; (S.B.); (H.-G.P.); (D.P.); (N.M.)
| | - Nuwan Madusanka
- Department of Computer Engineering, u-AHRC, Inje University, Gimhae 50834, Korea; (S.B.); (H.-G.P.); (D.P.); (N.M.)
| | - Nam-Hoon Cho
- Department of Pathology, Yonsei University Hospital, Seoul 03722, Korea;
| | - Heung-Kook Choi
- Department of Computer Engineering, u-AHRC, Inje University, Gimhae 50834, Korea; (S.B.); (H.-G.P.); (D.P.); (N.M.)
| |
Collapse
|
13
|
Lende TH, Austdal M, Varhaugvik AE, Skaland I, Gudlaugsson E, Kvaløy JT, Akslen LA, Søiland H, Janssen EAM, Baak JPA. Influence of pre-operative oral carbohydrate loading vs. standard fasting on tumor proliferation and clinical outcome in breast cancer patients ─ a randomized trial. BMC Cancer 2019; 19:1076. [PMID: 31703648 PMCID: PMC6842165 DOI: 10.1186/s12885-019-6275-z] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2019] [Accepted: 10/18/2019] [Indexed: 12/18/2022] Open
Abstract
Background Conflicting results have been reported on the influence of carbohydrates in breast cancer. Objective To determine the influence of pre-operative per-oral carbohydrate load on proliferation in breast tumors. Design Randomized controlled trial. Setting University hospital with primary and secondary care functions in South-West Norway. Patients Sixty-one patients with operable breast cancer from a population-based cohort. Intervention Per-oral carbohydrate load (preOp™) 18 and 2–4 h before surgery (n = 26) or standard pre-operative fasting with free consumption of tap water (n = 35). Measurements The primary outcome was post-operative tumor proliferation measured by the mitotic activity index (MAI). The secondary outcomes were changes in the levels of serum insulin, insulin-c-peptide, glucose, IGF-1, and IGFBP3; patients’ well-being, and clinical outcome over a median follow-up of 88 months (range 33–97 months). Results In the estrogen receptor (ER) positive subgroup (n = 50), high proliferation (MAI ≥ 10) occurred more often in the carbohydrate group (CH) than in the fasting group (p = 0.038). The CH group was more frequently progesterone receptor (PR) negative (p = 0.014). The CH group had a significant increase in insulin (+ 24.31 mIE/L, 95% CI 15.34 mIE/L to 33.27 mIE/L) and insulin c-peptide (+ 1.39 nM, 95% CI 1.03 nM to 1.77 nM), but reduced IGFBP3 levels (− 0.26 nM; 95% CI − 0.46 nM to − 0.051 nM) compared to the fasting group. CH-intervention ER-positive patients had poorer relapse-free survival (73%) than the fasting group (100%; p = 0.012; HR = 9.3, 95% CI, 1.1 to 77.7). In the ER-positive patients, only tumor size (p = 0.021; HR = 6.07, 95% CI 1.31 to 28.03) and the CH/fasting subgrouping (p = 0.040; HR = 9.30, 95% CI 1.11 to 77.82) had independent prognostic value. The adverse clinical outcome of carbohydrate loading occurred only in T2 patients with relapse-free survival of 100% in the fasting group vs. 33% in the CH group (p = 0.015; HR = inf). The CH group reported less pain on days 5 and 6 than the control group (p < 0.001) but otherwise exhibited no factors related to well-being. Limitation Only applicable to T2 tumors in patients with ER-positive breast cancer. Conclusions Pre-operative carbohydrate load increases proliferation and PR-negativity in ER-positive patients and worsens clinical outcome in ER-positive T2 patients. Trial registration CliniTrials.gov; NCT03886389. Retrospectively registered March 22, 2019.
Collapse
Affiliation(s)
- Tone Hoel Lende
- Department of Breast & Endocrine Surgery, Stavanger University Hospital, Helse Stavanger HF, P.O. Box 8100, N-4068, Stavanger, Norway. .,Centre for Cancer Biomarkers CCBIO, Department of Clinical Medicine, Faculty of Medicine and Dentistry, University of Bergen, Jonas Lies vei 87, N-5012, Bergen, Norway.
| | - Marie Austdal
- Department of Research, Stavanger University Hospital, Helse Stavanger HF, P.O. Box 8100, N-4068, Stavanger, Norway.,Department of Pathology, Stavanger University Hospital, Helse Stavanger HF, P.O. Box 8100, N-4068, Stavanger, Norway
| | - Anne Elin Varhaugvik
- Department of Pathology, Stavanger University Hospital, Helse Stavanger HF, P.O. Box 8100, N-4068, Stavanger, Norway.,Department of Pathology, Helse Møre og Romsdal HF, P.O. Box 1600, N-6026, Ålesund, Norway
| | - Ivar Skaland
- Department of Pathology, Stavanger University Hospital, Helse Stavanger HF, P.O. Box 8100, N-4068, Stavanger, Norway
| | - Einar Gudlaugsson
- Department of Pathology, Stavanger University Hospital, Helse Stavanger HF, P.O. Box 8100, N-4068, Stavanger, Norway
| | - Jan Terje Kvaløy
- Department of Research, Stavanger University Hospital, Helse Stavanger HF, P.O. Box 8100, N-4068, Stavanger, Norway.,Department of Mathematics and Physics, University of Stavanger, P.O. Box 8600 Forus, N-4036, Stavanger, Norway
| | - Lars A Akslen
- Centre for Cancer Biomarkers CCBIO, Department of Clinical Medicine, Faculty of Medicine and Dentistry, University of Bergen, Jonas Lies vei 87, N-5012, Bergen, Norway.,Gades Institute, Laboratory Medicine Pathology, University of Bergen, Jonas Lies vei 87, N-5012, Bergen, Norway
| | - Håvard Søiland
- Department of Breast & Endocrine Surgery, Stavanger University Hospital, Helse Stavanger HF, P.O. Box 8100, N-4068, Stavanger, Norway.,Department of Clinical Science, University of Bergen, Jonas Lies vei 87, N-5012, Bergen, Norway
| | - Emiel A M Janssen
- Department of Pathology, Stavanger University Hospital, Helse Stavanger HF, P.O. Box 8100, N-4068, Stavanger, Norway.,Department of Mathematics and Physics, University of Stavanger, P.O. Box 8600 Forus, N-4036, Stavanger, Norway
| | - Jan P A Baak
- Department of Pathology, Stavanger University Hospital, Helse Stavanger HF, P.O. Box 8100, N-4068, Stavanger, Norway.,, Risavegen 66, N-4056, Tananger, Norway.,, Vierhuysen 6, 1921 SB, Akersloot, Netherlands
| |
Collapse
|
14
|
Deep learning assisted mitotic counting for breast cancer. J Transl Med 2019; 99:1596-1606. [PMID: 31222166 DOI: 10.1038/s41374-019-0275-0] [Citation(s) in RCA: 60] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2019] [Revised: 04/06/2019] [Accepted: 04/08/2019] [Indexed: 11/09/2022] Open
Abstract
As part of routine histological grading, for every invasive breast cancer the mitotic count is assessed by counting mitoses in the (visually selected) region with the highest proliferative activity. Because this procedure is prone to subjectivity, the present study compares visual mitotic counting with deep learning based automated mitotic counting and fully automated hotspot selection. Two cohorts were used in this study. Cohort A comprised 90 prospectively included tumors which were selected based on the mitotic frequency scores given during routine glass slide diagnostics. This pathologist additionally assessed the mitotic count in these tumors in whole slide images (WSI) within a preselected hotspot. A second observer performed the same procedures on this cohort. The preselected hotspot was generated by a convolutional neural network (CNN) trained to detect all mitotic figures in digitized hematoxylin and eosin (H&E) sections. The second cohort comprised a multicenter, retrospective TNBC cohort (n = 298), of which the mitotic count was assessed by three independent observers on glass slides. The same CNN was applied on this cohort and the absolute number of mitotic figures in the hotspot was compared to the averaged mitotic count of the observers. Baseline interobserver agreement for glass slide assessment in cohort A was good (kappa 0.689; 95% CI 0.580-0.799). Using the CNN generated hotspot in WSI, the agreement score increased to 0.814 (95% CI 0.719-0.909). Automated counting by the CNN in comparison with observers counting in the predefined hotspot region yielded an average kappa of 0.724. We conclude that manual mitotic counting is not affected by assessment modality (glass slides, WSI) and that counting mitotic figures in WSI is feasible. Using a predefined hotspot area considerably improves reproducibility. Also, fully automated assessment of mitotic score appears to be feasible without introducing additional bias or variability.
Collapse
|