1
|
Tafavvoghi M, Bongo LA, Shvetsov N, Busund LTR, Møllersen K. Publicly available datasets of breast histopathology H&E whole-slide images: A scoping review. J Pathol Inform 2024; 15:100363. [PMID: 38405160 PMCID: PMC10884505 DOI: 10.1016/j.jpi.2024.100363] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 11/24/2023] [Accepted: 01/23/2024] [Indexed: 02/27/2024] Open
Abstract
Advancements in digital pathology and computing resources have made a significant impact in the field of computational pathology for breast cancer diagnosis and treatment. However, access to high-quality labeled histopathological images of breast cancer is a big challenge that limits the development of accurate and robust deep learning models. In this scoping review, we identified the publicly available datasets of breast H&E-stained whole-slide images (WSIs) that can be used to develop deep learning algorithms. We systematically searched 9 scientific literature databases and 9 research data repositories and found 17 publicly available datasets containing 10 385 H&E WSIs of breast cancer. Moreover, we reported image metadata and characteristics for each dataset to assist researchers in selecting proper datasets for specific tasks in breast cancer computational pathology. In addition, we compiled 2 lists of breast H&E patches and private datasets as supplementary resources for researchers. Notably, only 28% of the included articles utilized multiple datasets, and only 14% used an external validation set, suggesting that the performance of other developed models may be susceptible to overestimation. The TCGA-BRCA was used in 52% of the selected studies. This dataset has a considerable selection bias that can impact the robustness and generalizability of the trained algorithms. There is also a lack of consistent metadata reporting of breast WSI datasets that can be an issue in developing accurate deep learning models, indicating the necessity of establishing explicit guidelines for documenting breast WSI dataset characteristics and metadata.
Collapse
Affiliation(s)
- Masoud Tafavvoghi
- Department of Community Medicine, Uit The Arctic University of Norway, Tromsø, Norway
| | - Lars Ailo Bongo
- Department of Computer Science, Uit The Arctic University of Norway, Tromsø, Norway
| | - Nikita Shvetsov
- Department of Computer Science, Uit The Arctic University of Norway, Tromsø, Norway
| | | | - Kajsa Møllersen
- Department of Community Medicine, Uit The Arctic University of Norway, Tromsø, Norway
| |
Collapse
|
2
|
Budginaite E, Magee DR, Kloft M, Woodruff HC, Grabsch HI. Computational methods for metastasis detection in lymph nodes and characterization of the metastasis-free lymph node microarchitecture: A systematic-narrative hybrid review. J Pathol Inform 2024; 15:100367. [PMID: 38455864 PMCID: PMC10918266 DOI: 10.1016/j.jpi.2024.100367] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Revised: 01/31/2024] [Accepted: 01/31/2024] [Indexed: 03/09/2024] Open
Abstract
Background Histological examination of tumor draining lymph nodes (LNs) plays a vital role in cancer staging and prognostication. However, as soon as a LN is classed as metastasis-free, no further investigation will be performed and thus, potentially clinically relevant information detectable in tumor-free LNs is currently not captured. Objective To systematically study and critically assess methods for the analysis of digitized histological LN images described in published research. Methods A systematic search was conducted in several public databases up to December 2023 using relevant search terms. Studies using brightfield light microscopy images of hematoxylin and eosin or immunohistochemically stained LN tissue sections aiming to detect and/or segment LNs, their compartments or metastatic tumor using artificial intelligence (AI) were included. Dataset, AI methodology, cancer type, and study objective were compared between articles. Results A total of 7201 articles were collected and 73 articles remained for detailed analyses after article screening. Of the remaining articles, 86% aimed at LN metastasis identification, 8% aimed at LN compartment segmentation, and remaining focused on LN contouring. Furthermore, 78% of articles used patch classification and 22% used pixel segmentation models for analyses. Five out of six studies (83%) of metastasis-free LNs were performed on publicly unavailable datasets, making quantitative article comparison impossible. Conclusions Multi-scale models mimicking multiple microscopy zooms show promise for computational LN analysis. Large-scale datasets are needed to establish the clinical relevance of analyzing metastasis-free LN in detail. Further research is needed to identify clinically interpretable metrics for LN compartment characterization.
Collapse
Affiliation(s)
- Elzbieta Budginaite
- Department of Pathology, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
- Department of Precision Medicine, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
| | | | - Maximilian Kloft
- Department of Pathology, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
- Department of Internal Medicine, Justus-Liebig-University, Giessen, Germany
| | - Henry C. Woodruff
- Department of Precision Medicine, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
| | - Heike I. Grabsch
- Department of Pathology, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
- Pathology and Data Analytics, Leeds Institute of Medical Research at St James’s, University of Leeds, Leeds, UK
| |
Collapse
|
3
|
Saqi A, Liu Y, Politis MG, Salvatore M, Jambawalikar S. Combined expert-in-the-loop-random forest multiclass segmentation U-net based artificial intelligence model: evaluation of non-small cell lung cancer in fibrotic and non-fibrotic microenvironments. J Transl Med 2024; 22:640. [PMID: 38978066 PMCID: PMC11232199 DOI: 10.1186/s12967-024-05394-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Accepted: 06/12/2024] [Indexed: 07/10/2024] Open
Abstract
BACKGROUND The tumor microenvironment (TME) plays a key role in lung cancer initiation, proliferation, invasion, and metastasis. Artificial intelligence (AI) methods could potentially accelerate TME analysis. The aims of this study were to (1) assess the feasibility of using hematoxylin and eosin (H&E)-stained whole slide images (WSI) to develop an AI model for evaluating the TME and (2) to characterize the TME of adenocarcinoma (ADCA) and squamous cell carcinoma (SCCA) in fibrotic and non-fibrotic lung. METHODS The cohort was derived from chest CT scans of patients presenting with lung neoplasms, with and without background fibrosis. WSI images were generated from slides of all 76 available pathology cases with ADCA (n = 53) or SCCA (n = 23) in fibrotic (n = 47) or non-fibrotic (n = 29) lung. Detailed ground-truth annotations, including of stroma (i.e., fibrosis, vessels, inflammation), necrosis and background, were performed on WSI and optimized via an expert-in-the-loop (EITL) iterative procedure using a lightweight [random forest (RF)] classifier. A convolution neural network (CNN)-based model was used to achieve tissue-level multiclass segmentation. The model was trained on 25 annotated WSI from 13 cases of ADCA and SCCA within and without fibrosis and then applied to the 76-case cohort. The TME analysis included tumor stroma ratio (TSR), tumor fibrosis ratio (TFR), tumor inflammation ratio (TIR), tumor vessel ratio (TVR), tumor necrosis ratio (TNR), and tumor background ratio (TBR). RESULTS The model's overall classification for precision, sensitivity, and F1-score were 94%, 90%, and 91%, respectively. Statistically significant differences were noted in TSR (p = 0.041) and TFR (p = 0.001) between fibrotic and non-fibrotic ADCA. Within fibrotic lung, statistically significant differences were present in TFR (p = 0.039), TIR (p = 0.003), TVR (p = 0.041), TNR (p = 0.0003), and TBR (p = 0.020) between ADCA and SCCA. CONCLUSION The combined EITL-RF CNN model using only H&E WSI can facilitate multiclass evaluation and quantification of the TME. There are significant differences in the TME of ADCA and SCCA present within or without background fibrosis. Future studies are needed to determine the significance of TME on prognosis and treatment.
Collapse
Affiliation(s)
- Anjali Saqi
- Department of Pathology and Cell Biology, Columbia University Irving Medical Center, 630 West 168th Street, New York, NY, VC14-215, 10032, USA.
| | - Yucheng Liu
- Department of Radiation Physics, Atlantic Health System, New Jersey, NJ, USA
| | - Michelle Garlin Politis
- Department of Pathology and Cell Biology, Columbia University Irving Medical Center, 630 West 168th Street, New York, NY, VC14-215, 10032, USA
| | - Mary Salvatore
- Department of Radiology, Columbia University Irving Medical Center, New York, NY, USA
| | - Sachin Jambawalikar
- Department of Radiology, Columbia University Irving Medical Center, New York, NY, USA
| |
Collapse
|
4
|
McGenity C, Clarke EL, Jennings C, Matthews G, Cartlidge C, Freduah-Agyemang H, Stocken DD, Treanor D. Artificial intelligence in digital pathology: a systematic review and meta-analysis of diagnostic test accuracy. NPJ Digit Med 2024; 7:114. [PMID: 38704465 PMCID: PMC11069583 DOI: 10.1038/s41746-024-01106-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2023] [Accepted: 04/12/2024] [Indexed: 05/06/2024] Open
Abstract
Ensuring diagnostic performance of artificial intelligence (AI) before introduction into clinical practice is essential. Growing numbers of studies using AI for digital pathology have been reported over recent years. The aim of this work is to examine the diagnostic accuracy of AI in digital pathology images for any disease. This systematic review and meta-analysis included diagnostic accuracy studies using any type of AI applied to whole slide images (WSIs) for any disease. The reference standard was diagnosis by histopathological assessment and/or immunohistochemistry. Searches were conducted in PubMed, EMBASE and CENTRAL in June 2022. Risk of bias and concerns of applicability were assessed using the QUADAS-2 tool. Data extraction was conducted by two investigators and meta-analysis was performed using a bivariate random effects model, with additional subgroup analyses also performed. Of 2976 identified studies, 100 were included in the review and 48 in the meta-analysis. Studies were from a range of countries, including over 152,000 whole slide images (WSIs), representing many diseases. These studies reported a mean sensitivity of 96.3% (CI 94.1-97.7) and mean specificity of 93.3% (CI 90.5-95.4). There was heterogeneity in study design and 99% of studies identified for inclusion had at least one area at high or unclear risk of bias or applicability concerns. Details on selection of cases, division of model development and validation data and raw performance data were frequently ambiguous or missing. AI is reported as having high diagnostic accuracy in the reported areas but requires more rigorous evaluation of its performance.
Collapse
Affiliation(s)
- Clare McGenity
- University of Leeds, Leeds, UK.
- Leeds Teaching Hospitals NHS Trust, Leeds, UK.
| | - Emily L Clarke
- University of Leeds, Leeds, UK
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | - Charlotte Jennings
- University of Leeds, Leeds, UK
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | | | | | | | | | - Darren Treanor
- University of Leeds, Leeds, UK
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
- Department of Clinical Pathology and Department of Clinical and Experimental Medicine, Linköping University, Linköping, Sweden
- Centre for Medical Image Science and Visualization (CMIV), Linköping University, Linköping, Sweden
| |
Collapse
|
5
|
Bai Y, Li W, An J, Xia L, Chen H, Zhao G, Gao Z. Masked autoencoders with handcrafted feature predictions: Transformer for weakly supervised esophageal cancer classification. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 244:107936. [PMID: 38016392 DOI: 10.1016/j.cmpb.2023.107936] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Revised: 10/28/2023] [Accepted: 11/19/2023] [Indexed: 11/30/2023]
Abstract
BACKGROUND AND OBJECTIVE Esophageal cancer is a serious disease with a high prevalence in Eastern Asia. Histopathology tissue analysis stands as the gold standard in diagnosing esophageal cancer. In recent years, there has been a shift towards digitizing histopathological images into whole slide images (WSIs), progressively integrating them into cancer diagnostics. However, the gigapixel sizes of WSIs present significant storage and processing challenges, and they often lack localized annotations. To address this issue, multi-instance learning (MIL) has been introduced for WSI classification, utilizing weakly supervised learning for diagnosis analysis. By applying the principles of MIL to WSI analysis, it is possible to reduce the workload of pathologists by facilitating the generation of localized annotations. Nevertheless, the approach's effectiveness is hindered by the traditional simple aggregation operation and the domain shift resulting from the prevalent use of convolutional feature extractors pretrained on ImageNet. METHODS We propose a MIL-based framework for WSI analysis and cancer classification. Concurrently, we introduce employing self-supervised learning, which obviates the need for manual annotation and demonstrates versatility in various tasks, to pretrain feature extractors. This method enhances the extraction of representative features from esophageal WSI for MIL, ensuring more robust and accurate performance. RESULTS We build a comprehensive dataset of whole esophageal slide images and conduct extensive experiments utilizing this dataset. The performance on our dataset demonstrates the efficiency of our proposed MIL framework and the pretraining process, with our framework outperforming existing methods, achieving an accuracy of 93.07% and AUC (area under the curve) of 95.31%. CONCLUSION This work proposes an effective MIL method to classify WSI of esophageal cancer. The promising results indicate that our cancer classification framework holds great potential in promoting the automatic whole esophageal slide image analysis.
Collapse
Affiliation(s)
- Yunhao Bai
- the School of Electrical and Information Engineering, Tianjin University, Tianjin, China
| | - Wenqi Li
- Department of Pathology, Key Laboratory of Cancer Prevention and Therapy, National Clinical Research Center for Cancer, Tianjin Medical University Cancer Institute and Hospital, Tianjin, China
| | - Jianpeng An
- the School of Electrical and Information Engineering, Tianjin University, Tianjin, China
| | - Lili Xia
- the School of Electrical and Information Engineering, Tianjin University, Tianjin, China
| | - Huazhen Chen
- the School of Electrical and Information Engineering, Tianjin University, Tianjin, China
| | - Gang Zhao
- Department of Pathology, Key Laboratory of Cancer Prevention and Therapy, National Clinical Research Center for Cancer, Tianjin Medical University Cancer Institute and Hospital, Tianjin, China
| | - Zhongke Gao
- the School of Electrical and Information Engineering, Tianjin University, Tianjin, China.
| |
Collapse
|
6
|
An automatic entropy method to efficiently mask histology whole-slide images. Sci Rep 2023; 13:4321. [PMID: 36922520 PMCID: PMC10017682 DOI: 10.1038/s41598-023-29638-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2022] [Accepted: 02/08/2023] [Indexed: 03/18/2023] Open
Abstract
Tissue segmentation of histology whole-slide images (WSI) remains a critical task in automated digital pathology workflows for both accurate disease diagnosis and deep phenotyping for research purposes. This is especially challenging when the tissue structure of biospecimens is relatively porous and heterogeneous, such as for atherosclerotic plaques. In this study, we developed a unique approach called 'EntropyMasker' based on image entropy to tackle the fore- and background segmentation (masking) task in histology WSI. We evaluated our method on 97 high-resolution WSI of human carotid atherosclerotic plaques in the Athero-Express Biobank Study, constituting hematoxylin and eosin and 8 other staining types. Using multiple benchmarking metrics, we compared our method with four widely used segmentation methods: Otsu's method, Adaptive mean, Adaptive Gaussian and slideMask and observed that our method had the highest sensitivity and Jaccard similarity index. We envision EntropyMasker to fill an important gap in WSI preprocessing, machine learning image analysis pipelines, and enable disease phenotyping beyond the field of atherosclerosis.
Collapse
|
7
|
Liu X, Yuan P, Li R, Zhang D, An J, Ju J, Liu C, Ren F, Hou R, Li Y, Yang J. Predicting breast cancer recurrence and metastasis risk by integrating color and texture features of histopathological images and machine learning technologies. Comput Biol Med 2022; 146:105569. [DOI: 10.1016/j.compbiomed.2022.105569] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Revised: 04/24/2022] [Accepted: 04/25/2022] [Indexed: 12/11/2022]
|
8
|
Fine-Tuned DenseNet-169 for Breast Cancer Metastasis Prediction Using FastAI and 1-Cycle Policy. SENSORS 2022; 22:s22082988. [PMID: 35458972 PMCID: PMC9025766 DOI: 10.3390/s22082988] [Citation(s) in RCA: 54] [Impact Index Per Article: 27.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/16/2022] [Revised: 04/09/2022] [Accepted: 04/12/2022] [Indexed: 12/02/2022]
Abstract
Lymph node metastasis in breast cancer may be accurately predicted using a DenseNet-169 model. However, the current system for identifying metastases in a lymph node is manual and tedious. A pathologist well-versed with the process of detection and characterization of lymph nodes goes through hours investigating histological slides. Furthermore, because of the massive size of most whole-slide images (WSI), it is wise to divide a slide into batches of small image patches and apply methods independently on each patch. The present work introduces a novel method for the automated diagnosis and detection of metastases from whole slide images using the Fast AI framework and the 1-cycle policy. Additionally, it compares this new approach to previous methods. The proposed model has surpassed other state-of-art methods with more than 97.4% accuracy. In addition, a mobile application is developed for prompt and quick response. It collects user information and models to diagnose metastases present in the early stages of cancer. These results indicate that the suggested model may assist general practitioners in accurately analyzing breast cancer situations, hence preventing future complications and mortality. With digital image processing, histopathologic interpretation and diagnostic accuracy have improved considerably.
Collapse
|
9
|
Ye Z, Zhang Y, Liang Y, Lang J, Zhang X, Zang G, Yuan D, Tian G, Xiao M, Yang J. Cervical Cancer Metastasis and Recurrence Risk Prediction Based on Deep Convolutional Neural Network. Curr Bioinform 2022. [DOI: 10.2174/1574893616666210708143556] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
Abstract
Background:
Evaluating the risk of metastasis and recurrence of a cervical cancer patient is
critical for appropriate adjuvant therapy. However, current risk assessment models usually involve the
testing of tens to thousands of genes from patients’ tissue samples, which is expensive and timeconsuming.
Therefore, computer-aided diagnosis and prognosis prediction based on Hematoxylin and Eosin
(H&E) pathological images have received much attention recently.
Objective:
The prognosis of whether patients will have metastasis and recurrence can support accurate
treatment for patients in advance and help reduce patient loss. It is also important for guiding treatment
after surgery to be able to quickly and accurately predict the risk of metastasis and recurrence of a cervical
cancer patient.
Method:
To address this problem, we propose a hybrid method. Transfer learning is used to extract features,
and it is combined with traditional machine learning in order to analyze and determine whether
patients have the risks of metastasis and recurrence. First, the proposed model retrieved relevant patches
using a color-based method from H&E pathological images, which were then subjected to image preprocessing
steps such as image normalization and color homogenization. Based on the labeled patched
images, the Xception model with good classification performance was selected, and deep features of
patched pathological images were automatically extracted with transfer learning. After that, the extracted
features were combined to train a random forest model to predict the label of a new patched image.
Finally, a majority voting method was developed to predict the metastasis and recurrence risk of a patient
based on the predictions of patched images from the whole-slide H&E image.
Results:
In our experiment, the proposed model yielded an area under the receiver operating characteristic
curve of 0.82 for the whole-slide image. The experimental results showed that the high-level features
extracted by the deep convolutional neural network from the whole-slide image can be used to predict
the risk of recurrence and metastasis after surgical resection and help identify patients who might receive
additional benefit from adjuvant therapy.
Conclusion:
This paper explored the feasibility of predicting the risk of metastasis and recurrence from
cervical cancer whole slide H&E images through deep learning and random forest methods.
Collapse
Affiliation(s)
- Zixuan Ye
- School of Computer, Hunan University of Technology, Zhuzhou Hunan 412007, China
| | | | - Yuebin Liang
- Geneis (Beijing) Co. Ltd., Beijing 100102, China
| | - Jidong Lang
- Geneis (Beijing) Co. Ltd., Beijing 100102, China
| | - Xiaoli Zhang
- School of Computer, Hunan University of Technology, Zhuzhou Hunan 412007, China
| | | | - Dawei Yuan
- Geneis (Beijing) Co. Ltd., Beijing 100102, China
| | - Geng Tian
- School of Computer, Hunan University of Technology, Zhuzhou Hunan 412007, China
- Geneis (Beijing) Co. Ltd., Beijing 100102, China
| | - Mansheng Xiao
- School of Computer, Hunan University of Technology, Zhuzhou Hunan 412007, China
| | - Jialiang Yang
- School of Computer, Hunan University of Technology, Zhuzhou Hunan 412007, China
- Geneis (Beijing) Co. Ltd., Beijing 100102, China
- Academician Workstation, Changsha
Medical University, Changsha Hunan 410219, China
| |
Collapse
|
10
|
A comprehensive review of computer-aided whole-slide image analysis: from datasets to feature extraction, segmentation, classification and detection approaches. Artif Intell Rev 2022. [DOI: 10.1007/s10462-021-10121-0] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
|
11
|
Ruusuvuori P, Valkonen M, Kartasalo K, Valkonen M, Visakorpi T, Nykter M, Latonen L. Spatial analysis of histology in 3D: quantification and visualization of organ and tumor level tissue environment. Heliyon 2022; 8:e08762. [PMID: 35128089 PMCID: PMC8800033 DOI: 10.1016/j.heliyon.2022.e08762] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2021] [Revised: 11/24/2021] [Accepted: 01/11/2022] [Indexed: 10/25/2022] Open
Abstract
Histological changes in tissue are of primary importance in pathological research and diagnosis. Automated histological analysis requires ability to computationally separate pathological alterations from normal tissue. Conventional histopathological assessments are performed from individual tissue sections, leading to the loss of three-dimensional context of the tissue. Yet, the tissue context and spatial determinants are critical in several pathologies, such as in understanding growth patterns of cancer in its local environment. Here, we develop computational methods for visualization and quantitative assessment of histopathological alterations in three dimensions. First, we reconstruct the 3D representation of the whole organ from serial sectioned tissue. Then, we proceed to analyze the histological characteristics and regions of interest in 3D. As our example cases, we use whole slide images representing hematoxylin-eosin stained whole mouse prostates in a Pten+/- mouse prostate tumor model. We show that quantitative assessment of tumor sizes, shapes, and separation between spatial locations within the organ enable characterizing and grouping tumors. Further, we show that 3D visualization of tissue with computationally quantified features provides an intuitive way to observe tissue pathology. Our results underline the heterogeneity in composition and cellular organization within individual tumors. As an example, we show how prostate tumors have nuclear density gradients indicating areas of tumor growth directions and reflecting varying pressure from the surrounding tissue. The methods presented here are applicable to any tissue and different types of pathologies. This work provides a proof-of-principle for gaining a comprehensive view from histology by studying it quantitatively in 3D.
Collapse
Affiliation(s)
- Pekka Ruusuvuori
- Institute of Biomedicine, University of Turku, Turku, Finland
- Faculty of Medicine and Health Technology, Tampere University, Finland
| | - Masi Valkonen
- Institute of Biomedicine, University of Turku, Turku, Finland
| | - Kimmo Kartasalo
- Faculty of Medicine and Health Technology, Tampere University, Finland
| | - Mira Valkonen
- Faculty of Medicine and Health Technology, Tampere University, Finland
| | - Tapio Visakorpi
- Faculty of Medicine and Health Technology, Tampere University, Finland
- Tays Cancer Center, Tampere University Hospital, Tampere, Finland
- Fimlab Laboratories Ltd, Tampere University Hospital, Tampere, Finland
| | - Matti Nykter
- Faculty of Medicine and Health Technology, Tampere University, Finland
- Tays Cancer Center, Tampere University Hospital, Tampere, Finland
| | - Leena Latonen
- Institute of Biomedicine, University of Eastern Finland, Kuopio, Finland
| |
Collapse
|
12
|
Brockmoeller S, Echle A, Ghaffari Laleh N, Eiholm S, Malmstrøm ML, Plato Kuhlmann T, Levic K, Grabsch HI, West NP, Saldanha OL, Kouvidi K, Bono A, Heij LR, Brinker TJ, Gögenür I, Quirke P, Kather JN. Deep Learning identifies inflamed fat as a risk factor for lymph node metastasis in early colorectal cancer. J Pathol 2021; 256:269-281. [PMID: 34738636 DOI: 10.1002/path.5831] [Citation(s) in RCA: 47] [Impact Index Per Article: 15.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2021] [Revised: 10/18/2021] [Accepted: 11/01/2021] [Indexed: 11/07/2022]
Abstract
The spread of early-stage (T1 and T2) adenocarcinomas to loco-regional lymph nodes is a key event in disease progression of colorectal cancer (CRC). The cellular mechanisms behind this event are not completely understood and existing predictive biomarkers are imperfect. Here, we used an end-to-end Deep Learning algorithm to identify risk factors for lymph node metastasis (LNM) status in digitized histopathology slides of the primary CRC and its surrounding tissue. In two large population-based cohorts, we show that this system can predict the presence of more than one LNM in pT2 CRC patients with an area under the receiver operating curve (AUROC) of 0.733 (0.67-0.758) and patients with any LNM with an AUROC of 0.711 (0.597-0.797). Similarly, in pT1 CRC patients, the presence of more than one LNM or any LNM was predictable with an AUROC of 0.733 (0.644-0.778) and 0.567 (0.542-0.597), respectively. Based on these findings, we used the Deep Learning system to guide human pathology experts towards highly predictive regions for LNM in the whole slide images. This hybrid human observer and Deep Learning approach identified inflamed adipose tissue as the highest predictive feature for LNM presence. Our study is a first proof of concept that artificial intelligence (AI) systems may be able to discover potentially new biological mechanisms in cancer progression. Our Deep Learning algorithm is publicly available and can be used for biomarker discovery in any disease setting. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Scarlet Brockmoeller
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Amelie Echle
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
| | | | - Susanne Eiholm
- Department of Pathology, Zealand University Hospital, University of Copenhagen, Roskilde, Denmark
| | | | | | - Katarina Levic
- Department of Surgery, Herlev University Hospital, Copenhagen, Denmark
| | - Heike Irmgard Grabsch
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
- Department of Pathology, GROW School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, The Netherlands
| | - Nicholas P West
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | | | - Katerina Kouvidi
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Aurora Bono
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Lara R Heij
- Department of Pathology, GROW School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, The Netherlands
- Institute of Pathology, University Hospital RWTH Aachen, Aachen, Germany
- Department of Surgery and Transplantation, University Hospital RWTH Aachen, Aachen, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, National Center for Tumour Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Ismayil Gögenür
- Department of Surgery, Zealand University Hospital, University of Copenhagen, Køge, Denmark
- Gastrounit - Surgical Division, Center for Surgical Research, Copenhagen University Hospital Hvidovre, Copenhagen, Denmark
| | - Philip Quirke
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Jakob Nikolas Kather
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
- Medical Oncology, National Center of Tumour Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany
| |
Collapse
|
13
|
Pérez-Bueno F, Vega M, Sales MA, Aneiros-Fernández J, Naranjo V, Molina R, Katsaggelos AK. Blind color deconvolution, normalization, and classification of histological images using general super Gaussian priors and Bayesian inference. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 211:106453. [PMID: 34649072 DOI: 10.1016/j.cmpb.2021.106453] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/06/2021] [Accepted: 10/01/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND AND OBJECTIVE Color variations in digital histopathology severely impact the performance of computer-aided diagnosis systems. They are due to differences in the staining process and acquisition system, among other reasons. Blind color deconvolution techniques separate multi-stained images into single stained bands which, once normalized, can be used to eliminate these negative color variations and improve the performance of machine learning tasks. METHODS In this work, we decompose the observed RGB image in its hematoxylin and eosin components. We apply Bayesian modeling and inference based on the use of Super Gaussian sparse priors for each stain together with prior closeness to a given reference color-vector matrix. The hematoxylin and eosin components are then used for image normalization and classification of histological images. The proposed framework is tested on stain separation, image normalization, and cancer classification problems. The results are measured using the peak signal to noise ratio, normalized median intensity and the area under ROC curve on five different databases. RESULTS The obtained results show the superiority of our approach to current state-of-the-art blind color deconvolution techniques. In particular, the fidelity to the tissue improves 1,27 dB in mean PSNR. The normalized median intensity shows a good normalization quality of the proposed approach on the tested datasets. Finally, in cancer classification experiments the area under the ROC curve improves from 0.9491 to 0.9656 and from 0.9279 to 0.9541 on Camelyon-16 and Camelyon-17, respectively, when the original and processed images are used. Furthermore, these figures of merits are better than those obtained by the methods compared with. CONCLUSIONS The proposed framework for blind color deconvolution, normalization and classification of images guarantees fidelity to the tissue structure and can be used both for normalization and classification. In addition, color deconvolution enables the use of the optical density space for classification, which improves the classification performance.
Collapse
Affiliation(s)
- Fernando Pérez-Bueno
- Dpto. Ciencias de la Computación e Inteligencia Artificial, Universidad de Granada, Spain.
| | - Miguel Vega
- Dpto. de Lenguajes y Sistemas Informáticos, Universidad de Granada, Spain.
| | - María A Sales
- Anatomical Pathology Service, University Clinical Hospital of Valencia, Valencia, Spain.
| | - José Aneiros-Fernández
- Intercenter Unit of Pathological Anatomy, San Cecilio University Hospital, Granada, Spain.
| | - Valery Naranjo
- Dpto. de Comunicaciones, Universidad Politécnica de Valencia, Spain.
| | - Rafael Molina
- Dpto. Ciencias de la Computación e Inteligencia Artificial, Universidad de Granada, Spain.
| | - Aggelos K Katsaggelos
- Dept. of Electrical Engineering and Computer Science, Northwestern University, Evanston, IL, USA.
| |
Collapse
|
14
|
Ghalkhani E, Akbari MT, Izadi P, Mahmoodzadeh H, Kamali F. Assessment of DAPK1 and CAVIN3 Gene Promoter Methylation in Breast Invasive Ductal Carcinoma and Metastasis. CELL JOURNAL 2021; 23:397-405. [PMID: 34455714 PMCID: PMC8405083 DOI: 10.22074/cellj.2021.7251] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/03/2019] [Accepted: 01/26/2020] [Indexed: 12/15/2022]
Abstract
Objective Metastasis might be latent or occur several years after primary tumor removal. Currently used methods for detection of distant metastasis have still some limitations. Blood tests may improve sensitivity and specificity of currently used screening procedures. The present study was designed to investigate promoter methylation status of DAPK1 and CAVIN3 genes in plasma circulating free DNA (cfDNA) samples in Iranian invasive ductal carcinoma (IDC) patients. We also investigated association of two gene promoter methylations with breast cancer (BC) and metastatic BC was also assessed. Materials and Methods In this case-control study, MethySYBR assay was performed to determine DAPK1 and CAVIN3 promoter methylation status in breast IDC from 90 patients and 30 controls. Based on clinicopathological information, patient samples subdivided into stage I, II/III and IV groups (each group contained 30 individuals). Results According to the results an increased promoter methylation level of the DAPK1 gene in BC patients was observed. It was found that as disease progressed, the percentage of methylation was changed while it was not significant. Methylation changes in metastatic and non-metastatic BC revealed that methylation levels were significantly increased in metastatic than non-metastatic group. Analysis revealed that promoter methylation of CAVIN3 gene in BC patients was significantly increased. The observed methylation changes from less to more invasive stages were not significant in the CAVIN3 gene. Moreover, promoter methylation was changed in metastatic rather than non-metastatic condition, although it was not significant. Conclusion Promoter hypermethylation of c and CAVIN3 genes in plasma are associated with the risk of BC and they can be potential diagnostic biomarkers along with current methods. Additionally, association of aberrant DAPK1 promoter methylation with metastasis suggests its potential usage as a non-invasive strategy for metastatic BC diagnosis.
Collapse
Affiliation(s)
- Esmat Ghalkhani
- Department of Medical Genetics, Faculty of Medical Sciences, Tarbiat Modares University, Tehran, Iran
| | - Mohammad Taghi Akbari
- Department of Medical Genetics, Faculty of Medical Sciences, Tarbiat Modares University, Tehran, Iran.
| | - Pantea Izadi
- Department of Medical Genetics, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - Habibollah Mahmoodzadeh
- Department of Surgery, Cancer Institute of Iran, Tehran University of Medical Sciences, Tehran, Iran
| | - Fatemeh Kamali
- Iran National Tumor Bank, Cancer Institute of Iran, Tehran, University of Medical Sciences, Tehran, Iran
| |
Collapse
|
15
|
Ruan J, Zhu Z, Wu C, Ye G, Zhou J, Yue J. A fast and effective detection framework for whole-slide histopathology image analysis. PLoS One 2021; 16:e0251521. [PMID: 33979398 PMCID: PMC8115773 DOI: 10.1371/journal.pone.0251521] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2021] [Accepted: 04/27/2021] [Indexed: 11/25/2022] Open
Abstract
Pathologists generally pan, focus, zoom and scan tissue biopsies either under microscopes or on digital images for diagnosis. With the rapid development of whole-slide digital scanners for histopathology, computer-assisted digital pathology image analysis has attracted increasing clinical attention. Thus, the working style of pathologists is also beginning to change. Computer-assisted image analysis systems have been developed to help pathologists perform basic examinations. This paper presents a novel lightweight detection framework for automatic tumor detection in whole-slide histopathology images. We develop the Double Magnification Combination (DMC) classifier, which is a modified DenseNet-40 to make patch-level predictions with only 0.3 million parameters. To improve the detection performance of multiple instances, we propose an improved adaptive sampling method with superpixel segmentation and introduce a new heuristic factor, local sampling density, as the convergence condition of iterations. In postprocessing, we use a CNN model with 4 convolutional layers to regulate the patch-level predictions based on the predictions of adjacent sampling points and use linear interpolation to generate a tumor probability heatmap. The entire framework was trained and validated using the dataset from the Camelyon16 Grand Challenge and Hubei Cancer Hospital. In our experiments, the average AUC was 0.95 in the test set for pixel-level detection.
Collapse
Affiliation(s)
- Jun Ruan
- School of Information Engineering, Wuhan University of Technology, Wuhan, China
| | - Zhikui Zhu
- School of Information Engineering, Wuhan University of Technology, Wuhan, China
| | - Chenchen Wu
- School of Information Engineering, Wuhan University of Technology, Wuhan, China
| | - Guanglu Ye
- School of Information Engineering, Wuhan University of Technology, Wuhan, China
| | - Jingfan Zhou
- School of Information Engineering, Wuhan University of Technology, Wuhan, China
| | - Junqiu Yue
- Department of Pathology, Huazhong University of Science and Technology, Tongji Medical College, Hubei Cancer Hospital, Wuhan, China
- * E-mail:
| |
Collapse
|
16
|
Abstract
Histopathological images (HIs) are the gold standard for evaluating some types of tumors for cancer diagnosis. The analysis of such images is time and resource-consuming and very challenging even for experienced pathologists, resulting in inter-observer and intra-observer disagreements. One of the ways of accelerating such an analysis is to use computer-aided diagnosis (CAD) systems. This paper presents a review on machine learning methods for histopathological image analysis, including shallow and deep learning methods. We also cover the most common tasks in HI analysis, such as segmentation and feature extraction. Besides, we present a list of publicly available and private datasets that have been used in HI research.
Collapse
|
17
|
Integrative Data Augmentation with U-Net Segmentation Masks Improves Detection of Lymph Node Metastases in Breast Cancer Patients. Cancers (Basel) 2020; 12:cancers12102934. [PMID: 33053723 PMCID: PMC7601653 DOI: 10.3390/cancers12102934] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2020] [Revised: 10/01/2020] [Accepted: 10/09/2020] [Indexed: 11/17/2022] Open
Abstract
Simple Summary In recent years many successful models have been developed to perform various tasks in digital histopathology, yet, there is still a reluctance to fully embrace the new technologies in clinical settings. One of the reasons for this is that although these models have achieved high performance at the patch-level, their performance at the image-level can still be underwhelming. Through this study, our main objective was to investigate whether integrating multiple extracted histological features to the input image had potential to further improve the performance of classifier models at the patch-level. Ideally, by achieving 100% accuracy at the patch-level, one can achieve 100% accuracy at the image-level. We hope that our research will entice the community to develop new strategies to further improve performance of existing state-of-the-art models, and facilitate their adoption in the clinics. Abstract Deep learning models have potential to improve performance of automated computer-assisted diagnosis tools in digital histopathology and reduce subjectivity. The main objective of this study was to further improve diagnostic potential of convolutional neural networks (CNNs) in detection of lymph node metastasis in breast cancer patients by integrative augmentation of input images with multiple segmentation channels. For this retrospective study, we used the PatchCamelyon dataset, consisting of 327,680 histopathology images of lymph node sections from breast cancer. Images had labels for the presence or absence of metastatic tissue. In addition, we used four separate histopathology datasets with annotations for nucleus, mitosis, tubule, and epithelium to train four instances of U-net. Then our baseline model was trained with and without additional segmentation channels and their performances were compared. Integrated gradient was used to visualize model attribution. The model trained with concatenation/integration of original input plus four additional segmentation channels, which we refer to as ConcatNet, was superior (AUC 0.924) compared to baseline with or without augmentations (AUC 0.854; 0.884). Baseline model trained with one additional segmentation channel showed intermediate performance (AUC 0.870-0.895). ConcatNet had sensitivity of 82.0% and specificity of 87.8%, which was an improvement in performance over the baseline (sensitivity of 74.6%; specificity of 80.4%). Integrated gradients showed that models trained with additional segmentation channels had improved focus on particular areas of the image containing aberrant cells. Augmenting images with additional segmentation channels improved baseline model performance as well as its ability to focus on discrete areas of the image.
Collapse
|
18
|
Chen Y, Janowczyk A, Madabhushi A. Quantitative Assessment of the Effects of Compression on Deep Learning in Digital Pathology Image Analysis. JCO Clin Cancer Inform 2020; 4:221-233. [PMID: 32155093 PMCID: PMC7113072 DOI: 10.1200/cci.19.00068] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/15/2020] [Indexed: 11/20/2022] Open
Abstract
PURPOSE Deep learning (DL), a class of approaches involving self-learned discriminative features, is increasingly being applied to digital pathology (DP) images for tasks such as disease identification and segmentation of tissue primitives (eg, nuclei, glands, lymphocytes). One application of DP is in telepathology, which involves digitally transmitting DP slides over the Internet for secondary diagnosis by an expert at a remote location. Unfortunately, the places benefiting most from telepathology often have poor Internet quality, resulting in prohibitive transmission times of DP images. Image compression may help, but the degree to which image compression affects performance of DL algorithms has been largely unexplored. METHODS We investigated the effects of image compression on the performance of DL strategies in the context of 3 representative use cases involving segmentation of nuclei (n = 137), segmentation of lymph node metastasis (n = 380), and lymphocyte detection (n = 100). For each use case, test images at various levels of compression (JPEG compression quality score ranging from 1-100 and JPEG2000 compression peak signal-to-noise ratio ranging from 18-100 dB) were evaluated by a DL classifier. Performance metrics including F1 score and area under the receiver operating characteristic curve were computed at the various compression levels. RESULTS Our results suggest that DP images can be compressed by 85% while still maintaining the performance of the DL algorithms at 95% of what is achievable without any compression. Interestingly, the maximum compression level sustainable by DL algorithms is similar to where pathologists also reported difficulties in providing accurate interpretations. CONCLUSION Our findings seem to suggest that in low-resource settings, DP images can be significantly compressed before transmission for DL-based telepathology applications.
Collapse
Affiliation(s)
| | - Andrew Janowczyk
- Case Western Reserve University, Cleveland, OH
- Precision Oncology Center, Lausanne University Hospital, Lausanne, Switzerland
| | - Anant Madabhushi
- Case Western Reserve University, Cleveland, OH
- Louis Stokes Cleveland Veterans Affair Medical Center, Cleveland, OH
| |
Collapse
|
19
|
Liu R, Elhalawani H, Radwan Mohamed AS, Elgohari B, Court L, Zhu H, Fuller CD. Stability analysis of CT radiomic features with respect to segmentation variation in oropharyngeal cancer. Clin Transl Radiat Oncol 2020; 21:11-18. [PMID: 31886423 PMCID: PMC6920497 DOI: 10.1016/j.ctro.2019.11.005] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2019] [Revised: 11/24/2019] [Accepted: 11/25/2019] [Indexed: 01/02/2023] Open
Abstract
INTRODUCTION Accurate segmentation of tumors and quantification of tumor features are important for cancer detection, diagnosis, monitoring, and planning therapeutic intervention. Due to inherent noise components in multi-parametric imaging and inter-observer and intra-observer variations, it is common that various segmentation methods may produce large segmentation errors in tumor volumes and their associated radiomic features. The purpose of this study is to carry out the stability analysis for radiomic features with respect to segmentation variation in oropharyngeal cancer (OPC). METHODS In this study, 436 contrast-enhanced computed tomography (CT) axial images were collected from patients with OPC. In order to derive various segmentations of tumor volumes, two additional segmentations were obtained via resizing the original segmented regions of interest (ROIs) based on their geometric information on the boundary. For three ROI image groups, we calculated 109 radiomic features. Then, a logistic regression model was built to investigate the correlation between the radiomic features extracted from GTVp and the response to chemotherapy and radiation in terms of overall survival (OS). Finally, in order to evaluate the stability of each feature with respect to segmentation results, based on the prediction probabilities, we assessed the inter-rater reliability and reproducibility by calculating the intra-class correlation coefficients (ICC) and concordance correlation coefficients (CCC). RESULTS Most radiomic features in this study varied a lot when the ROIs were not well segmented. For both the representation agreement and predictive agreement, the ICC and CCC were below 0.5 for all the features. We still found some robust features with relatively high ICC and CCC compared to most features. For example, 25percentile (ICC = 0.38, CCC = 0.37 in representation agreement and ICC = CCC = 0.27 in predictive agreement) is a quantile based feature, which is robust to the extremely high or low values; and Hu_1_std (ICC = 0.31, CCC = 0.31 in representation agreement) is a feature calculated based on the first Hu moment, which is invariant to the transformation of ROIs. CONCLUSION In OPC studies, the tumor segmentation variation affects the radiomic features from CT images in terms of both representation and prediction. Some features that are robust to the extreme values or invariant to the transformation of ROIs may be treated as radiomic markers to assist with OPC treatment monitoring and prognostic prediction.
Collapse
Affiliation(s)
- Rongjie Liu
- Department of Statistics, Rice University, Houston, TX 77005, USA
| | - Hesham Elhalawani
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Abdallah Sherif Radwan Mohamed
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- MD Anderson UTHealth Graduate School of Biomedical Sciences, Houston, TX 77030, USA
- Department of Clinical Oncology and Nuclear Medicine, Faculty of Medicine, University of Alexandria, Alexandria, Egypt
| | - Baher Elgohari
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- Department of Clinical Oncology and Nuclear Medicine, Faculty of Medicine, University of Almansoura, Almansoura, Egypt
| | - Laurence Court
- MD Anderson UTHealth Graduate School of Biomedical Sciences, Houston, TX 77030, USA
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Hongtu Zhu
- Department of Biostatistics, The University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Clifton David Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- MD Anderson UTHealth Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| |
Collapse
|
20
|
Chuang WY, Chang SH, Yu WH, Yang CK, Yeh CJ, Ueng SH, Liu YJ, Chen TD, Chen KH, Hsieh YY, Hsia Y, Wang TH, Hsueh C, Kuo CF, Yeh CY. Successful Identification of Nasopharyngeal Carcinoma in Nasopharyngeal Biopsies Using Deep Learning. Cancers (Basel) 2020; 12:cancers12020507. [PMID: 32098314 PMCID: PMC7072217 DOI: 10.3390/cancers12020507] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2020] [Revised: 02/16/2020] [Accepted: 02/19/2020] [Indexed: 12/14/2022] Open
Abstract
Pathologic diagnosis of nasopharyngeal carcinoma (NPC) can be challenging since most cases are nonkeratinizing carcinoma with little differentiation and many admixed lymphocytes. Our aim was to evaluate the possibility to identify NPC in nasopharyngeal biopsies using deep learning. A total of 726 nasopharyngeal biopsies were included. Among them, 100 cases were randomly selected as the testing set, 20 cases as the validation set, and all other 606 cases as the training set. All three datasets had equal numbers of NPC cases and benign cases. Manual annotation was performed. Cropped square image patches of 256 × 256 pixels were used for patch-level training, validation, and testing. The final patch-level algorithm effectively identified NPC patches, with an area under the receiver operator characteristic curve (AUC) of 0.9900. Using gradient-weighted class activation mapping, we demonstrated that the identification of NPC patches was based on morphologic features of tumor cells. At the second stage, whole-slide images were sequentially cropped into patches, inferred with the patch-level algorithm, and reconstructed into images with a smaller size for training, validation, and testing. Finally, the AUC was 0.9848 for slide-level identification of NPC. Our result shows for the first time that deep learning algorithms can identify NPC.
Collapse
Affiliation(s)
- Wen-Yu Chuang
- Department of Pathology, Chang Gung Memorial Hospital and Chang Gung University, No. 5, Fuxing Street, Guishan District, Taoyuan City 333, Taiwan; (W.-Y.C.); (C.-J.Y.); (S.-H.U.); (Y.-J.L.); (T.-D.C.); (K.-H.C.); (Y.-Y.H.); (Y.H.); (C.H.)
- Center for Vascularized Composite Allotransplantation, Chang Gung Memorial Hospital, No. 5, Fuxing Street, Guishan District, Taoyuan City 333, Taiwan
| | - Shang-Hung Chang
- Center for Big Data Analytics and Statistics, Chang Gung Memorial Hospital, No. 5, Fuxing Street, Guishan District, Taoyuan City 333, Taiwan;
| | - Wei-Hsiang Yu
- aetherAI, Co., Ltd., No. 3-2, Yuan-Qu Street, Nangang District, Taipei City 115, Taiwan; (W.-H.Y.); (C.-K.Y.)
| | - Cheng-Kun Yang
- aetherAI, Co., Ltd., No. 3-2, Yuan-Qu Street, Nangang District, Taipei City 115, Taiwan; (W.-H.Y.); (C.-K.Y.)
| | - Chi-Ju Yeh
- Department of Pathology, Chang Gung Memorial Hospital and Chang Gung University, No. 5, Fuxing Street, Guishan District, Taoyuan City 333, Taiwan; (W.-Y.C.); (C.-J.Y.); (S.-H.U.); (Y.-J.L.); (T.-D.C.); (K.-H.C.); (Y.-Y.H.); (Y.H.); (C.H.)
| | - Shir-Hwa Ueng
- Department of Pathology, Chang Gung Memorial Hospital and Chang Gung University, No. 5, Fuxing Street, Guishan District, Taoyuan City 333, Taiwan; (W.-Y.C.); (C.-J.Y.); (S.-H.U.); (Y.-J.L.); (T.-D.C.); (K.-H.C.); (Y.-Y.H.); (Y.H.); (C.H.)
- Chang Gung Molecular Medicine Research Center, Chang Gung University, No. 259, Wenhua First Road, Guishan District, Taoyuan City 333, Taiwan
| | - Yu-Jen Liu
- Department of Pathology, Chang Gung Memorial Hospital and Chang Gung University, No. 5, Fuxing Street, Guishan District, Taoyuan City 333, Taiwan; (W.-Y.C.); (C.-J.Y.); (S.-H.U.); (Y.-J.L.); (T.-D.C.); (K.-H.C.); (Y.-Y.H.); (Y.H.); (C.H.)
| | - Tai-Di Chen
- Department of Pathology, Chang Gung Memorial Hospital and Chang Gung University, No. 5, Fuxing Street, Guishan District, Taoyuan City 333, Taiwan; (W.-Y.C.); (C.-J.Y.); (S.-H.U.); (Y.-J.L.); (T.-D.C.); (K.-H.C.); (Y.-Y.H.); (Y.H.); (C.H.)
| | - Kuang-Hua Chen
- Department of Pathology, Chang Gung Memorial Hospital and Chang Gung University, No. 5, Fuxing Street, Guishan District, Taoyuan City 333, Taiwan; (W.-Y.C.); (C.-J.Y.); (S.-H.U.); (Y.-J.L.); (T.-D.C.); (K.-H.C.); (Y.-Y.H.); (Y.H.); (C.H.)
| | - Yi-Yin Hsieh
- Department of Pathology, Chang Gung Memorial Hospital and Chang Gung University, No. 5, Fuxing Street, Guishan District, Taoyuan City 333, Taiwan; (W.-Y.C.); (C.-J.Y.); (S.-H.U.); (Y.-J.L.); (T.-D.C.); (K.-H.C.); (Y.-Y.H.); (Y.H.); (C.H.)
| | - Yi Hsia
- Department of Pathology, Chang Gung Memorial Hospital and Chang Gung University, No. 5, Fuxing Street, Guishan District, Taoyuan City 333, Taiwan; (W.-Y.C.); (C.-J.Y.); (S.-H.U.); (Y.-J.L.); (T.-D.C.); (K.-H.C.); (Y.-Y.H.); (Y.H.); (C.H.)
- Department of Pathology, MacKay Memorial Hospital, No. 92, Section 2, Zhongshan North Road, Zhongshan District, Taipei City 104, Taiwan
| | - Tong-Hong Wang
- Tissue Bank, Chang Gung Memorial Hospital, No. 5, Fuxing Street, Guishan District, Taoyuan City 333, Taiwan;
| | - Chuen Hsueh
- Department of Pathology, Chang Gung Memorial Hospital and Chang Gung University, No. 5, Fuxing Street, Guishan District, Taoyuan City 333, Taiwan; (W.-Y.C.); (C.-J.Y.); (S.-H.U.); (Y.-J.L.); (T.-D.C.); (K.-H.C.); (Y.-Y.H.); (Y.H.); (C.H.)
- Chang Gung Molecular Medicine Research Center, Chang Gung University, No. 259, Wenhua First Road, Guishan District, Taoyuan City 333, Taiwan
| | - Chang-Fu Kuo
- Center for Artificial Intelligence in Medicine, Chang Gung Memorial Hospital, No. 5, Fuxing Street, Guishan District, Taoyuan City 333, Taiwan;
| | - Chao-Yuan Yeh
- aetherAI, Co., Ltd., No. 3-2, Yuan-Qu Street, Nangang District, Taipei City 115, Taiwan; (W.-H.Y.); (C.-K.Y.)
- Correspondence: ; Tel.: +886-2-27856892
| |
Collapse
|
21
|
Automated detection and quantification of breast cancer brain metastases in an animal model using democratized machine learning tools. Sci Rep 2019; 9:17333. [PMID: 31758004 PMCID: PMC6874643 DOI: 10.1038/s41598-019-53911-x] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2019] [Accepted: 11/06/2019] [Indexed: 02/07/2023] Open
Abstract
Advances in digital whole-slide imaging and machine learning (ML) provide new opportunities for automated examination and quantification of histopathological slides to support pathologists and biologists. However, implementation of ML tools often requires advanced skills in computer science that may not be immediately available in the traditional wet-lab environment. Here, we propose a simple and accessible workflow to automate detection and quantification of brain epithelial metastases on digitized histological slides. We leverage 100 Hematoxylin & Eosin (H&E)-stained whole slide images (WSIs) from 25 Balb/c mice with various level of brain metastatic tumor burden. A supervised training of the Trainable Weka Segmentation (TWS) from Fiji was achieved from annotated WSIs. Upon comparison with manually drawn regions, it is apparent that the algorithm learned to identify and segment cancer cell-specific nuclei and normal brain tissue. Our approach resulted in a robust and highly concordant correlation between automated metastases quantification of brain metastases and manual human assessment (R2 = 0.8783; P < 0.0001). This simple approach is amenable to other similar analyses, including that of human tissues. Widespread adoption of these tools aims to democratize ML and improve precision in traditionally qualitative tasks in histopathology-based research.
Collapse
|
22
|
Takamatsu M, Yamamoto N, Kawachi H, Chino A, Saito S, Ueno M, Ishikawa Y, Takazawa Y, Takeuchi K. Prediction of early colorectal cancer metastasis by machine learning using digital slide images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2019; 178:155-161. [PMID: 31416544 DOI: 10.1016/j.cmpb.2019.06.022] [Citation(s) in RCA: 35] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/10/2019] [Revised: 06/08/2019] [Accepted: 06/21/2019] [Indexed: 06/10/2023]
Abstract
BACKGROUND AND OBJECTIVES Prediction of lymph node metastasis (LNM) for early colorectal cancer (CRC) is critical for determining treatment strategies after endoscopic resection. Some histologic parameters for predicting LNM have been established, but evaluator error and inter-observer disagreement are unsolved issues. Here we describe an LNM prediction algorithm for submucosal invasive (T1) CRC based on machine learning. METHODS We conducted a retrospective single-institution study of 397 T1 CRCs. Several morphologic parameters were extracted from whole slide images of cytokeratin immunohistochemistry using Image J. A random forest algorithm for a training dataset (n = 277) was executed and used to predict LNM for the test dataset (n = 120). The results were compared with conventional histologic evaluation of hematoxylin-eosin staining. RESULTS Machine learning showed better LNM predictive ability than the conventional method on some datasets. Cross validation revealed no significant difference between the methods. Machine learning resulted in fewer false-negative cases than the conventional method. CONCLUSIONS Machine learning on whole slide images is a potential alternative for determining treatment strategies for T1 CRC.
Collapse
Affiliation(s)
- Manabu Takamatsu
- Division of Pathology, The Cancer Institute; Department of Pathology, The Cancer Institute Hospital, Japanese Foundation for Cancer Research, Tokyo, Japan.
| | - Noriko Yamamoto
- Division of Pathology, The Cancer Institute; Department of Pathology, The Cancer Institute Hospital, Japanese Foundation for Cancer Research, Tokyo, Japan
| | - Hiroshi Kawachi
- Division of Pathology, The Cancer Institute; Department of Pathology, The Cancer Institute Hospital, Japanese Foundation for Cancer Research, Tokyo, Japan
| | - Akiko Chino
- Department of Endoscopy, The Cancer Institute Hospital, Japanese Foundation for Cancer Research, Tokyo, Japan
| | - Shoichi Saito
- Department of Endoscopy, The Cancer Institute Hospital, Japanese Foundation for Cancer Research, Tokyo, Japan
| | - Masashi Ueno
- Department of Colorectal Surgery, The Cancer Institute Hospital, Japanese Foundation for Cancer Research, Tokyo, Japan
| | - Yuichi Ishikawa
- Division of Pathology, The Cancer Institute; Department of Pathology, The Cancer Institute Hospital, Japanese Foundation for Cancer Research, Tokyo, Japan
| | - Yutaka Takazawa
- Division of Pathology, The Cancer Institute; Department of Pathology, The Cancer Institute Hospital, Japanese Foundation for Cancer Research, Tokyo, Japan
| | - Kengo Takeuchi
- Division of Pathology, The Cancer Institute; Department of Pathology, The Cancer Institute Hospital, Japanese Foundation for Cancer Research, Tokyo, Japan
| |
Collapse
|
23
|
Esteban ÁE, López-Pérez M, Colomer A, Sales MA, Molina R, Naranjo V. A new optical density granulometry-based descriptor for the classification of prostate histological images using shallow and deep Gaussian processes. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2019; 178:303-317. [PMID: 31416557 DOI: 10.1016/j.cmpb.2019.07.003] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/29/2019] [Revised: 06/28/2019] [Accepted: 07/03/2019] [Indexed: 06/10/2023]
Abstract
BACKGROUND AND OBJECTIVE Prostate cancer is one of the most common male tumors. The increasing use of whole slide digital scanners has led to an enormous interest in the application of machine learning techniques to histopathological image classification. Here we introduce a novel family of morphological descriptors which, extracted in the appropriate image space and combined with shallow and deep Gaussian process based classifiers, improves early prostate cancer diagnosis. METHOD We decompose the acquired RGB image in its RGB and optical density hematoxylin and eosin components. Then, we define two novel granulometry-based descriptors which work in both, RGB and optical density, spaces but perform better when used on the latter. In this space they clearly encapsulate knowledge used by pathologists to identify cancer lesions. The obtained features become the inputs to shallow and deep Gaussian process classifiers which achieve an accurate prediction of cancer. RESULTS We have used a real and unique dataset. The dataset is composed of 60 Whole Slide Images. For a five fold cross validation, shallow and deep Gaussian Processes obtain area under ROC curve values higher than 0.98. They outperform current state of the art patch based shallow classifiers and are very competitive to the best performing deep learning method. Models were also compared on 17 Whole Slide test Images using the FROC curve. With the cost of one false positive, the best performing method, the one layer Gaussian process, identifies 83.87% (sensitivity) of all annotated cancer in the Whole Slide Image. This result corroborates the quality of the extracted features, no more than a layer is needed to achieve excellent generalization results. CONCLUSION Two new descriptors to extract morphological features from histological images have been proposed. They collect very relevant information for cancer detection. From these descriptors, shallow and deep Gaussian Processes are capable of extracting the complex structure of prostate histological images. The new space/descriptor/classifier paradigm outperforms state-of-art shallow classifiers. Furthermore, despite being much simpler, it is competitive to state-of-art CNN architectures both on the proposed SICAPv1 database and on an external database.
Collapse
Affiliation(s)
- Ángel E Esteban
- Institute of Research and Innovation in Bioengineering, I3B, Polytechnic University of Valencia, Valencia, Spain.
| | - Miguel López-Pérez
- Department of Computer Science and Artificial Intelligence, University of Granada, Granada, Spain.
| | - Adrián Colomer
- Institute of Research and Innovation in Bioengineering, I3B, Polytechnic University of Valencia, Valencia, Spain.
| | - María A Sales
- Anatomical Pathology Service, University Clinical Hospital of Valencia, Valencia, Spain.
| | - Rafael Molina
- Department of Computer Science and Artificial Intelligence, University of Granada, Granada, Spain.
| | - Valery Naranjo
- Institute of Research and Innovation in Bioengineering, I3B, Polytechnic University of Valencia, Valencia, Spain.
| |
Collapse
|
24
|
Campanella G, Hanna MG, Geneslaw L, Miraflor A, Werneck Krauss Silva V, Busam KJ, Brogi E, Reuter VE, Klimstra DS, Fuchs TJ. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat Med 2019; 25:1301-1309. [PMID: 31308507 DOI: 10.1038/s41591-019-0508-1] [Citation(s) in RCA: 1001] [Impact Index Per Article: 200.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2018] [Accepted: 06/03/2019] [Indexed: 02/07/2023]
Abstract
The development of decision support systems for pathology and their deployment in clinical practice have been hindered by the need for large manually annotated datasets. To overcome this problem, we present a multiple instance learning-based deep learning system that uses only the reported diagnoses as labels for training, thereby avoiding expensive and time-consuming pixel-wise manual annotations. We evaluated this framework at scale on a dataset of 44,732 whole slide images from 15,187 patients without any form of data curation. Tests on prostate cancer, basal cell carcinoma and breast cancer metastases to axillary lymph nodes resulted in areas under the curve above 0.98 for all cancer types. Its clinical application would allow pathologists to exclude 65-75% of slides while retaining 100% sensitivity. Our results show that this system has the ability to train accurate classification models at unprecedented scale, laying the foundation for the deployment of computational decision support systems in clinical practice.
Collapse
Affiliation(s)
- Gabriele Campanella
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA.,Weill Cornell Graduate School of Medical Sciences, New York, NY, USA
| | - Matthew G Hanna
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Luke Geneslaw
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Allen Miraflor
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | | | - Klaus J Busam
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Edi Brogi
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Victor E Reuter
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - David S Klimstra
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Thomas J Fuchs
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA. .,Weill Cornell Graduate School of Medical Sciences, New York, NY, USA.
| |
Collapse
|
25
|
Abstract
Histopathology plays a central role in diagnosis of many diseases including solid cancers. Efforts are underway to transform this subjective art to an objective and quantitative science. Coherent Raman imaging (CRI), a label-free imaging modality with sub-cellular spatial resolution and molecule-specific contrast possesses characteristics which could support the qualitative-to-quantitative transition of histopathology. In this work we briefly survey major themes related to modernization of histopathology, review applications of CRI to histopathology and, finally, discuss potential roles for CRI in the transformation of histopathology that is already underway.
Collapse
|
26
|
Simon O, Yacoub R, Jain S, Tomaszewski JE, Sarder P. Multi-radial LBP Features as a Tool for Rapid Glomerular Detection and Assessment in Whole Slide Histopathology Images. Sci Rep 2018; 8:2032. [PMID: 29391542 PMCID: PMC5795004 DOI: 10.1038/s41598-018-20453-7] [Citation(s) in RCA: 47] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2017] [Accepted: 01/18/2018] [Indexed: 12/03/2022] Open
Abstract
We demonstrate a simple and effective automated method for the localization of glomeruli in large (~1 gigapixel) histopathological whole-slide images (WSIs) of thin renal tissue sections and biopsies, using an adaptation of the well-known local binary patterns (LBP) image feature vector to train a support vector machine (SVM) model. Our method offers high precision (>90%) and reasonable recall (>70%) for glomeruli from WSIs, is readily adaptable to glomeruli from multiple species, including mouse, rat, and human, and is robust to diverse slide staining methods. Using 5 Intel(R) Core(TM) i7-4790 CPUs with 40 GB RAM, our method typically requires ~15 sec for training and ~2 min to extract glomeruli reproducibly from a WSI. Deploying a deep convolutional neural network trained for glomerular recognition in tandem with the SVM suffices to reduce false positives to below 3%. We also apply our LBP-based descriptor to successfully detect pathologic changes in a mouse model of diabetic nephropathy. We envision potential clinical and laboratory applications for this approach in the study and diagnosis of glomerular disease, and as a means of greatly accelerating the construction of feature sets to fuel deep learning studies into tissue structure and pathology.
Collapse
Affiliation(s)
- Olivier Simon
- Department of Pathology and Anatomical Sciences, University at Buffalo, Buffalo, USA
| | - Rabi Yacoub
- Division of Nephrology, Department of Medicine, University at Buffalo, Buffalo, USA
| | - Sanjay Jain
- Division of Nephrology, Department of Medicine, Washington University School of Medicine, St. Louis, USA
| | - John E Tomaszewski
- Department of Pathology and Anatomical Sciences, University at Buffalo, Buffalo, USA
| | - Pinaki Sarder
- Department of Pathology and Anatomical Sciences, University at Buffalo, Buffalo, USA.
| |
Collapse
|