1
|
Baidar Bakht A, Javed S, Gilani SQ, Karki H, Muneeb M, Werghi N. DeepBLS: Deep Feature-Based Broad Learning System for Tissue Phenotyping in Colorectal Cancer WSIs. J Digit Imaging 2023; 36:1653-1662. [PMID: 37059892 PMCID: PMC10406762 DOI: 10.1007/s10278-023-00797-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Revised: 02/09/2023] [Accepted: 02/10/2023] [Indexed: 04/16/2023] Open
Abstract
Tissue phenotyping is a fundamental step in computational pathology for the analysis of tumor micro-environment in whole slide images (WSIs). Automatic tissue phenotyping in whole slide images (WSIs) of colorectal cancer (CRC) assists pathologists in better cancer grading and prognostication. In this paper, we propose a novel algorithm for the identification of distinct tissue components in colon cancer histology images by blending a comprehensive learning system with deep features extraction in the current work. Firstly, we extracted the features from the pre-trained VGG19 network which are then transformed into mapped features space for nodes enhancement generation. Utilizing both mapped features and enhancement nodes, the proposed algorithm classifies seven distinct tissue components including stroma, tumor, complex stroma, necrotic, normal benign, lymphocytes, and smooth muscle. To validate our proposed model, the experiments are performed on two publicly available colorectal cancer histology datasets. We showcase that our approach achieves a remarkable performance boost surpassing existing state-of-the-art methods by (1.3% AvTP, 2% F1) and (7% AvTP, 6% F1) on CRCD-1, and CRCD-2, respectively.
Collapse
Affiliation(s)
- Ahsan Baidar Bakht
- Electrical and Computer Engineering Department, Khalifa University, 12778 Abu Dhabi, United Arab Emirates
| | - Sajid Javed
- Electrical and Computer Engineering Department, Khalifa University, 12778 Abu Dhabi, United Arab Emirates
| | - Syed Qasim Gilani
- Department of Electrical Engineering and Computer Science, Florida Atlantic University, Boca Raton, 33431 USA
| | - Hamad Karki
- Mechanical Engineering Department, Khalifa University, 12778 Abu Dhabi, United Arab Emirates
| | - Muhammad Muneeb
- Electrical and Computer Engineering Department, Khalifa University, 12778 Abu Dhabi, United Arab Emirates
| | - Naoufel Werghi
- Electrical and Computer Engineering Department, Khalifa University, 12778 Abu Dhabi, United Arab Emirates
| |
Collapse
|
2
|
Firmbach D, Benz M, Kuritcyn P, Bruns V, Lang-Schwarz C, Stuebs FA, Merkel S, Leikauf LS, Braunschweig AL, Oldenburger A, Gloßner L, Abele N, Eck C, Matek C, Hartmann A, Geppert CI. Tumor-Stroma Ratio in Colorectal Cancer-Comparison between Human Estimation and Automated Assessment. Cancers (Basel) 2023; 15:2675. [PMID: 37345012 DOI: 10.3390/cancers15102675] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Revised: 04/27/2023] [Accepted: 05/02/2023] [Indexed: 06/23/2023] Open
Abstract
The tumor-stroma ratio (TSR) has been repeatedly shown to be a prognostic factor for survival prediction of different cancer types. However, an objective and reliable determination of the tumor-stroma ratio remains challenging. We present an easily adaptable deep learning model for accurately segmenting tumor regions in hematoxylin and eosin (H&E)-stained whole slide images (WSIs) of colon cancer patients into five distinct classes (tumor, stroma, necrosis, mucus, and background). The tumor-stroma ratio can be determined in the presence of necrotic or mucinous areas. We employ a few-shot model, eventually aiming for the easy adaptability of our approach to related segmentation tasks or other primaries, and compare the results to a well-established state-of-the art approach (U-Net). Both models achieve similar results with an overall accuracy of 86.5% and 86.7%, respectively, indicating that the adaptability does not lead to a significant decrease in accuracy. Moreover, we comprehensively compare with TSR estimates of human observers and examine in detail discrepancies and inter-rater reliability. Adding a second survey for segmentation quality on top of a first survey for TSR estimation, we found that TSR estimations of human observers are not as reliable a ground truth as previously thought.
Collapse
Affiliation(s)
- Daniel Firmbach
- Digital Health Systems Department, Fraunhofer-Institute for Integrated Circuits IIS, Am Wolfsmantel 33, 91058 Erlangen, Germany
- Institute of Pathology, University Hospital Erlangen, FAU Erlangen-Nuremberg, Krankenhausstr. 8-10, 91054 Erlangen, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC), University Hospital Erlangen, FAU Erlangen-Nuremberg, Östliche Stadtmauerstr. 30, 91054 Erlangen, Germany
| | - Michaela Benz
- Digital Health Systems Department, Fraunhofer-Institute for Integrated Circuits IIS, Am Wolfsmantel 33, 91058 Erlangen, Germany
| | - Petr Kuritcyn
- Digital Health Systems Department, Fraunhofer-Institute for Integrated Circuits IIS, Am Wolfsmantel 33, 91058 Erlangen, Germany
| | - Volker Bruns
- Digital Health Systems Department, Fraunhofer-Institute for Integrated Circuits IIS, Am Wolfsmantel 33, 91058 Erlangen, Germany
| | - Corinna Lang-Schwarz
- Institute of Pathology, Hospital Bayreuth, Preuschwitzer Str. 101, 95445 Bayreuth, Germany
| | - Frederik A Stuebs
- Comprehensive Cancer Center Erlangen-EMN (CCC), University Hospital Erlangen, FAU Erlangen-Nuremberg, Östliche Stadtmauerstr. 30, 91054 Erlangen, Germany
- Department of Obstetrics and Gynaecology, University Hospital Erlangen, FAU Erlangen-Nuremberg, Universitätsstraße 21-23, 91054 Erlangen, Germany
| | - Susanne Merkel
- Comprehensive Cancer Center Erlangen-EMN (CCC), University Hospital Erlangen, FAU Erlangen-Nuremberg, Östliche Stadtmauerstr. 30, 91054 Erlangen, Germany
- Department of Surgery, University Hospital Erlangen, FAU Erlangen-Nuremberg, Krankenhausstr. 12, 91054 Erlangen, Germany
| | - Leah-Sophie Leikauf
- Institute of Pathology, University Hospital Erlangen, FAU Erlangen-Nuremberg, Krankenhausstr. 8-10, 91054 Erlangen, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC), University Hospital Erlangen, FAU Erlangen-Nuremberg, Östliche Stadtmauerstr. 30, 91054 Erlangen, Germany
| | - Anna-Lea Braunschweig
- Institute of Pathology, University Hospital Erlangen, FAU Erlangen-Nuremberg, Krankenhausstr. 8-10, 91054 Erlangen, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC), University Hospital Erlangen, FAU Erlangen-Nuremberg, Östliche Stadtmauerstr. 30, 91054 Erlangen, Germany
| | - Angelika Oldenburger
- Institute of Pathology, University Hospital Erlangen, FAU Erlangen-Nuremberg, Krankenhausstr. 8-10, 91054 Erlangen, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC), University Hospital Erlangen, FAU Erlangen-Nuremberg, Östliche Stadtmauerstr. 30, 91054 Erlangen, Germany
| | - Laura Gloßner
- Institute of Pathology, University Hospital Erlangen, FAU Erlangen-Nuremberg, Krankenhausstr. 8-10, 91054 Erlangen, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC), University Hospital Erlangen, FAU Erlangen-Nuremberg, Östliche Stadtmauerstr. 30, 91054 Erlangen, Germany
| | - Niklas Abele
- Institute of Pathology, University Hospital Erlangen, FAU Erlangen-Nuremberg, Krankenhausstr. 8-10, 91054 Erlangen, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC), University Hospital Erlangen, FAU Erlangen-Nuremberg, Östliche Stadtmauerstr. 30, 91054 Erlangen, Germany
| | - Christine Eck
- Institute of Pathology, University Hospital Erlangen, FAU Erlangen-Nuremberg, Krankenhausstr. 8-10, 91054 Erlangen, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC), University Hospital Erlangen, FAU Erlangen-Nuremberg, Östliche Stadtmauerstr. 30, 91054 Erlangen, Germany
| | - Christian Matek
- Institute of Pathology, University Hospital Erlangen, FAU Erlangen-Nuremberg, Krankenhausstr. 8-10, 91054 Erlangen, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC), University Hospital Erlangen, FAU Erlangen-Nuremberg, Östliche Stadtmauerstr. 30, 91054 Erlangen, Germany
| | - Arndt Hartmann
- Institute of Pathology, University Hospital Erlangen, FAU Erlangen-Nuremberg, Krankenhausstr. 8-10, 91054 Erlangen, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC), University Hospital Erlangen, FAU Erlangen-Nuremberg, Östliche Stadtmauerstr. 30, 91054 Erlangen, Germany
| | - Carol I Geppert
- Institute of Pathology, University Hospital Erlangen, FAU Erlangen-Nuremberg, Krankenhausstr. 8-10, 91054 Erlangen, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC), University Hospital Erlangen, FAU Erlangen-Nuremberg, Östliche Stadtmauerstr. 30, 91054 Erlangen, Germany
| |
Collapse
|
3
|
Artificial Intelligence-Based Tissue Phenotyping in Colorectal Cancer Histopathology Using Visual and Semantic Features Aggregation. MATHEMATICS 2022. [DOI: 10.3390/math10111909] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
Tissue phenotyping of the tumor microenvironment has a decisive role in digital profiling of intra-tumor heterogeneity, epigenetics, and progression of cancer. Most of the existing methods for tissue phenotyping often rely on time-consuming and error-prone manual procedures. Recently, with the advent of advanced technologies, these procedures have been automated using artificial intelligence techniques. In this paper, a novel deep histology heterogeneous feature aggregation network (HHFA-Net) is proposed based on visual and semantic information fusion for the detection of tissue phenotypes in colorectal cancer (CRC). We adopted and tested various data augmentation techniques to avoid computationally expensive stain normalization procedures and handle limited and imbalanced data problems. Three publicly available datasets are used in the experiments: CRC tissue phenotyping (CRC-TP), CRC histology (CRCH), and colon cancer histology (CCH). The proposed HHFA-Net achieves higher accuracies than the state-of-the-art methods for tissue phenotyping in CRC histopathology images.
Collapse
|
4
|
Fine-Tuned DenseNet-169 for Breast Cancer Metastasis Prediction Using FastAI and 1-Cycle Policy. SENSORS 2022; 22:s22082988. [PMID: 35458972 PMCID: PMC9025766 DOI: 10.3390/s22082988] [Citation(s) in RCA: 54] [Impact Index Per Article: 27.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/16/2022] [Revised: 04/09/2022] [Accepted: 04/12/2022] [Indexed: 12/02/2022]
Abstract
Lymph node metastasis in breast cancer may be accurately predicted using a DenseNet-169 model. However, the current system for identifying metastases in a lymph node is manual and tedious. A pathologist well-versed with the process of detection and characterization of lymph nodes goes through hours investigating histological slides. Furthermore, because of the massive size of most whole-slide images (WSI), it is wise to divide a slide into batches of small image patches and apply methods independently on each patch. The present work introduces a novel method for the automated diagnosis and detection of metastases from whole slide images using the Fast AI framework and the 1-cycle policy. Additionally, it compares this new approach to previous methods. The proposed model has surpassed other state-of-art methods with more than 97.4% accuracy. In addition, a mobile application is developed for prompt and quick response. It collects user information and models to diagnose metastases present in the early stages of cancer. These results indicate that the suggested model may assist general practitioners in accurately analyzing breast cancer situations, hence preventing future complications and mortality. With digital image processing, histopathologic interpretation and diagnostic accuracy have improved considerably.
Collapse
|
5
|
Domain generalization on medical imaging classification using episodic training with task augmentation. Comput Biol Med 2021; 141:105144. [PMID: 34971982 DOI: 10.1016/j.compbiomed.2021.105144] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2021] [Revised: 12/12/2021] [Accepted: 12/13/2021] [Indexed: 12/22/2022]
Abstract
Medical imaging datasets usually exhibit domain shift due to the variations of scanner vendors, imaging protocols, etc. This raises the concern about the generalization capacity of machine learning models. Domain generalization (DG), which aims to learn a model from multiple source domains such that it can be directly generalized to unseen test domains, seems particularly promising to medical imaging community. To address DG, recent model-agnostic meta-learning (MAML) has been introduced, which transfers the knowledge from previous training tasks to facilitate the learning of novel testing tasks. However, in clinical practice, there are usually only a few annotated source domains available, which decreases the capacity of training task generation and thus increases the risk of overfitting to training tasks in the paradigm. In this paper, we propose a novel DG scheme of episodic training with task augmentation on medical imaging classification. Based on meta-learning, we develop the paradigm of episodic training to construct the knowledge transfer from episodic training-task simulation to the real testing task of DG. Motivated by the limited number of source domains in real-world medical deployment, we consider the unique task-level overfitting and we propose task augmentation to enhance the variety during training task generation to alleviate it. With the established learning framework, we further exploit a novel meta-objective to regularize the deep embedding of training domains. To validate the effectiveness of the proposed method, we perform experiments on histopathological images and abdominal CT images.
Collapse
|
6
|
Zeid MAE, El-Bahnasy K, Abo-Youssef SE. Multiclass Colorectal Cancer Histology Images Classification Using Vision Transformers. 2021 TENTH INTERNATIONAL CONFERENCE ON INTELLIGENT COMPUTING AND INFORMATION SYSTEMS (ICICIS) 2021. [DOI: 10.1109/icicis52592.2021.9694125] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/01/2023]
Affiliation(s)
- Magdy Abd-Elghany Zeid
- Obour High Institute for Management and Informatics,Computer Science Department,Cairo,Egypt
| | - Khaled El-Bahnasy
- Obour High Institute for Management and Informatics,Computer Science Department,Cairo,Egypt
| | - S. E. Abo-Youssef
- Al-Azhar University,Faculty of Science,Mathematics and Computer Science Department,Cairo,Egypt
| |
Collapse
|
7
|
Nastały P, Smentoch J, Popęda M, Martini E, Maiuri P, Żaczek AJ, Sowa M, Matuszewski M, Szade J, Kalinowski L, Niemira M, Brandt B, Eltze E, Semjonow A, Bednarz-Knoll N. Low Tumor-to-Stroma Ratio Reflects Protective Role of Stroma against Prostate Cancer Progression. J Pers Med 2021; 11:1088. [PMID: 34834440 PMCID: PMC8622253 DOI: 10.3390/jpm11111088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2021] [Revised: 10/21/2021] [Accepted: 10/23/2021] [Indexed: 12/09/2022] Open
Abstract
Tumor-to-stroma ratio (TSR) is a prognostic factor that expresses the relative amounts of tumor and intratumoral stroma. In this study, its clinical and molecular relevance was evaluated in prostate cancer (PCa). The feasibility of automated quantification was tested in digital scans of tissue microarrays containing 128 primary tumors from 72 PCa patients stained immunohistochemically for epithelial cell adhesion molecule (EpCAM), followed by validation in a cohort of 310 primary tumors from 209 PCa patients. In order to investigate the gene expression differences between tumors with low and high TSR, we applied multigene expression analysis (nCounter® PanCancer Progression Panel, NanoString) of 42 tissue samples. TSR scores were categorized into low (<1 TSR) and high (≥1 TSR). In the pilot cohort, 31 patients (43.1%) were categorized as low and 41 (56.9%) as high TSR score, whereas 48 (23.0%) patients from the validation cohort were classified as low TSR and 161 (77.0%) as high. In both cohorts, high TSR appeared to indicate the shorter time to biochemical recurrence in PCa patients (Log-rank test, p = 0.04 and p = 0.01 for the pilot and validation cohort, respectively). Additionally, in the multivariate analysis of the validation cohort, TSR predicted BR independent of other factors, i.e., pT, pN, and age (p = 0.04, HR 2.75, 95%CI 1.07-7.03). Our data revealed that tumors categorized into low and high TSR score show differential expression of various genes; the genes upregulated in tumors with low TSR score were mostly associated with extracellular matrix and cell adhesion regulation. Taken together, this study shows that high stroma content can play a protective role in PCa. Automatic EpCAM-based quantification of TSR might improve prognostication in personalized medicine for PCa.
Collapse
Affiliation(s)
- Paulina Nastały
- Laboratory of Translational Oncology, Medical University of Gdańsk, 80-210 Gdańsk, Poland; (P.N.); (J.S.); (M.P.); (A.J.Ż.)
- FIRC (Italian Foundation for Cancer Research), Institute of Molecular Oncology (IFOM), 20139 Milan, Italy; (E.M.); (P.M.)
| | - Julia Smentoch
- Laboratory of Translational Oncology, Medical University of Gdańsk, 80-210 Gdańsk, Poland; (P.N.); (J.S.); (M.P.); (A.J.Ż.)
| | - Marta Popęda
- Laboratory of Translational Oncology, Medical University of Gdańsk, 80-210 Gdańsk, Poland; (P.N.); (J.S.); (M.P.); (A.J.Ż.)
| | - Emanuele Martini
- FIRC (Italian Foundation for Cancer Research), Institute of Molecular Oncology (IFOM), 20139 Milan, Italy; (E.M.); (P.M.)
| | - Paolo Maiuri
- FIRC (Italian Foundation for Cancer Research), Institute of Molecular Oncology (IFOM), 20139 Milan, Italy; (E.M.); (P.M.)
| | - Anna J. Żaczek
- Laboratory of Translational Oncology, Medical University of Gdańsk, 80-210 Gdańsk, Poland; (P.N.); (J.S.); (M.P.); (A.J.Ż.)
| | - Marek Sowa
- Department of Urology, Medical University of Gdańsk, 80-214 Gdańsk, Poland; (M.S.); (M.M.)
| | - Marcin Matuszewski
- Department of Urology, Medical University of Gdańsk, 80-214 Gdańsk, Poland; (M.S.); (M.M.)
| | - Jolanta Szade
- Department of Pathomorphology, Medical University of Gdańsk, 80-214 Gdańsk, Poland;
| | - Leszek Kalinowski
- Department of Medical Laboratory Diagnostics-Biobank, Medical University of Gdańsk, 80-210 Gdańsk, Poland;
- Biobanking and Biomolecular Resources Research Infrastructure (BBMRI.pl), 80-214 Gdańsk, Poland
| | - Magdalena Niemira
- Clinical Research Centre, Medical University of Bialystok, 15-276 Bialystok, Poland;
| | - Burkhard Brandt
- Institute of Clinical Chemistry, University Medical Centre Schleswig-Holstein, 24105 Kiel, Germany;
| | - Elke Eltze
- Institute of Pathology Saarbruecken-Rastpfuhl, 66113 Saarbruecken, Germany;
| | - Axel Semjonow
- Department of Urology, Prostate Center, University Clinic Münster, 48149 Münster, Germany;
| | - Natalia Bednarz-Knoll
- Laboratory of Translational Oncology, Medical University of Gdańsk, 80-210 Gdańsk, Poland; (P.N.); (J.S.); (M.P.); (A.J.Ż.)
| |
Collapse
|
8
|
Qi Q, Lin X, Chen C, Xie W, Huang Y, Ding X, Liu X, Yu Y. Curriculum Feature Alignment Domain Adaptation for Epithelium-Stroma Classification in Histopathological Images. IEEE J Biomed Health Inform 2021; 25:1163-1172. [PMID: 32881698 DOI: 10.1109/jbhi.2020.3021558] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
In recent years, deep learning methods have received more attention in epithelial-stroma (ES) classification tasks. Traditional deep learning methods assume that the training and test data have the same distribution, an assumption that is seldom satisfied in complex imaging procedures. Unsupervised domain adaptation (UDA) transfers knowledge from a labelled source domain to a completely unlabeled target domain, and is more suitable for ES classification tasks to avoid tedious annotation. However, existing UDA methods for this task ignore the semantic alignment across domains. In this paper, we propose a Curriculum Feature Alignment Network (CFAN) to gradually align discriminative features across domains through selecting effective samples from the target domain and minimizing intra-class differences. Specifically, we developed the Curriculum Transfer Strategy (CTS) and Adaptive Centroid Alignment (ACA) steps to train our model iteratively. We validated the method using three independent public ES datasets, and experimental results demonstrate that our method achieves better performance in ES classification compared with commonly used deep learning methods and existing deep domain adaptation methods.
Collapse
|
9
|
Two Ensemble-CNN Approaches for Colorectal Cancer Tissue Type Classification. J Imaging 2021; 7:jimaging7030051. [PMID: 34460707 PMCID: PMC8321410 DOI: 10.3390/jimaging7030051] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2021] [Revised: 02/16/2021] [Accepted: 02/26/2021] [Indexed: 02/06/2023] Open
Abstract
In recent years, automatic tissue phenotyping has attracted increasing interest in the Digital Pathology (DP) field. For Colorectal Cancer (CRC), tissue phenotyping can diagnose the cancer and differentiate between different cancer grades. The development of Whole Slide Images (WSIs) has provided the required data for creating automatic tissue phenotyping systems. In this paper, we study different hand-crafted feature-based and deep learning methods using two popular multi-classes CRC-tissue-type databases: Kather-CRC-2016 and CRC-TP. For the hand-crafted features, we use two texture descriptors (LPQ and BSIF) and their combination. In addition, two classifiers are used (SVM and NN) to classify the texture features into distinct CRC tissue types. For the deep learning methods, we evaluate four Convolutional Neural Network (CNN) architectures (ResNet-101, ResNeXt-50, Inception-v3, and DenseNet-161). Moreover, we propose two Ensemble CNN approaches: Mean-Ensemble-CNN and NN-Ensemble-CNN. The experimental results show that the proposed approaches outperformed the hand-crafted feature-based methods, CNN architectures and the state-of-the-art methods in both databases.
Collapse
|
10
|
Cusano C, Napoletano P, Schettini R. T1K+: A Database for Benchmarking Color Texture Classification and Retrieval Methods. SENSORS 2021; 21:s21031010. [PMID: 33540828 PMCID: PMC7867336 DOI: 10.3390/s21031010] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/18/2020] [Revised: 01/25/2021] [Accepted: 01/26/2021] [Indexed: 11/16/2022]
Abstract
In this paper we present T1K+, a very large, heterogeneous database of high-quality texture images acquired under variable conditions. T1K+ contains 1129 classes of textures ranging from natural subjects to food, textile samples, construction materials, etc. T1K+ allows the design of experiments especially aimed at understanding the specific issues related to texture classification and retrieval. To help the exploration of the database, all the 1129 classes are hierarchically organized in 5 thematic categories and 266 sub-categories. To complete our study, we present an evaluation of hand-crafted and learned visual descriptors in supervised texture classification tasks.
Collapse
Affiliation(s)
- Claudio Cusano
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Via Ferrata 1, 27100 Pavia, Italy;
| | - Paolo Napoletano
- Department of Informatics, Systems and Communication, University of Milan, Bicocca, Viale Sarca 336, 20126 Milan, Italy;
- Correspondence:
| | - Raimondo Schettini
- Department of Informatics, Systems and Communication, University of Milan, Bicocca, Viale Sarca 336, 20126 Milan, Italy;
| |
Collapse
|
11
|
Liu GH, Zhang BW, Qian G, Wang B, Mao B, Bichindaritz I. Bioimage-Based Prediction of Protein Subcellular Location in Human Tissue with Ensemble Features and Deep Networks. IEEE/ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS 2020; 17:1966-1980. [PMID: 31107658 DOI: 10.1109/tcbb.2019.2917429] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Prediction of protein subcellular location has currently become a hot topic because it has been proven to be useful for understanding both the disease mechanisms and novel drug design. With the rapid development of automated microscopic imaging technology in recent years, classification methods of bioimage-based protein subcellular location have attracted considerable attention for images can describe the protein distribution intuitively and in detail. In the current study, a prediction method of protein subcellular location was proposed based on multi-view image features that are extracted from three different views, including the four texture features of the original image, the global and local features of the protein extracted from the protein channel images after color segmentation, and the global features of DNA extracted from the DNA channel image. Finally, the extracted features were combined together to improve the performance of subcellular localization prediction. From the performance comparison of different combination features under the same classifier, the best ensemble features could be obtained. In this work, a classifier based on Stacked Auto-encoders and the random forest was also put forward. To improve the prediction results, the deep network was combined with the traditional statistical classification methods. Stringent cross-validation and independent validation tests on the benchmark dataset demonstrated the efficacy of the proposed method.
Collapse
|
12
|
Zhao K, Li Z, Yao S, Wang Y, Wu X, Xu Z, Wu L, Huang Y, Liang C, Liu Z. Artificial intelligence quantified tumour-stroma ratio is an independent predictor for overall survival in resectable colorectal cancer. EBioMedicine 2020; 61:103054. [PMID: 33039706 PMCID: PMC7648125 DOI: 10.1016/j.ebiom.2020.103054] [Citation(s) in RCA: 68] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2020] [Revised: 09/13/2020] [Accepted: 09/17/2020] [Indexed: 12/11/2022] Open
Abstract
Background An artificial intelligence method could accelerate the clinical implementation of tumour-stroma ratio (TSR), which has prognostic relevance in colorectal cancer (CRC). We, therefore, developed a deep learning model for the fully automated TSR quantification on routine haematoxylin and eosin (HE) stained whole-slide images (WSI) and further investigated its prognostic validity for patient stratification. Methods We trained a convolutional neural network (CNN) model using transfer learning, with its nine-class tissue classification performance evaluated in two independent test sets. Patch-level segmentation on WSI HE slides was performed using the model, with TSR subsequently derived. A discovery (N=499) and validation cohort (N=315) were used to evaluate the prognostic value of TSR for overall survival (OS). Findings The CNN-quantified TSR was a prognostic factor, independently of other clinicopathologic characteristics, with stroma-high associated with reduced OS in the discovery (HR 1.72, 95% CI 1.24-2.37, P=0.001) and validation cohort (2.08, 1.26-3.42, 0.004). Integrating TSR into a Cox model with other risk factors showed improved prognostic capability. Interpretation We developed a deep learning model to quantify TSR based on histologic WSI of CRC and demonstrated its prognostic validity for patient stratification for OS in two independent CRC patient cohorts. This fully automatic approach allows for the objective and standardised application while reducing pathologists' workload. Thus, it can potentially be of significant aid in clinical prognosis prediction and decision-making. Funding National Key Research and Development Program of China, National Science Fund for Distinguished Young Scholar, and National Science Foundation for Young Scientists of China.
Collapse
Affiliation(s)
- Ke Zhao
- School of Medicine, South China University of Technology, Guangzhou 510006, China; Department of Radiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, 106 Zhongshan Er Road, Guangzhou 510080, China
| | - Zhenhui Li
- Department of Radiology, The Third Affiliated Hospital of Kunming Medical University, Yunnan Cancer Hospital, Yunnan Cancer Center, Kunming 650118, China
| | - Su Yao
- Department of Pathology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou 510080, China
| | - Yingyi Wang
- Department of Radiology, Zhuhai People's Hospital, Zhuhai Hospital Affiliated with Jinan University, Zhuhai 519000, China
| | - Xiaomei Wu
- Department of Radiology, The Sixth Affiliated Hospital of Sun Yat-sen University, Guangzhou 510665, China
| | - Zeyan Xu
- School of Medicine, South China University of Technology, Guangzhou 510006, China
| | - Lin Wu
- Department of Pathology, The Third Affiliated Hospital of Kunming Medical University, Yunnan Cancer Hospital, Yunnan Cancer Center, Kunming 650118, China
| | - Yanqi Huang
- Department of Radiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, 106 Zhongshan Er Road, Guangzhou 510080, China; The Second School of Clinical Medicine, Southern Medical University, Guangzhou 510515, China.
| | - Changhong Liang
- Department of Radiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, 106 Zhongshan Er Road, Guangzhou 510080, China.
| | - Zaiyi Liu
- Department of Radiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, 106 Zhongshan Er Road, Guangzhou 510080, China.
| |
Collapse
|
13
|
Javed S, Mahmood A, Werghi N, Benes K, Rajpoot N. Multiplex Cellular Communities in Multi-Gigapixel Colorectal Cancer Histology Images for Tissue Phenotyping. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2020; PP:9204-9219. [PMID: 32966218 DOI: 10.1109/tip.2020.3023795] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
In computational pathology, automated tissue phenotyping in cancer histology images is a fundamental tool for profiling tumor microenvironments. Current tissue phenotyping methods use features derived from image patches which may not carry biological significance. In this work, we propose a novel multiplex cellular community-based algorithm for tissue phenotyping integrating cell-level features within a graph-based hierarchical framework. We demonstrate that such integration offers better performance compared to prior deep learning and texture-based methods as well as to cellular community based methods using uniplex networks. To this end, we construct celllevel graphs using texture, alpha diversity and multi-resolution deep features. Using these graphs, we compute cellular connectivity features which are then employed for the construction of a patch-level multiplex network. Over this network, we compute multiplex cellular communities using a novel objective function. The proposed objective function computes a low-dimensional subspace from each cellular network and subsequently seeks a common low-dimensional subspace using the Grassmann manifold. We evaluate our proposed algorithm on three publicly available datasets for tissue phenotyping, demonstrating a significant improvement over existing state-of-the-art methods.
Collapse
|
14
|
Wang J, Yang F, Liu W, Sun J, Han Y, Li D, Gkoutos GV, Zhu Y, Chen Y. Radiomic Analysis of Native T 1 Mapping Images Discriminates Between MYH7 and MYBPC3-Related Hypertrophic Cardiomyopathy. J Magn Reson Imaging 2020; 52:1714-1721. [PMID: 32525266 DOI: 10.1002/jmri.27209] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2020] [Revised: 05/08/2020] [Accepted: 05/12/2020] [Indexed: 02/05/2023] Open
Abstract
BACKGROUND The phenotype via conventional cardiac MRI analysis of MYH7 (β-myosin heavy chain)- and MYBPC3 (β-myosin-binding protein C)-associated hypertrophic cardiomyopathy (HCM) groups is similar. Few studies exist on the genotypic-phenotypic association as assessed by machine learning in HCM patients. PURPOSE To explore the phenotypic differences based on radiomics analysis of T1 mapping images between MYH7 and MYBPC3-associated HCM subgroups. STUDY TYPE Prospective observational study. SUBJECTS In all, 102 HCM patients with pathogenic, or likely pathogenic mutation, in MYH7 (n = 68) or MYBPC3 (n = 34) genes. FIELD STRENGTH/SEQUENCE Cardiac MRI was performed at 3.0T with balanced steady-state free precession (bSSFP), phase-sensitive inversion recovery (PSIR) late gadolinium enhancement (LGE), and modified Look-Locker inversion recovery (MOLLI) T1 mapping sequences. ASSESSMENT All patients underwent next-generation sequencing and Sanger genetic sequencing. Left ventricular native T1 and LGE were analyzed. One hundred and fifty-seven radiomic features were extracted and modeled using a support vector machine (SVM) combined with principal component analysis (PCA). Each subgroup was randomly split 4:1 (feature selection / test validation). STATISTICAL TESTS Mann-Whitney U-tests and Student's t-tests were performed to assess differences between subgroups. A receiver operating characteristic (ROC) curve was used to assess the model's ability to stratify patients based on radiomic features. RESULTS There were no significant differences between MYH7- and MYBPC3-associated HCM subgroups based on traditional native T1 values (global, basal, and middle short-axis slice native T1 ; P = 0.760, 0.914, and 0.178, respectively). However, the SVM model combined with PCA achieved an accuracy and area under the curve (AUC) of 92.0% and 0.968 (95% confidence interval [CI]: 0.968-0.971), respectively. For the test validation dataset, the accuracy and AUC were 85.5% and 0.886 (95% CI: 0.881-0.901), respectively. DATA CONCLUSION Radiomic analysis of native T1 mapping images may be able to discriminate between MYH7- and MYBPC3-associated HCM patients, exceeding the performance of conventional native T1 values. LEVEL OF EVIDENCE 3 TECHNICAL EFFICACY STAGE: 2 J. MAGN. RESON. IMAGING 2020;52:1714-1721.
Collapse
Affiliation(s)
- Jie Wang
- Department of Cardiology, West China Hospital, Sichuan University, Chengdu, China
- College of Medical and Dental Sciences, Institute of Cancer and Genomic Sciences, University of Birmingham, Birmingham, UK
| | - Fuyao Yang
- Department of Cardiology, West China Hospital, Sichuan University, Chengdu, China
| | - Wentao Liu
- Medical Big Data Center, West China Hospital, Sichuan University, Chengdu, P. R. China
| | - Jiayu Sun
- Department of Radiology, West China Hospital, Sichuan University, Chengdu, P. R. China
| | - Yuchi Han
- Department of Medicine (Cardiovascular Division), University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Dong Li
- Division of Hospital Medicine, Emory University School of Medicine, Atlanta, Georgia, USA
| | - Georgios V Gkoutos
- College of Medical and Dental Sciences, Institute of Cancer and Genomic Sciences, University of Birmingham, Birmingham, UK
- MRC Health Data Research UK (HDR UK), London, UK
| | - Yanjie Zhu
- Paul C. Lauterbur Research Centre for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, P. R. China
| | - Yucheng Chen
- Department of Cardiology, West China Hospital, Sichuan University, Chengdu, China
- Department of Radiology, West China Hospital, Sichuan University, Chengdu, P. R. China
- Center of Rare diseases, West China Hospital, Sichuan University, Chengdu, P. R. China
| |
Collapse
|
15
|
Javed S, Mahmood A, Fraz MM, Koohbanani NA, Benes K, Tsang YW, Hewitt K, Epstein D, Snead D, Rajpoot N. Cellular community detection for tissue phenotyping in colorectal cancer histology images. Med Image Anal 2020; 63:101696. [PMID: 32330851 DOI: 10.1016/j.media.2020.101696] [Citation(s) in RCA: 55] [Impact Index Per Article: 13.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Revised: 02/18/2020] [Accepted: 04/02/2020] [Indexed: 02/01/2023]
Abstract
Classification of various types of tissue in cancer histology images based on the cellular compositions is an important step towards the development of computational pathology tools for systematic digital profiling of the spatial tumor microenvironment. Most existing methods for tissue phenotyping are limited to the classification of tumor and stroma and require large amount of annotated histology images which are often not available. In the current work, we pose the problem of identifying distinct tissue phenotypes as finding communities in cellular graphs or networks. First, we train a deep neural network for cell detection and classification into five distinct cellular components. Considering the detected nuclei as nodes, potential cell-cell connections are assigned using Delaunay triangulation resulting in a cell-level graph. Based on this cell graph, a feature vector capturing potential cell-cell connection of different types of cells is computed. These feature vectors are used to construct a patch-level graph based on chi-square distance. We map patch-level nodes to the geometric space by representing each node as a vector of geodesic distances from other nodes in the network and iteratively drifting the patch nodes in the direction of positive density gradients towards maximum density regions. The proposed algorithm is evaluated on a publicly available dataset and another new large-scale dataset consisting of 280K patches of seven tissue phenotypes. The estimated communities have significant biological meanings as verified by the expert pathologists. A comparison with current state-of-the-art methods reveals significant performance improvement in tissue phenotyping.
Collapse
Affiliation(s)
- Sajid Javed
- Department of Computer Science, University of Warwick, Coventry, CV4 7AL, UK; Khalifa University Center for Autonomous Robotic Systems (KUCARS), Abu Dhabi, P.O. Box 127788, UAE
| | - Arif Mahmood
- Department of Computer Science, Information Technology University, Lahore, Pakistan
| | - Muhammad Moazam Fraz
- Department of Computer Science, University of Warwick, Coventry, CV4 7AL, UK; National University of Science and Technology (NUST), Islamabad, Pakistan
| | | | - Ksenija Benes
- Department of Pathology, University Hospitals Coventry & Warwickshire NHS Trust, Walsgrave, Coventry, CV2 2DX, UK
| | - Yee-Wah Tsang
- Department of Pathology, University Hospitals Coventry & Warwickshire NHS Trust, Walsgrave, Coventry, CV2 2DX, UK
| | - Katherine Hewitt
- Department of Pathology, University Hospitals Coventry & Warwickshire NHS Trust, Walsgrave, Coventry, CV2 2DX, UK
| | - David Epstein
- Mathematics Institute, University of Warwick, Coventry, CV4 7AL, UK
| | - David Snead
- Department of Pathology, University Hospitals Coventry & Warwickshire NHS Trust, Walsgrave, Coventry, CV2 2DX, UK
| | - Nasir Rajpoot
- Department of Computer Science, University of Warwick, Coventry, CV4 7AL, UK; Department of Pathology, University Hospitals Coventry & Warwickshire NHS Trust, Walsgrave, Coventry, CV2 2DX, UK; The Alan Turing Institute, London, UK.
| |
Collapse
|
16
|
Halicek M, Shahedi M, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Head and Neck Cancer Detection in Digitized Whole-Slide Histology Using Convolutional Neural Networks. Sci Rep 2019; 9:14043. [PMID: 31575946 PMCID: PMC6773771 DOI: 10.1038/s41598-019-50313-x] [Citation(s) in RCA: 49] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2019] [Accepted: 09/10/2019] [Indexed: 01/01/2023] Open
Abstract
Primary management for head and neck cancers, including squamous cell carcinoma (SCC), involves surgical resection with negative cancer margins. Pathologists guide surgeons during these operations by detecting cancer in histology slides made from the excised tissue. In this study, 381 digitized, histological whole-slide images (WSI) from 156 patients with head and neck cancer were used to train, validate, and test an inception-v4 convolutional neural network. The proposed method is able to detect and localize primary head and neck SCC on WSI with an AUC of 0.916 for patients in the SCC testing group and 0.954 for patients in the thyroid carcinoma testing group. Moreover, the proposed method is able to diagnose WSI with cancer versus normal slides with an AUC of 0.944 and 0.995 for the SCC and thyroid carcinoma testing groups, respectively. For comparison, we tested the proposed, diagnostic method on an open-source dataset of WSI from sentinel lymph nodes with breast cancer metastases, CAMELYON 2016, to obtain patch-based cancer localization and slide-level cancer diagnoses. The experimental design yields a robust method with potential to help create a tool to increase efficiency and accuracy of pathologists detecting head and neck cancers in histological images.
Collapse
Affiliation(s)
- Martin Halicek
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX, USA.,Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, USA
| | - Maysam Shahedi
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX, USA
| | - James V Little
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Atlanta, GA, USA
| | - Amy Y Chen
- Department of Otolaryngology, Emory University School of Medicine, Atlanta, GA, USA
| | - Larry L Myers
- Department of Otolaryngology, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Baran D Sumer
- Department of Otolaryngology, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Baowei Fei
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX, USA. .,Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, TX, USA. .,Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX, USA.
| |
Collapse
|
17
|
Qaiser T, Tsang YW, Taniyama D, Sakamoto N, Nakane K, Epstein D, Rajpoot N. Fast and accurate tumor segmentation of histology images using persistent homology and deep convolutional features. Med Image Anal 2019; 55:1-14. [PMID: 30991188 DOI: 10.1016/j.media.2019.03.014] [Citation(s) in RCA: 78] [Impact Index Per Article: 15.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2018] [Revised: 03/27/2019] [Accepted: 03/30/2019] [Indexed: 12/17/2022]
Abstract
Tumor segmentation in whole-slide images of histology slides is an important step towards computer-assisted diagnosis. In this work, we propose a tumor segmentation framework based on the novel concept of persistent homology profiles (PHPs). For a given image patch, the homology profiles are derived by efficient computation of persistent homology, which is an algebraic tool from homology theory. We propose an efficient way of computing topological persistence of an image, alternative to simplicial homology. The PHPs are devised to distinguish tumor regions from their normal counterparts by modeling the atypical characteristics of tumor nuclei. We propose two variants of our method for tumor segmentation: one that targets speed without compromising accuracy and the other that targets higher accuracy. The fast version is based on a selection of exemplar image patches from a convolution neural network (CNN) and patch classification by quantifying the divergence between the PHPs of exemplars and the input image patch. Detailed comparative evaluation shows that the proposed algorithm is significantly faster than competing algorithms while achieving comparable results. The accurate version combines the PHPs and high-level CNN features and employs a multi-stage ensemble strategy for image patch labeling. Experimental results demonstrate that the combination of PHPs and CNN features outperform competing algorithms. This study is performed on two independently collected colorectal datasets containing adenoma, adenocarcinoma, signet, and healthy cases. Collectively, the accurate tumor segmentation produces the highest average patch-level F1-score, as compared with competing algorithms, on malignant and healthy cases from both the datasets. Overall the proposed framework highlights the utility of persistent homology for histopathology image analysis.
Collapse
Affiliation(s)
- Talha Qaiser
- Department of Computer Science, University of Warwick, UK.
| | - Yee-Wah Tsang
- Department of Pathology, University Hospitals Coventry and Warwickshire, UK
| | - Daiki Taniyama
- Department of Molecular Pathology, Hiroshima University Institute of Biomedical and Health Sciences, Japan
| | - Naoya Sakamoto
- Department of Molecular Pathology, Hiroshima University Institute of Biomedical and Health Sciences, Japan
| | - Kazuaki Nakane
- Graduate School of Medicine, Division of Health Science, Osaka University, Japan
| | | | - Nasir Rajpoot
- Department of Computer Science, University of Warwick, UK; Department of Pathology, University Hospitals Coventry and Warwickshire, UK; The Alan Turing Institute, UK.
| |
Collapse
|
18
|
Computer aided quantification of intratumoral stroma yields an independent prognosticator in rectal cancer. Cell Oncol (Dordr) 2019; 42:331-341. [PMID: 30825182 DOI: 10.1007/s13402-019-00429-z] [Citation(s) in RCA: 76] [Impact Index Per Article: 15.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/07/2019] [Indexed: 12/19/2022] Open
Abstract
PURPOSE Tumor-stroma ratio (TSR) serves as an independent prognostic factor in colorectal cancer and other solid malignancies. The recent introduction of digital pathology in routine tissue diagnostics holds opportunities for automated TSR analysis. We investigated the potential of computer-aided quantification of intratumoral stroma in rectal cancer whole-slide images. METHODS Histological slides from 129 rectal adenocarcinoma patients were analyzed by two experts who selected a suitable stroma hot-spot and visually assessed TSR. A semi-automatic method based on deep learning was trained to segment all relevant tissue types in rectal cancer histology and subsequently applied to the hot-spots provided by the experts. Patients were assigned to a 'stroma-high' or 'stroma-low' group by both TSR methods (visual and automated). This allowed for prognostic comparison between the two methods in terms of disease-specific and disease-free survival times. RESULTS With stroma-low as baseline, automated TSR was found to be prognostic independent of age, gender, pT-stage, lymph node status, tumor grade, and whether adjuvant therapy was given, both for disease-specific survival (hazard ratio = 2.48 (95% confidence interval 1.29-4.78)) and for disease-free survival (hazard ratio = 2.05 (95% confidence interval 1.11-3.78)). Visually assessed TSR did not serve as an independent prognostic factor in multivariate analysis. CONCLUSIONS This work shows that TSR is an independent prognosticator in rectal cancer when assessed automatically in user-provided stroma hot-spots. The deep learning-based technology presented here may be a significant aid to pathologists in routine diagnostics.
Collapse
|
19
|
Halicek M, Shahedi M, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Detection of Squamous Cell Carcinoma in Digitized Histological Images from the Head and Neck Using Convolutional Neural Networks. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2019; 10956. [PMID: 32476700 DOI: 10.1117/12.2512570] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Primary management for head and neck squamous cell carcinoma (SCC) involves surgical resection with negative cancer margins. Pathologists guide surgeons during these operations by detecting SCC in histology slides made from the excised tissue. In this study, 192 digitized histological images from 84 head and neck SCC patients were used to train, validate, and test an inception-v4 convolutional neural network. The proposed method performs with an AUC of 0.91 and 0.92 for the validation and testing group. The careful experimental design yields a robust method with potential to help create a tool to increase efficiency and accuracy of pathologists for detecting SCC in histological images.
Collapse
Affiliation(s)
- Martin Halicek
- Department of Bioengineering, University of Texas at Dallas, Dallas, TX, USA.,Georgia Inst. of Tech. & Emory Univ., Dept. of Biomedical Engineering, Atlanta, GA.,Medical College of Georgia, Augusta University, Augusta, GA
| | - Maysam Shahedi
- Department of Bioengineering, University of Texas at Dallas, Dallas, TX, USA
| | - James V Little
- Emory Univ. School of Medicine, Dept. of Pathology & Laboratory Medicine, Atlanta, GA
| | - Amy Y Chen
- Emory University School of Medicine, Dept. of Otolaryngology, Atlanta, GA
| | - Larry L Myers
- University of Texas Southwestern Medical Center, Dept. of Otolaryngology, Dallas, TX
| | - Baran D Sumer
- University of Texas Southwestern Medical Center, Dept. of Otolaryngology, Dallas, TX
| | - Baowei Fei
- Department of Bioengineering, University of Texas at Dallas, Dallas, TX, USA.,Univ. of Texas Southwestern Medical Center, Advanced Imaging Research Center, Dallas, TX.,University of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX
| |
Collapse
|
20
|
Awan R, Al-Maadeed S, Al-Saady R. Using spectral imaging for the analysis of abnormalities for colorectal cancer: When is it helpful? PLoS One 2018; 13:e0197431. [PMID: 29874262 PMCID: PMC5991384 DOI: 10.1371/journal.pone.0197431] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2017] [Accepted: 05/02/2018] [Indexed: 12/16/2022] Open
Abstract
The spectral imaging technique has been shown to provide more discriminative information than the RGB images and has been proposed for a range of problems. There are many studies demonstrating its potential for the analysis of histopathology images for abnormality detection but there have been discrepancies among previous studies as well. Many multispectral based methods have been proposed for histopathology images but the significance of the use of whole multispectral cube versus a subset of bands or a single band is still arguable. We performed comprehensive analysis using individual bands and different subsets of bands to determine the effectiveness of spectral information for determining the anomaly in colorectal images. Our multispectral colorectal dataset consists of four classes, each represented by infra-red spectrum bands in addition to the visual spectrum bands. We performed our analysis of spectral imaging by stratifying the abnormalities using both spatial and spectral information. For our experiments, we used a combination of texture descriptors with an ensemble classification approach that performed best on our dataset. We applied our method to another dataset and got comparable results with those obtained using the state-of-the-art method and convolutional neural network based method. Moreover, we explored the relationship of the number of bands with the problem complexity and found that higher number of bands is required for a complex task to achieve improved performance. Our results demonstrate a synergy between infra-red and visual spectrum by improving the classification accuracy (by 6%) on incorporating the infra-red representation. We also highlight the importance of how the dataset should be divided into training and testing set for evaluating the histopathology image-based approaches, which has not been considered in previous studies on multispectral histopathology images.
Collapse
Affiliation(s)
- Ruqayya Awan
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
- * E-mail:
| | - Somaya Al-Maadeed
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| | | |
Collapse
|
21
|
Yu X, Zheng H, Liu C, Huang Y, Ding X. Classify epithelium-stroma in histopathological images based on deep transferable network. J Microsc 2018; 271:164-173. [PMID: 29676794 DOI: 10.1111/jmi.12705] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2017] [Revised: 03/12/2018] [Accepted: 03/28/2018] [Indexed: 11/28/2022]
Abstract
Recently, the deep learning methods have received more attention in histopathological image analysis. However, the traditional deep learning methods assume that training data and test data have the same distributions, which causes certain limitations in real-world histopathological applications. However, it is costly to recollect a large amount of labeled histology data to train a new neural network for each specified image acquisition procedure even for similar tasks. In this paper, an unsupervised domain adaptation is introduced into a typical deep convolutional neural network (CNN) model to mitigate the repeating of the labels. The unsupervised domain adaptation is implemented by adding two regularisation terms, namely the feature-based adaptation and entropy minimisation, to the object function of a widely used CNN model called the AlexNet. Three independent public epithelium-stroma datasets were used to verify the proposed method. The experimental results have demonstrated that in the epithelium-stroma classification, the proposed method can achieve better performance than the commonly used deep learning methods and some existing deep domain adaptation methods. Therefore, the proposed method can be considered as a better option for the real-world applications of histopathological image analysis because there is no requirement for recollection of large-scale labeled data for every specified domain.
Collapse
Affiliation(s)
- X Yu
- Fujian key Laboratory of Sensing and Computing for Smart City, Xiamen Unviersity, Xiamen, Fujian, China
- School of Information Science and Engineering, Xiamen University, Xiamen, Fujian, China
| | - H Zheng
- Fujian key Laboratory of Sensing and Computing for Smart City, Xiamen Unviersity, Xiamen, Fujian, China
- School of Information Science and Engineering, Xiamen University, Xiamen, Fujian, China
| | - C Liu
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania, U.S.A
| | - Y Huang
- Fujian key Laboratory of Sensing and Computing for Smart City, Xiamen Unviersity, Xiamen, Fujian, China
- School of Information Science and Engineering, Xiamen University, Xiamen, Fujian, China
| | - X Ding
- Fujian key Laboratory of Sensing and Computing for Smart City, Xiamen Unviersity, Xiamen, Fujian, China
- School of Information Science and Engineering, Xiamen University, Xiamen, Fujian, China
| |
Collapse
|
22
|
van Pelt GW, Sandberg TP, Morreau H, Gelderblom H, van Krieken JHJM, Tollenaar RAEM, Mesker WE. The tumour-stroma ratio in colon cancer: the biological role and its prognostic impact. Histopathology 2018; 73:197-206. [PMID: 29457843 DOI: 10.1111/his.13489] [Citation(s) in RCA: 81] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
The tumour microenvironment consists of a complex mixture of non-neoplastic cells, including fibroblasts, immune cells and endothelial cells embedded in the proteins of the extracellular matrix. The tumour microenvironment plays an active role in tumour behaviour. By interacting with cancer cells, it influences disease progression and the metastatic capacity of the tumour. Tumours with a high amount of stroma correspond to poor patient prognosis. The tumour-stroma ratio (TSR) is a strong independent prognostic tool in colon cancer and provides additional value to the current clinically used tumour-node-metastasis classification. The TSR is assessed on conventional haematoxylin and eosin-stained paraffin sections at the invasive front of the tumour. Here we review studies demonstrating the prognostic significance of the TSR in solid epithelial tumours with a focus on colon cancer. Moreover, the biological role of the tumour microenvironment during tumour progression and invasion will be discussed, as well as the attempts to target the tumour stroma for therapeutic purposes. We suggest that the TSR can be implemented with little effort and without additional costs in current routine pathology diagnostics owing to its simplicity and reliability.
Collapse
Affiliation(s)
- Gabi W van Pelt
- Department of Surgery, Leiden University Medical Centre, Leiden, the Netherlands
| | - Tessa P Sandberg
- Department of Surgery, Leiden University Medical Centre, Leiden, the Netherlands
| | - Hans Morreau
- Department of Pathology, Leiden University Medical Centre, Leiden, the Netherlands
| | - Hans Gelderblom
- Department of Clinical Oncology, Leiden University Medical Centre, Leiden, the Netherlands
| | - J Han J M van Krieken
- Department of Pathology, Radboud University Medical Centre, Nijmegen, the Netherlands
| | - Rob A E M Tollenaar
- Department of Surgery, Leiden University Medical Centre, Leiden, the Netherlands
| | - Wilma E Mesker
- Department of Surgery, Leiden University Medical Centre, Leiden, the Netherlands
| |
Collapse
|
23
|
Epithelium and Stroma Identification in Histopathological Images Using Unsupervised and Semi-Supervised Superpixel-Based Segmentation. J Imaging 2017. [DOI: 10.3390/jimaging3040061] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2023] Open
|
24
|
Epithelium-Stroma Classification via Convolutional Neural Networks and Unsupervised Domain Adaptation in Histopathological Images. IEEE J Biomed Health Inform 2017; 21:1625-1632. [DOI: 10.1109/jbhi.2017.2691738] [Citation(s) in RCA: 44] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
25
|
Thibault G, Azimi V, Johnson B, Jorgens D, Link J, Margolin A, Gray JW. Quantitative analysis of histological tissue image based on cytological profiles and spatial statistics. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2017; 2016:1175-1178. [PMID: 28324942 DOI: 10.1109/embc.2016.7590914] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
The cellular heterogeneity and complex tissue architecture of most tumor samples is a major obstacle in image analysis on standard hematoxylin and eosin-stained (H&E) tissue sections. A mixture of cancer and normal cells complicates the interpretation of their cytological profiles. Furthermore, spatial arrangement and architectural organization of cells are generally not reflected in cellular characteristics analysis. To address these challenges, first we describe an automatic nuclei segmentation of H&E tissue sections. In the task of deconvoluting cellular heterogeneity, we adopt Landmark based Spectral Clustering (LSC) to group individual nuclei in such a way that nuclei in the same group are more similar. We next devise spatial statistics for analyzing spatial arrangement and organization, which are not detectable by individual cellular characteristics. Our quantitative, spatial statistics analysis could benefit H&E section analysis by refining and complementing cellular characteristics analysis.
Collapse
|
26
|
Wan T, Cao J, Chen J, Qin Z. Automated grading of breast cancer histopathology using cascaded ensemble with combination of multi-level image features. Neurocomputing 2017. [DOI: 10.1016/j.neucom.2016.05.084] [Citation(s) in RCA: 72] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
|
27
|
Chang YH, Thibault G, Johnson B, Margolin A, Gray JW. Integrative Analysis on Histopathological Image for Identifying Cellular Heterogeneity. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2017; 10140. [PMID: 30364826 DOI: 10.1117/12.2250428] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
This study has brought together image processing, clustering and spatial pattern analysis to quantitatively analyze hematoxylin and eosin-stained (H&E) tissue sections. A mixture of tumor and normal cells (intratumoral heterogeneity) as well as complex tissue architectures of most samples complicate the interpretation of their cytological profiles. To address these challenges, we develop a simple but effective methodology for quantitative analysis for H&E section. We adopt comparative analyses of spatial point patterns to characterize spatial distribution of different nuclei types and complement cellular characteristics analysis. We demonstrate that tumor and normal cell regions exhibit significant differences of lymphocytes spatial distribution or lymphocyte infiltration pattern.
Collapse
Affiliation(s)
- Young Hwan Chang
- Biomedical Engineering, Oregon Health and Science University, Portland, OR, USA
| | - Guillaume Thibault
- Biomedical Engineering, Oregon Health and Science University, Portland, OR, USA
| | - Brett Johnson
- Biomedical Engineering, Oregon Health and Science University, Portland, OR, USA
| | - Adam Margolin
- Biomedical Engineering, Oregon Health and Science University, Portland, OR, USA
| | - Joe W Gray
- Biomedical Engineering, Oregon Health and Science University, Portland, OR, USA
| |
Collapse
|
28
|
Kather JN, Weis CA, Bianconi F, Melchers SM, Schad LR, Gaiser T, Marx A, Zöllner FG. Multi-class texture analysis in colorectal cancer histology. Sci Rep 2016; 6:27988. [PMID: 27306927 PMCID: PMC4910082 DOI: 10.1038/srep27988] [Citation(s) in RCA: 148] [Impact Index Per Article: 18.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2016] [Accepted: 05/25/2016] [Indexed: 02/08/2023] Open
Abstract
Automatic recognition of different tissue types in histological images is an essential part in the digital pathology toolbox. Texture analysis is commonly used to address this problem; mainly in the context of estimating the tumour/stroma ratio on histological samples. However, although histological images typically contain more than two tissue types, only few studies have addressed the multi-class problem. For colorectal cancer, one of the most prevalent tumour types, there are in fact no published results on multiclass texture separation. In this paper we present a new dataset of 5,000 histological images of human colorectal cancer including eight different types of tissue. We used this set to assess the classification performance of a wide range of texture descriptors and classifiers. As a result, we found an optimal classification strategy that markedly outperformed traditional methods, improving the state of the art for tumour-stroma separation from 96.9% to 98.6% accuracy and setting a new standard for multiclass tissue separation (87.4% accuracy for eight classes). We make our dataset of histological images publicly available under a Creative Commons license and encourage other researchers to use it as a benchmark for their studies.
Collapse
Affiliation(s)
- Jakob Nikolas Kather
- Institute of Pathology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany
- Institute of Computer Assisted Clinical Medicine, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Cleo-Aron Weis
- Institute of Pathology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany
| | | | - Susanne M. Melchers
- Department of Dermatology, Venereology and Allergology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany
| | - Lothar R. Schad
- Institute of Computer Assisted Clinical Medicine, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Timo Gaiser
- Institute of Pathology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany
| | - Alexander Marx
- Institute of Pathology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany
| | - Frank Gerrit Zöllner
- Institute of Computer Assisted Clinical Medicine, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| |
Collapse
|
29
|
Xu J, Luo X, Wang G, Gilmore H, Madabhushi A. A Deep Convolutional Neural Network for segmenting and classifying epithelial and stromal regions in histopathological images. Neurocomputing 2016; 191:214-223. [PMID: 28154470 PMCID: PMC5283391 DOI: 10.1016/j.neucom.2016.01.034] [Citation(s) in RCA: 217] [Impact Index Per Article: 27.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
Epithelial (EP) and stromal (ST) are two types of tissues in histological images. Automated segmentation or classification of EP and ST tissues is important when developing computerized system for analyzing the tumor microenvironment. In this paper, a Deep Convolutional Neural Networks (DCNN) based feature learning is presented to automatically segment or classify EP and ST regions from digitized tumor tissue microarrays (TMAs). Current approaches are based on handcraft feature representation, such as color, texture, and Local Binary Patterns (LBP) in classifying two regions. Compared to handcrafted feature based approaches, which involve task dependent representation, DCNN is an end-to-end feature extractor that may be directly learned from the raw pixel intensity value of EP and ST tissues in a data driven fashion. These high-level features contribute to the construction of a supervised classifier for discriminating the two types of tissues. In this work we compare DCNN based models with three handcraft feature extraction based approaches on two different datasets which consist of 157 Hematoxylin and Eosin (H&E) stained images of breast cancer and 1376 immunohistological (IHC) stained images of colorectal cancer, respectively. The DCNN based feature learning approach was shown to have a F1 classification score of 85%, 89%, and 100%, accuracy (ACC) of 84%, 88%, and 100%, and Matthews Correlation Coefficient (MCC) of 86%, 77%, and 100% on two H&E stained (NKI and VGH) and IHC stained data, respectively. Our DNN based approach was shown to outperform three handcraft feature extraction based approaches in terms of the classification of EP and ST regions.
Collapse
Affiliation(s)
- Jun Xu
- Jiangsu Key Laboratory of Big Data Analysis Technique, Nanjing University of Information Science and Technology, Nanjing 210044, China
| | - Xiaofei Luo
- Jiangsu Key Laboratory of Big Data Analysis Technique, Nanjing University of Information Science and Technology, Nanjing 210044, China
| | - Guanhao Wang
- Jiangsu Key Laboratory of Big Data Analysis Technique, Nanjing University of Information Science and Technology, Nanjing 210044, China
| | - Hannah Gilmore
- Institute for Pathology, University Hospitals Case Medical Center, Case Western Reserve University, OH 44106-7207, USA
| | - Anant Madabhushi
- Department of Biomedical Engineering, Case Western Reserve University, OH 44106, USA
| |
Collapse
|