101
|
Analysis of cancer in histological images: employing an approach based on genetic algorithm. Pattern Anal Appl 2020. [DOI: 10.1007/s10044-020-00931-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
102
|
Yazdi M, Erfankhah H. Multiclass histology image retrieval, classification using Riesz transform and local binary pattern features. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2020. [DOI: 10.1080/21681163.2020.1761885] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Affiliation(s)
- Mehran Yazdi
- Faculty of Electrical and Computer Engineering, Shiraz University, Shiraz, Iran
| | - Hamed Erfankhah
- Faculty of Electrical and Computer Engineering, Shiraz University, Shiraz, Iran
| |
Collapse
|
103
|
Xu H, Park S, Hwang TH. Computerized Classification of Prostate Cancer Gleason Scores from Whole Slide Images. IEEE/ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS 2020; 17:1871-1882. [PMID: 31536012 DOI: 10.1109/tcbb.2019.2941195] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Histological Gleason grading of tumor patterns is one of the most powerful prognostic predictors in prostate cancer. However, manual analysis and grading performed by pathologists are typically subjective and time-consuming. In this paper, we present an automatic technique for Gleason grading of prostate cancer from H&E stained whole slide pathology images using a set of novel completed and statistical local binary pattern (CSLBP) descriptors. First, the technique divides the whole slide image (WSI) into a set of small image tiles, where salient tumor tiles with high nuclei densities are selected for analysis. The CSLBP texture features that encode pixel intensity variations from circularly surrounding neighborhoods are extracted from salient image tiles to characterize different Gleason patterns. Finally, the CSLBP texture features computed from all tiles are integrated and utilized by the multi-class support vector machine (SVM) that assigns patient slides with different Gleason scores such as 6, 7, or ≥ 8. Experiments have been performed on 312 different patient cases selected from the cancer genome atlas (TCGA) and have achieved superior performances over state-of-the-art texture descriptors and baseline methods including deep learning models for prostate cancer Gleason grading.
Collapse
|
104
|
Zhao K, Li Z, Li Y, Yao S, Huang Y, Wang Y, Zhang F, Wu L, Chen X, Liang C, Liu Z. Hist-Immune signature: a prognostic factor in colorectal cancer using immunohistochemical slide image analysis. Oncoimmunology 2020; 9:1841935. [PMID: 33194320 PMCID: PMC7605350 DOI: 10.1080/2162402x.2020.1841935] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2020] [Accepted: 10/21/2020] [Indexed: 01/08/2023] Open
Abstract
Computerized image analysis for whole-slide images has been shown to improve efficiency, accuracy, and consistency in histopathology evaluations. We aimed to assess whether immunohistochemistry (IHC) image quantitative features can reflect the immune status and provide prognostic information for colorectal cancer patients. A fully automated pipeline was designed to extract histogram features from IHC digital images in a training set (N = 243). A Hist-Immune signature was generated with selected features using the LASSO Cox model. The results were validated using internal (N = 147) and external (N = 76) validation sets. The five-feature-based Hist-Immune signature was significantly associated with overall survival in training (HR 2.72, 95% CI 1.68-4.41, P < .001), internal (2.86, 1.28-6.39, 0.010), and external (2.30, 1.02-6.16, 0.044) validation sets. The full model constructed by integrating the Hist-Immune signature and clinicopathological factors had good discrimination ability (C-index 0.727, 95% CI 0.678-0.776), confirmed using internal (0.703, 0.621-0.784) and external (0.756, 0.653-0.859) validation sets. Our findings indicate that the Hist-Immune signature constructed based on the quantitative features could reflect the immune status of patients with colorectal cancer, which might advocate change in risk stratification and consequent precision medicine.
Collapse
Affiliation(s)
- Ke Zhao
- Department of Radiology, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
- School of Medicine, South China University of Technology, Guangzhou, China
| | - Zhenhui Li
- Department of Radiology, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Kunming, China
| | - Yong Li
- Department of General Surgery, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
| | - Su Yao
- Department of Pathology, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
| | - Yanqi Huang
- Department of Radiology, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
| | - Yingyi Wang
- Department of Radiology, Zhuhai People’s Hospital, Zhuhai Hospital Affiliated with Jinan University, Zhuhai, China
| | - Fang Zhang
- School of Computer Science Engineering, South China University of Technology, Guangzhou, China
| | - Lin Wu
- Department of Pathology, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Kunming, China
| | - Xin Chen
- Department of Radiology, Guangzhou First People’s Hospital, School of Medicine, South China University of Technology, Guangzhou, China
| | - Changhong Liang
- Department of Radiology, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
| | - Zaiyi Liu
- Department of Radiology, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
| |
Collapse
|
105
|
Sikaroudi M, Safarpoor A, Ghojogh B, Shafiei S, Crowley M, Tizhoosh HR. Supervision and Source Domain Impact on Representation Learning: A Histopathology Case Study. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:1400-1403. [PMID: 33018251 DOI: 10.1109/embc44109.2020.9176279] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
As many algorithms depend on a suitable representation of data, learning unique features is considered a crucial task. Although supervised techniques using deep neural networks have boosted the performance of representation learning, the need for a large sets of labeled data limits the application of such methods. As an example, high-quality delineations of regions of interest in the field of pathology is a tedious and time-consuming task due to the large image dimensions. In this work, we explored the performance of a deep neural network and triplet loss in the area of representation learning. We investigated the notion of similarity and dissimilarity in pathology whole-slide images and compared different setups from unsupervised and semi-supervised to supervised learning in our experiments. Additionally, different approaches were tested, applying few-shot learning on two publicly available pathology image datasets. We achieved high accuracy and generalization when the learned representations were applied to two different pathology datasets.
Collapse
|
106
|
Wetteland R, Engan K, Eftestøl T, Kvikstad V, Janssen EAM. A Multiscale Approach for Whole-Slide Image Segmentation of five Tissue Classes in Urothelial Carcinoma Slides. Technol Cancer Res Treat 2020. [PMCID: PMC7570776 DOI: 10.1177/1533033820946787] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022] Open
Abstract
In pathology labs worldwide, we see an increasing number of tissue samples that need to be assessed without the same increase in the number of pathologists. Computational pathology, where digital scans of histological samples called whole-slide images (WSI) are processed by computational tools, can be of help for the pathologists and is gaining research interests. Most research effort has been given to classify slides as being cancerous or not, localization of cancerous regions, and to the “big-four” in cancer: breast, lung, prostate, and bowel. Urothelial carcinoma, the most common form of bladder cancer, is expensive to follow up due to a high risk of recurrence, and grading systems have a high degree of inter- and intra-observer variability. The tissue samples of urothelial carcinoma contain a mixture of damaged tissue, blood, stroma, muscle, and urothelium, where it is mainly muscle and urothelium that is diagnostically relevant. A coarse segmentation of these tissue types would be useful to i) guide pathologists to the diagnostic relevant areas of the WSI, and ii) use as input in a computer-aided diagnostic (CAD) system. However, little work has been done on segmenting tissue types in WSIs, and on computational pathology for urothelial carcinoma in particular. In this work, we are using convolutional neural networks (CNN) for multiscale tile-wise classification and coarse segmentation, including both context and detail, by using three magnification levels: 25x, 100x, and 400x. 28 models were trained on weakly labeled data from 32 WSIs, where the best model got an F1-score of 96.5% across six classes. The multiscale models were consistently better than the single-scale models, demonstrating the benefit of combining multiple scales. No tissue-class ground-truth for complete WSIs exist, but the best models were used to segment seven unseen WSIs where the results were manually inspected by a pathologist and are considered as very promising.
Collapse
Affiliation(s)
- Rune Wetteland
- Department of Electrical Engineering and Computer Science, University of Stavanger, Norway
| | - Kjersti Engan
- Department of Electrical Engineering and Computer Science, University of Stavanger, Norway
| | - Trygve Eftestøl
- Department of Electrical Engineering and Computer Science, University of Stavanger, Norway
| | - Vebjørn Kvikstad
- Department of Pathology, Stavanger University Hospital, Norway
- Department of Chemistry, Bioscience and Environmental Engineering, University of Stavanger, Norway
| | - Emiel A. M. Janssen
- Department of Pathology, Stavanger University Hospital, Norway
- Department of Chemistry, Bioscience and Environmental Engineering, University of Stavanger, Norway
| |
Collapse
|
107
|
Huang Z, Gan Y, Lye T, Zhang H, Laine A, Angelini ED, Hendon C. Heterogeneity Measurement of Cardiac Tissues Leveraging Uncertainty Information from Image Segmentation. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION : MICCAI ... INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION 2020; 12261:782-791. [PMID: 34169298 PMCID: PMC8220598 DOI: 10.1007/978-3-030-59710-8_76] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/03/2024]
Abstract
Identifying arrhythmia substrates and quantifying their heterogeneity has great potential to provide critical guidance for radio frequency ablation. However, quantitative analysis of heterogeneity on cardiac optical coherence tomography (OCT) images is lacking. In this paper, we conduct the first study on quantifying cardiac tissue heterogeneity from human OCT images. Our proposed method applies a dropout-based Monte Carlo sampling technique to measure the model uncertainty. The heterogeneity information is extracted by decoupling the intra/inter-tissue heterogeneity and tissue boundary uncertainty from the uncertainty measurement. We empirically demonstrate that our model can highlight the subtle features from OCT images, and the heterogeneity information extracted is positively correlated with the tissue heterogeneity information from corresponding histology images.
Collapse
Affiliation(s)
- Ziyi Huang
- Department of Electrical Engineering, Columbia University, New York, NY, USA
| | - Yu Gan
- Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL, USA
| | - Theresa Lye
- Department of Electrical Engineering, Columbia University, New York, NY, USA
| | - Haofeng Zhang
- Department of Industrial Engineering and Operations Research, Columbia University, New York, NY, USA
| | - Andrew Laine
- Department of Biomedical Engineering, Columbia University, New York, NY, USA
| | - Elsa D Angelini
- Department of Biomedical Engineering, Columbia University, New York, NY, USA
- NIHR Imperial Biomedical Research Centre, ITMAT Data Science Group, Imperial College London, London, UK
| | - Christine Hendon
- Department of Electrical Engineering, Columbia University, New York, NY, USA
| |
Collapse
|
108
|
Javed S, Mahmood A, Werghi N, Benes K, Rajpoot N. Multiplex Cellular Communities in Multi-Gigapixel Colorectal Cancer Histology Images for Tissue Phenotyping. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2020; PP:9204-9219. [PMID: 32966218 DOI: 10.1109/tip.2020.3023795] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
In computational pathology, automated tissue phenotyping in cancer histology images is a fundamental tool for profiling tumor microenvironments. Current tissue phenotyping methods use features derived from image patches which may not carry biological significance. In this work, we propose a novel multiplex cellular community-based algorithm for tissue phenotyping integrating cell-level features within a graph-based hierarchical framework. We demonstrate that such integration offers better performance compared to prior deep learning and texture-based methods as well as to cellular community based methods using uniplex networks. To this end, we construct celllevel graphs using texture, alpha diversity and multi-resolution deep features. Using these graphs, we compute cellular connectivity features which are then employed for the construction of a patch-level multiplex network. Over this network, we compute multiplex cellular communities using a novel objective function. The proposed objective function computes a low-dimensional subspace from each cellular network and subsequently seeks a common low-dimensional subspace using the Grassmann manifold. We evaluate our proposed algorithm on three publicly available datasets for tissue phenotyping, demonstrating a significant improvement over existing state-of-the-art methods.
Collapse
|
109
|
Chan L, Hosseini MS, Plataniotis KN. A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains. Int J Comput Vis 2020. [DOI: 10.1007/s11263-020-01373-4] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
110
|
Sabol P, Sinčák P, Hartono P, Kočan P, Benetinová Z, Blichárová A, Verbóová Ľ, Štammová E, Sabolová-Fabianová A, Jašková A. Explainable classifier for improving the accountability in decision-making for colorectal cancer diagnosis from histopathological images. J Biomed Inform 2020; 109:103523. [PMID: 32758538 DOI: 10.1016/j.jbi.2020.103523] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2020] [Revised: 07/22/2020] [Accepted: 07/27/2020] [Indexed: 10/23/2022]
Abstract
Pathologists are responsible for cancer type diagnoses from histopathological cancer tissues. However, it is known that microscopic examination is tedious and time-consuming. In recent years, a long list of machine learning approaches to image classification and whole-slide segmentation has been developed to support pathologists. Although many showed exceptional performances, the majority of them are not able to rationalize their decisions. In this study, we developed an explainable classifier to support decision making for medical diagnoses. The proposed model does not provide an explanation about the causality between the input and the decisions, but offers a human-friendly explanation about the plausibility of the decision. Cumulative Fuzzy Class Membership Criterion (CFCMC) explains its decisions in three ways: through a semantical explanation about the possibilities of misclassification, showing the training sample responsible for a certain prediction and showing training samples from conflicting classes. In this paper, we explain about the mathematical structure of the classifier, which is not designed to be used as a fully automated diagnosis tool but as a support system for medical experts. We also report on the accuracy of the classifier against real world histopathological data for colorectal cancer. We also tested the acceptability of the system through clinical trials by 14 pathologists. We show that the proposed classifier is comparable to state of the art neural networks in accuracy, but more importantly it is more acceptable to be used by human experts as a diagnosis tool in the medical domain.
Collapse
Affiliation(s)
- Patrik Sabol
- Department of Cybernetics and Artificial Intelligence, Technical University of Košice, Košice, Slovakia.
| | - Peter Sinčák
- Department of Cybernetics and Artificial Intelligence, Technical University of Košice, Košice, Slovakia.
| | | | - Pavel Kočan
- Department of Pathology, Pavol Jozef Šafárik University in Košice, Košice, Slovakia
| | - Zuzana Benetinová
- Department of Pathology, Pavol Jozef Šafárik University in Košice, Košice, Slovakia
| | - Alžbeta Blichárová
- Department of Pathology, Pavol Jozef Šafárik University in Košice, Košice, Slovakia
| | - Ľudmila Verbóová
- Department of Pathology, Pavol Jozef Šafárik University in Košice, Košice, Slovakia
| | - Erika Štammová
- Department of Pathology, Pavol Jozef Šafárik University in Košice, Košice, Slovakia
| | | | - Anna Jašková
- The Faculty of Arts of Prešov University, Prešov, Slovakia
| |
Collapse
|
111
|
Ensemble of convolutional neural networks for bioimage classification. APPLIED COMPUTING AND INFORMATICS 2020. [DOI: 10.1016/j.aci.2018.06.002] [Citation(s) in RCA: 37] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
This work presents a system based on an ensemble of Convolutional Neural Networks (CNNs) and descriptors for bioimage classification that has been validated on different datasets of color images. The proposed system represents a very simple yet effective way of boosting the performance of trained CNNs by composing multiple CNNs into an ensemble and combining scores by sum rule. Several types of ensembles are considered, with different CNN topologies along with different learning parameter sets. The proposed system not only exhibits strong discriminative power but also generalizes well over multiple datasets thanks to the combination of multiple descriptors based on different feature types, both learned and handcrafted. Separate classifiers are trained for each descriptor, and the entire set of classifiers is combined by sum rule. Results show that the proposed system obtains state-of-the-art performance across four different bioimage and medical datasets. The MATLAB code of the descriptors will be available at https://github.com/LorisNanni.
Collapse
|
112
|
Dif N, Elberrichi Z. A New Deep Learning Model Selection Method for Colorectal Cancer Classification. INTERNATIONAL JOURNAL OF SWARM INTELLIGENCE RESEARCH 2020. [DOI: 10.4018/ijsir.2020070105] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/05/2023]
Abstract
Deep learning is one of the most commonly used techniques in computer-aided diagnosis systems. Their exploitation for histopathological image analysis is important because of the complex morphology of whole slide images. However, the main limitation of these methods is the restricted number of available medical images, which can lead to an overfitting problem. Many studies have suggested the use of static ensemble learning methods to address this issue. This article aims to propose a new dynamic ensemble deep learning method. First, it generates a set of models based on the transfer learning strategy from deep neural networks. Then, the relevant subset of models is selected by the particle swarm optimization algorithm and combined by voting or averaging methods. The proposed approach was tested on a histopathological dataset for colorectal cancer classification, based on seven types of CNNs. The method has achieved accurate results (94.52%) by the Resnet121 model and the voting strategy, which provides important insights into the efficiency of dynamic ensembling in deep learning.
Collapse
Affiliation(s)
- Nassima Dif
- EEDIS Laboraory, Djillali Liabes University, Sidi Bel Abbes, Algeria
| | | |
Collapse
|
113
|
Zhang F, Yao S, Li Z, Liang C, Zhao K, Huang Y, Gao Y, Qu J, Li Z, Liu Z. Predicting treatment response to neoadjuvant chemoradiotherapy in local advanced rectal cancer by biopsy digital pathology image features. Clin Transl Med 2020; 10:e110. [PMID: 32594660 PMCID: PMC7403709 DOI: 10.1002/ctm2.110] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2020] [Revised: 06/04/2020] [Accepted: 06/05/2020] [Indexed: 12/24/2022] Open
Affiliation(s)
- Fang Zhang
- School of Computer Science and Engineering, South China University of Technology, Guangzhou, Guangdong, China.,Department of Radiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, Guangdong, China
| | - Su Yao
- Department of Pathology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, Guangdong, China
| | - Zhi Li
- Department of Pathology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, Guangdong, China
| | - Changhong Liang
- Department of Radiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, Guangdong, China
| | - Ke Zhao
- Department of Radiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, Guangdong, China.,School of Medicine, South China University of Technology, Guangzhou, Guangdong, China
| | - Yanqi Huang
- Department of Radiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, Guangdong, China.,Southern Medical University, Guangzhou, Guangdong, China
| | - Ying Gao
- School of Computer Science and Engineering, South China University of Technology, Guangzhou, Guangdong, China
| | - Jinrong Qu
- Department of Radiology, the Affiliated Cancer Hospital of Zhengzhou University, Henan Cancer Hospital, Zhengzhou, Henan, China
| | - Zhenhui Li
- Department of Radiology, The Third Affiliated Hospital of Kunming Medical University, Yunnan Cancer Hospita, Yunnan Cancer Center, Kunming, Yunnan, China
| | - Zaiyi Liu
- Department of Radiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, Guangdong, China
| |
Collapse
|
114
|
Characterization of interstitial diffuse fibrosis patterns using texture analysis of myocardial native T1 mapping. PLoS One 2020; 15:e0233694. [PMID: 32479518 PMCID: PMC7263579 DOI: 10.1371/journal.pone.0233694] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2019] [Accepted: 05/11/2020] [Indexed: 11/19/2022] Open
Abstract
Background The pattern of myocardial fibrosis differs significantly between different cardiomyopathies. Fibrosis in hypertrophic cardiomyopathy (HCM) is characteristically as patchy and regional but in dilated cardiomyopathy (DCM) as diffuse and global. We sought to investigate if texture analyses on myocardial native T1 mapping can differentiate between fibrosis patterns in patients with HCM and DCM. Methods We prospectively acquired native myocardial T1 mapping images for 321 subjects (55±15 years, 70% male): 65 control, 116 HCM, and 140 DCM patients. To quantify different fibrosis patterns, four sets of texture descriptors were used to extract 152 texture features from native T1 maps. Seven features were sequentially selected to identify HCM- and DCM-specific patterns in 70% of data (training dataset). Pattern reproducibility and generalizability were tested on the rest of data (testing dataset) using support vector machines (SVM) and regression models. Results Pattern-derived texture features were capable to identify subjects in HCM, DCM, and controls cohorts with 202/237(85.2%) accuracy of all subjects in the training dataset using 10-fold cross-validation on SVM (AUC = 0.93, 0.93, and 0.93 for controls, HCM and DCM, respectively), while pattern-independent global native T1 mapping was poorly capable to identify those subjects with 121/237(51.1%) accuracy (AUC = 0.78, 0.51, and 0.74) (P<0.001 for all). The pattern-derived features were reproducible with excellent intra- and inter-observer reliability and generalizable on the testing dataset with 75/84(89.3%) accuracy. Conclusion Texture analysis of myocardial native T1 mapping can characterize fibrosis patterns in HCM and DCM patients and provides additional information beyond average native T1 values.
Collapse
|
115
|
Szczotka AB, Shakir DI, Ravì D, Clarkson MJ, Pereira SP, Vercauteren T. Learning from irregularly sampled data for endomicroscopy super-resolution: a comparative study of sparse and dense approaches. Int J Comput Assist Radiol Surg 2020; 15:1167-1175. [PMID: 32415459 PMCID: PMC7316691 DOI: 10.1007/s11548-020-02170-7] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2019] [Accepted: 04/14/2020] [Indexed: 12/27/2022]
Abstract
Purpose Probe-based confocal laser endomicroscopy (pCLE) enables performing an optical biopsy via a probe. pCLE probes consist of multiple optical fibres arranged in a bundle, which taken together generate signals in an irregularly sampled pattern. Current pCLE reconstruction is based on interpolating irregular signals onto an over-sampled Cartesian grid, using a naive linear interpolation. It was shown that convolutional neural networks (CNNs) could improve pCLE image quality. Yet classical CNNs may be suboptimal in regard to irregular data.
Methods We compare pCLE reconstruction and super-resolution (SR) methods taking irregularly sampled or reconstructed pCLE images as input. We also propose to embed a Nadaraya–Watson (NW) kernel regression into the CNN framework as a novel trainable CNN layer. We design deep learning architectures allowing for reconstructing high-quality pCLE images directly from the irregularly sampled input data. We created synthetic sparse pCLE images to evaluate our methodology.
Results The results were validated through an image quality assessment based on a combination of the following metrics: peak signal-to-noise ratio and the structural similarity index. Our analysis indicates that both dense and sparse CNNs outperform the reconstruction method currently used in the clinic.
Conclusion The main contributions of our study are a comparison of sparse and dense approach in pCLE image reconstruction. We also implement trainable generalised NW kernel regression as a novel sparse approach. We also generated synthetic data for training pCLE SR. Electronic supplementary material The online version of this article (10.1007/s11548-020-02170-7) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
| | | | - Daniele Ravì
- Centre for Medical Image Computing (CMIC), Department of Computer Science, University College London, London, UK
| | - Matthew J Clarkson
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
| | - Stephen P Pereira
- UCL Institute for Liver and Digestive Health, University College London, London, UK
| | - Tom Vercauteren
- Department of Surgical & Interventional Engineering, King's College London, London, UK
| |
Collapse
|
116
|
Das D, Mahanta LB, Ahmed S, Baishya BK. Classification of childhood medulloblastoma into WHO-defined multiple subtypes based on textural analysis. J Microsc 2020; 279:26-38. [PMID: 32271463 DOI: 10.1111/jmi.12893] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2019] [Revised: 03/20/2020] [Accepted: 03/30/2020] [Indexed: 11/28/2022]
Abstract
Childhood medulloblastoma is a case of a childhood brain tumour that requires close attention due to the low survival rate. Effective prognosis depends a lot on accurate detection of its subtype. The present study proposes a texture-based computer-aided categorization of childhood medulloblastoma samples. According to the World Health Organization, it has four subtypes (desmoplastic, classic, nodular and large). Classification is done in two levels: (i) normal and abnormal and (ii) its four subtypes. The system is evaluated on indigenous patient samples collected from the region. The main objective of database generation is to create a data set of childhood medulloblastoma samples since there exists no available benchmark data set. The proposed framework for automated classification is based on the architectural property and the distribution of cells. Five texture features were extracted for the feature set, namely: grey-level co-occurrence matrix, grey-level run length matrix, first-order histogram features, local binary pattern and Tamura features. The performance of each feature set was evaluated, both individually and in combinations, using five different classifiers. Fivefold cross-validation was used for training and testing the data set. Experiments on both individual feature sets and combinations (best-2, best-3, best-4 and all-5) of feature sets were evaluated based on the accuracy of performance. It was revealed that the combined best-4 feature set resulted in the highest accuracy of 91.3%. The precision, recall and specificity were 0.913, 0.913 and 0.97, respectively. Significantly, it implied that the all-5 feature set is not necessary to have a useful classification. Feature reduction by principal component analysis resulted in increased accuracy of 96.7%. LAY DESCRIPTION: Childhood medulloblastoma is a case of childhood brain tumour that requires high attention due to a low survival rate. Effective prognosis depends a lot on accurate detection of its subtype. The present study proposes a texture-based computer-aided categorization of childhood medulloblastoma samples. According to the World Health Organization (W.H.O), it has four subtypes (desmoplastic, classic, nodular, and large). Classification is done in two levels: i) normal and abnormal ii) its four subtypes. The system is evaluated on indigenous patient samples collected from the region. The main objective of database generation is to create a data set of childhood medulloblastoma samples since there exists no available benchmark data set. The proposed framework is a model for the automatic classification of the samples. The tissue samples obtained post-operation by doctors are converted into images, and then necessary algorithms are applied so that certain features describing each group of the image are known and studied for classification. Later these images are classified using the image features into the subtypes of abnormal samples.
Collapse
Affiliation(s)
- Daisy Das
- Institute of Advanced Study in Science and Technology, Guwahati, India
| | - Lipi B Mahanta
- Institute of Advanced Study in Science and Technology, Guwahati, India
| | - Shabnam Ahmed
- Guwahati Neurological Research Center, Sixmile, Guwahati, India
| | | |
Collapse
|
117
|
Javed S, Mahmood A, Fraz MM, Koohbanani NA, Benes K, Tsang YW, Hewitt K, Epstein D, Snead D, Rajpoot N. Cellular community detection for tissue phenotyping in colorectal cancer histology images. Med Image Anal 2020; 63:101696. [PMID: 32330851 DOI: 10.1016/j.media.2020.101696] [Citation(s) in RCA: 54] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Revised: 02/18/2020] [Accepted: 04/02/2020] [Indexed: 02/01/2023]
Abstract
Classification of various types of tissue in cancer histology images based on the cellular compositions is an important step towards the development of computational pathology tools for systematic digital profiling of the spatial tumor microenvironment. Most existing methods for tissue phenotyping are limited to the classification of tumor and stroma and require large amount of annotated histology images which are often not available. In the current work, we pose the problem of identifying distinct tissue phenotypes as finding communities in cellular graphs or networks. First, we train a deep neural network for cell detection and classification into five distinct cellular components. Considering the detected nuclei as nodes, potential cell-cell connections are assigned using Delaunay triangulation resulting in a cell-level graph. Based on this cell graph, a feature vector capturing potential cell-cell connection of different types of cells is computed. These feature vectors are used to construct a patch-level graph based on chi-square distance. We map patch-level nodes to the geometric space by representing each node as a vector of geodesic distances from other nodes in the network and iteratively drifting the patch nodes in the direction of positive density gradients towards maximum density regions. The proposed algorithm is evaluated on a publicly available dataset and another new large-scale dataset consisting of 280K patches of seven tissue phenotypes. The estimated communities have significant biological meanings as verified by the expert pathologists. A comparison with current state-of-the-art methods reveals significant performance improvement in tissue phenotyping.
Collapse
Affiliation(s)
- Sajid Javed
- Department of Computer Science, University of Warwick, Coventry, CV4 7AL, UK; Khalifa University Center for Autonomous Robotic Systems (KUCARS), Abu Dhabi, P.O. Box 127788, UAE
| | - Arif Mahmood
- Department of Computer Science, Information Technology University, Lahore, Pakistan
| | - Muhammad Moazam Fraz
- Department of Computer Science, University of Warwick, Coventry, CV4 7AL, UK; National University of Science and Technology (NUST), Islamabad, Pakistan
| | | | - Ksenija Benes
- Department of Pathology, University Hospitals Coventry & Warwickshire NHS Trust, Walsgrave, Coventry, CV2 2DX, UK
| | - Yee-Wah Tsang
- Department of Pathology, University Hospitals Coventry & Warwickshire NHS Trust, Walsgrave, Coventry, CV2 2DX, UK
| | - Katherine Hewitt
- Department of Pathology, University Hospitals Coventry & Warwickshire NHS Trust, Walsgrave, Coventry, CV2 2DX, UK
| | - David Epstein
- Mathematics Institute, University of Warwick, Coventry, CV4 7AL, UK
| | - David Snead
- Department of Pathology, University Hospitals Coventry & Warwickshire NHS Trust, Walsgrave, Coventry, CV2 2DX, UK
| | - Nasir Rajpoot
- Department of Computer Science, University of Warwick, Coventry, CV4 7AL, UK; Department of Pathology, University Hospitals Coventry & Warwickshire NHS Trust, Walsgrave, Coventry, CV2 2DX, UK; The Alan Turing Institute, London, UK.
| |
Collapse
|
118
|
Nanni L, Lumini A, Ghidoni S, Maguolo G. Stochastic Selection of Activation Layers for Convolutional Neural Networks. SENSORS 2020; 20:s20061626. [PMID: 32183334 PMCID: PMC7147370 DOI: 10.3390/s20061626] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/14/2020] [Revised: 03/11/2020] [Accepted: 03/12/2020] [Indexed: 11/16/2022]
Abstract
In recent years, the field of deep learning has achieved considerable success in pattern recognition, image segmentation, and many other classification fields. There are many studies and practical applications of deep learning on images, video, or text classification. Activation functions play a crucial role in discriminative capabilities of the deep neural networks and the design of new "static" or "dynamic" activation functions is an active area of research. The main difference between "static" and "dynamic" functions is that the first class of activations considers all the neurons and layers as identical, while the second class learns parameters of the activation function independently for each layer or even each neuron. Although the "dynamic" activation functions perform better in some applications, the increased number of trainable parameters requires more computational time and can lead to overfitting. In this work, we propose a mixture of "static" and "dynamic" activation functions, which are stochastically selected at each layer. Our idea for model design is based on a method for changing some layers along the lines of different functional blocks of the best performing CNN models, with the aim of designing new models to be used as stand-alone networks or as a component of an ensemble. We propose to replace each activation layer of a CNN (usually a ReLU layer) by a different activation function stochastically drawn from a set of activation functions: in this way, the resulting CNN has a different set of activation function layers.
Collapse
Affiliation(s)
- Loris Nanni
- Department of Information Enginering, University of Padua, viale Gradenigo 6, 35131 Padua, Italy; (S.G.); (G.M.)
- Correspondence:
| | - Alessandra Lumini
- DISI, Università di Bologna, Via dell’università 50, 47521 Cesena, Italy;
| | - Stefano Ghidoni
- Department of Information Enginering, University of Padua, viale Gradenigo 6, 35131 Padua, Italy; (S.G.); (G.M.)
| | - Gianluca Maguolo
- Department of Information Enginering, University of Padua, viale Gradenigo 6, 35131 Padua, Italy; (S.G.); (G.M.)
| |
Collapse
|
119
|
Uckermann O, Galli R, Mark G, Meinhardt M, Koch E, Schackert G, Steiner G, Kirsch M. Label-free multiphoton imaging allows brain tumor recognition based on texture analysis-a study of 382 tumor patients. Neurooncol Adv 2020; 2:vdaa035. [PMID: 32642692 PMCID: PMC7212881 DOI: 10.1093/noajnl/vdaa035] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/05/2023] Open
Abstract
Background Label-free multiphoton microscopy has been suggested for intraoperative recognition and delineation of brain tumors. For any future clinical application, appropriate approaches for image acquisition and analysis have to be developed. Moreover, an evaluation of the reliability of the approach, taking into account inter- and intrapatient variability, is needed. Methods Coherent anti-Stokes Raman scattering (CARS), two-photon excited fluorescence (TPEF), and second-harmonic generation were acquired on cryosections of brain tumors of 382 patients and 28 human nontumor brain samples. Texture parameters of those images were calculated and used as input for linear discriminant analysis. Results The combined analysis of texture parameters of the CARS and TPEF signal proved to be most suited for the discrimination of nontumor brain versus brain tumors (low- and high-grade astrocytoma, oligodendroglioma, glioblastoma, recurrent glioblastoma, brain metastases of lung, colon, renal, and breast cancer and of malignant melanoma) leading to a correct rate of 96% (sensitivity: 96%, specificity: 100%). To approximate the clinical setting, the results were validated on 42 fresh, unfixed tumor biopsies. 82% of the tumors and, most important, all of the nontumor samples were correctly recognized. An image resolution of 1 µm was sufficient to distinguish brain tumors and nontumor brain. Moreover, the vast majority of single fields of view of each patient’s sample were correctly classified with high probabilities, which is important for clinical translation. Conclusion Label-free multiphoton imaging might allow fast and accurate intraoperative delineation of primary and secondary brain tumors in combination with endoscopic systems.
Collapse
Affiliation(s)
- Ortrud Uckermann
- Neurosurgery, University Hospital Carl Gustav Carus, TU Dresden, Dresden, Germany
| | - Roberta Galli
- Clinical Sensoring and Monitoring, Department of Anesthesiology and Intensive Care Medicine, Faculty of Medicine, TU Dresden, Dresden, Germany
| | - Georg Mark
- Neurosurgery, University Hospital Carl Gustav Carus, TU Dresden, Dresden, Germany
| | - Matthias Meinhardt
- Neuropathology, Institute of Pathology, University Hospital Carl Gustav Carus, TU Dresden, Dresden, Germany
| | - Edmund Koch
- Clinical Sensoring and Monitoring, Department of Anesthesiology and Intensive Care Medicine, Faculty of Medicine, TU Dresden, Dresden, Germany
| | - Gabriele Schackert
- Neurosurgery, University Hospital Carl Gustav Carus, TU Dresden, Dresden, Germany
| | - Gerald Steiner
- Clinical Sensoring and Monitoring, Department of Anesthesiology and Intensive Care Medicine, Faculty of Medicine, TU Dresden, Dresden, Germany
| | - Matthias Kirsch
- Neurosurgery, University Hospital Carl Gustav Carus, TU Dresden, Dresden, Germany
| |
Collapse
|
120
|
Rachapudi V, Lavanya Devi G. Improved convolutional neural network based histopathological image classification. EVOLUTIONARY INTELLIGENCE 2020. [DOI: 10.1007/s12065-020-00367-y] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
121
|
Interpretable multimodal deep learning for real-time pan-tissue pan-disease pathology search on social media. Mod Pathol 2020; 33:2169-2185. [PMID: 32467650 PMCID: PMC7581495 DOI: 10.1038/s41379-020-0540-1] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2020] [Revised: 03/17/2020] [Accepted: 03/17/2020] [Indexed: 02/07/2023]
Abstract
Pathologists are responsible for rapidly providing a diagnosis on critical health issues. Challenging cases benefit from additional opinions of pathologist colleagues. In addition to on-site colleagues, there is an active worldwide community of pathologists on social media for complementary opinions. Such access to pathologists worldwide has the capacity to improve diagnostic accuracy and generate broader consensus on next steps in patient care. From Twitter we curate 13,626 images from 6,351 tweets from 25 pathologists from 13 countries. We supplement the Twitter data with 113,161 images from 1,074,484 PubMed articles. We develop machine learning and deep learning models to (i) accurately identify histopathology stains, (ii) discriminate between tissues, and (iii) differentiate disease states. Area Under Receiver Operating Characteristic (AUROC) is 0.805-0.996 for these tasks. We repurpose the disease classifier to search for similar disease states given an image and clinical covariates. We report precision@k = 1 = 0.7618 ± 0.0018 (chance 0.397 ± 0.004, mean ±stdev ). The classifiers find that texture and tissue are important clinico-visual features of disease. Deep features trained only on natural images (e.g., cats and dogs) substantially improved search performance, while pathology-specific deep features and cell nuclei features further improved search to a lesser extent. We implement a social media bot (@pathobot on Twitter) to use the trained classifiers to aid pathologists in obtaining real-time feedback on challenging cases. If a social media post containing pathology text and images mentions the bot, the bot generates quantitative predictions of disease state (normal/artifact/infection/injury/nontumor, preneoplastic/benign/low-grade-malignant-potential, or malignant) and lists similar cases across social media and PubMed. Our project has become a globally distributed expert system that facilitates pathological diagnosis and brings expertise to underserved regions or hospitals with less expertise in a particular disease. This is the first pan-tissue pan-disease (i.e., from infection to malignancy) method for prediction and search on social media, and the first pathology study prospectively tested in public on social media. We will share data through http://pathobotology.org . We expect our project to cultivate a more connected world of physicians and improve patient care worldwide.
Collapse
|
122
|
Jayachandran S, Ghosh A. Deep Transfer Learning for Texture Classification in Colorectal Cancer Histology. ARTIFICIAL NEURAL NETWORKS IN PATTERN RECOGNITION 2020. [DOI: 10.1007/978-3-030-58309-5_14] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
|
123
|
Tellez D, Litjens G, Bándi P, Bulten W, Bokhorst JM, Ciompi F, van der Laak J. Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology. Med Image Anal 2019; 58:101544. [DOI: 10.1016/j.media.2019.101544] [Citation(s) in RCA: 179] [Impact Index Per Article: 35.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2019] [Revised: 06/25/2019] [Accepted: 08/19/2019] [Indexed: 11/17/2022]
|
124
|
Correlating Changes in the Epithelial Gland Tissue With Advancing Colorectal Cancer Histologic Grade, Using IHC Stained for AIB1 Expression Biopsy Material. Appl Immunohistochem Mol Morphol 2019; 27:749-757. [DOI: 10.1097/pai.0000000000000691] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
|
125
|
Rączkowska A, Możejko M, Zambonelli J, Szczurek E. ARA: accurate, reliable and active histopathological image classification framework with Bayesian deep learning. Sci Rep 2019; 9:14347. [PMID: 31586139 PMCID: PMC6778075 DOI: 10.1038/s41598-019-50587-1] [Citation(s) in RCA: 48] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2019] [Accepted: 09/16/2019] [Indexed: 02/07/2023] Open
Abstract
Machine learning algorithms hold the promise to effectively automate the analysis of histopathological images that are routinely generated in clinical practice. Any machine learning method used in the clinical diagnostic process has to be extremely accurate and, ideally, provide a measure of uncertainty for its predictions. Such accurate and reliable classifiers need enough labelled data for training, which requires time-consuming and costly manual annotation by pathologists. Thus, it is critical to minimise the amount of data needed to reach the desired accuracy by maximising the efficiency of training. We propose an accurate, reliable and active (ARA) image classification framework and introduce a new Bayesian Convolutional Neural Network (ARA-CNN) for classifying histopathological images of colorectal cancer. The model achieves exceptional classification accuracy, outperforming other models trained on the same dataset. The network outputs an uncertainty measurement for each tested image. We show that uncertainty measures can be used to detect mislabelled training samples and can be employed in an efficient active learning workflow. Using a variational dropout-based entropy measure of uncertainty in the workflow speeds up the learning process by roughly 45%. Finally, we utilise our model to segment whole-slide images of colorectal tissue and compute segmentation-based spatial statistics.
Collapse
Affiliation(s)
- Alicja Rączkowska
- Faculty of Mathematics, Informatics and Mechanics, University of Warsaw, Warsaw, Poland
| | - Marcin Możejko
- Faculty of Mathematics, Informatics and Mechanics, University of Warsaw, Warsaw, Poland
| | - Joanna Zambonelli
- Department of Pathology, Medical University of Warsaw, Warsaw, Poland
| | - Ewa Szczurek
- Faculty of Mathematics, Informatics and Mechanics, University of Warsaw, Warsaw, Poland.
| |
Collapse
|
126
|
Tosta TAA, de Faria PR, Servato JPS, Neves LA, Roberto GF, Martins AS, do Nascimento MZ. Unsupervised method for normalization of hematoxylin-eosin stain in histological images. Comput Med Imaging Graph 2019; 77:101646. [DOI: 10.1016/j.compmedimag.2019.101646] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2018] [Revised: 06/27/2019] [Accepted: 08/01/2019] [Indexed: 11/24/2022]
|
127
|
Samsi S, Jones M, Kepner J, Reuther A. Colorization of H&E stained tissue using Deep Learning. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2019; 2018:640-643. [PMID: 30440478 DOI: 10.1109/embc.2018.8512419] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
Histopathology is a critical tool in the diagnosis and stratification of cancer. Digital Pathology involves the scanning of stained and fixed tissue samples to produce highresolution images that can be used for computer-aided diagnosis and research. A common challenge in digital pathology related to the quality and characteristics of staining, which can vary widely from center to center and also within the same institution depending on the age of the stain and other human factors. In this paper we examine the use of deep learning models for colorizing H&E stained tissue images and compare the results with traditional image processing/statistical approaches that have been developed for standardizing or normalizing histopathology images. We adapt existing deep learning models that have been developed for colorizing natural images and compare the results with models developed specifically for digital pathology. Our results show that deep learning approaches can standardize the colorization of H&E images. The performance as measured by the chi-square statistic shows that the deep learning approach can be nearly as good as current state-of-the art normalization methods.
Collapse
|
128
|
Texture Segmentation: An Objective Comparison between Five Traditional Algorithms and a Deep-Learning U-Net Architecture. APPLIED SCIENCES-BASEL 2019. [DOI: 10.3390/app9183900] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Abstract
This paper compares a series of traditional and deep learning methodologies for the segmentation of textures. Six well-known texture composites first published by Randen and Husøy were used to compare traditional segmentation techniques (co-occurrence, filtering, local binary patterns, watershed, multiresolution sub-band filtering) against a deep-learning approach based on the U-Net architecture. For the latter, the effects of depth of the network, number of epochs and different optimisation algorithms were investigated. Overall, the best results were provided by the deep-learning approach. However, the best results were distributed within the parameters, and many configurations provided results well below the traditional techniques.
Collapse
|
129
|
Semi-automated analysis of digital whole slides from humanized lung-cancer xenograft models for checkpoint inhibitor response prediction. Oncotarget 2019; 10:4587-4597. [PMID: 31360306 PMCID: PMC6642041 DOI: 10.18632/oncotarget.27069] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2019] [Accepted: 06/21/2019] [Indexed: 11/25/2022] Open
Abstract
We propose a deep learning workflow for the classification of hematoxylin and eosin stained histological whole-slide images of non-small-cell lung cancer. The workflow includes automatic extraction of meta-features for the characterization of the tumor. We show that the tissue-classification produces state-of-the-art results with an average F1-score of 83%. Manual supervision indicates that experts, in practice, accept a far higher percentage of predictions. Furthermore, the extracted meta-features are validated via visualization revealing relevant biomedical relations between the different tissue classes. In a hypothetical decision-support scenario, these meta-features can be used to discriminate the tumor response with regard to available treatment options with an estimated accuracy of 84%. This workflow supports large-scale analysis of tissue obtained in preclinical animal experiments, enables reproducible quantification of tissue classes and immune system markers, and paves the way towards discovery of novel features predicting response in translational immune-oncology research.
Collapse
|
130
|
Burgermeister E, Battaglin F, Eladly F, Wu W, Herweck F, Schulte N, Betge J, Härtel N, Kather JN, Weis CA, Gaiser T, Marx A, Weiss C, Hofheinz R, Miller IS, Loupakis F, Lenz HJ, Byrne AT, Ebert MP. Aryl hydrocarbon receptor nuclear translocator-like (ARNTL/BMAL1) is associated with bevacizumab resistance in colorectal cancer via regulation of vascular endothelial growth factor A. EBioMedicine 2019; 45:139-154. [PMID: 31300350 PMCID: PMC6642438 DOI: 10.1016/j.ebiom.2019.07.004] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2019] [Revised: 06/26/2019] [Accepted: 07/02/2019] [Indexed: 02/07/2023] Open
Abstract
BACKGROUND The identification of new biomarkers and the development of novel, targetable contexts of vulnerability are of urgent clinical need in drug-resistant metastatic colorectal cancer (mCRC). Aryl-Hydrocarbon-Receptor-Nuclear-Translocator-Like (ARNTL/BMAL1) is a circadian clock-regulated transcription factor promoting expression of genes involved in angiogenesis and tumour progression. We hypothesised that BMAL1 increases expression of the vascular endothelial growth factor A VEGFA gene and, thereby, confers resistance to anti-angiogenic therapy with bevacizumab (Beva), a clinically used antibody for neutralization of VEGFA. METHODS PCR and immunohistochemistry were employed to assess BMAL1 expression in mice (C57BL/6 J Apcmin/+; BALB/c nu/nu xenografts) and CRC patients under combination chemotherapy with Beva. BMAL1 single nucleotide gene polymorphisms (SNPs) were analysed by DNA-microarray in clinical samples. BMAL1 functions were studied in human CRC cell lines using colorimetric growth, DNA-binding and reporter assays. FINDINGS In murine CRCs, high BMAL1 expression correlated with poor preclinical response to Beva treatment. In CRC patients' tumours (n = 74), high BMAL1 expression was associated with clinical non-response to combination chemotherapy with Beva (*p = .0061) and reduced progression-free survival (PFS) [*p = .0223, Hazard Ratio (HR) = 1.69]. BMAL1 SNPs also correlated with shorter PFS (rs7396943, rs7938307, rs2279287) and overall survival (OS) [rs11022780, *p = .014, HR = 1.61]. Mechanistically, Nuclear-Receptor-Subfamily-1-Group-D-Member-1 (NR1D1/REVERBA) bound a - 672 bp Retinoic-Acid-Receptor-Related-Orphan-Receptor-Alpha-responsive-element (RORE) adjacent to a BMAL1 DNA-binding motif (E-box) in the VEGFA gene promoter, resulting in increased VEGFA synthesis and proliferation of human CRC cell lines. INTERPRETATION BMAL1 was associated with Beva resistance in CRC. Inhibition of REVERBA-BMAL1 signalling may prevent resistance to anti-angiogenic therapy. FUND: This work was in part supported by the European Commission Seventh Framework Programme (Contract No. 278981 [ANGIOPREDICT]).
Collapse
Affiliation(s)
- Elke Burgermeister
- Department of Medicine II, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany.
| | - Francesca Battaglin
- Division of Medical Oncology, Norris Comprehensive Cancer Center, Keck School of Medicine, University of Southern California, CA, United States; Unit of Medical Oncology 1, Clinical and Experimental Oncology Department, Veneto Institute of Oncology IOV - IRCCS, Padua, Italy
| | - Fagr Eladly
- Department of Medicine II, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| | - Wen Wu
- Department of Medicine II, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| | - Frank Herweck
- Department of Medicine II, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| | - Nadine Schulte
- Department of Medicine II, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| | - Johannes Betge
- Department of Medicine II, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| | - Nicolai Härtel
- Department of Medicine II, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| | - Jakob N Kather
- Division of Gastroenterology, Hepatology and Hepatobiliary Oncology, University Hospital RWTH Aachen, Aachen, Germany
| | - Cleo-Aron Weis
- Institute of Pathology, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| | - Timo Gaiser
- Institute of Pathology, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| | - Alexander Marx
- Institute of Pathology, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| | - Christel Weiss
- Department of Medical Statistics, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| | - Ralf Hofheinz
- Department of Medicine III, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| | - Ian S Miller
- Department of Physiology & Medical Physics, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Fotios Loupakis
- Unit of Medical Oncology 1, Clinical and Experimental Oncology Department, Veneto Institute of Oncology IOV - IRCCS, Padua, Italy
| | - Heinz-Josef Lenz
- Division of Medical Oncology, Norris Comprehensive Cancer Center, Keck School of Medicine, University of Southern California, CA, United States
| | - Annette T Byrne
- Department of Physiology & Medical Physics, Royal College of Surgeons in Ireland, Dublin, Ireland; UCD School of Biomolecular and Biomedical Science, Conway Institute of Biomolecular and Biomedical Research, University College Dublin, Dublin, Ireland
| | - Matthias P Ebert
- Department of Medicine II, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| |
Collapse
|
131
|
Kather JN, Pearson AT, Halama N, Jäger D, Krause J, Loosen SH, Marx A, Boor P, Tacke F, Neumann UP, Grabsch HI, Yoshikawa T, Brenner H, Chang-Claude J, Hoffmeister M, Trautwein C, Luedde T. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat Med 2019; 25:1054-1056. [PMID: 31160815 PMCID: PMC7423299 DOI: 10.1038/s41591-019-0462-y] [Citation(s) in RCA: 618] [Impact Index Per Article: 123.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2019] [Accepted: 04/19/2019] [Indexed: 11/09/2022]
Abstract
Microsatellite instability determines whether patients with gastrointestinal cancer respond exceptionally well to immunotherapy. However, in clinical practice, not every patient is tested for MSI, because this requires additional genetic or immunohistochemical tests. Here we show that deep residual learning can predict MSI directly from H&E histology, which is ubiquitously available. This approach has the potential to provide immunotherapy to a much broader subset of patients with gastrointestinal cancer.
Collapse
Affiliation(s)
- Jakob Nikolas Kather
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany.
- German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Heidelberg, Germany.
- Applied Tumor Immunity, German Cancer Research Center (DKFZ), Heidelberg, Germany.
- Hematology/Oncology, Department of Medicine, University of Chicago, Chicago, IL, USA.
- Medical Oncology, National Center for Tumor Diseases (NCT), Heidelberg, Germany.
| | - Alexander T Pearson
- Hematology/Oncology, Department of Medicine, University of Chicago, Chicago, IL, USA
| | - Niels Halama
- German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Medical Oncology, National Center for Tumor Diseases (NCT), Heidelberg, Germany
- Translational Immunotherapy, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Dirk Jäger
- German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Applied Tumor Immunity, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Medical Oncology, National Center for Tumor Diseases (NCT), Heidelberg, Germany
| | - Jeremias Krause
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
| | - Sven H Loosen
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
| | - Alexander Marx
- Institute of Pathology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany
| | - Peter Boor
- Institute of Pathology and Department of Nephrology, University Hospital RWTH Aachen, Aachen, Germany
| | - Frank Tacke
- Hepatology and Gastroenterology, Charité University Medicine, Berlin, Germany
| | - Ulf Peter Neumann
- Visceral and Transplant Surgery, University Hospital RWTH Aachen, Aachen, Germany
| | - Heike I Grabsch
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
- Pathology and GROW School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, the Netherlands
| | - Takaki Yoshikawa
- Department of Gastrointestinal Surgery, Kanagawa Cancer Center, Yokohama, Japan
- Department of Gastric Surgery, National Cancer Center Hospital, Tokyo, Japan
| | - Hermann Brenner
- German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Division of Preventive Oncology, German Cancer Research Center (DKFZ) and National Center for Tumor Diseases (NCT), Heidelberg, Germany
| | - Jenny Chang-Claude
- Division of Cancer Epidemiology, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Cancer Epidemiology Group, University Cancer Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Michael Hoffmeister
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Christian Trautwein
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
| | - Tom Luedde
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany.
| |
Collapse
|
132
|
Dragić M, Zarić M, Mitrović N, Nedeljković N, Grković I. Application of Gray Level Co-Occurrence Matrix Analysis as a New Method for Enzyme Histochemistry Quantification. MICROSCOPY AND MICROANALYSIS : THE OFFICIAL JOURNAL OF MICROSCOPY SOCIETY OF AMERICA, MICROBEAM ANALYSIS SOCIETY, MICROSCOPICAL SOCIETY OF CANADA 2019; 25:690-698. [PMID: 30714562 DOI: 10.1017/s1431927618016306] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Enzyme histochemistry is a valuable histological method which provides a connection between morphology, activity, and spatial localization of investigated enzymes. Even though the method relies purely on arbitrary evaluations performed by the human eye, it is still wildly accepted and used in histo(patho)logy. Texture analysis emerged as an excellent tool for image quantification of subtle differences reflected in both spatial discrepancies and gray level values of pixels. The current study of texture analysis utilizes the gray-level co-occurrence matrix as a method for quantification of differences between ecto-5'-nucleotidase activities in healthy hippocampal tissue and tissue with marked neurodegeneration. We used the angular second moment, contrast (CON), correlation, inverse difference moment (INV), and entropy for texture analysis and receiver operating characteristic analysis with immunoblot and qualitative assessment of enzyme histochemistry as a validation. Our results strongly argue that co-occurrence matrix analysis could be used for the determination of fine differences in the enzyme activities with the possibility to ascribe those differences to regions or specific cell types. In addition, it emerged that INV and CON are especially useful parameters for this type of enzyme histochemistry analysis. We concluded that texture analysis is a reliable method for quantification of this descriptive technique, thus removing biases and adding it a quantitative dimension.
Collapse
Affiliation(s)
- Milorad Dragić
- Department for General Physiology and Biophysics,Faculty of Biology,University of Belgrade,Belgrade,Studentski trg 3,11001 Belgrade,Serbia
| | - Marina Zarić
- Department of Molecular Biology and Endocrinology,Vinča Institute of Nuclear Sciences, University of Belgrade,Mike Petrovića Alasa 12-14,11001 Belgrade,Serbia
| | - Nataša Mitrović
- Department of Molecular Biology and Endocrinology,Vinča Institute of Nuclear Sciences, University of Belgrade,Mike Petrovića Alasa 12-14,11001 Belgrade,Serbia
| | - Nadežda Nedeljković
- Department for General Physiology and Biophysics,Faculty of Biology,University of Belgrade,Belgrade,Studentski trg 3,11001 Belgrade,Serbia
| | - Ivana Grković
- Department of Molecular Biology and Endocrinology,Vinča Institute of Nuclear Sciences, University of Belgrade,Mike Petrovića Alasa 12-14,11001 Belgrade,Serbia
| |
Collapse
|
133
|
Signature maps for automatic identification of prostate cancer from colorimetric analysis of H&E- and IHC-stained histopathological specimens. Sci Rep 2019; 9:6992. [PMID: 31061447 PMCID: PMC6502869 DOI: 10.1038/s41598-019-43486-y] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2019] [Accepted: 04/24/2019] [Indexed: 01/16/2023] Open
Abstract
Prostate cancer (PCa) is a major cause of cancer death among men. The histopathological examination of post-surgical prostate specimens and manual annotation of PCa not only allow for detailed assessment of disease characteristics and extent, but also supply the ground truth for developing of computer-aided diagnosis (CAD) systems for PCa detection before definitive treatment. As manual cancer annotation is tedious and subjective, there have been a number of publications describing methods for automating the procedure via the analysis of digitized whole-slide images (WSIs). However, these studies have focused only on the analysis of WSIs stained with hematoxylin and eosin (H&E), even though there is additional information that could be obtained from immunohistochemical (IHC) staining. In this work, we propose a framework for automating the annotation of PCa that is based on automated colorimetric analysis of both H&E and IHC WSIs stained with a triple-antibody cocktail against high-molecular weight cytokeratin (HMWCK), p63, and α-methylacyl CoA racemase (AMACR). The analysis outputs were then used to train a regression model to estimate the distribution of cancerous epithelium within slides. The approach yielded an AUC of 0.951, sensitivity of 87.1%, and specificity of 90.7% as compared to slide-level annotations, and generalized well to cancers of all grades.
Collapse
|
134
|
Qaiser T, Tsang YW, Taniyama D, Sakamoto N, Nakane K, Epstein D, Rajpoot N. Fast and accurate tumor segmentation of histology images using persistent homology and deep convolutional features. Med Image Anal 2019; 55:1-14. [PMID: 30991188 DOI: 10.1016/j.media.2019.03.014] [Citation(s) in RCA: 78] [Impact Index Per Article: 15.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2018] [Revised: 03/27/2019] [Accepted: 03/30/2019] [Indexed: 12/17/2022]
Abstract
Tumor segmentation in whole-slide images of histology slides is an important step towards computer-assisted diagnosis. In this work, we propose a tumor segmentation framework based on the novel concept of persistent homology profiles (PHPs). For a given image patch, the homology profiles are derived by efficient computation of persistent homology, which is an algebraic tool from homology theory. We propose an efficient way of computing topological persistence of an image, alternative to simplicial homology. The PHPs are devised to distinguish tumor regions from their normal counterparts by modeling the atypical characteristics of tumor nuclei. We propose two variants of our method for tumor segmentation: one that targets speed without compromising accuracy and the other that targets higher accuracy. The fast version is based on a selection of exemplar image patches from a convolution neural network (CNN) and patch classification by quantifying the divergence between the PHPs of exemplars and the input image patch. Detailed comparative evaluation shows that the proposed algorithm is significantly faster than competing algorithms while achieving comparable results. The accurate version combines the PHPs and high-level CNN features and employs a multi-stage ensemble strategy for image patch labeling. Experimental results demonstrate that the combination of PHPs and CNN features outperform competing algorithms. This study is performed on two independently collected colorectal datasets containing adenoma, adenocarcinoma, signet, and healthy cases. Collectively, the accurate tumor segmentation produces the highest average patch-level F1-score, as compared with competing algorithms, on malignant and healthy cases from both the datasets. Overall the proposed framework highlights the utility of persistent homology for histopathology image analysis.
Collapse
Affiliation(s)
- Talha Qaiser
- Department of Computer Science, University of Warwick, UK.
| | - Yee-Wah Tsang
- Department of Pathology, University Hospitals Coventry and Warwickshire, UK
| | - Daiki Taniyama
- Department of Molecular Pathology, Hiroshima University Institute of Biomedical and Health Sciences, Japan
| | - Naoya Sakamoto
- Department of Molecular Pathology, Hiroshima University Institute of Biomedical and Health Sciences, Japan
| | - Kazuaki Nakane
- Graduate School of Medicine, Division of Health Science, Osaka University, Japan
| | | | - Nasir Rajpoot
- Department of Computer Science, University of Warwick, UK; Department of Pathology, University Hospitals Coventry and Warwickshire, UK; The Alan Turing Institute, UK.
| |
Collapse
|
135
|
Dealing with Lack of Training Data for Convolutional Neural Networks: The Case of Digital Pathology. ELECTRONICS 2019. [DOI: 10.3390/electronics8030256] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Thanks to their capability to learn generalizable descriptors directly from images, deep Convolutional Neural Networks (CNNs) seem the ideal solution to most pattern recognition problems. On the other hand, to learn the image representation, CNNs need huge sets of annotated samples that are unfeasible in many every-day scenarios. This is the case, for example, of Computer-Aided Diagnosis (CAD) systems for digital pathology, where additional challenges are posed by the high variability of the cancerous tissue characteristics. In our experiments, state-of-the-art CNNs trained from scratch on histological images were less accurate and less robust to variability than a traditional machine learning framework, highlighting all the issues of fully training deep networks with limited data from real patients. To solve this problem, we designed and compared three transfer learning frameworks, leveraging CNNs pre-trained on non-medical images. This approach obtained very high accuracy, requiring much less computational resource for the training. Our findings demonstrate that transfer learning is a solution to the automated classification of histological samples and solves the problem of designing accurate and computationally-efficient CAD systems with limited training data.
Collapse
|
136
|
Kather JN, Krisam J, Charoentong P, Luedde T, Herpel E, Weis CA, Gaiser T, Marx A, Valous NA, Ferber D, Jansen L, Reyes-Aldasoro CC, Zörnig I, Jäger D, Brenner H, Chang-Claude J, Hoffmeister M, Halama N. Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study. PLoS Med 2019; 16:e1002730. [PMID: 30677016 PMCID: PMC6345440 DOI: 10.1371/journal.pmed.1002730] [Citation(s) in RCA: 394] [Impact Index Per Article: 78.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/23/2018] [Accepted: 12/17/2018] [Indexed: 11/19/2022] Open
Abstract
BACKGROUND For virtually every patient with colorectal cancer (CRC), hematoxylin-eosin (HE)-stained tissue slides are available. These images contain quantitative information, which is not routinely used to objectively extract prognostic biomarkers. In the present study, we investigated whether deep convolutional neural networks (CNNs) can extract prognosticators directly from these widely available images. METHODS AND FINDINGS We hand-delineated single-tissue regions in 86 CRC tissue slides, yielding more than 100,000 HE image patches, and used these to train a CNN by transfer learning, reaching a nine-class accuracy of >94% in an independent data set of 7,180 images from 25 CRC patients. With this tool, we performed automated tissue decomposition of representative multitissue HE images from 862 HE slides in 500 stage I-IV CRC patients in the The Cancer Genome Atlas (TCGA) cohort, a large international multicenter collection of CRC tissue. Based on the output neuron activations in the CNN, we calculated a "deep stroma score," which was an independent prognostic factor for overall survival (OS) in a multivariable Cox proportional hazard model (hazard ratio [HR] with 95% confidence interval [CI]: 1.99 [1.27-3.12], p = 0.0028), while in the same cohort, manual quantification of stromal areas and a gene expression signature of cancer-associated fibroblasts (CAFs) were only prognostic in specific tumor stages. We validated these findings in an independent cohort of 409 stage I-IV CRC patients from the "Darmkrebs: Chancen der Verhütung durch Screening" (DACHS) study who were recruited between 2003 and 2007 in multiple institutions in Germany. Again, the score was an independent prognostic factor for OS (HR 1.63 [1.14-2.33], p = 0.008), CRC-specific OS (HR 2.29 [1.5-3.48], p = 0.0004), and relapse-free survival (RFS; HR 1.92 [1.34-2.76], p = 0.0004). A prospective validation is required before this biomarker can be implemented in clinical workflows. CONCLUSIONS In our retrospective study, we show that a CNN can assess the human tumor microenvironment and predict prognosis directly from histopathological images.
Collapse
Affiliation(s)
- Jakob Nikolas Kather
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany
- German Cancer Consortium (DKTK), Heidelberg, Germany
- Applied Tumor Immunity, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Division of Gastroenterology, Hepatology and Hepatobiliary Oncology, University Hospital RWTH Aachen, Aachen, Germany
| | - Johannes Krisam
- Institute of Medical Biometry and Informatics, University Hospital Heidelberg, Heidelberg, Germany
| | - Pornpimol Charoentong
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany
- Applied Tumor Immunity, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Tom Luedde
- Division of Gastroenterology, Hepatology and Hepatobiliary Oncology, University Hospital RWTH Aachen, Aachen, Germany
| | - Esther Herpel
- Institute of Pathology, Heidelberg University, Heidelberg, Germany
- Tissue Bank of the National Center for Tumor Diseases (NCT), Heidelberg, Germany
| | - Cleo-Aron Weis
- Institute of Pathology, University Medical Center Mannheim, Mannheim, Germany
| | - Timo Gaiser
- Institute of Pathology, University Medical Center Mannheim, Mannheim, Germany
| | - Alexander Marx
- Institute of Pathology, University Medical Center Mannheim, Mannheim, Germany
| | - Nektarios A Valous
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany
- Applied Tumor Immunity, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Dyke Ferber
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany
- Applied Tumor Immunity, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Lina Jansen
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | | | - Inka Zörnig
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany
- Applied Tumor Immunity, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Dirk Jäger
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany
- German Cancer Consortium (DKTK), Heidelberg, Germany
- Applied Tumor Immunity, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Hermann Brenner
- German Cancer Consortium (DKTK), Heidelberg, Germany
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Division of Preventive Oncology, German Cancer Research Center (DKFZ) and National Center for Tumor Diseases (NCT), Heidelberg, Germany
| | - Jenny Chang-Claude
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Michael Hoffmeister
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Niels Halama
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany
- German Cancer Consortium (DKTK), Heidelberg, Germany
- Applied Tumor Immunity, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Translational Immunotherapy, German Cancer Research Center (DKFZ), Heidelberg, Germany
| |
Collapse
|
137
|
Segato Dos Santos LF, Neves LA, Rozendo GB, Ribeiro MG, Zanchetta do Nascimento M, Azevedo Tosta TA. Multidimensional and fuzzy sample entropy (SampEn MF) for quantifying H&E histological images of colorectal cancer. Comput Biol Med 2018; 103:148-160. [PMID: 30368171 DOI: 10.1016/j.compbiomed.2018.10.013] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2018] [Revised: 09/22/2018] [Accepted: 10/13/2018] [Indexed: 12/23/2022]
Abstract
In this study, we propose to use a method based on the combination of sample entropy with multiscale and multidimensional approaches, along with a fuzzy function. The model was applied to quantify and classify H&E histological images of colorectal cancer. The multiscale approach was defined by analysing windows of different sizes and variations in tolerance for determining pattern similarity. The multidimensional strategy was performed by considering each pixel in the colour image as an n-dimensional vector, which was analysed from the Minkowski distance. The fuzzy strategy was a Gaussian function used to verify the pertinence of the distances between windows. The result was a method capable of computing similarities between pixels contained in windows of various sizes, as well as the information present in the colour channels. The power of quantification was tested in a public colorectal image dataset, which was composed of both benign and malignant classes. The results were given as inputs for classifiers of different categories and analysed by applying the k-fold cross-validation and holdout methods. The derived performances indicate that the proposed association was capable of distinguishing the benign and malignant groups, with values that surpassed those results obtained with important techniques available in the Literature. The best performance was an AUC value of 0.983, an important result, mainly when we consider the difficulties of clinical practice for the diagnosis of the colorectal cancer.
Collapse
Affiliation(s)
- Luiz Fernando Segato Dos Santos
- Department of Computer Science and Statistics (DCCE), São Paulo State University (UNESP), Rua Cristóvão Colombo, 2265, 15054-000, São José do Rio Preto, São Paulo, Brazil.
| | - Leandro Alves Neves
- Department of Computer Science and Statistics (DCCE), São Paulo State University (UNESP), Rua Cristóvão Colombo, 2265, 15054-000, São José do Rio Preto, São Paulo, Brazil.
| | - Guilherme Botazzo Rozendo
- Department of Computer Science and Statistics (DCCE), São Paulo State University (UNESP), Rua Cristóvão Colombo, 2265, 15054-000, São José do Rio Preto, São Paulo, Brazil.
| | - Matheus Gonçalves Ribeiro
- Department of Computer Science and Statistics (DCCE), São Paulo State University (UNESP), Rua Cristóvão Colombo, 2265, 15054-000, São José do Rio Preto, São Paulo, Brazil.
| | - Marcelo Zanchetta do Nascimento
- Faculty of Computation (FACOM), Federal University of Uberlândia (UFU), Avenida João Neves de Ávila 2121, Bl.B, 38400-902, Uberlândia, Minas Gerais, Brazil.
| | - Thaína Aparecida Azevedo Tosta
- Center of Mathematics, Computing and Cognition, Federal University of ABC (UFABC), Avenida dos Estados, 5001, 09210-580, Santo André, São Paulo, Brazil.
| |
Collapse
|
138
|
Tiwari S. An Analysis in Tissue Classification for Colorectal Cancer Histology Using Convolution Neural Network and Colour Models. INTERNATIONAL JOURNAL OF INFORMATION SYSTEM MODELING AND DESIGN 2018. [DOI: 10.4018/ijismd.2018100101] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Computer vision-based identification of different tissue categories in histological images is a critical application of the computer-assisted diagnosis (CAD). Computer-assisted diagnosis systems support to reduce the cost and increase the efficiency of this process. Traditional image classification approaches depend on feature extraction methods designed for a specific problem based on domain information. Deep learning approaches are becoming important alternatives with advance of machine learning technologies to overcome the numerous difficulties of the feature-based approaches. A method for the classification of histological images of human colorectal cancer containing seven different types of tissue using convolutional neural network (CNN) is proposed in this article. The method is evaluated using four different colour models in absence and presence of Gaussian noise. The highest classification accuracies are achieved with HVI colour model, which is 95.8% in nonexistence and 78.5% in existence of noise respectively.
Collapse
|
139
|
Kather JN, Suarez-Carmona M, Charoentong P, Weis CA, Hirsch D, Bankhead P, Horning M, Ferber D, Kel I, Herpel E, Schott S, Zörnig I, Utikal J, Marx A, Gaiser T, Brenner H, Chang-Claude J, Hoffmeister M, Jäger D, Halama N. Topography of cancer-associated immune cells in human solid tumors. eLife 2018; 7:36967. [PMID: 30179157 PMCID: PMC6133554 DOI: 10.7554/elife.36967] [Citation(s) in RCA: 171] [Impact Index Per Article: 28.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2018] [Accepted: 08/30/2018] [Indexed: 12/14/2022] Open
Abstract
Lymphoid and myeloid cells are abundant in the tumor microenvironment, can be quantified by immunohistochemistry and shape the disease course of human solid tumors. Yet, there is no comprehensive understanding of spatial immune infiltration patterns (‘topography’) across cancer entities and across various immune cell types. In this study, we systematically measure the topography of multiple immune cell types in 965 histological tissue slides from N = 177 patients in a pan-cancer cohort. We provide a definition of inflamed (‘hot’), non-inflamed (‘cold’) and immune excluded patterns and investigate how these patterns differ between immune cell types and between cancer types. In an independent cohort of N = 287 colorectal cancer patients, we show that hot, cold and excluded topographies for effector lymphocytes (CD8) and tumor-associated macrophages (CD163) alone are not prognostic, but that a bivariate classification system can stratify patients. Our study adds evidence to consider immune topographies as biomarkers for patients with solid tumors.
Collapse
Affiliation(s)
- Jakob Nikolas Kather
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany.,German Cancer Consortium, Heidelberg, Germany.,Applied Tumor Immunity, German Cancer Research Center, Heidelberg, Germany.,Division of Gastroenterology, Hepatology and Hepatobiliary Oncology, University Hospital RWTH Aachen, Aachen, Germany
| | - Meggy Suarez-Carmona
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany.,Applied Tumor Immunity, German Cancer Research Center, Heidelberg, Germany
| | - Pornpimol Charoentong
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany.,Applied Tumor Immunity, German Cancer Research Center, Heidelberg, Germany
| | - Cleo-Aron Weis
- Department of Pathology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany
| | - Daniela Hirsch
- Department of Pathology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany
| | - Peter Bankhead
- Northern Ireland Molecular Pathology Laboratory, Centre for Cancer Research and Cell Biology, Queen's University Belfast, Northern Ireland, United Kingdom
| | - Marcel Horning
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany
| | - Dyke Ferber
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany.,Applied Tumor Immunity, German Cancer Research Center, Heidelberg, Germany
| | - Ivan Kel
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany
| | - Esther Herpel
- Institute of Pathology, University Hospital Heidelberg, Heidelberg, Germany.,Tissue Bank of the National Center for Tumor Diseases, Heidelberg, Germany
| | - Sarah Schott
- Department of Gynecology, University Hospital Heidelberg, Heidelberg, Germany
| | - Inka Zörnig
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany.,Applied Tumor Immunity, German Cancer Research Center, Heidelberg, Germany
| | - Jochen Utikal
- Skin Cancer Unit, German Cancer Research Center, Heidelberg, Germany.,Department of Dermatology, Venereology and Allergology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany
| | - Alexander Marx
- Department of Pathology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany
| | - Timo Gaiser
- Department of Pathology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany
| | - Herrmann Brenner
- German Cancer Consortium, Heidelberg, Germany.,Division of Clinical Epidemiology and Aging Research, German Cancer Research Center, Heidelberg, Germany.,Division of Preventive Oncology, German Cancer Research Center and National Center for Tumor Diseases, Heidelberg, Germany
| | - Jenny Chang-Claude
- Division of Cancer Epidemiology, German Cancer Research Centre, Heidelberg, Germany.,Cancer Epidemiology Group, University Cancer Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Michael Hoffmeister
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center, Heidelberg, Germany
| | - Dirk Jäger
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany.,German Cancer Consortium, Heidelberg, Germany.,Applied Tumor Immunity, German Cancer Research Center, Heidelberg, Germany
| | - Niels Halama
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany.,German Cancer Consortium, Heidelberg, Germany.,Applied Tumor Immunity, German Cancer Research Center, Heidelberg, Germany
| |
Collapse
|
140
|
Van Eycke YR, Balsat C, Verset L, Debeir O, Salmon I, Decaestecker C. Segmentation of glandular epithelium in colorectal tumours to automatically compartmentalise IHC biomarker quantification: A deep learning approach. Med Image Anal 2018; 49:35-45. [PMID: 30081241 DOI: 10.1016/j.media.2018.07.004] [Citation(s) in RCA: 36] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2017] [Revised: 06/29/2018] [Accepted: 07/05/2018] [Indexed: 12/18/2022]
Abstract
In this paper, we propose a method for automatically annotating slide images from colorectal tissue samples. Our objective is to segment glandular epithelium in histological images from tissue slides submitted to different staining techniques, including usual haematoxylin-eosin (H&E) as well as immunohistochemistry (IHC). The proposed method makes use of Deep Learning and is based on a new convolutional network architecture. Our method achieves better performances than the state of the art on the H&E images of the GlaS challenge contest, whereas it uses only the haematoxylin colour channel extracted by colour deconvolution from the RGB images in order to extend its applicability to IHC. The network only needs to be fine-tuned on a small number of additional examples to be accurate on a new IHC dataset. Our approach also includes a new method of data augmentation to achieve good generalisation when working with different experimental conditions and different IHC markers. We show that our methodology enables to automate the compartmentalisation of the IHC biomarker analysis, results concurring highly with manual annotations.
Collapse
Affiliation(s)
- Yves-Rémi Van Eycke
- DIAPath, Center for Microscopy and Molecular Imaging, Université Libre de Bruxelles (ULB), CPI 305/1, Rue Adrienne Bolland, 8, 6041 Gosselies, Belgium; Laboratories of Image, Signal processing & Acoustics, Université Libre de Bruxelles (ULB), CPI 165/57, Avenue Franklin Roosevelt 50, Brussels 1050 , Belgium.
| | - Cédric Balsat
- DIAPath, Center for Microscopy and Molecular Imaging, Université Libre de Bruxelles (ULB), CPI 305/1, Rue Adrienne Bolland, 8, 6041 Gosselies, Belgium
| | - Laurine Verset
- Department of Pathology, Erasme Hospital, Université Libre de Bruxelles (ULB), Route de Lennik 808, Brussels 1070, Belgium
| | - Olivier Debeir
- Laboratories of Image, Signal processing & Acoustics, Université Libre de Bruxelles (ULB), CPI 165/57, Avenue Franklin Roosevelt 50, Brussels 1050 , Belgium; MIP, Center for Microscopy and Molecular Imaging, Université Libre de Bruxelles (ULB), CPI 305/1, Rue Adrienne Bolland, 8, 6041 Gosselies, Belgium
| | - Isabelle Salmon
- DIAPath, Center for Microscopy and Molecular Imaging, Université Libre de Bruxelles (ULB), CPI 305/1, Rue Adrienne Bolland, 8, 6041 Gosselies, Belgium; Department of Pathology, Erasme Hospital, Université Libre de Bruxelles (ULB), Route de Lennik 808, Brussels 1070, Belgium
| | - Christine Decaestecker
- DIAPath, Center for Microscopy and Molecular Imaging, Université Libre de Bruxelles (ULB), CPI 305/1, Rue Adrienne Bolland, 8, 6041 Gosselies, Belgium; Laboratories of Image, Signal processing & Acoustics, Université Libre de Bruxelles (ULB), CPI 165/57, Avenue Franklin Roosevelt 50, Brussels 1050 , Belgium.
| |
Collapse
|
141
|
Qu J, Hiruta N, Terai K, Nosato H, Murakawa M, Sakanashi H. Gastric Pathology Image Classification Using Stepwise Fine-Tuning for Deep Neural Networks. JOURNAL OF HEALTHCARE ENGINEERING 2018; 2018:8961781. [PMID: 30034677 PMCID: PMC6033298 DOI: 10.1155/2018/8961781] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/23/2018] [Revised: 05/14/2018] [Accepted: 05/27/2018] [Indexed: 02/06/2023]
Abstract
Deep learning using convolutional neural networks (CNNs) is a distinguished tool for many image classification tasks. Due to its outstanding robustness and generalization, it is also expected to play a key role to facilitate advanced computer-aided diagnosis (CAD) for pathology images. However, the shortage of well-annotated pathology image data for training deep neural networks has become a major issue at present because of the high-cost annotation upon pathologist's professional observation. Faced with this problem, transfer learning techniques are generally used to reinforcing the capacity of deep neural networks. In order to further boost the performance of the state-of-the-art deep neural networks and alleviate insufficiency of well-annotated data, this paper presents a novel stepwise fine-tuning-based deep learning scheme for gastric pathology image classification and establishes a new type of target-correlative intermediate datasets. Our proposed scheme is deemed capable of making the deep neural network imitating the pathologist's perception manner and of acquiring pathology-related knowledge in advance, but with very limited extra cost in data annotation. The experiments are conducted with both well-annotated gastric pathology data and the proposed target-correlative intermediate data on several state-of-the-art deep neural networks. The results congruously demonstrate the feasibility and superiority of our proposed scheme for boosting the classification performance.
Collapse
Affiliation(s)
- Jia Qu
- Department of Intelligent Interaction Technologies, University of Tsukuba, Tsukuba 305-8573, Japan
| | - Nobuyuki Hiruta
- Department of Surgical Pathology, Toho University Sakura Medical Center, Sakura 285-8741, Japan
| | - Kensuke Terai
- Department of Surgical Pathology, Toho University Sakura Medical Center, Sakura 285-8741, Japan
| | - Hirokazu Nosato
- Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba 305-8560, Japan
| | - Masahiro Murakawa
- Department of Intelligent Interaction Technologies, University of Tsukuba, Tsukuba 305-8573, Japan
- Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba 305-8560, Japan
| | - Hidenori Sakanashi
- Department of Intelligent Interaction Technologies, University of Tsukuba, Tsukuba 305-8573, Japan
- Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba 305-8560, Japan
| |
Collapse
|
142
|
Histopathological image classification with bilinear convolutional neural networks. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2018; 2017:4050-4053. [PMID: 29060786 DOI: 10.1109/embc.2017.8037745] [Citation(s) in RCA: 49] [Impact Index Per Article: 8.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
The computer-aided quantitative analysis for histopathological images has attracted considerable attention. The stain decomposition on histopathological images is usually recommended to address the issue of co-localization or aliasing of tissue substances. Although the convolutional neural networks (CNN) is a popular deep learning algorithm for various tasks on histopathological image analysis, it is only directly performed on histopathological images without considering stain decomposition. The bilinear CNN (BCNN) is a new CNN model for fine-grained classification. BCNN consists of two CNNs, whose convolutional-layer outputs are multiplied with outer product at each spatial location. In this work, we propose a novel BCNN-based method for classification of histopathological images, which first decomposes histopathological images into hematoxylin and eosin stain components, and then perform BCNN on the decomposed images to fuse and improve the feature representation performance. The experimental results on the colorectal cancer histopathological image dataset with eight classes indicate that the proposed BCNN-based algorithm is superior to the traditional CNN.
Collapse
|
143
|
Awan R, Al-Maadeed S, Al-Saady R. Using spectral imaging for the analysis of abnormalities for colorectal cancer: When is it helpful? PLoS One 2018; 13:e0197431. [PMID: 29874262 PMCID: PMC5991384 DOI: 10.1371/journal.pone.0197431] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2017] [Accepted: 05/02/2018] [Indexed: 12/16/2022] Open
Abstract
The spectral imaging technique has been shown to provide more discriminative information than the RGB images and has been proposed for a range of problems. There are many studies demonstrating its potential for the analysis of histopathology images for abnormality detection but there have been discrepancies among previous studies as well. Many multispectral based methods have been proposed for histopathology images but the significance of the use of whole multispectral cube versus a subset of bands or a single band is still arguable. We performed comprehensive analysis using individual bands and different subsets of bands to determine the effectiveness of spectral information for determining the anomaly in colorectal images. Our multispectral colorectal dataset consists of four classes, each represented by infra-red spectrum bands in addition to the visual spectrum bands. We performed our analysis of spectral imaging by stratifying the abnormalities using both spatial and spectral information. For our experiments, we used a combination of texture descriptors with an ensemble classification approach that performed best on our dataset. We applied our method to another dataset and got comparable results with those obtained using the state-of-the-art method and convolutional neural network based method. Moreover, we explored the relationship of the number of bands with the problem complexity and found that higher number of bands is required for a complex task to achieve improved performance. Our results demonstrate a synergy between infra-red and visual spectrum by improving the classification accuracy (by 6%) on incorporating the infra-red representation. We also highlight the importance of how the dataset should be divided into training and testing set for evaluating the histopathology image-based approaches, which has not been considered in previous studies on multispectral histopathology images.
Collapse
Affiliation(s)
- Ruqayya Awan
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
- * E-mail:
| | - Somaya Al-Maadeed
- Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| | | |
Collapse
|
144
|
Multiclass classification for skin cancer profiling based on the integration of heterogeneous gene expression series. PLoS One 2018; 13:e0196836. [PMID: 29750795 PMCID: PMC5947894 DOI: 10.1371/journal.pone.0196836] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2017] [Accepted: 04/20/2018] [Indexed: 11/19/2022] Open
Abstract
Most of the research studies developed applying microarray technology to the characterization of different pathological states of any disease may fail in reaching statistically significant results. This is largely due to the small repertoire of analysed samples, and to the limitation in the number of states or pathologies usually addressed. Moreover, the influence of potential deviations on the gene expression quantification is usually disregarded. In spite of the continuous changes in omic sciences, reflected for instance in the emergence of new Next-Generation Sequencing-related technologies, the existing availability of a vast amount of gene expression microarray datasets should be properly exploited. Therefore, this work proposes a novel methodological approach involving the integration of several heterogeneous skin cancer series, and a later multiclass classifier design. This approach is thus a way to provide the clinicians with an intelligent diagnosis support tool based on the use of a robust set of selected biomarkers, which simultaneously distinguishes among different cancer-related skin states. To achieve this, a multi-platform combination of microarray datasets from Affymetrix and Illumina manufacturers was carried out. This integration is expected to strengthen the statistical robustness of the study as well as the finding of highly-reliable skin cancer biomarkers. Specifically, the designed operation pipeline has allowed the identification of a small subset of 17 differentially expressed genes (DEGs) from which to distinguish among 7 involved skin states. These genes were obtained from the assessment of a number of potential batch effects on the gene expression data. The biological interpretation of these genes was inspected in the specific literature to understand their underlying information in relation to skin cancer. Finally, in order to assess their possible effectiveness in cancer diagnosis, a cross-validation Support Vector Machines (SVM)-based classification including feature ranking was performed. The accuracy attained exceeded the 92% in overall recognition of the 7 different cancer-related skin states. The proposed integration scheme is expected to allow the co-integration with other state-of-the-art technologies such as RNA-seq.
Collapse
|
145
|
Nanni L, Brahnam S, Ghidoni S, Lumini A. Bioimage Classification with Handcrafted and Learned Features. IEEE/ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS 2018; 16:874-885. [PMID: 29994096 DOI: 10.1109/tcbb.2018.2821127] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Bioimage classification is increasingly becoming more important in many biological studies including those that require accurate cell phenotype recognition, subcellular localization, and histopathological classification. In this paper, we present a new General Purpose (GenP) bioimage classification method that can be applied to a large range of classification problems. The GenP system we propose is an ensemble that combines multiple texture features (both handcrafted and learned descriptors) for superior and generalizable discriminative power. Our ensemble obtains a boosting of performance by combining local features, dense sampling features, and deep learning features. Each descriptor is used to train a different Support Vector Machine that is then combined by sum rule. We evaluate our method on a diverse set of bioimage classification tasks each represented by a benchmark database, including some of those available in the IICBU 2008 database. Each bioimage classification task represents a typical subcellular, cellular, and tissue level classification problem. Our evaluation on these datasets demonstrates that the proposed GenP bioimage ensemble obtains state-of-the-art performance without any ad-hoc dataset tuning of the parameters (thereby avoiding any risk of overfitting/overtraining). To reproduce the experiments reported in this paper, the MATLAB code of all the descriptors is available at https://github.com/LorisNanni and https://www.dropbox.com/s/bguw035yrqz0pwp/ElencoCode.docx?dl=0.
Collapse
|
146
|
Kather JN, Halama N, Jaeger D. Genomics and emerging biomarkers for immunotherapy of colorectal cancer. Semin Cancer Biol 2018; 52:189-197. [PMID: 29501787 DOI: 10.1016/j.semcancer.2018.02.010] [Citation(s) in RCA: 95] [Impact Index Per Article: 15.8] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2017] [Revised: 02/19/2018] [Accepted: 02/28/2018] [Indexed: 02/06/2023]
Abstract
Colorectal cancer (CRC) is a common and lethal disease with a high therapeutic need. For most patients with metastatic CRC, chemotherapy is the only viable option. Currently, immunotherapy is restricted to the particular genetic subgroup of mismatch-repair deficient (MMRd)/microsatellite instable (MSI) CRC. Anti-PD1 therapy was recently FDA-approved as a second-line treatment in this subgroup. However, in a metastatic setting, these MMRd/MSI tumors are vastly outnumbered by mismatch-repair proficient (MMRp)/microsatellite stable (MSS) tumors. These MMRp/MSS tumors do not meaningfully respond to any traditional immunotherapy approach including checkpoint blockade, adoptive cell transfer and vaccination. This resistance to immunotherapy is due to a complex tumor microenvironment that counteracts antitumor immunity through a combination of poorly antigenic tumor cells and an immunosuppressive tumor microenvironment. To find ways of overcoming immunotherapy resistance in the majority of CRC patients, it is necessary to analyze the immunological makeup in an in-depth and personalized way and in the context of their tumor genetic makeup. Flexible, biomarker-guided early-phase immunotherapy trials are needed to optimize this workflow. In this review, we detail key mechanisms for immune evasion and emerging immune biomarkers for personalized immunotherapy in CRC. Also, we present a template for biomarker-guided clinical trials that are needed to move new immunotherapy approaches closer to clinical application.
Collapse
Affiliation(s)
- Jakob Nikolas Kather
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany; German Cancer Consortium (DKTK), Heidelberg, Germany; Applied Tumor Immunity, German Cancer Research Center (DKFZ), Heidelberg, Germany.
| | - Niels Halama
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany; German Cancer Consortium (DKTK), Heidelberg, Germany; Applied Tumor Immunity, German Cancer Research Center (DKFZ), Heidelberg, Germany.
| | - Dirk Jaeger
- Department of Medical Oncology and Internal Medicine VI, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany; German Cancer Consortium (DKTK), Heidelberg, Germany; Applied Tumor Immunity, German Cancer Research Center (DKFZ), Heidelberg, Germany.
| |
Collapse
|
147
|
Komura D, Ishikawa S. Machine Learning Methods for Histopathological Image Analysis. Comput Struct Biotechnol J 2018; 16:34-42. [PMID: 30275936 PMCID: PMC6158771 DOI: 10.1016/j.csbj.2018.01.001] [Citation(s) in RCA: 354] [Impact Index Per Article: 59.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2017] [Revised: 12/03/2017] [Accepted: 01/14/2018] [Indexed: 12/12/2022] Open
Abstract
Abundant accumulation of digital histopathological images has led to the increased demand for their analysis, such as computer-aided diagnosis using machine learning techniques. However, digital pathological images and related tasks have some issues to be considered. In this mini-review, we introduce the application of digital pathological image analysis using machine learning algorithms, address some problems specific to such analysis, and propose possible solutions.
Collapse
Affiliation(s)
- Daisuke Komura
- Department of Genomic Pathology, Medical Research Institute, Tokyo Medical and Dental University, Tokyo, Japan
| | | |
Collapse
|
148
|
Sarkar R, Acton ST. SDL: Saliency-Based Dictionary Learning Framework for Image Similarity. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2018; 27:749-763. [PMID: 29053460 DOI: 10.1109/tip.2017.2763829] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
In image classification, obtaining adequate data to learn a robust classifier has often proven to be difficult in several scenarios. Classification of histological tissue images for health care analysis is a notable application in this context due to the necessity of surgery, biopsy or autopsy. To adequately exploit limited training data in classification, we propose a saliency guided dictionary learning method and subsequently an image similarity technique for histo-pathological image classification. Salient object detection from images aids in the identification of discriminative image features. We leverage the saliency values for the local image regions to learn a dictionary and respective sparse codes for an image, such that the more salient features are reconstructed with smaller error. The dictionary learned from an image gives a compact representation of the image itself and is capable of representing images with similar content, with comparable sparse codes. We employ this idea to design a similarity measure between a pair of images, where local image features of one image, are encoded with the dictionary learned from the other and vice versa. To effectively utilize the learned dictionary, we take into account the contribution of each dictionary atom in the sparse codes to generate a global image representation for image comparison. The efficacy of the proposed method was evaluated using three tissue data sets that consist of mammalian kidney, lung and spleen tissue, breast cancer, and colon cancer tissue images. From the experiments, we observe that our methods outperform the state of the art with an increase of 14.2% in the average classification accuracy over all data sets.
Collapse
|
149
|
Automatic classification of colorectal and prostatic histologic tumor images using multiscale multispectral local binary pattern texture features and stacked generalization. Neurocomputing 2018. [DOI: 10.1016/j.neucom.2017.05.010] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
|
150
|
Tesic V, Perovic M, Zaletel I, Jovanovic M, Puskas N, Ruzdijic S, Kanazir S. A single high dose of dexamethasone increases GAP-43 and synaptophysin in the hippocampus of aged rats. Exp Gerontol 2017; 98:62-69. [DOI: 10.1016/j.exger.2017.08.010] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2017] [Revised: 07/06/2017] [Accepted: 08/08/2017] [Indexed: 01/14/2023]
|