101
|
Rączkowska A, Możejko M, Zambonelli J, Szczurek E. ARA: accurate, reliable and active histopathological image classification framework with Bayesian deep learning. Sci Rep 2019; 9:14347. [PMID: 31586139 PMCID: PMC6778075 DOI: 10.1038/s41598-019-50587-1] [Citation(s) in RCA: 48] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2019] [Accepted: 09/16/2019] [Indexed: 02/07/2023] Open
Abstract
Machine learning algorithms hold the promise to effectively automate the analysis of histopathological images that are routinely generated in clinical practice. Any machine learning method used in the clinical diagnostic process has to be extremely accurate and, ideally, provide a measure of uncertainty for its predictions. Such accurate and reliable classifiers need enough labelled data for training, which requires time-consuming and costly manual annotation by pathologists. Thus, it is critical to minimise the amount of data needed to reach the desired accuracy by maximising the efficiency of training. We propose an accurate, reliable and active (ARA) image classification framework and introduce a new Bayesian Convolutional Neural Network (ARA-CNN) for classifying histopathological images of colorectal cancer. The model achieves exceptional classification accuracy, outperforming other models trained on the same dataset. The network outputs an uncertainty measurement for each tested image. We show that uncertainty measures can be used to detect mislabelled training samples and can be employed in an efficient active learning workflow. Using a variational dropout-based entropy measure of uncertainty in the workflow speeds up the learning process by roughly 45%. Finally, we utilise our model to segment whole-slide images of colorectal tissue and compute segmentation-based spatial statistics.
Collapse
Affiliation(s)
- Alicja Rączkowska
- Faculty of Mathematics, Informatics and Mechanics, University of Warsaw, Warsaw, Poland
| | - Marcin Możejko
- Faculty of Mathematics, Informatics and Mechanics, University of Warsaw, Warsaw, Poland
| | - Joanna Zambonelli
- Department of Pathology, Medical University of Warsaw, Warsaw, Poland
| | - Ewa Szczurek
- Faculty of Mathematics, Informatics and Mechanics, University of Warsaw, Warsaw, Poland.
| |
Collapse
|
102
|
Microvascularity detection and quantification in glioma: a novel deep-learning-based framework. J Transl Med 2019; 99:1515-1526. [PMID: 31201368 DOI: 10.1038/s41374-019-0272-3] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2018] [Revised: 03/25/2019] [Accepted: 04/18/2019] [Indexed: 01/08/2023] Open
Abstract
Microvascularity is highly correlated with the grading and subtyping of gliomas, making this one of its most important histological features. Accurate quantitative analysis of microvessels is helpful for the development of a targeted therapy for antiangiogenesis. The deep-learning algorithm is by far the most effective segmentation and detection model and enables location and recognition of complex microvascular networks in large images obtained from hematoxylin and eosin (H&E) stained specimens. We proposed an automated deep-learning-based method to detect and quantify the microvascularity in glioma and applied it to comprehensive clinical analyses. A total of 350 glioma patients were enrolled in our study, for which digitalized imaging of H&E stained slides were reviewed, molecular diagnosis was performed and follow-up was investigated. The microvascular features were compared according to their histologic types, molecular types, and patients' prognosis. The results show that the proposed method can quantify microvascular characteristics automatically and effectively. Significant increases of microvascular density and microvascular area were observed in glioblastomas (95% p < 0.001 in density, 170% p < 0.001 in area) in comparison with other histologic types; increases were also observed in cases with TERT-mut only (68% p < 0.001 in density, 54% p < 0.001 in area) compared with other molecular types. Survival analysis showed that microvascular features can be used to cluster cases into two groups with different survival periods (hazard ratio [HR] 2.843, log-rank <0.001), which indicates the quantified microvascular features may potentially be alternative signatures for revealing patients' prognosis. This deep-learning-based method may be a useful tool in routine clinical practice for precise diagnosis and antiangiogenic treatment.
Collapse
|
103
|
DeNeRD: high-throughput detection of neurons for brain-wide analysis with deep learning. Sci Rep 2019; 9:13828. [PMID: 31554830 PMCID: PMC6761257 DOI: 10.1038/s41598-019-50137-9] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2019] [Accepted: 09/03/2019] [Indexed: 11/08/2022] Open
Abstract
Mapping the structure of the mammalian brain at cellular resolution is a challenging task and one that requires capturing key anatomical features at the appropriate level of analysis. Although neuroscientific methods have managed to provide significant insights at the micro and macro level, in order to obtain a whole-brain analysis at a cellular resolution requires a meso-scopic approach. A number of methods can be currently used to detect and count cells, with, nevertheless, significant limitations when analyzing data of high complexity. To overcome some of these constraints, we introduce a fully automated Artificial Intelligence (AI)-based method for whole-brain image processing to Detect Neurons in different brain Regions during Development (DeNeRD). We demonstrate a high performance of our deep neural network in detecting neurons labeled with different genetic markers in a range of imaging planes and imaging modalities.
Collapse
|
104
|
Xing F, Xie Y, Shi X, Chen P, Zhang Z, Yang L. Towards pixel-to-pixel deep nucleus detection in microscopy images. BMC Bioinformatics 2019; 20:472. [PMID: 31521104 PMCID: PMC6744696 DOI: 10.1186/s12859-019-3037-5] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2018] [Accepted: 08/21/2019] [Indexed: 12/21/2022] Open
Abstract
BACKGROUND Nucleus is a fundamental task in microscopy image analysis and supports many other quantitative studies such as object counting, segmentation, tracking, etc. Deep neural networks are emerging as a powerful tool for biomedical image computing; in particular, convolutional neural networks have been widely applied to nucleus/cell detection in microscopy images. However, almost all models are tailored for specific datasets and their applicability to other microscopy image data remains unknown. Some existing studies casually learn and evaluate deep neural networks on multiple microscopy datasets, but there are still several critical, open questions to be addressed. RESULTS We analyze the applicability of deep models specifically for nucleus detection across a wide variety of microscopy image data. More specifically, we present a fully convolutional network-based regression model and extensively evaluate it on large-scale digital pathology and microscopy image datasets, which consist of 23 organs (or cancer diseases) and come from multiple institutions. We demonstrate that for a specific target dataset, training with images from the same types of organs might be usually necessary for nucleus detection. Although the images can be visually similar due to the same staining technique and imaging protocol, deep models learned with images from different organs might not deliver desirable results and would require model fine-tuning to be on a par with those trained with target data. We also observe that training with a mixture of target and other/non-target data does not always mean a higher accuracy of nucleus detection, and it might require proper data manipulation during model training to achieve good performance. CONCLUSIONS We conduct a systematic case study on deep models for nucleus detection in a wide variety of microscopy images, aiming to address several important but previously understudied questions. We present and extensively evaluate an end-to-end, pixel-to-pixel fully convolutional regression network and report a few significant findings, some of which might have not been reported in previous studies. The model performance analysis and observations would be helpful to nucleus detection in microscopy images.
Collapse
Affiliation(s)
- Fuyong Xing
- Department of Biostatistics and Informatics, and the Data Science to Patient Value initiative, University of Colorado Anschutz Medical Campus, 13001 E 17th Pl, Aurora, Colorado 80045, United States
| | - Yuanpu Xie
- J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida, 1275 Center Drive, Gainesville, Florida 32611, United States
| | - Xiaoshuang Shi
- J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida, 1275 Center Drive, Gainesville, Florida 32611, United States
| | - Pingjun Chen
- J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida, 1275 Center Drive, Gainesville, Florida 32611, United States
| | - Zizhao Zhang
- Department of Computer and Information Science and Engineering, University of Florida, 432 Newell Drive, Gainesville, Florida 32611, United States
| | - Lin Yang
- J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida, 1275 Center Drive, Gainesville, Florida 32611, United States
- Department of Computer and Information Science and Engineering, University of Florida, 432 Newell Drive, Gainesville, Florida 32611, United States
| |
Collapse
|
105
|
Yin W, Hu Y, Yi S, He J. A segmentation method combining probability map and boundary based on multiple fully convolutional networks and repetitive training. Phys Med Biol 2019; 64:185003. [PMID: 30808019 DOI: 10.1088/1361-6560/ab0a90] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
Cell nuclei image segmentation technology can help researchers observe each cell's stress response to drug treatment. However, it is still a challenge to accurately segment the adherent cell nuclei. At present, image segmentation based on a fully convolutional network (FCN) is attracting researchers' attention. We propose a multiple FCN architecture and repetitive training (M-FCN-RT) method to learn features of cell nucleus images. In M-FCN-RT, the multiple FCN (M-FCN) architecture is composed of several single FCNs (S-FCNs) with the same structure, and each FCN is used to learn the specific features of image datasets. In this paper, the M-FCN contains three FCNs; FCN1-2, FCN3 and FCNB. FCN1-2 and FCN3 are respectively used to learn the spatial features of cell nuclei for generating probability maps to indicate nucleus regions of an image; FCNB (boundary FCN) is used to learn the edge features of cell nuclei for generating the nucleus boundary. For the training of each FCN, we propose a repetitive training (RT) method to improve the classification accuracy of the model. To segment cell nuclei, we finally propose an algorithm combining the probability map and boundary (PMB) to segment the adherent nuclei. This paper uses a public opening nucleus image dataset to train, verify and evaluate the proposed M-FCN-RT and PMB methods. Our M-FCN-RT method achieves a high Dice similarity coefficient (DSC) of 92.11%, 95.64% and 87.99% on the three types of sub-datasets respectively for probability maps. In addition, segmentation experimental results show the PMB method is more effective and efficient compared with other methods.
Collapse
Affiliation(s)
- Wenshe Yin
- School of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, People's Republic of China
| | | | | | | |
Collapse
|
106
|
Caicedo JC, Roth J, Goodman A, Becker T, Karhohs KW, Broisin M, Molnar C, McQuin C, Singh S, Theis FJ, Carpenter AE. Evaluation of Deep Learning Strategies for Nucleus Segmentation in Fluorescence Images. Cytometry A 2019; 95:952-965. [PMID: 31313519 PMCID: PMC6771982 DOI: 10.1002/cyto.a.23863] [Citation(s) in RCA: 132] [Impact Index Per Article: 26.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2018] [Revised: 05/31/2019] [Accepted: 06/23/2019] [Indexed: 12/12/2022]
Abstract
Identifying nuclei is often a critical first step in analyzing microscopy images of cells and classical image processing algorithms are most commonly used for this task. Recent developments in deep learning can yield superior accuracy, but typical evaluation metrics for nucleus segmentation do not satisfactorily capture error modes that are relevant in cellular images. We present an evaluation framework to measure accuracy, types of errors, and computational efficiency; and use it to compare deep learning strategies and classical approaches. We publicly release a set of 23,165 manually annotated nuclei and source code to reproduce experiments and run the proposed evaluation methodology. Our evaluation framework shows that deep learning improves accuracy and can reduce the number of biologically relevant errors by half. © 2019 The Authors. Cytometry Part A published by Wiley Periodicals, Inc. on behalf of International Society for Advancement of Cytometry.
Collapse
Affiliation(s)
- Juan C. Caicedo
- Imaging PlatformBroad Institute of MIT and HarvardCambridgeMassachusetts
| | - Jonathan Roth
- Imaging PlatformBroad Institute of MIT and HarvardCambridgeMassachusetts
- Institute of Computational BiologyGerman Research Center for Environmental HealthMunichGermany
| | - Allen Goodman
- Imaging PlatformBroad Institute of MIT and HarvardCambridgeMassachusetts
| | - Tim Becker
- Imaging PlatformBroad Institute of MIT and HarvardCambridgeMassachusetts
| | - Kyle W. Karhohs
- Imaging PlatformBroad Institute of MIT and HarvardCambridgeMassachusetts
| | - Matthieu Broisin
- Imaging PlatformBroad Institute of MIT and HarvardCambridgeMassachusetts
- Biomedical Imaging GroupEcole polytechnique fédérale de LausanneLausanneSwitzerland
| | - Csaba Molnar
- Imaging PlatformBroad Institute of MIT and HarvardCambridgeMassachusetts
- Biological Research Centre of the Hungarian Academy of SciencesSzegedHungary
| | - Claire McQuin
- Imaging PlatformBroad Institute of MIT and HarvardCambridgeMassachusetts
| | - Shantanu Singh
- Imaging PlatformBroad Institute of MIT and HarvardCambridgeMassachusetts
| | - Fabian J. Theis
- Institute of Computational BiologyGerman Research Center for Environmental HealthMunichGermany
| | - Anne E. Carpenter
- Imaging PlatformBroad Institute of MIT and HarvardCambridgeMassachusetts
| |
Collapse
|
107
|
Tofighi M, Guo T, Vanamala JKP, Monga V. Prior Information Guided Regularized Deep Learning for Cell Nucleus Detection. IEEE TRANSACTIONS ON MEDICAL IMAGING 2019; 38:2047-2058. [PMID: 30703016 DOI: 10.1109/tmi.2019.2895318] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Cell nuclei detection is a challenging research topic because of limitations in cellular image quality and diversity of nuclear morphology, i.e., varying nuclei shapes, sizes, and overlaps between multiple cell nuclei. This has been a topic of enduring interest with promising recent success shown by deep learning methods. These methods train convolutional neural networks (CNNs) with a training set of input images and known, labeled nuclei locations. Many such methods are supplemented by spatial or morphological processing. Using a set of canonical cell nuclei shapes, prepared with the help of a domain expert, we develop a new approach that we call shape priors (SPs) with CNNs (SPs-CNN). We further extend the network to introduce an SP layer and then allowing it to become trainable (i.e., optimizable). We call this network as tunable SP-CNN (TSP-CNN). In summary, we present new network structures that can incorporate "expected behavior" of nucleus shapes via two components: learnable layers that perform the nucleus detection and a fixed processing part that guides the learning with prior information. Analytically, we formulate two new regularization terms that are targeted at: 1) learning the shapes and 2) reducing false positives while simultaneously encouraging detection inside the cell nucleus boundary. Experimental results on two challenging datasets reveal that the proposed SP-CNN and TSP-CNN can outperform the state-of-the-art alternatives.
Collapse
|
108
|
Wang S, Zhu Y, Yu L, Chen H, Lin H, Wan X, Fan X, Heng PA. RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification. Med Image Anal 2019; 58:101549. [PMID: 31499320 DOI: 10.1016/j.media.2019.101549] [Citation(s) in RCA: 82] [Impact Index Per Article: 16.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2019] [Revised: 08/24/2019] [Accepted: 08/29/2019] [Indexed: 12/11/2022]
Abstract
The whole slide histopathology images (WSIs) play a critical role in gastric cancer diagnosis. However, due to the large scale of WSIs and various sizes of the abnormal area, how to select informative regions and analyze them are quite challenging during the automatic diagnosis process. The multi-instance learning based on the most discriminative instances can be of great benefit for whole slide gastric image diagnosis. In this paper, we design a recalibrated multi-instance deep learning method (RMDL) to address this challenging problem. We first select the discriminative instances, and then utilize these instances to diagnose diseases based on the proposed RMDL approach. The designed RMDL network is capable of capturing instance-wise dependencies and recalibrating instance features according to the importance coefficient learned from the fused features. Furthermore, we build a large whole-slide gastric histopathology image dataset with detailed pixel-level annotations. Experimental results on the constructed gastric dataset demonstrate the significant improvement on the accuracy of our proposed framework compared with other state-of-the-art multi-instance learning methods. Moreover, our method is general and can be extended to other diagnosis tasks of different cancer types based on WSIs.
Collapse
Affiliation(s)
- Shujun Wang
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China
| | - Yaxi Zhu
- Department of Pathology, The Sixth Affiliated Hospital of Sun Yat-sen University, China
| | - Lequan Yu
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China
| | - Hao Chen
- Imsight Medical Technology Co., Ltd., China.
| | - Huangjing Lin
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China; Imsight Medical Technology Co., Ltd., China
| | - Xiangbo Wan
- Department of Radiation Oncology, The Sixth Affiliated Hospital of Sun Yat-sen University, China
| | - Xinjuan Fan
- Department of Pathology, The Sixth Affiliated Hospital of Sun Yat-sen University, China.
| | - Pheng-Ann Heng
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China
| |
Collapse
|
109
|
Munir K, Elahi H, Ayub A, Frezza F, Rizzi A. Cancer Diagnosis Using Deep Learning: A Bibliographic Review. Cancers (Basel) 2019; 11:E1235. [PMID: 31450799 PMCID: PMC6770116 DOI: 10.3390/cancers11091235] [Citation(s) in RCA: 128] [Impact Index Per Article: 25.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2019] [Revised: 06/30/2019] [Accepted: 08/14/2019] [Indexed: 01/06/2023] Open
Abstract
In this paper, we first describe the basics of the field of cancer diagnosis, which includes steps of cancer diagnosis followed by the typical classification methods used by doctors, providing a historical idea of cancer classification techniques to the readers. These methods include Asymmetry, Border, Color and Diameter (ABCD) method, seven-point detection method, Menzies method, and pattern analysis. They are used regularly by doctors for cancer diagnosis, although they are not considered very efficient for obtaining better performance. Moreover, considering all types of audience, the basic evaluation criteria are also discussed. The criteria include the receiver operating characteristic curve (ROC curve), Area under the ROC curve (AUC), F1 score, accuracy, specificity, sensitivity, precision, dice-coefficient, average accuracy, and Jaccard index. Previously used methods are considered inefficient, asking for better and smarter methods for cancer diagnosis. Artificial intelligence and cancer diagnosis are gaining attention as a way to define better diagnostic tools. In particular, deep neural networks can be successfully used for intelligent image analysis. The basic framework of how this machine learning works on medical imaging is provided in this study, i.e., pre-processing, image segmentation and post-processing. The second part of this manuscript describes the different deep learning techniques, such as convolutional neural networks (CNNs), generative adversarial models (GANs), deep autoencoders (DANs), restricted Boltzmann's machine (RBM), stacked autoencoders (SAE), convolutional autoencoders (CAE), recurrent neural networks (RNNs), long short-term memory (LTSM), multi-scale convolutional neural network (M-CNN), multi-instance learning convolutional neural network (MIL-CNN). For each technique, we provide Python codes, to allow interested readers to experiment with the cited algorithms on their own diagnostic problems. The third part of this manuscript compiles the successfully applied deep learning models for different types of cancers. Considering the length of the manuscript, we restrict ourselves to the discussion of breast cancer, lung cancer, brain cancer, and skin cancer. The purpose of this bibliographic review is to provide researchers opting to work in implementing deep learning and artificial neural networks for cancer diagnosis a knowledge from scratch of the state-of-the-art achievements.
Collapse
Affiliation(s)
- Khushboo Munir
- Department of Information Engineering, Electronics and Telecommunications (DIET), Sapienza University of Rome, Via Eudossiana 18, 00184 Rome, Italy.
| | - Hassan Elahi
- Department of Mechanical and Aerospace Engineering (DIMA), Sapienza University of Rome, Via Eudossiana 18, 00184 Rome, Italy
| | - Afsheen Ayub
- Department of Basic and Applied Science for Engineering (SBAI), Sapienza University of Rome, Via Antonio Scarpa 14/16, 00161 Rome, Italy
| | - Fabrizio Frezza
- Department of Information Engineering, Electronics and Telecommunications (DIET), Sapienza University of Rome, Via Eudossiana 18, 00184 Rome, Italy
| | - Antonello Rizzi
- Department of Information Engineering, Electronics and Telecommunications (DIET), Sapienza University of Rome, Via Eudossiana 18, 00184 Rome, Italy
| |
Collapse
|
110
|
Kaushal C, Bhat S, Koundal D, Singla A. Recent Trends in Computer Assisted Diagnosis (CAD) System for Breast Cancer Diagnosis Using Histopathological Images. Ing Rech Biomed 2019. [DOI: 10.1016/j.irbm.2019.06.001] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
111
|
Cui Y, Zhang G, Liu Z, Xiong Z, Hu J. A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images. Med Biol Eng Comput 2019; 57:2027-2043. [PMID: 31346949 DOI: 10.1007/s11517-019-02008-8] [Citation(s) in RCA: 55] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2018] [Accepted: 06/24/2019] [Indexed: 12/12/2022]
Abstract
This paper addresses the task of nuclei segmentation in high-resolution histopathology images. We propose an automatic end-to-end deep neural network algorithm for segmentation of individual nuclei. A nucleus-boundary model is introduced to predict nuclei and their boundaries simultaneously using a fully convolutional neural network. Given a color-normalized image, the model directly outputs an estimated nuclei map and a boundary map. A simple, fast, and parameter-free post-processing procedure is performed on the estimated nuclei map to produce the final segmented nuclei. An overlapped patch extraction and assembling method is also designed for seamless prediction of nuclei in large whole-slide images. We also show the effectiveness of data augmentation methods for nuclei segmentation task. Our experiments showed our method outperforms prior state-of-the-art methods. Moreover, it is efficient that one 1000×1000 image can be segmented in less than 5 s. This makes it possible to precisely segment the whole-slide image in acceptable time. The source code is available at https://github.com/easycui/nuclei_segmentation . Graphical Abstract The neural network for nuclei segmentation.
Collapse
Affiliation(s)
- Yuxin Cui
- Department of Computer Science and Technology, University of South Carolina, Columbia, SC, 29208, USA
| | - Guiying Zhang
- Department of Medical Information Engineering, Zunyi Medical University, Zunyi, China
| | - Zhonghao Liu
- Department of Computer Science and Technology, University of South Carolina, Columbia, SC, 29208, USA
| | - Zheng Xiong
- Department of Computer Science and Technology, University of South Carolina, Columbia, SC, 29208, USA
| | - Jianjun Hu
- Department of Computer Science and Technology, University of South Carolina, Columbia, SC, 29208, USA.
| |
Collapse
|
112
|
Quick Leukocyte Nucleus Segmentation in Leukocyte Counting. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2019; 2019:3072498. [PMID: 31308855 PMCID: PMC6594331 DOI: 10.1155/2019/3072498] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/23/2019] [Revised: 05/13/2019] [Accepted: 05/21/2019] [Indexed: 12/03/2022]
Abstract
The leukocyte nucleus quick segmentation is one of the key techniques in leukocyte real-time online scanning of human blood smear. We propose a quick leukocyte nucleus segmentation method based on the component difference in RGB color space. By analyzing the captured microscopic images of the peripheral blood smears from the autoscanning microscope, it is found that the difference values between B component and G component (B − G values) in the regions of the leukocyte nuclei and the platelets are much bigger than those in the other regions, even in the regions including the stains. So, the B − G values can segment the leukocyte nuclei and the platelets with an appropriate empirical threshold because the platelets are much smaller than the leukocyte nuclei, so the leukocyte nuclei can be segmented by size filtering. Also, only an 8 bit subtraction operation is performed for the B − G values, and it can improve the leukocyte nucleus segmentation speed significantly. Experimental results show that the proposed method performs well for the five types of leukocyte segmentation with a quick speed. It is very suitable for the real-time peripheral blood smear autoscanning test application. In addition, the five types of leukocytes can be counted accurately.
Collapse
|
113
|
Lotfollahi M, Berisha S, Saadatifard L, Montier L, Žiburkus J, Mayerich D. Three-dimensional GPU-accelerated active contours for automated localization of cells in large images. PLoS One 2019; 14:e0215843. [PMID: 31173591 PMCID: PMC6555506 DOI: 10.1371/journal.pone.0215843] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2018] [Accepted: 04/09/2019] [Indexed: 01/17/2023] Open
Abstract
Cell segmentation in microscopy is a challenging problem, since cells are often asymmetric and densely packed. Successful cell segmentation algorithms rely identifying seed points, and are highly sensitive to variablility in cell size. In this paper, we present an efficient and highly parallel formulation for symmetric three-dimensional contour evolution that extends previous work on fast two-dimensional snakes. We provide a formulation for optimization on 3D images, as well as a strategy for accelerating computation on consumer graphics hardware. The proposed software takes advantage of Monte-Carlo sampling schemes in order to speed up convergence and reduce thread divergence. Experimental results show that this method provides superior performance for large 2D and 3D cell localization tasks when compared to existing methods on large 3D brain images.
Collapse
Affiliation(s)
- Mahsa Lotfollahi
- Department of Electrical and Computer engineering, University of Houston, Houston, TX, United States of America
| | - Sebastian Berisha
- Department of Electrical and Computer engineering, University of Houston, Houston, TX, United States of America
| | - Leila Saadatifard
- Department of Electrical and Computer engineering, University of Houston, Houston, TX, United States of America
| | - Laura Montier
- Department of Biology and Biochemistry, University of Houston, TX, United States of America
| | - Jokūbas Žiburkus
- Department of Biology and Biochemistry, University of Houston, TX, United States of America
| | - David Mayerich
- Department of Electrical and Computer engineering, University of Houston, Houston, TX, United States of America
- * E-mail:
| |
Collapse
|
114
|
Niazi MKK, Parwani AV, Gurcan MN. Digital pathology and artificial intelligence. Lancet Oncol 2019; 20:e253-e261. [PMID: 31044723 PMCID: PMC8711251 DOI: 10.1016/s1470-2045(19)30154-8] [Citation(s) in RCA: 490] [Impact Index Per Article: 98.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2019] [Revised: 02/28/2019] [Accepted: 03/13/2019] [Indexed: 02/06/2023]
Abstract
In modern clinical practice, digital pathology has a crucial role and is increasingly a technological requirement in the scientific laboratory environment. The advent of whole-slide imaging, availability of faster networks, and cheaper storage solutions has made it easier for pathologists to manage digital slide images and share them for clinical use. In parallel, unprecedented advances in machine learning have enabled the synergy of artificial intelligence and digital pathology, which offers image-based diagnosis possibilities that were once limited only to radiology and cardiology. Integration of digital slides into the pathology workflow, advanced algorithms, and computer-aided diagnostic techniques extend the frontiers of the pathologist's view beyond a microscopic slide and enable true utilisation and integration of knowledge that is beyond human limits and boundaries, and we believe there is clear potential for artificial intelligence breakthroughs in the pathology setting. In this Review, we discuss advancements in digital slide-based image diagnosis for cancer along with some challenges and opportunities for artificial intelligence in digital pathology.
Collapse
Affiliation(s)
| | - Anil V Parwani
- Department of Pathology, The Ohio State University, Columbus, OH, USA
| | - Metin N Gurcan
- Center for Biomedical Informatics, Wake Forest School of Medicine, Winston-Salem, NC, USA
| |
Collapse
|
115
|
NHL Pathological Image Classification Based on Hierarchical Local Information and GoogLeNet-Based Representations. BIOMED RESEARCH INTERNATIONAL 2019; 2019:1065652. [PMID: 31016181 PMCID: PMC6448331 DOI: 10.1155/2019/1065652] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/14/2018] [Accepted: 03/04/2019] [Indexed: 01/31/2023]
Abstract
Background Accurate classification for different non-Hodgkin lymphomas (NHL) is one of the main challenges in clinical pathological diagnosis due to its intrinsic complexity. Therefore, this paper proposes an effective classification model for three types of NHL pathological images, including mantle cell lymphoma (MCL), follicular lymphoma (FL), and chronic lymphocytic leukemia (CLL). Methods There are three main parts with respect to our model. First, NHL pathological images stained by hematoxylin and eosin (H&E) are transferred into blue ratio (BR) and Lab spaces, respectively. Then specific patch-level textural and statistical features are extracted from BR images and color features are obtained from Lab images both using a hierarchical way, yielding a set of hand-crafted representations corresponding to different image spaces. A random forest classifier is subsequently trained for patch-level classification. Second, H&E images are cropped and fed into a pretrained google inception net (GoogLeNet) for learning high-level representations and a softmax classifier is used for patch-level classification. Finally, three image-level classification strategies based on patch-level results are discussed including a novel method for calculating the weighted sum of patch results. Different classification results are fused at both feature 1 and image levels to obtain a more satisfactory result. Results The proposed model is evaluated on a public IICBU Malignant Lymphoma Dataset and achieves an improved overall accuracy of 0.991 and area under the receiver operating characteristic curve of 0.998. Conclusion The experimentations demonstrate the significantly increased classification performance of the proposed model, indicating that it is a suitable classification approach for NHL pathological images.
Collapse
|
116
|
Zhang P, Wang F, Teodoro G, Liang Y, Roy M, Brat D, Kong J. Effective nuclei segmentation with sparse shape prior and dynamic occlusion constraint for glioblastoma pathology images. J Med Imaging (Bellingham) 2019; 6:017502. [PMID: 30891467 DOI: 10.1117/1.jmi.6.1.017502] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2018] [Accepted: 02/19/2019] [Indexed: 11/14/2022] Open
Abstract
We propose a segmentation method for nuclei in glioblastoma histopathologic images based on a sparse shape prior guided variational level set framework. By spectral clustering and sparse coding, a set of shape priors is exploited to accommodate complicated shape variations. We automate the object contour initialization by a seed detection algorithm and deform contours by minimizing an energy functional that incorporates a shape term in a sparse shape prior representation, an adaptive contour occlusion penalty term, and a boundary term encouraging contours to converge to strong edges. As a result, our approach is able to deal with mutual occlusions and detect contours of multiple intersected nuclei simultaneously. Our method is applied to several whole-slide histopathologic image datasets for nuclei segmentation. The proposed method is compared with other state-of-the-art methods and demonstrates good accuracy for nuclei detection and segmentation, suggesting its promise to support biomedical image-based investigations.
Collapse
Affiliation(s)
- Pengyue Zhang
- Stony Brook University, Department of Computer Science, Stony Brook, New York, United States
| | - Fusheng Wang
- Stony Brook University, Department of Biomedical Informatics and Computer Science, Stony Brook, New York, United States
| | - George Teodoro
- University of Brasìlia, Department of Computer Science, Brasìlia, Brazil
| | - Yanhui Liang
- Google Inc., Mountain View, California, United States
| | - Mousumi Roy
- Stony Brook University, Department of Computer Science, Stony Brook, New York, United States
| | - Daniel Brat
- Northwestern University, Department of Pathology, Chicago, Illinois, United States
| | - Jun Kong
- Emory University, Department of Computer Science and Biomedical Informatics, Atlanta, Georgia, United States.,Georgia State University, Department of Mathematics and Statistics, Atlanta, Georgia, United States
| |
Collapse
|
117
|
Abdolhoseini M, Kluge MG, Walker FR, Johnson SJ. Segmentation of Heavily Clustered Nuclei from Histopathological Images. Sci Rep 2019; 9:4551. [PMID: 30872619 PMCID: PMC6418222 DOI: 10.1038/s41598-019-38813-2] [Citation(s) in RCA: 34] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2018] [Accepted: 12/10/2018] [Indexed: 01/27/2023] Open
Abstract
Automated cell nucleus segmentation is the key to gain further insight into cell features and functionality which support computer-aided pathology in early diagnosis of diseases such as breast cancer and brain tumour. Despite considerable advances in automated segmentation, it still remains a challenging task to split heavily clustered nuclei due to intensity variations caused by noise and uneven absorption of stains. To address this problem, we propose a novel method applicable to variety of histopathological images stained for different proteins, with high speed, accuracy and level of automation. Our algorithm is initiated by applying a new locally adaptive thresholding method on watershed regions. Followed by a new splitting technique based on multilevel thresholding and the watershed algorithm to separate clustered nuclei. Finalized by a model-based merging step to eliminate oversegmentation and a model-based correction step to improve segmentation results and eliminate small objects. We have applied our method to three image datasets: breast cancer stained for hematoxylin and eosin (H&E), Drosophila Kc167 cells stained for DNA to label nuclei, and mature neurons stained for NeuN. Evaluated results show our method outperforms the state-of-the-art methods in terms of accuracy, precision, F1-measure, and computational time.
Collapse
Affiliation(s)
- Mahmoud Abdolhoseini
- The University of Newcastle, School of Electrical Engineering and Computing, Callaghan, NSW, 2308, Australia.
| | - Murielle G Kluge
- The University of Newcastle, School of Biomedical Sciences and Pharmacy, Callaghan, NSW, 2308, Australia.,The Hunter Medical Research Institute, New Lambton, NSW, 2305, Australia
| | - Frederick R Walker
- The University of Newcastle, School of Biomedical Sciences and Pharmacy, Callaghan, NSW, 2308, Australia.,The Hunter Medical Research Institute, New Lambton, NSW, 2305, Australia
| | - Sarah J Johnson
- The University of Newcastle, School of Electrical Engineering and Computing, Callaghan, NSW, 2308, Australia.,The Hunter Medical Research Institute, New Lambton, NSW, 2305, Australia
| |
Collapse
|
118
|
An Enhancement of Deep Learning Algorithm for Brain Tumor Segmentation Using Kernel Based CNN with M-SVM. J Med Syst 2019; 43:84. [PMID: 30810822 DOI: 10.1007/s10916-019-1223-7] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2018] [Accepted: 02/21/2019] [Indexed: 01/09/2023]
Abstract
The brain tumor can be created by uncontrollable increase of abnormal cells in tissue of brain and it has two kinds of tumors: one is benign and another one is malignant tumor. The benign brain tumor does not affect the adjacent normal and healthy tissue but the malignant tumor can affect the neighboring tissues of brain that can lead to the death of person. An early detection of brain tumor can be required to protect the survival of patients. Usually, the brain tumor is detected using MRI scanning method. However, the radiologists are not providing the effective tumor segmentation in MRI image due to the irregular shape of tumors and position of tumor in the brain. Accurate brain tumor segmentation is needed to locate the tumor and it is used to give the correct treatment for a patient and it provides the key to the doctor who must execute the surgery for patient. In this paper, a novel deep learning algorithm (kernel based CNN) with M-SVM is presented to segment the tumor automatically and efficiently. This presented work contains several steps that are preprocessing, feature extraction, image classification and tumor segmentation of brain. The MRI image is smoothed and enhanced by Laplacian of Gaussian filtering method (LoG) with Contrast Limited Adaptive Histrogram Equalization (CLAHE) and feature can be extracted from it based on tumor shape position, shape and surface features in brain. Consequently, the image classification is done using M-SVM depending on the selected features. From MRI image, the tumor is segmented with help of kernel based CNN method.. Experimental results of proposed method can show that this presented technique can executes brain tumor segmentation accurately reaching almost 84% in evaluation with existing algorithms.
Collapse
|
119
|
Wang B, Lei Y, Tian S, Wang T, Liu Y, Patel P, Jani AB, Mao H, Curran WJ, Liu T, Yang X. Deeply supervised 3D fully convolutional networks with group dilated convolution for automatic MRI prostate segmentation. Med Phys 2019; 46:1707-1718. [PMID: 30702759 DOI: 10.1002/mp.13416] [Citation(s) in RCA: 119] [Impact Index Per Article: 23.8] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2018] [Revised: 01/18/2019] [Accepted: 01/24/2019] [Indexed: 12/15/2022] Open
Abstract
PURPOSE Reliable automated segmentation of the prostate is indispensable for image-guided prostate interventions. However, the segmentation task is challenging due to inhomogeneous intensity distributions, variation in prostate anatomy, among other problems. Manual segmentation can be time-consuming and is subject to inter- and intraobserver variation. We developed an automated deep learning-based method to address this technical challenge. METHODS We propose a three-dimensional (3D) fully convolutional networks (FCN) with deep supervision and group dilated convolution to segment the prostate on magnetic resonance imaging (MRI). In this method, a deeply supervised mechanism was introduced into a 3D FCN to effectively alleviate the common exploding or vanishing gradients problems in training deep models, which forces the update process of the hidden layer filters to favor highly discriminative features. A group dilated convolution which aggregates multiscale contextual information for dense prediction was proposed to enlarge the effective receptive field of convolutional neural networks, which improve the prediction accuracy of prostate boundary. In addition, we introduced a combined loss function including cosine and cross entropy, which measures similarity and dissimilarity between segmented and manual contours, to further improve the segmentation accuracy. Prostate volumes manually segmented by experienced physicians were used as a gold standard against which our segmentation accuracy was measured. RESULTS The proposed method was evaluated on an internal dataset comprising 40 T2-weighted prostate MR volumes. Our method achieved a Dice similarity coefficient (DSC) of 0.86 ± 0.04, a mean surface distance (MSD) of 1.79 ± 0.46 mm, 95% Hausdorff distance (95%HD) of 7.98 ± 2.91 mm, and absolute relative volume difference (aRVD) of 15.65 ± 10.82. A public dataset (PROMISE12) including 50 T2-weighted prostate MR volumes was also employed to evaluate our approach. Our method yielded a DSC of 0.88 ± 0.05, MSD of 1.02 ± 0.35 mm, 95% HD of 9.50 ± 5.11 mm, and aRVD of 8.93 ± 7.56. CONCLUSION We developed a novel deeply supervised deep learning-based approach with a group dilated convolution to automatically segment the MRI prostate, demonstrated its clinical feasibility, and validated its accuracy against manual segmentation. The proposed technique could be a useful tool for image-guided interventions in prostate cancer.
Collapse
Affiliation(s)
- Bo Wang
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30322, USA.,School of Physics and Electronic-Electrical Engineering, Ningxia University, Yinchuan, Ningxia, 750021, P.R. China
| | - Yang Lei
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30322, USA
| | - Sibo Tian
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30322, USA
| | - Tonghe Wang
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30322, USA
| | - Yingzi Liu
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30322, USA
| | - Pretesh Patel
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30322, USA
| | - Ashesh B Jani
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30322, USA
| | - Hui Mao
- Department of Radiology and Imaging Sciences and Winship Cancer Institute, Emory University, Atlanta, GA, 30322, USA
| | - Walter J Curran
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30322, USA
| | - Tian Liu
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30322, USA
| | - Xiaofeng Yang
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30322, USA
| |
Collapse
|
120
|
Xu J, Gong L, Wang G, Lu C, Gilmore H, Zhang S, Madabhushi A. Convolutional neural network initialized active contour model with adaptive ellipse fitting for nuclear segmentation on breast histopathological images. J Med Imaging (Bellingham) 2019; 6:017501. [PMID: 30840729 DOI: 10.1117/1.jmi.6.1.017501] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2018] [Accepted: 01/07/2019] [Indexed: 11/14/2022] Open
Abstract
Automated detection and segmentation of nuclei from high-resolution histopathological images is a challenging problem owing to the size and complexity of digitized histopathologic images. In the context of breast cancer, the modified Bloom-Richardson Grading system is highly correlated with the morphological and topological nuclear features are highly correlated with Modified Bloom-Richardson grading. Therefore, to develop a computer-aided prognosis system, automated detection and segmentation of nuclei are critical prerequisite steps. We present a method for automated detection and segmentation of breast cancer nuclei named a convolutional neural network initialized active contour model with adaptive ellipse fitting (CoNNACaeF). The CoNNACaeF model is able to detect and segment nuclei simultaneously, which consist of three different modules: convolutional neural network (CNN) for accurate nuclei detection, (2) region-based active contour (RAC) model for subsequent nuclear segmentation based on the initial CNN-based detection of nuclear patches, and (3) adaptive ellipse fitting for overlapping solution of clumped nuclear regions. The performance of the CoNNACaeF model is evaluated on three different breast histological data sets, comprising a total of 257 H&E-stained images. The model is shown to have improved detection accuracy of F-measure 80.18%, 85.71%, and 80.36% and average area under precision-recall curves (AveP) 77%, 82%, and 74% on a total of 3 million nuclei from 204 whole slide images from three different datasets. Additionally, CoNNACaeF yielded an F-measure at 74.01% and 85.36%, respectively, for two different breast cancer datasets. The CoNNACaeF model also outperformed the three other state-of-the-art nuclear detection and segmentation approaches, which are blue ratio initialized local region active contour, iterative radial voting initialized local region active contour, and maximally stable extremal region initialized local region active contour models.
Collapse
Affiliation(s)
- Jun Xu
- Nanjing University of Information Science and Technology, Jiangsu Key Laboratory of Big Data Analysis Technique, Nanjing, China
| | - Lei Gong
- Nanjing University of Information Science and Technology, Jiangsu Key Laboratory of Big Data Analysis Technique, Nanjing, China
| | - Guanhao Wang
- Nanjing University of Information Science and Technology, Jiangsu Key Laboratory of Big Data Analysis Technique, Nanjing, China
| | - Cheng Lu
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Hannah Gilmore
- University Hospitals Case Medical Center, Case Western Reserve University, Institute for Pathology, Cleveland, Ohio, United States
| | - Shaoting Zhang
- University of North Carolina at Charlotte, Department of Computer Science, Charlotte, North Carolina, United States
| | - Anant Madabhushi
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States.,Louis Stokes Cleveland Veterans Administration Medical Center, Cleveland, Ohio, United States
| |
Collapse
|
121
|
Dabeer S, Khan MM, Islam S. Cancer diagnosis in histopathological image: CNN based approach. INFORMATICS IN MEDICINE UNLOCKED 2019. [DOI: 10.1016/j.imu.2019.100231] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022] Open
|
122
|
Lao Y, David J, Torosian A, Placencio V, Wang Y, Hendifar A, Yang W, Tuli R. Combined morphologic and metabolic pipeline for Positron emission tomography/computed tomography based radiotherapy response evaluation in locally advanced pancreatic adenocarcinoma. PHYSICS & IMAGING IN RADIATION ONCOLOGY 2019; 9:28-34. [PMID: 32190750 PMCID: PMC7079767 DOI: 10.1016/j.phro.2018.12.003] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
A novel morphologic and metabolic combined pipeline for PA response evaluation. The derived metric outperformed traditional imaging metrics in risk stratification. May serve as a new image biomarker to characterize heterogeneous tumor response.
Background and purpose Adaptive radiation planning for pancreatic adenocarcinoma (PA) relies on accurate treatment response assessment, while traditional response evaluation criteria inefficiently characterize tumors with complex morphological features or intrinsically low metabolism. To better assess treatment response of PA, we quantify and compare regional morphological and metabolic features of the 3D pre- and post-radiation therapy (RT) tumor models. Materials and methods Thirty-one PA patients with pre and post-RT Positron emission tomography/computed tomography (PET/CT) scans were evaluated. 3D meshes of pre- and post-RT tumors were generated and registered to establish vertex-wise correspondence. To assess tumor response, Mahalanobis distances (Mdist|Fusion) between pre- and post-RT tumor surfaces with anatomic and metabolic fused vectors were calculated for each patient. Mdist|Fusion was evaluated by overall survival (OS) prediction and survival risk classification. As a comparison, the same analyses were conducted on traditional imaging/physiological predictors, and distances measurements based on metabolic and morphological features only. Results Among all the imaging/physiological parameters, Mdist|Fusion was shown to be the best predictor of OS (HR = 0.52, p = 0.008), while other parameters failed to reach significance. Moreover, Mdist|Fusion outperformed traditional morphologic and metabolic measurements in patient risk stratification, either alone (HR = 11.51, p < 0.001) or combined with age (HR = 9.04, p < 0.001). Conclusions We introduced a PET/CT-based novel morphologic and metabolic pipeline for response evaluation in locally advanced PA. The fused Mdist|Fusion outperformed traditional morphologic, metabolic, and physiological measurements in OS prediction and risk stratification. The novel fusion model may serve as a new imaging-marker to more accurately characterize the heterogeneous tumor RT response.
Collapse
Affiliation(s)
- Yi Lao
- Department of Radiation Oncology, Cedars-Sinai Medical Center, Los Angeles, USA
| | - John David
- Department of Radiation Oncology, Cedars-Sinai Medical Center, Los Angeles, USA
| | - Arman Torosian
- Department of Radiation Oncology, Cedars-Sinai Medical Center, Los Angeles, USA
| | - Veronica Placencio
- Department of Radiation Oncology, Cedars-Sinai Medical Center, Los Angeles, USA.,School of Computing, Informatics, Decision Systems and Engineering, Arizona State University, Tempe, AZ, USA
| | - Yalin Wang
- School of Computing, Informatics, Decision Systems and Engineering, Arizona State University, Tempe, AZ, USA
| | - Andrew Hendifar
- Department of Surgery, Cedars-Sinai Medical Center, Los Angeles, USA
| | - Wensha Yang
- Department of Radiation Oncology, Cedars-Sinai Medical Center, Los Angeles, USA
| | - Richard Tuli
- Department of Radiation Oncology, Cedars-Sinai Medical Center, Los Angeles, USA
| |
Collapse
|
123
|
Gupta A, Mallick P, Sharma O, Gupta R, Duggal R. PCSeg: Color model driven probabilistic multiphase level set based tool for plasma cell segmentation in multiple myeloma. PLoS One 2018; 13:e0207908. [PMID: 30540767 PMCID: PMC6291116 DOI: 10.1371/journal.pone.0207908] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2017] [Accepted: 11/08/2018] [Indexed: 02/03/2023] Open
Abstract
Plasma cell segmentation is the first stage of a computer assisted automated diagnostic tool for multiple myeloma (MM). Owing to large variability in biological cell types, a method for one cell type cannot be applied directly on the other cell types. In this paper, we present PCSeg Tool for plasma cell segmentation from microscopic medical images. These images were captured from bone marrow aspirate slides of patients with MM. PCSeg has a robust pipeline consisting of a pre-processing step, the proposed modified multiphase level set method followed by post-processing steps including the watershed and circular Hough transform to segment clusters of cells of interest and to remove unwanted cells. Our modified level set method utilizes prior information about the probability densities of regions of interest (ROIs) in the color spaces and provides a solution to the minimal-partition problem to segment ROIs in one of the level sets of a two-phase level set formulation. PCSeg tool is tested on a number of microscopic images and provides good segmentation results on single cells as well as efficient segmentation of plasma cell clusters.
Collapse
Affiliation(s)
- Anubha Gupta
- SBILab, Department of ECE, Indraprastha Institute of Information Technology-Delhi (IIIT-Delhi), New Delhi, India
- * E-mail:
| | - Pramit Mallick
- Department of Computer Science, Courant Institute of Mathematical Sciences, New York University, New York City, New York, United States of America
| | - Ojaswa Sharma
- Department of CSE, Indraprastha Institute of Information Technology-Delhi (IIIT-Delhi), New Delhi, India
| | - Ritu Gupta
- Laboratory Oncology Unit, Dr. B. R.A. IRCH, All India Institute of Medical Sciences (AIIMS), New Delhi, India
| | - Rahul Duggal
- SBILab, Department of ECE, Indraprastha Institute of Information Technology-Delhi (IIIT-Delhi), New Delhi, India
| |
Collapse
|
124
|
Song J, Xiao L, Lian Z. Contour-Seed Pairs Learning-Based Framework for Simultaneously Detecting and Segmenting Various Overlapping Cells/Nuclei in Microscopy Images. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2018; 27:5759-5774. [PMID: 30028701 DOI: 10.1109/tip.2018.2857001] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
In this paper, we propose a novel contour-seed pairs learning-based framework for robust and automated cell/nucleus segmentation. Automated granular object segmentation in microscopy images has significant clinical importance for pathology grading of the cell carcinoma and gene expression. The focus of the past literature is dominated by either segmenting a certain type of cells/nuclei or simply splitting the clustered objects without contours inference of them. Our method addresses these issues by formulating the detection and segmentation tasks in terms of a unified regression problem, where a cascade sparse regression chain model is trained and then applied to return object locations and entire boundaries of clustered objects. In particular, we first learn a set of online convolutional features in each layer. Then, in the proposed cascade sparse regression chain, with the input from the learned features, we iteratively update the locations and clustered object boundaries until convergence. In this way, the boundary evidences of each individual object can be easily delineated and be further fed to a complete contour inference procedure optimized by the minimum description length principle. For any probe image, our method enables to analyze free-lying and overlapping cells with complex shapes. Experimental results show that the proposed method is very generic and performs well on contour inferences of various cell/nucleus types. Compared with the current segmentation techniques, our approach achieves state-of-the-art performances on four challenging datasets, i.e., the kidney renal cell carcinoma histopathology dataset, Drosophila Kc167 cellular dataset, differential interference contrast red blood cell dataset, and cervical cytology dataset.
Collapse
|
125
|
M M, P S. MRI Brain Tumour Segmentation Using Hybrid Clustering and Classification by Back Propagation Algorithm. Asian Pac J Cancer Prev 2018; 19:3257-3263. [PMID: 30486629 PMCID: PMC6318394 DOI: 10.31557/apjcp.2018.19.11.3257] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022] Open
Abstract
Generally the segmentation refers, the partitioning of an image into smaller regions to identify or locate the region of abnormality. Even though image segmentation is the challenging task in medical applications, due to contrary image, local observations of an image, noise image, non uniform texture of the images and so on. Many techniques are available for image segmentation, but still it requires to introduce an efficient, fast medical image segmentation methods. This research article introduces an efficient image segmentation method based on K means clustering integrated with a spatial Fuzzy C means clustering algorithms. The suggested technique combines the advantages of the two methods. K means segmentation requires minimum computation time, but spatial Fuzzy C means provides high accuracy for image segmentation. The performance of the proposed method is evaluated in terms of accuracy, PSNR and processing time. It also provides good implementation results for MRI brain image segmentation with high accuracy and minimal execution time. After completing the segmentation the of abnormal part of the input MRI brain image, it is compulsory to classify the image is normal or abnormal. There are many classifiers like a self organizing map, Back propagation algorithm, support vector machine etc., The algorithm helps to classify the abnormalities like benign or malignant brain tumour in case of MRI brain image. The abnormality is detected based on the extracted features from an input image. Discrete wavelet transform helps to find the hidden information from the MRI brain image. The extracted features are trained by Back Propagation Algorithm to classify the abnormalities of MRI brain image.
Collapse
Affiliation(s)
- Malathi M
- Department of Electronics and Instrumentation, Saveetha Engineering College, Chennai, India.
| | | |
Collapse
|
126
|
To MNN, Vu DQ, Turkbey B, Choyke PL, Kwak JT. Deep dense multi-path neural network for prostate segmentation in magnetic resonance imaging. Int J Comput Assist Radiol Surg 2018; 13:1687-1696. [PMID: 30088208 PMCID: PMC6177294 DOI: 10.1007/s11548-018-1841-4] [Citation(s) in RCA: 38] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2018] [Accepted: 07/27/2018] [Indexed: 01/22/2023]
Abstract
PURPOSE We propose an approach of 3D convolutional neural network to segment the prostate in MR images. METHODS A 3D deep dense multi-path convolutional neural network that follows the framework of the encoder-decoder design is proposed. The encoder is built based upon densely connected layers that learn the high-level feature representation of the prostate. The decoder interprets the features and predicts the whole prostate volume by utilizing a residual layout and grouped convolution. A set of sub-volumes of MR images, centered at the prostate, is generated and fed into the proposed network for training purpose. The performance of the proposed network is compared to previously reported approaches. RESULTS Two independent datasets were employed to assess the proposed network. In quantitative evaluations, the proposed network achieved 95.11 and 89.01 Dice coefficients for the two datasets. The segmentation results were robust to variations in MR images. In comparison experiments, the segmentation performance of the proposed network was comparable to the previously reported approaches. In qualitative evaluations, the segmentation results by the proposed network were well matched to the ground truth provided by human experts. CONCLUSIONS The proposed network is capable of segmenting the prostate in an accurate and robust manner. This approach can be applied to other types of medical images.
Collapse
Affiliation(s)
- Minh Nguyen Nhat To
- Department of Computer Science and Engineering, Sejong University, Seoul, 05006, South Korea
| | - Dang Quoc Vu
- Department of Computer Science and Engineering, Sejong University, Seoul, 05006, South Korea
| | - Baris Turkbey
- Molecular Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD, 20892, USA
| | - Peter L Choyke
- Molecular Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD, 20892, USA
| | - Jin Tae Kwak
- Department of Computer Science and Engineering, Sejong University, Seoul, 05006, South Korea.
| |
Collapse
|
127
|
Fu Y, Mazur TR, Wu X, Liu S, Chang X, Lu Y, Li HH, Kim H, Roach MC, Henke L, Yang D. A novel MRI segmentation method using CNN-based correction network for MRI-guided adaptive radiotherapy. Med Phys 2018; 45:5129-5137. [PMID: 30269345 DOI: 10.1002/mp.13221] [Citation(s) in RCA: 78] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2018] [Revised: 08/20/2018] [Accepted: 09/21/2018] [Indexed: 12/14/2022] Open
Abstract
PURPOSE The purpose of this study was to expedite the contouring process for MRI-guided adaptive radiotherapy (MR-IGART), a convolutional neural network (CNN) deep-learning (DL) model is proposed to accurately segment the liver, kidneys, stomach, bowel and duodenum in 3D MR images. METHODS Images and structure contours for 120 patients were collected retrospectively. Treatment sites included pancreas, liver, stomach, adrenal gland, and prostate. The proposed DL model contains a voxel-wise label prediction CNN and a correction network which consists of two sub-networks. The prediction CNN and sub-networks in the correction network each includes a dense block which consists of twelve densely connected convolutional layers. The correction network was designed to improve the voxel-wise labeling accuracy of a CNN by learning and enforcing implicit anatomical constraints in the segmentation process. Its sub-networks learn to fix the erroneous classification of its previous network by taking as input both the original images and the softmax probability maps generated from its previous sub-network. The parameters of each sub-network were trained independently using piecewise training. The model was trained on 100 datasets, validated on 10 datasets and tested on the remaining 10 datasets. Dice coefficient, Hausdorff distance (HD) were calculated to evaluate the segmentation accuracy. RESULTS The proposed DL model was able to segment the organs with good accuracy. The correction network outperformed the conditional random field (CRF), a most comparable method that is usually applied as a post-processing step. For the 10 testing patients, the average Dice coefficients were 95.3 ± 0.73, 93.1 ± 2.22, 85.0 ± 3.75, 86.6 ± 2.69, and 65.5 ± 8.90 for liver, kidneys, stomach, bowel, and duodenum, respectively. The mean Hausdorff Distance (HD) were 5.41 ± 2.34, 6.23 ± 4.59, 6.88 ± 4.89, 5.90 ± 4.05, and 7.99 ± 6.84 mm, respectively. Manual contouring, as to correct the automatic segmentation results, was four times as fast as manual contouring from scratch. CONCLUSION The proposed method can automatically segment the liver, kidneys, stomach, bowel, and duodenum in 3D MR images with good accuracy. It is useful to expedite the manual contouring for MR-IGART.
Collapse
Affiliation(s)
- Yabo Fu
- Department of Radiation Oncology, School of Medicine, Washington University in Saint Louis, St.Louis, MO, 63110, USA
| | - Thomas R Mazur
- Department of Radiation Oncology, School of Medicine, Washington University in Saint Louis, St.Louis, MO, 63110, USA
| | - Xue Wu
- Department of Radiation Oncology, School of Medicine, Washington University in Saint Louis, St.Louis, MO, 63110, USA
| | - Shi Liu
- Department of Radiation Oncology, Stanford University, Stanford, CA, 94305, USA
| | - Xiao Chang
- Department of Radiation Oncology, School of Medicine, Washington University in Saint Louis, St.Louis, MO, 63110, USA
| | - Yonggang Lu
- Department of radiology, Medical College of Wisconsin, Milwaukee, WI, 53226, USA
| | - H Harold Li
- Department of Radiation Oncology, School of Medicine, Washington University in Saint Louis, St.Louis, MO, 63110, USA
| | - Hyun Kim
- Department of Radiation Oncology, School of Medicine, Washington University in Saint Louis, St.Louis, MO, 63110, USA
| | - Michael C Roach
- Department of Radiation Oncology, School of Medicine, Washington University in Saint Louis, St.Louis, MO, 63110, USA
| | - Lauren Henke
- Department of Radiation Oncology, School of Medicine, Washington University in Saint Louis, St.Louis, MO, 63110, USA
| | - Deshan Yang
- Department of Radiation Oncology, School of Medicine, Washington University in Saint Louis, St.Louis, MO, 63110, USA
| |
Collapse
|
128
|
Coudray N, Ocampo PS, Sakellaropoulos T, Narula N, Snuderl M, Fenyö D, Moreira AL, Razavian N, Tsirigos A. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med 2018; 24:1559-1567. [PMID: 30224757 PMCID: PMC9847512 DOI: 10.1038/s41591-018-0177-5] [Citation(s) in RCA: 1306] [Impact Index Per Article: 217.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2017] [Accepted: 07/06/2018] [Indexed: 02/06/2023]
Abstract
Visual inspection of histopathology slides is one of the main methods used by pathologists to assess the stage, type and subtype of lung tumors. Adenocarcinoma (LUAD) and squamous cell carcinoma (LUSC) are the most prevalent subtypes of lung cancer, and their distinction requires visual inspection by an experienced pathologist. In this study, we trained a deep convolutional neural network (inception v3) on whole-slide images obtained from The Cancer Genome Atlas to accurately and automatically classify them into LUAD, LUSC or normal lung tissue. The performance of our method is comparable to that of pathologists, with an average area under the curve (AUC) of 0.97. Our model was validated on independent datasets of frozen tissues, formalin-fixed paraffin-embedded tissues and biopsies. Furthermore, we trained the network to predict the ten most commonly mutated genes in LUAD. We found that six of them-STK11, EGFR, FAT1, SETBP1, KRAS and TP53-can be predicted from pathology images, with AUCs from 0.733 to 0.856 as measured on a held-out population. These findings suggest that deep-learning models can assist pathologists in the detection of cancer subtype or gene mutations. Our approach can be applied to any cancer type, and the code is available at https://github.com/ncoudray/DeepPATH .
Collapse
Affiliation(s)
- Nicolas Coudray
- Applied Bioinformatics Laboratories, New York University School of Medicine, NY 10016, USA,Skirball Institute, Dept. of Cell Biology, New York University School of Medicine, NY 10016, USA
| | | | - Theodore Sakellaropoulos
- School of Mechanical Engineering, National Technical University of Athens, Zografou 15780, Greece
| | - Navneet Narula
- Department of Pathology, New York University School of Medicine, NY 10016, USA
| | - Matija Snuderl
- Department of Pathology, New York University School of Medicine, NY 10016, USA
| | - David Fenyö
- Institute for Systems Genetics, New York University School of Medicine, NY 10016, USA,Department of Biochemistry and molecular Pharmacology, New York University School of Medicine, NY 10016, USA
| | - Andre L. Moreira
- Department of Pathology, New York University School of Medicine, NY 10016, USA,Center for Biospecimen Research and Development, New York University, NY 10016, USA
| | - Narges Razavian
- Department of Population Health and the Center for Healthcare Innovation and Delivery Science, New York University School of Medicine, NY 10016, USA,To whom correspondence should be addressed. Tel: +1 646 501 2693; ; Correspondence may also be addressed to Narges Razavian. Tel: +1 212 263 2234,
| | - Aristotelis Tsirigos
- Applied Bioinformatics Laboratories, New York University School of Medicine, NY 10016, USA,Department of Pathology, New York University School of Medicine, NY 10016, USA,To whom correspondence should be addressed. Tel: +1 646 501 2693; ; Correspondence may also be addressed to Narges Razavian. Tel: +1 212 263 2234,
| |
Collapse
|
129
|
Mouelhi A, Rmili H, Ali JB, Sayadi M, Doghri R, Mrad K. Fast unsupervised nuclear segmentation and classification scheme for automatic allred cancer scoring in immunohistochemical breast tissue images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2018; 165:37-51. [PMID: 30337080 DOI: 10.1016/j.cmpb.2018.08.005] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/12/2018] [Revised: 07/22/2018] [Accepted: 08/08/2018] [Indexed: 06/08/2023]
Abstract
BACKGROUND AND OBJECTIVE This paper presents an improved scheme able to perform accurate segmentation and classification of cancer nuclei in immunohistochemical (IHC) breast tissue images in order to provide quantitative evaluation of estrogen or progesterone (ER/PR) receptor status that will assist pathologists in cancer diagnostic process. METHODS The proposed segmentation method is based on adaptive local thresholding and an enhanced morphological procedure, which are applied to extract all stained nuclei regions and to split overlapping nuclei. In fact, a new segmentation approach is presented here for cell nuclei detection from the IHC image using a modified Laplacian filter and an improved watershed algorithm. Stromal cells are then removed from the segmented image using an adaptive criterion in order to get fast tumor nuclei recognition. Finally, unsupervised classification of cancer nuclei is obtained by the combination of four common color separation techniques for a subsequent Allred cancer scoring. RESULTS Experimental results on various IHC tissue images of different cancer affected patients, demonstrate the effectiveness of the proposed scheme when compared to the manual scoring of pathological experts. A statistical analysis is performed on the whole image database between immuno-score of manual and automatic method, and compared with the scores that have reached using other state-of-art segmentation and classification strategies. According to the performance evaluation, we recorded more than 98% for both accuracy of detected nuclei and image cancer scoring over the truths provided by experienced pathologists which shows the best correlation with the expert's score (Pearson's correlation coefficient = 0.993, p-value < 0.005) and the lowest computational total time of 72.3 s/image (±1.9) compared to recent studied methods. CONCLUSIONS The proposed scheme can be easily applied for any histopathological diagnostic process that needs stained nuclear quantification and cancer grading. Moreover, the reduced processing time and manual interactions of our procedure can facilitate its implementation in a real-time device to construct a fully online evaluation system of IHC tissue images.
Collapse
MESH Headings
- Algorithms
- Breast Neoplasms/classification
- Breast Neoplasms/diagnostic imaging
- Breast Neoplasms/metabolism
- Carcinoma, Ductal, Breast/classification
- Carcinoma, Ductal, Breast/diagnostic imaging
- Carcinoma, Ductal, Breast/metabolism
- Cell Nucleus/classification
- Cell Nucleus/metabolism
- Cell Nucleus/pathology
- Female
- Humans
- Image Interpretation, Computer-Assisted/methods
- Image Interpretation, Computer-Assisted/statistics & numerical data
- Immunohistochemistry/methods
- Immunohistochemistry/statistics & numerical data
- Receptors, Estrogen/metabolism
- Receptors, Progesterone/metabolism
- Staining and Labeling
- Unsupervised Machine Learning
Collapse
Affiliation(s)
- Aymen Mouelhi
- University of Tunis, ENSIT, LR13ES03 SIME, Montfleury 1008, Tunisia.
| | - Hana Rmili
- University of Tunis El-Manar, ISTMT, Laboratory of Biophysics and Medical Technologies, Tunisia.
| | - Jaouher Ben Ali
- University of Tunis, ENSIT, LR13ES03 SIME, Montfleury 1008, Tunisia; FEMTO-ST Institute, AS2M department, UMR CNRS 6174 - UFC / ENSMM /UTBM, Besançon 25000, France.
| | - Mounir Sayadi
- University of Tunis, ENSIT, LR13ES03 SIME, Montfleury 1008, Tunisia.
| | - Raoudha Doghri
- Salah Azaiez Institute of Oncology, Morbid Anatomy Service, bd du 9 avril, Bab Saadoun, Tunis 1006, Tunisia.
| | - Karima Mrad
- Salah Azaiez Institute of Oncology, Morbid Anatomy Service, bd du 9 avril, Bab Saadoun, Tunis 1006, Tunisia.
| |
Collapse
|
130
|
Li S, Jiang H, Yao YD, Pang W, Sun Q, Kuang L. Structure convolutional extreme learning machine and case-based shape template for HCC nucleus segmentation. Neurocomputing 2018. [DOI: 10.1016/j.neucom.2018.05.013] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
|
131
|
Xing F, Xie Y, Su H, Liu F, Yang L. Deep Learning in Microscopy Image Analysis: A Survey. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2018; 29:4550-4568. [PMID: 29989994 DOI: 10.1109/tnnls.2017.2766168] [Citation(s) in RCA: 147] [Impact Index Per Article: 24.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/21/2023]
Abstract
Computerized microscopy image analysis plays an important role in computer aided diagnosis and prognosis. Machine learning techniques have powered many aspects of medical investigation and clinical practice. Recently, deep learning is emerging as a leading machine learning tool in computer vision and has attracted considerable attention in biomedical image analysis. In this paper, we provide a snapshot of this fast-growing field, specifically for microscopy image analysis. We briefly introduce the popular deep neural networks and summarize current deep learning achievements in various tasks, such as detection, segmentation, and classification in microscopy image analysis. In particular, we explain the architectures and the principles of convolutional neural networks, fully convolutional networks, recurrent neural networks, stacked autoencoders, and deep belief networks, and interpret their formulations or modelings for specific tasks on various microscopy images. In addition, we discuss the open challenges and the potential trends of future research in microscopy image analysis using deep learning.
Collapse
|
132
|
Höfener H, Homeyer A, Weiss N, Molin J, Lundström CF, Hahn HK. Deep learning nuclei detection: A simple approach can deliver state-of-the-art results. Comput Med Imaging Graph 2018; 70:43-52. [PMID: 30286333 DOI: 10.1016/j.compmedimag.2018.08.010] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2018] [Revised: 07/13/2018] [Accepted: 08/23/2018] [Indexed: 11/28/2022]
Abstract
BACKGROUND Deep convolutional neural networks have become a widespread tool for the detection of nuclei in histopathology images. Many implementations share a basic approach that includes generation of an intermediate map indicating the presence of a nucleus center, which we refer to as PMap. Nevertheless, these implementations often still differ in several parameters, resulting in different detection qualities. METHODS We identified several essential parameters and configured the basic PMap approach using combinations of them. We thoroughly evaluated and compared various configurations on multiple datasets with respect to detection quality, efficiency and training effort. RESULTS Post-processing of the PMap was found to have the largest impact on detection quality. Also, two different network architectures were identified that improve either detection quality or runtime performance. The best-performing configuration yields f1-measures of 0.816 on H&E stained images of colorectal adenocarcinomas and 0.819 on Ki-67 stained images of breast tumor tissue. On average, it was fully trained in less than 15,000 iterations and processed 4.15 megapixels per second at prediction time. CONCLUSIONS The basic PMap approach is greatly affected by certain parameters. Our evaluation provides guidance on their impact and best settings. When configured properly, this simple and efficient approach can yield equal detection quality as more complex and time-consuming state-of-the-art approaches.
Collapse
Affiliation(s)
| | - André Homeyer
- Fraunhofer MEVIS, Am Fallturm 1, 28359, Bremen, Germany.
| | - Nick Weiss
- Fraunhofer MEVIS, Am Fallturm 1, 28359, Bremen, Germany.
| | - Jesper Molin
- Sectra AB, Teknikringen 20, 58330, Linköping, Sweden.
| | - Claes F Lundström
- Sectra AB, Teknikringen 20, 58330, Linköping, Sweden; Center for Medical Image Science and Visualization, Linköping University, 58183, Linköping, Sweden.
| | - Horst K Hahn
- Fraunhofer MEVIS, Am Fallturm 1, 28359, Bremen, Germany; Jacobs University, Campus Ring 1, 28759, Bremen, Germany.
| |
Collapse
|
133
|
Khoshdeli M, Winkelmaier G, Parvin B. Fusion of encoder-decoder deep networks improves delineation of multiple nuclear phenotypes. BMC Bioinformatics 2018; 19:294. [PMID: 30086715 PMCID: PMC6081825 DOI: 10.1186/s12859-018-2285-0] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2018] [Accepted: 07/16/2018] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Nuclear segmentation is an important step for profiling aberrant regions of histology sections. If nuclear segmentation can be resolved, then new biomarkers of nuclear phenotypes and their organization can be predicted for the application of precision medicine. However, segmentation is a complex problem as a result of variations in nuclear geometry (e.g., size, shape), nuclear type (e.g., epithelial, fibroblast), nuclear phenotypes (e.g., vesicular, aneuploidy), and overlapping nuclei. The problem is further complicated as a result of variations in sample preparation (e.g., fixation, staining). Our hypothesis is that (i) deep learning techniques can learn complex phenotypic signatures that rise in tumor sections, and (ii) fusion of different representations (e.g., regions, boundaries) contributes to improved nuclear segmentation. RESULTS We have demonstrated that training of deep encoder-decoder convolutional networks overcomes complexities associated with multiple nuclear phenotypes, where we evaluate alternative architecture of deep learning for an improved performance against the simplicity of the design. In addition, improved nuclear segmentation is achieved by color decomposition and combining region- and boundary-based features through a fusion network. The trained models have been evaluated against approximately 19,000 manually annotated nuclei, and object-level Precision, Recall, F1-score and Standard Error are reported with the best F1-score being 0.91. Raw training images, annotated images, processed images, and source codes are released as a part of the Additional file 1. CONCLUSIONS There are two intrinsic barriers in nuclear segmentation to H&E stained images, which correspond to the diversity of nuclear phenotypes and perceptual boundaries between adjacent cells. We demonstrate that (i) the encoder-decoder architecture can learn complex phenotypes that include the vesicular type; (ii) delineation of overlapping nuclei is enhanced by fusion of region- and edge-based networks; (iii) fusion of ENets produces an improved result over the fusion of UNets; and (iv) fusion of networks is better than multitask learning. We suggest that our protocol enables processing a large cohort of whole slide images for applications in precision medicine.
Collapse
Affiliation(s)
- Mina Khoshdeli
- Electrical and Biomedical Department, University of Nevada, Reno, 1664 N. Virginia, Reno, USA
| | - Garrett Winkelmaier
- Electrical and Biomedical Department, University of Nevada, Reno, 1664 N. Virginia, Reno, USA
| | - Bahram Parvin
- Electrical and Biomedical Department, University of Nevada, Reno, 1664 N. Virginia, Reno, USA
| |
Collapse
|
134
|
Deep learning in mammography and breast histology, an overview and future trends. Med Image Anal 2018; 47:45-67. [DOI: 10.1016/j.media.2018.03.006] [Citation(s) in RCA: 160] [Impact Index Per Article: 26.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2017] [Revised: 01/03/2018] [Accepted: 03/14/2018] [Indexed: 12/20/2022]
|
135
|
Sornapudi S, Stanley RJ, Stoecker WV, Almubarak H, Long R, Antani S, Thoma G, Zuna R, Frazier SR. Deep Learning Nuclei Detection in Digitized Histology Images by Superpixels. J Pathol Inform 2018; 9:5. [PMID: 29619277 PMCID: PMC5869967 DOI: 10.4103/jpi.jpi_74_17] [Citation(s) in RCA: 37] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2017] [Accepted: 01/17/2018] [Indexed: 01/08/2023] Open
Abstract
Background Advances in image analysis and computational techniques have facilitated automatic detection of critical features in histopathology images. Detection of nuclei is critical for squamous epithelium cervical intraepithelial neoplasia (CIN) classification into normal, CIN1, CIN2, and CIN3 grades. Methods In this study, a deep learning (DL)-based nuclei segmentation approach is investigated based on gathering localized information through the generation of superpixels using a simple linear iterative clustering algorithm and training with a convolutional neural network. Results The proposed approach was evaluated on a dataset of 133 digitized histology images and achieved an overall nuclei detection (object-based) accuracy of 95.97%, with demonstrated improvement over imaging-based and clustering-based benchmark techniques. Conclusions The proposed DL-based nuclei segmentation Method with superpixel analysis has shown improved segmentation results in comparison to state-of-the-art methods.
Collapse
Affiliation(s)
- Sudhir Sornapudi
- Department of Electrical and Computer Engineering, Missouri University of Science and Technology, Rolla, USA
| | - Ronald Joe Stanley
- Department of Electrical and Computer Engineering, Missouri University of Science and Technology, Rolla, USA
| | | | - Haidar Almubarak
- Department of Electrical and Computer Engineering, Missouri University of Science and Technology, Rolla, USA
| | - Rodney Long
- DHHS, Lister Hill National Center for Biomedical Communications for National Library of Medicine, National Institutes of Health, Bethesda, MD, USA
| | - Sameer Antani
- DHHS, Lister Hill National Center for Biomedical Communications for National Library of Medicine, National Institutes of Health, Bethesda, MD, USA
| | - George Thoma
- DHHS, Lister Hill National Center for Biomedical Communications for National Library of Medicine, National Institutes of Health, Bethesda, MD, USA
| | - Rosemary Zuna
- Department of Pathology, University of Oklahoma Health Sciences Center, Oklahoma City, OK, USA
| | - Shelliane R Frazier
- Department of Surgical Pathology, University of Missouri Hospitals and Clinics, Columbia, USA
| |
Collapse
|
136
|
Liu C, Huang Y, Ozolek JA, Hanna MG, Singh R, Rohde GK. SetSVM: An Approach to Set Classification in Nuclei-Based Cancer Detection. IEEE J Biomed Health Inform 2018; 23:351-361. [PMID: 29994380 DOI: 10.1109/jbhi.2018.2803793] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Due to the importance of nuclear structure in cancer diagnosis, several predictive models have been described for diagnosing a wide variety of cancers based on nuclear morphology. In many computer-aided diagnosis (CAD) systems, cancer detection tasks can be generally formulated as set classification problems, which can not be directly solved by classifying single instances. In this paper, we propose a novel set classification approach SetSVM to build a predictive model by considering any nuclei set as a whole without specific assumptions. SetSVM features highly discriminative power in cancer detection challenges in the sense that it not only optimizes the classifier decision boundary but also transfers discriminative information to set representation learning. During model training, these two processes are unified in the support vector machine (SVM) maximum separation margin problem. Experiment results show that SetSVM provides significant improvements compared with five commonly used approaches in cancer detection tasks utilizing 260 patients in total across three different cancer types, namely, thyroid cancer, liver cancer, and melanoma. In addition, we show that SetSVM enables visual interpretation of discriminative nuclear characteristics representing the nuclei set. These features make SetSVM a potentially practical tool in building accurate and interpretable CAD systems for cancer detection.
Collapse
|
137
|
Xie Y, Xing F, Shi X, Kong X, Su H, Yang L. Efficient and robust cell detection: A structured regression approach. Med Image Anal 2018; 44:245-254. [PMID: 28797548 PMCID: PMC6051760 DOI: 10.1016/j.media.2017.07.003] [Citation(s) in RCA: 55] [Impact Index Per Article: 9.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2016] [Revised: 02/22/2017] [Accepted: 07/21/2017] [Indexed: 10/19/2022]
Abstract
Efficient and robust cell detection serves as a critical prerequisite for many subsequent biomedical image analysis methods and computer-aided diagnosis (CAD). It remains a challenging task due to touching cells, inhomogeneous background noise, and large variations in cell sizes and shapes. In addition, the ever-increasing amount of available datasets and the high resolution of whole-slice scanned images pose a further demand for efficient processing algorithms. In this paper, we present a novel structured regression model based on a proposed fully residual convolutional neural network for efficient cell detection. For each testing image, our model learns to produce a dense proximity map that exhibits higher responses at locations near cell centers. Our method only requires a few training images with weak annotations (just one dot indicating the cell centroids). We have extensively evaluated our method using four different datasets, covering different microscopy staining methods (e.g., H & E or Ki-67 staining) or image acquisition techniques (e.g., bright-filed image or phase contrast). Experimental results demonstrate the superiority of our method over existing state of the art methods in terms of both detection accuracy and running time.
Collapse
Affiliation(s)
- Yuanpu Xie
- Department of Biomedical Engineering, University of Florida, FL 32611 USA.
| | - Fuyong Xing
- Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL, 32611, USA
| | - Xiaoshuang Shi
- Department of Biomedical Engineering, University of Florida, FL 32611 USA
| | - Xiangfei Kong
- School of Electrical and Electronic Engineering, Nanyang Technological University, Nanyang Drive 637553 Singapore
| | - Hai Su
- Department of Biomedical Engineering, University of Florida, FL 32611 USA
| | - Lin Yang
- Department of Biomedical Engineering, University of Florida, FL 32611 USA; Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL, 32611, USA.
| |
Collapse
|
138
|
A convolutional neural network model for semantic segmentation of mitotic events in microscopy images. Neural Comput Appl 2018. [DOI: 10.1007/s00521-017-3333-9] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
|
139
|
Men K, Chen X, Zhang Y, Zhang T, Dai J, Yi J, Li Y. Deep Deconvolutional Neural Network for Target Segmentation of Nasopharyngeal Cancer in Planning Computed Tomography Images. Front Oncol 2017; 7:315. [PMID: 29376025 PMCID: PMC5770734 DOI: 10.3389/fonc.2017.00315] [Citation(s) in RCA: 121] [Impact Index Per Article: 17.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2017] [Accepted: 12/05/2017] [Indexed: 12/13/2022] Open
Abstract
BACKGROUND Radiotherapy is one of the main treatment methods for nasopharyngeal carcinoma (NPC). It requires exact delineation of the nasopharynx gross tumor volume (GTVnx), the metastatic lymph node gross tumor volume (GTVnd), the clinical target volume (CTV), and organs at risk in the planning computed tomography images. However, this task is time-consuming and operator dependent. In the present study, we developed an end-to-end deep deconvolutional neural network (DDNN) for segmentation of these targets. METHODS The proposed DDNN is an end-to-end architecture enabling fast training and testing. It consists of two important components: an encoder network and a decoder network. The encoder network was used to extract the visual features of a medical image and the decoder network was used to recover the original resolution by deploying deconvolution. A total of 230 patients diagnosed with NPC stage I or stage II were included in this study. Data from 184 patients were chosen randomly as a training set to adjust the parameters of DDNN, and the remaining 46 patients were the test set to assess the performance of the model. The Dice similarity coefficient (DSC) was used to quantify the segmentation results of the GTVnx, GTVnd, and CTV. In addition, the performance of DDNN was compared with the VGG-16 model. RESULTS The proposed DDNN method outperformed the VGG-16 in all the segmentation. The mean DSC values of DDNN were 80.9% for GTVnx, 62.3% for the GTVnd, and 82.6% for CTV, whereas VGG-16 obtained 72.3, 33.7, and 73.7% for the DSC values, respectively. CONCLUSION DDNN can be used to segment the GTVnx and CTV accurately. The accuracy for the GTVnd segmentation was relatively low due to the considerable differences in its shape, volume, and location among patients. The accuracy is expected to increase with more training data and combination of MR images. In conclusion, DDNN has the potential to improve the consistency of contouring and streamline radiotherapy workflows, but careful human review and a considerable amount of editing will be required.
Collapse
Affiliation(s)
- Kuo Men
- National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Xinyuan Chen
- National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Ye Zhang
- National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Tao Zhang
- National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Jianrong Dai
- National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Junlin Yi
- National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yexiong Li
- National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
140
|
Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42:60-88. [PMID: 28778026 DOI: 10.1016/j.media.2017.07.005] [Citation(s) in RCA: 4304] [Impact Index Per Article: 614.9] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2017] [Revised: 07/24/2017] [Accepted: 07/25/2017] [Indexed: 02/07/2023]
Affiliation(s)
- Geert Litjens
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - Thijs Kooi
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | | | | | - Francesco Ciompi
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Mohsen Ghafoorian
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | | | - Bram van Ginneken
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Clara I Sánchez
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| |
Collapse
|
141
|
Guan J, Li J, Liang S, Li R, Li X, Shi X, Huang C, Zhang J, Pan J, Jia H, Zhang L, Chen X, Liao X. NeuroSeg: automated cell detection and segmentation for in vivo two-photon Ca 2+ imaging data. Brain Struct Funct 2017; 223:519-533. [PMID: 29124351 DOI: 10.1007/s00429-017-1545-5] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2017] [Accepted: 10/15/2017] [Indexed: 11/28/2022]
Abstract
Two-photon Ca2+ imaging has become a popular approach for monitoring neuronal population activity with cellular or subcellular resolution in vivo. This approach allows for the recording of hundreds to thousands of neurons per animal and thus leads to a large amount of data to be processed. In particular, manually drawing regions of interest is the most time-consuming aspect of data analysis. However, the development of automated image analysis pipelines, which will be essential for dealing with the likely future deluge of imaging data, remains a major challenge. To address this issue, we developed NeuroSeg, an open-source MATLAB program that can facilitate the accurate and efficient segmentation of neurons in two-photon Ca2+ imaging data. We proposed an approach using a generalized Laplacian of Gaussian filter to detect cells and weighting-based segmentation to separate individual cells from the background. We tested this approach on an in vivo two-photon Ca2+ imaging dataset obtained from mouse cortical neurons with differently sized view fields. We show that this approach exhibits superior performance for cell detection and segmentation compared with the existing published tools. In addition, we integrated the previously reported, activity-based segmentation into our approach and found that this combined method was even more promising. The NeuroSeg software, including source code and graphical user interface, is freely available and will be a useful tool for in vivo brain activity mapping.
Collapse
Affiliation(s)
- Jiangheng Guan
- Brain Research Center, Third Military Medical University, Chongqing, 400038, China
| | - Jingcheng Li
- Brain Research Center, Third Military Medical University, Chongqing, 400038, China
| | - Shanshan Liang
- Brain Research Center, Third Military Medical University, Chongqing, 400038, China
| | - Ruijie Li
- Brain Research Center, Third Military Medical University, Chongqing, 400038, China
| | - Xingyi Li
- Brain Research Center, Third Military Medical University, Chongqing, 400038, China
| | - Xiaozhe Shi
- Brain Research Center, Third Military Medical University, Chongqing, 400038, China.,School of Life Sciences, Peking University, Beijing, 100871, China
| | - Ciyu Huang
- College of Computer and Information Science and College of Software, Southwest University, Chongqing, 400715, China
| | - Jianxiong Zhang
- Brain Research Center, Third Military Medical University, Chongqing, 400038, China
| | - Junxia Pan
- Brain Research Center, Third Military Medical University, Chongqing, 400038, China
| | - Hongbo Jia
- Brain Research Instrument Innovation Center, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, Jiangsu, China
| | - Le Zhang
- College of Computer and Information Science and College of Software, Southwest University, Chongqing, 400715, China
| | - Xiaowei Chen
- Brain Research Center, Third Military Medical University, Chongqing, 400038, China. .,CAS Center for Excellence in Brain Science and Intelligence Technology, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences, Shanghai, 200031, China.
| | - Xiang Liao
- Brain Research Center, Third Military Medical University, Chongqing, 400038, China.
| |
Collapse
|
142
|
Men K, Dai J, Li Y. Automatic segmentation of the clinical target volume and organs at risk in the planning CT for rectal cancer using deep dilated convolutional neural networks. Med Phys 2017; 44:6377-6389. [PMID: 28963779 DOI: 10.1002/mp.12602] [Citation(s) in RCA: 196] [Impact Index Per Article: 28.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2017] [Revised: 09/21/2017] [Accepted: 09/22/2017] [Indexed: 12/13/2022] Open
Abstract
PURPOSE Delineation of the clinical target volume (CTV) and organs at risk (OARs) is very important for radiotherapy but is time-consuming and prone to inter-observer variation. Here, we proposed a novel deep dilated convolutional neural network (DDCNN)-based method for fast and consistent auto-segmentation of these structures. METHODS Our DDCNN method was an end-to-end architecture enabling fast training and testing. Specifically, it employed a novel multiple-scale convolutional architecture to extract multiple-scale context features in the early layers, which contain the original information on fine texture and boundaries and which are very useful for accurate auto-segmentation. In addition, it enlarged the receptive fields of dilated convolutions at the end of networks to capture complementary context features. Then, it replaced the fully connected layers with fully convolutional layers to achieve pixel-wise segmentation. We used data from 278 patients with rectal cancer for evaluation. The CTV and OARs were delineated and validated by senior radiation oncologists in the planning computed tomography (CT) images. A total of 218 patients chosen randomly were used for training, and the remaining 60 for validation. The Dice similarity coefficient (DSC) was used to measure segmentation accuracy. RESULTS Performance was evaluated on segmentation of the CTV and OARs. In addition, the performance of DDCNN was compared with that of U-Net. The proposed DDCNN method outperformed the U-Net for all segmentations, and the average DSC value of DDCNN was 3.8% higher than that of U-Net. Mean DSC values of DDCNN were 87.7% for the CTV, 93.4% for the bladder, 92.1% for the left femoral head, 92.3% for the right femoral head, 65.3% for the intestine, and 61.8% for the colon. The test time was 45 s per patient for segmentation of all the CTV, bladder, left and right femoral heads, colon, and intestine. We also assessed our approaches and results with those in the literature: our system showed superior performance and faster speed. CONCLUSIONS These data suggest that DDCNN can be used to segment the CTV and OARs accurately and efficiently. It was invariant to the body size, body shape, and age of the patients. DDCNN could improve the consistency of contouring and streamline radiotherapy workflows.
Collapse
Affiliation(s)
- Kuo Men
- National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Jianrong Dai
- National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yexiong Li
- National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
143
|
Dou Q, Yu L, Chen H, Jin Y, Yang X, Qin J, Heng PA. 3D deeply supervised network for automated segmentation of volumetric medical images. Med Image Anal 2017; 41:40-54. [DOI: 10.1016/j.media.2017.05.001] [Citation(s) in RCA: 198] [Impact Index Per Article: 28.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2017] [Revised: 04/14/2017] [Accepted: 05/01/2017] [Indexed: 10/19/2022]
|
144
|
Song Y, Li Q, Huang H, Feng D, Chen M, Cai W. Low Dimensional Representation of Fisher Vectors for Microscopy Image Classification. IEEE TRANSACTIONS ON MEDICAL IMAGING 2017; 36:1636-1649. [PMID: 28358678 DOI: 10.1109/tmi.2017.2687466] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
Microscopy image classification is important in various biomedical applications, such as cancer subtype identification, and protein localization for high content screening. To achieve automated and effective microscopy image classification, the representative and discriminative capability of image feature descriptors is essential. To this end, in this paper, we propose a new feature representation algorithm to facilitate automated microscopy image classification. In particular, we incorporate Fisher vector (FV) encoding with multiple types of local features that are handcrafted or learned, and we design a separation-guided dimension reduction method to reduce the descriptor dimension while increasing its discriminative capability. Our method is evaluated on four publicly available microscopy image data sets of different imaging types and applications, including the UCSB breast cancer data set, MICCAI 2015 CBTC challenge data set, and IICBU malignant lymphoma, and RNAi data sets. Our experimental results demonstrate the advantage of the proposed low-dimensional FV representation, showing consistent performance improvement over the existing state of the art and the commonly used dimension reduction techniques.
Collapse
|
145
|
Kumar N, Verma R, Sharma S, Bhargava S, Vahadane A, Sethi A. A Dataset and a Technique for Generalized Nuclear Segmentation for Computational Pathology. IEEE TRANSACTIONS ON MEDICAL IMAGING 2017; 36:1550-1560. [PMID: 28287963 DOI: 10.1109/tmi.2017.2677499] [Citation(s) in RCA: 320] [Impact Index Per Article: 45.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/13/2023]
Abstract
Nuclear segmentation in digital microscopic tissue images can enable extraction of high-quality features for nuclear morphometrics and other analysis in computational pathology. Conventional image processing techniques, such as Otsu thresholding and watershed segmentation, do not work effectively on challenging cases, such as chromatin-sparse and crowded nuclei. In contrast, machine learning-based segmentation can generalize across various nuclear appearances. However, training machine learning algorithms requires data sets of images, in which a vast number of nuclei have been annotated. Publicly accessible and annotated data sets, along with widely agreed upon metrics to compare techniques, have catalyzed tremendous innovation and progress on other image classification problems, particularly in object recognition. Inspired by their success, we introduce a large publicly accessible data set of hematoxylin and eosin (H&E)-stained tissue images with more than 21000 painstakingly annotated nuclear boundaries, whose quality was validated by a medical doctor. Because our data set is taken from multiple hospitals and includes a diversity of nuclear appearances from several patients, disease states, and organs, techniques trained on it are likely to generalize well and work right out-of-the-box on other H&E-stained images. We also propose a new metric to evaluate nuclear segmentation results that penalizes object- and pixel-level errors in a unified manner, unlike previous metrics that penalize only one type of error. We also propose a segmentation technique based on deep learning that lays a special emphasis on identifying the nuclear boundaries, including those between the touching or overlapping nuclei, and works well on a diverse set of test images.
Collapse
|
146
|
An Advanced Deep Learning Approach for Ki-67 Stained Hotspot Detection and Proliferation Rate Scoring for Prognostic Evaluation of Breast Cancer. Sci Rep 2017; 7:3213. [PMID: 28607456 PMCID: PMC5468356 DOI: 10.1038/s41598-017-03405-5] [Citation(s) in RCA: 64] [Impact Index Per Article: 9.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2017] [Accepted: 04/26/2017] [Indexed: 02/08/2023] Open
Abstract
Being a non-histone protein, Ki-67 is one of the essential biomarkers for the immunohistochemical assessment of proliferation rate in breast cancer screening and grading. The Ki-67 signature is always sensitive to radiotherapy and chemotherapy. Due to random morphological, color and intensity variations of cell nuclei (immunopositive and immunonegative), manual/subjective assessment of Ki-67 scoring is error-prone and time-consuming. Hence, several machine learning approaches have been reported; nevertheless, none of them had worked on deep learning based hotspots detection and proliferation scoring. In this article, we suggest an advanced deep learning model for computerized recognition of candidate hotspots and subsequent proliferation rate scoring by quantifying Ki-67 appearance in breast cancer immunohistochemical images. Unlike existing Ki-67 scoring techniques, our methodology uses Gamma mixture model (GMM) with Expectation-Maximization for seed point detection and patch selection and deep learning, comprises with decision layer, for hotspots detection and proliferation scoring. Experimental results provide 93% precision, 0.88% recall and 0.91% F-score value. The model performance has also been compared with the pathologists’ manual annotations and recently published articles. In future, the proposed deep learning framework will be highly reliable and beneficial to the junior and senior pathologists for fast and efficient Ki-67 scoring.
Collapse
|
147
|
Zhang L, Nogues I, Summers RM, Liu S, Yao J. DeepPap: Deep Convolutional Networks for Cervical Cell Classification. IEEE J Biomed Health Inform 2017; 21:1633-1643. [PMID: 28541229 DOI: 10.1109/jbhi.2017.2705583] [Citation(s) in RCA: 143] [Impact Index Per Article: 20.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Automation-assisted cervical screening via Pap smear or liquid-based cytology (LBC) is a highly effective cell imaging based cancer detection tool, where cells are partitioned into "abnormal" and "normal" categories. However, the success of most traditional classification methods relies on the presence of accurate cell segmentations. Despite sixty years of research in this field, accurate segmentation remains a challenge in the presence of cell clusters and pathologies. Moreover, previous classification methods are only built upon the extraction of hand-crafted features, such as morphology and texture. This paper addresses these limitations by proposing a method to directly classify cervical cells-without prior segmentation-based on deep features, using convolutional neural networks (ConvNets). First, the ConvNet is pretrained on a natural image dataset. It is subsequently fine-tuned on a cervical cell dataset consisting of adaptively resampled image patches coarsely centered on the nuclei. In the testing phase, aggregation is used to average the prediction scores of a similar set of image patches. The proposed method is evaluated on both Pap smear and LBC datasets. Results show that our method outperforms previous algorithms in classification accuracy (98.3%), area under the curve (0.99) values, and especially specificity (98.3%), when applied to the Herlev benchmark Pap smear dataset and evaluated using five-fold cross validation. Similar superior performances are also achieved on the HEMLBC (H&E stained manual LBC) dataset. Our method is promising for the development of automation-assisted reading systems in primary cervical screening.
Collapse
|
148
|
Kost H, Homeyer A, Molin J, Lundström C, Hahn HK. Training Nuclei Detection Algorithms with Simple Annotations. J Pathol Inform 2017; 8:21. [PMID: 28584683 PMCID: PMC5450511 DOI: 10.4103/jpi.jpi_3_17] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2017] [Accepted: 03/17/2017] [Indexed: 12/27/2022] Open
Abstract
BACKGROUND Generating good training datasets is essential for machine learning-based nuclei detection methods. However, creating exhaustive nuclei contour annotations, to derive optimal training data from, is often infeasible. METHODS We compared different approaches for training nuclei detection methods solely based on nucleus center markers. Such markers contain less accurate information, especially with regard to nuclear boundaries, but can be produced much easier and in greater quantities. The approaches use different automated sample extraction methods to derive image positions and class labels from nucleus center markers. In addition, the approaches use different automated sample selection methods to improve the detection quality of the classification algorithm and reduce the run time of the training process. We evaluated the approaches based on a previously published generic nuclei detection algorithm and a set of Ki-67-stained breast cancer images. RESULTS A Voronoi tessellation-based sample extraction method produced the best performing training sets. However, subsampling of the extracted training samples was crucial. Even simple class balancing improved the detection quality considerably. The incorporation of active learning led to a further increase in detection quality. CONCLUSIONS With appropriate sample extraction and selection methods, nuclei detection algorithms trained on the basis of simple center marker annotations can produce comparable quality to algorithms trained on conventionally created training sets.
Collapse
Affiliation(s)
- Henning Kost
- Fraunhofer Institute for Medical Image Computing MEVIS, 28359 Bremen, Germany
| | - André Homeyer
- Fraunhofer Institute for Medical Image Computing MEVIS, 28359 Bremen, Germany
| | - Jesper Molin
- Department of Applied Information Technology, Chalmers University of Technology, 41258 Gothenburg, Sweden.,Sectra AB, 58330 Linköping, Sweden.,Center for Medical Image Science and Visualization, Linköping University, 58183 Linköping, Sweden
| | - Claes Lundström
- Sectra AB, 58330 Linköping, Sweden.,Center for Medical Image Science and Visualization, Linköping University, 58183 Linköping, Sweden
| | - Horst Karl Hahn
- Fraunhofer Institute for Medical Image Computing MEVIS, 28359 Bremen, Germany
| |
Collapse
|
149
|
Li S, Jiang H, Pang W. Joint multiple fully connected convolutional neural network with extreme learning machine for hepatocellular carcinoma nuclei grading. Comput Biol Med 2017; 84:156-167. [PMID: 28365546 DOI: 10.1016/j.compbiomed.2017.03.017] [Citation(s) in RCA: 36] [Impact Index Per Article: 5.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2016] [Revised: 03/14/2017] [Accepted: 03/17/2017] [Indexed: 12/27/2022]
Abstract
Accurate cell grading of cancerous tissue pathological image is of great importance in medical diagnosis and treatment. This paper proposes a joint multiple fully connected convolutional neural network with extreme learning machine (MFC-CNN-ELM) architecture for hepatocellular carcinoma (HCC) nuclei grading. First, in preprocessing stage, each grayscale image patch with the fixed size is obtained using center-proliferation segmentation (CPS) method and the corresponding labels are marked under the guidance of three pathologists. Next, a multiple fully connected convolutional neural network (MFC-CNN) is designed to extract the multi-form feature vectors of each input image automatically, which considers multi-scale contextual information of deep layer maps sufficiently. After that, a convolutional neural network extreme learning machine (CNN-ELM) model is proposed to grade HCC nuclei. Finally, a back propagation (BP) algorithm, which contains a new up-sample method, is utilized to train MFC-CNN-ELM architecture. The experiment comparison results demonstrate that our proposed MFC-CNN-ELM has superior performance compared with related works for HCC nuclei grading. Meanwhile, external validation using ICPR 2014 HEp-2 cell dataset shows the good generalization of our MFC-CNN-ELM architecture.
Collapse
Affiliation(s)
- Siqi Li
- Software College, Northeastern University, Shenyang 110819, China.
| | - Huiyan Jiang
- Software College, Northeastern University, Shenyang 110819, China.
| | - Wenbo Pang
- Software College, Northeastern University, Shenyang 110819, China.
| |
Collapse
|
150
|
Tareef A, Song Y, Cai W, Huang H, Chang H, Wang Y, Fulham M, Feng D, Chen M. Automatic segmentation of overlapping cervical smear cells based on local distinctive features and guided shape deformation. Neurocomputing 2017. [DOI: 10.1016/j.neucom.2016.09.070] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|