1
|
Juang CF, Chuang YW, Lin GW, Chung IF, Lo YC. Deep learning-based glomerulus detection and classification with generative morphology augmentation in renal pathology images. Comput Med Imaging Graph 2024; 115:102375. [PMID: 38599040 DOI: 10.1016/j.compmedimag.2024.102375] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Revised: 02/01/2024] [Accepted: 03/19/2024] [Indexed: 04/12/2024]
Abstract
Glomerulus morphology on renal pathology images provides valuable diagnosis and outcome prediction information. To provide better care, an efficient, standardized, and scalable method is urgently needed to optimize the time-consuming and labor-intensive interpretation process by renal pathologists. This paper proposes a deep convolutional neural network (CNN)-based approach to automatically detect and classify glomeruli with different stains in renal pathology images. In the glomerulus detection stage, this paper proposes a flattened Xception with a feature pyramid network (FX-FPN). The FX-FPN is employed as a backbone in the framework of faster region-based CNN to improve glomerulus detection performance. In the classification stage, this paper considers classifications of five glomerulus morphologies using a flattened Xception classifier. To endow the classifier with higher discriminability, this paper proposes a generative data augmentation approach for patch-based glomerulus morphology augmentation. New glomerulus patches of different morphologies are generated for data augmentation through the cycle-consistent generative adversarial network (CycleGAN). The single detection model shows the F1 score up to 0.9524 in H&E and PAS stains. The classification result shows that the average sensitivity and specificity are 0.7077 and 0.9316, respectively, by using the flattened Xception with the original training data. The sensitivity and specificity increase to 0.7623 and 0.9443, respectively, by using the generative data augmentation. Comparisons with different deep CNN models show the effectiveness and superiority of the proposed approach.
Collapse
Affiliation(s)
- Chia-Feng Juang
- Department of Electrical Engineering, National Chung Hsing University, Taichung 40227, Taiwan, ROC
| | - Ya-Wen Chuang
- Section of Nephrology, Department of Medicine, Taichung Veterans General Hospital, Taichung 40705, Taiwan, ROC; Graduate Institute of Biomedical Sciences, College of Medicine, China Medical University, Taichung 406040, Taiwan, ROC; School of Medicine, College of Medicine, China Medical University, Taichung 406040, Taiwan, ROC; Department of Post-Baccalaureate Medicine, College of Medicine, National Chung Hsing University, Taichung 40227, Taiwan, ROC
| | - Guan-Wen Lin
- Department of Electrical Engineering, National Chung Hsing University, Taichung 40227, Taiwan, ROC
| | - I-Fang Chung
- Institute of Biomedical Informatics, National Yang Ming Chiao Tung University, Taipei 112304, Taiwan, ROC.
| | - Ying-Chih Lo
- Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, MA 02115, USA; Harvard Medical School, Boston, MA 02115, USA.
| |
Collapse
|
2
|
van Rijthoven M, Obahor S, Pagliarulo F, van den Broek M, Schraml P, Moch H, van der Laak J, Ciompi F, Silina K. Multi-resolution deep learning characterizes tertiary lymphoid structures and their prognostic relevance in solid tumors. COMMUNICATIONS MEDICINE 2024; 4:5. [PMID: 38182879 PMCID: PMC10770129 DOI: 10.1038/s43856-023-00421-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Accepted: 11/30/2023] [Indexed: 01/07/2024] Open
Abstract
BACKGROUND Tertiary lymphoid structures (TLSs) are dense accumulations of lymphocytes in inflamed peripheral tissues, including cancer, and are associated with improved survival and response to immunotherapy in various solid tumors. Histological TLS quantification has been proposed as a novel predictive and prognostic biomarker, but lack of standardized methods of TLS characterization hampers assessment of TLS densities across different patients, diseases, and clinical centers. METHODS We introduce an approach based on HookNet-TLS, a multi-resolution deep learning model, for automated and unbiased TLS quantification and identification of germinal centers in routine hematoxylin and eosin stained digital pathology slides. We developed HookNet-TLS using n = 1019 manually annotated TCGA slides from clear cell renal cell carcinoma, muscle-invasive bladder cancer, and lung squamous cell carcinoma. RESULTS Here we show that HookNet-TLS automates TLS quantification across multiple cancer types achieving human-level performance and demonstrates prognostic associations similar to visual assessment. CONCLUSIONS HookNet-TLS has the potential to be used as a tool for objective quantification of TLS in routine H&E digital pathology slides. We make HookNet-TLS publicly available to promote its use in research.
Collapse
Affiliation(s)
- Mart van Rijthoven
- Pathology Department, Radboud University Medical Center, Nijmegen, Netherlands.
| | - Simon Obahor
- Institute of Experimental Immunology, University of Zurich, Zurich, Switzerland
| | - Fabio Pagliarulo
- Institute of Experimental Immunology, University of Zurich, Zurich, Switzerland
| | | | - Peter Schraml
- Department of Pathology and Molecular Pathology, University Hospital Zurich, Zurich, Switzerland
| | - Holger Moch
- Department of Pathology and Molecular Pathology, University Hospital Zurich, Zurich, Switzerland
- Faculty of Medicine, University of Zurich, Zurich, Switzerland
| | - Jeroen van der Laak
- Pathology Department, Radboud University Medical Center, Nijmegen, Netherlands
| | - Francesco Ciompi
- Pathology Department, Radboud University Medical Center, Nijmegen, Netherlands
| | - Karina Silina
- Institute of Pharmaceutical Sciences, Swiss Federal Institute of Technology, Zurich, Switzerland
| |
Collapse
|
3
|
Liu X, Wu Y, Chen Y, Hui D, Zhang J, Hao F, Lu Y, Cheng H, Zeng Y, Han W, Wang C, Li M, Zhou X, Zheng W. Diagnosis of diabetic kidney disease in whole slide images via AI-driven quantification of pathological indicators. Comput Biol Med 2023; 166:107470. [PMID: 37722173 DOI: 10.1016/j.compbiomed.2023.107470] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Revised: 08/29/2023] [Accepted: 09/04/2023] [Indexed: 09/20/2023]
Abstract
Diagnosis of diabetic kidney disease (DKD) mainly relies on screening the morphological variations and internal lesions of glomeruli from pathological kidney biopsy. The prominent pathological alterations of glomeruli for DKD include glomerular hypertrophy and nodular mesangial sclerosis. However, the qualitative judgment of these alterations is inaccurate and inconstant due to the intra- and inter-subject variability of pathologists. It is necessary to design artificial intelligence (AI) methods for accurate quantification of these pathological alterations and outcome prediction of DKD. In this work, we present an AI-driven framework to quantify the volume of glomeruli and degree of nodular mesangial sclerosis, respectively, based on an instance segmentation module and a novel weakly supervised Macro-Micro Aggregation (MMA) module. Subsequently, we construct classic machine learning models to predict the degree of DKD based on three selected pathological indicators via factor analysis. These corresponding modules are trained and tested on a total of 281 whole slide images (WSIs) digitized from two hospitals with different scanners. Our designed AI framework achieved inspiring results with 0.926 mIoU for glomerulus segmentation, and 0.899 F1 score for glomerulus classification in the external testing dataset. Meantime, the visualized results of the MMA module could reflect the location of the lesions. The performance of predicting disease achieved the F1 score of 0.917, which further proved the effectiveness of our AI-driven quantification of pathological indicators. Additionally, the interpretation of the machine learning model with the SHAP method showed similar accordance with the development of DKD in pathology. In conclusion, the proposed auxiliary diagnostic technologies have the feasibility for quantitative analysis of glomerular pathological tissues and alterations in DKD. Pathological quantitative indicators will also make it more convenient to provide doctors with assistance in clinical practice.
Collapse
Affiliation(s)
- Xueyu Liu
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Yongfei Wu
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China.
| | - Yilin Chen
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Dongna Hui
- Department of Nephrology, Shanxi Provincial People's Hospital, Taiyuan, Shanxi, China
| | - Jianan Zhang
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Fang Hao
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Yuanyue Lu
- Department of Nephrology, Shanxi Provincial People's Hospital, Taiyuan, Shanxi, China
| | - Hangbei Cheng
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Yue Zeng
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Weixia Han
- Department of Pathology, Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
| | - Chen Wang
- Department of Pathology, Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
| | - Ming Li
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Xiaoshuang Zhou
- Department of Nephrology, Shanxi Provincial People's Hospital, Taiyuan, Shanxi, China.
| | - Wen Zheng
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| |
Collapse
|
4
|
Besusparis J, Morkunas M, Laurinavicius A. A Spatially Guided Machine-Learning Method to Classify and Quantify Glomerular Patterns of Injury in Histology Images. J Imaging 2023; 9:220. [PMID: 37888327 PMCID: PMC10607091 DOI: 10.3390/jimaging9100220] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Revised: 09/26/2023] [Accepted: 10/09/2023] [Indexed: 10/28/2023] Open
Abstract
Introduction The diagnosis of glomerular diseases is primarily based on visual assessment of histologic patterns. Semi-quantitative scoring of active and chronic lesions is often required to assess individual characteristics of the disease. Reproducibility of the visual scoring systems remains debatable, while digital and machine-learning technologies present opportunities to detect, classify and quantify glomerular lesions, also considering their inter- and intraglomerular heterogeneity. MATERIALS AND METHODS We performed a cross-validated comparison of three modifications of a convolutional neural network (CNN)-based approach for recognition and intraglomerular quantification of nine main glomerular patterns of injury. Reference values provided by two nephropathologists were used for validation. For each glomerular image, visual attention heatmaps were generated with a probability of class attribution for further intraglomerular quantification. The quality of classifier-produced heatmaps was evaluated by intersection over union metrics (IoU) between predicted and ground truth localization heatmaps. RESULTS A proposed spatially guided modification of the CNN classifier achieved the highest glomerular pattern classification accuracies, with area under curve (AUC) values up to 0.981. With regards to heatmap overlap area and intraglomerular pattern quantification, the spatially guided classifier achieved a significantly higher generalized mean IoU value compared to single-multiclass and multiple-binary classifiers. CONCLUSIONS We propose a spatially guided CNN classifier that in our experiments reveals the potential to achieve high accuracy for the localization of intraglomerular patterns.
Collapse
Affiliation(s)
- Justinas Besusparis
- Faculty of Medicine, Vilnius University, M.K.Ciurlionio 21, LT-03101 Vilnius, Lithuania; (M.M.); (A.L.)
- National Center of Pathology, Affiliate of Vilnius University Hospital Santaros Clinics, P. Baublio 5, LT-08406 Vilnius, Lithuania
| | - Mindaugas Morkunas
- Faculty of Medicine, Vilnius University, M.K.Ciurlionio 21, LT-03101 Vilnius, Lithuania; (M.M.); (A.L.)
- National Center of Pathology, Affiliate of Vilnius University Hospital Santaros Clinics, P. Baublio 5, LT-08406 Vilnius, Lithuania
| | - Arvydas Laurinavicius
- Faculty of Medicine, Vilnius University, M.K.Ciurlionio 21, LT-03101 Vilnius, Lithuania; (M.M.); (A.L.)
- National Center of Pathology, Affiliate of Vilnius University Hospital Santaros Clinics, P. Baublio 5, LT-08406 Vilnius, Lithuania
| |
Collapse
|
5
|
Kaur G, Garg M, Gupta S, Juneja S, Rashid J, Gupta D, Shah A, Shaikh A. Automatic Identification of Glomerular in Whole-Slide Images Using a Modified UNet Model. Diagnostics (Basel) 2023; 13:3152. [PMID: 37835895 PMCID: PMC10572820 DOI: 10.3390/diagnostics13193152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 09/23/2023] [Accepted: 09/25/2023] [Indexed: 10/15/2023] Open
Abstract
Glomeruli are interconnected capillaries in the renal cortex that are responsible for blood filtration. Damage to these glomeruli often signifies the presence of kidney disorders like glomerulonephritis and glomerulosclerosis, which can ultimately lead to chronic kidney disease and kidney failure. The timely detection of such conditions is essential for effective treatment. This paper proposes a modified UNet model to accurately detect glomeruli in whole-slide images of kidney tissue. The UNet model was modified by changing the number of filters and feature map dimensions from the first to the last layer to enhance the model's capacity for feature extraction. Moreover, the depth of the UNet model was also improved by adding one more convolution block to both the encoder and decoder sections. The dataset used in the study comprised 20 large whole-side images. Due to their large size, the images were cropped into 512 × 512-pixel patches, resulting in a dataset comprising 50,486 images. The proposed model performed well, with 95.7% accuracy, 97.2% precision, 96.4% recall, and 96.7% F1-score. These results demonstrate the proposed model's superior performance compared to the original UNet model, the UNet model with EfficientNetb3, and the current state-of-the-art. Based on these experimental findings, it has been determined that the proposed model accurately identifies glomeruli in extracted kidney patches.
Collapse
Affiliation(s)
- Gurjinder Kaur
- Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura 140401, Punjab, India; (G.K.); (M.G.); (S.G.); (D.G.)
| | - Meenu Garg
- Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura 140401, Punjab, India; (G.K.); (M.G.); (S.G.); (D.G.)
| | - Sheifali Gupta
- Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura 140401, Punjab, India; (G.K.); (M.G.); (S.G.); (D.G.)
| | - Sapna Juneja
- Kulliyyah of Information and Communication Technology, International Islamic University Malaysia, Kuala Lumpur 53100, Malaysia;
| | - Junaid Rashid
- Department of Data Science, Sejong University, Seoul 05006, Republic of Korea;
| | - Deepali Gupta
- Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura 140401, Punjab, India; (G.K.); (M.G.); (S.G.); (D.G.)
| | - Asadullah Shah
- Kulliyyah of Information and Communication Technology, International Islamic University Malaysia, Kuala Lumpur 53100, Malaysia;
| | - Asadullah Shaikh
- Department of Information Systems, College of Computer Science and Information Systems, Najran University, Najran 55461, Saudi Arabia;
| |
Collapse
|
6
|
Singh Samant S, Chauhan A, DN J, Singh V. Glomerulus Detection Using Segmentation Neural Networks. J Digit Imaging 2023; 36:1633-1642. [PMID: 37020148 PMCID: PMC10406769 DOI: 10.1007/s10278-022-00764-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2022] [Revised: 08/29/2022] [Accepted: 12/15/2022] [Indexed: 04/07/2023] Open
Abstract
Digital pathology is vital for the correct diagnosis of kidney before transplantation or kidney disease identification. One of the key challenges in kidney diagnosis is glomerulus detection in kidney tissue segments. In this study, we propose a deep learning-based method for glomerulus detection from digitized kidney slide segments. The proposed method applies models based on convolutional neural networks to detect image segments containing the glomerulus region. We employ various networks such as ResNets, UNet, LinkNet, and EfficientNet to train the models. In our experiments on a network trained on the NIH HuBMAP kidney whole slide image dataset, the proposed method achieves the highest scores with Dice coefficient of 0.942.
Collapse
Affiliation(s)
- Surender Singh Samant
- Department of Computer Science and Engineering, Graphic Era (Deemed to be University), Dehradun, 248002 Uttarkhand India
| | - Arun Chauhan
- Department of Computer Science and Engineering, Graphic Era (Deemed to be University), Dehradun, 248002 Uttarkhand India
| | - Jagadish DN
- Department of Electronics and Communication Engineering, Indian Institute of Information Technology, Dharwad, Karnataka 580009 India
| | - Vijay Singh
- Department of Computer Science and Engineering, Graphic Era (Deemed to be University), Dehradun, 248002 Uttarkhand India
| |
Collapse
|
7
|
Bokhorst JM, Nagtegaal ID, Zlobec I, Dawson H, Sheahan K, Simmer F, Kirsch R, Vieth M, Lugli A, van der Laak J, Ciompi F. Semi-Supervised Learning to Automate Tumor Bud Detection in Cytokeratin-Stained Whole-Slide Images of Colorectal Cancer. Cancers (Basel) 2023; 15:cancers15072079. [PMID: 37046742 PMCID: PMC10093661 DOI: 10.3390/cancers15072079] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2023] [Revised: 03/24/2023] [Accepted: 03/29/2023] [Indexed: 04/03/2023] Open
Abstract
Tumor budding is a histopathological biomarker associated with metastases and adverse survival outcomes in colorectal carcinoma (CRC) patients. It is characterized by the presence of single tumor cells or small clusters of cells within the tumor or at the tumor-invasion front. In order to obtain a tumor budding score for a patient, the region with the highest tumor bud density must first be visually identified by a pathologist, after which buds will be counted in the chosen hotspot field. The automation of this process will expectedly increase efficiency and reproducibility. Here, we present a deep learning convolutional neural network model that automates the above procedure. For model training, we used a semi-supervised learning method, to maximize the detection performance despite the limited amount of labeled training data. The model was tested on an independent dataset in which human- and machine-selected hotspots were mapped in relation to each other and manual and machine detected tumor bud numbers in the manually selected fields were compared. We report the results of the proposed method in comparison with visual assessment by pathologists. We show that the automated tumor bud count achieves a prognostic value comparable with visual estimation, while based on an objective and reproducible quantification. We also explore novel metrics to quantify buds such as density and dispersion and report their prognostic value. We have made the model available for research use on the grand-challenge platform.
Collapse
|
8
|
A multi-task deep learning framework for perineural invasion recognition in gastric cancer whole slide images. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104261] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
9
|
Kawazoe Y, Shimamoto K, Yamaguchi R, Nakamura I, Yoneda K, Shinohara E, Shintani-Domoto Y, Ushiku T, Tsukamoto T, Ohe K. Computational Pipeline for Glomerular Segmentation and Association of the Quantified Regions with Prognosis of Kidney Function in IgA Nephropathy. Diagnostics (Basel) 2022; 12:diagnostics12122955. [PMID: 36552963 PMCID: PMC9776670 DOI: 10.3390/diagnostics12122955] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Revised: 11/20/2022] [Accepted: 11/20/2022] [Indexed: 11/29/2022] Open
Abstract
The histopathological findings of the glomeruli from whole slide images (WSIs) of a renal biopsy play an important role in diagnosing and grading kidney disease. This study aimed to develop an automated computational pipeline to detect glomeruli and to segment the histopathological regions inside of the glomerulus in a WSI. In order to assess the significance of this pipeline, we conducted a multivariate regression analysis to determine whether the quantified regions were associated with the prognosis of kidney function in 46 cases of immunoglobulin A nephropathy (IgAN). The developed pipelines showed a mean intersection over union (IoU) of 0.670 and 0.693 for five classes (i.e., background, Bowman's space, glomerular tuft, crescentic, and sclerotic regions) against the WSI of its facility, and 0.678 and 0.609 against the WSI of the external facility. The multivariate analysis revealed that the predicted sclerotic regions, even those that were predicted by the external model, had a significant negative impact on the slope of the estimated glomerular filtration rate after biopsy. This is the first study to demonstrate that the quantified sclerotic regions that are predicted by an automated computational pipeline for the segmentation of the histopathological glomerular components on WSIs impact the prognosis of kidney function in patients with IgAN.
Collapse
Affiliation(s)
- Yoshimasa Kawazoe
- Artificial Intelligence in Healthcare, Graduate School of Medicine, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
- Correspondence: ; Tel.: +81-3-5800-9077
| | - Kiminori Shimamoto
- Artificial Intelligence in Healthcare, Graduate School of Medicine, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| | - Ryohei Yamaguchi
- Ohshima Memorial Kisen Hospital, 3-5-15, Misaki, Chiba 274-0812, Japan
| | - Issei Nakamura
- NTT DOCOMO, Inc., Sanno Park Tower, 2-11-1, Nagata-cho, Chiyoda-ku, Tokyo 100-6150, Japan
| | - Kota Yoneda
- Department of Reproductive, Developmental, and Aging Sciences, Graduate School of Medicine, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| | - Emiko Shinohara
- Artificial Intelligence in Healthcare, Graduate School of Medicine, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| | - Yukako Shintani-Domoto
- Department of Diagnostic Pathology, Nippon Medical School Hospital, 1-1-5, Sendagi, Bunkyo-ku, Tokyo 113-8602, Japan
| | - Tetsuo Ushiku
- Department of Pathology, Graduate School of Medicine, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| | - Tatsuo Tsukamoto
- Department of Nephrology and Dialysis, Tazuke Kofukai Medical Research Institute, Kitano Hospital, 2-4-20, Ohgimachi, Kita-ku, Osaka 530-8480, Japan
| | - Kazuhiko Ohe
- Department of Biomedical Informatics, Graduate School of Medicine, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| |
Collapse
|
10
|
Yao T, Lu Y, Long J, Jha A, Zhu Z, Asad Z, Yang H, Fogo AB, Huo Y. Glo-In-One: holistic glomerular detection, segmentation, and lesion characterization with large-scale web image mining. J Med Imaging (Bellingham) 2022; 9:052408. [PMID: 35747553 DOI: 10.1117/1.jmi.9.5.052408] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2021] [Accepted: 05/31/2022] [Indexed: 11/14/2022] Open
Abstract
Purpose: The quantitative detection, segmentation, and characterization of glomeruli from high-resolution whole slide imaging (WSI) play essential roles in the computer-assisted diagnosis and scientific research in digital renal pathology. Historically, such comprehensive quantification requires extensive programming skills to be able to handle heterogeneous and customized computational tools. To bridge the gap of performing glomerular quantification for non-technical users, we develop the Glo-In-One toolkit to achieve holistic glomerular detection, segmentation, and characterization via a single line of command. Additionally, we release a large-scale collection of 30,000 unlabeled glomerular images to further facilitate the algorithmic development of self-supervised deep learning. Approach: The inputs of the Glo-In-One toolkit are WSIs, while the outputs are (1) WSI-level multi-class circle glomerular detection results (which can be directly manipulated with ImageScope), (2) glomerular image patches with segmentation masks, and (3) different lesion types. In the current version, the fine-grained global glomerulosclerosis (GGS) characterization is provided, including assessed-solidified-GSS (associated with hypertension-related injury), disappearing-GSS (a further end result of the SGGS becoming contiguous with fibrotic interstitium), and obsolescent-GSS (nonspecific GGS increasing with aging) glomeruli. To leverage the performance of the Glo-In-One toolkit, we introduce self-supervised deep learning to glomerular quantification via large-scale web image mining. Results: The GGS fine-grained classification model achieved a decent performance compared with baseline supervised methods while only using 10% of the annotated data. The glomerular detection achieved an average precision of 0.627 with circle representations, while the glomerular segmentation achieved a 0.955 patch-wise Dice dimilarity coefficient. Conclusion: We develop and release an open-source Glo-In-One toolkit, a software with holistic glomerular detection, segmentation, and lesion characterization. This toolkit is user-friendly to non-technical users via a single line of command. The toolbox and the 30,000 web mined glomerular images have been made publicly available at https://github.com/hrlblab/Glo-In-One.
Collapse
Affiliation(s)
- Tianyuan Yao
- Vanderbilt University, Department of Computer Science, Nashville, Tennessee, United States
| | - Yuzhe Lu
- Vanderbilt University, Department of Computer Science, Nashville, Tennessee, United States
| | - Jun Long
- Central South University, Big Data Institute, Changsha, China
| | - Aadarsh Jha
- Vanderbilt University, Department of Computer Science, Nashville, Tennessee, United States
| | - Zheyu Zhu
- Vanderbilt University, Department of Computer Science, Nashville, Tennessee, United States
| | - Zuhayr Asad
- Vanderbilt University, Department of Computer Science, Nashville, Tennessee, United States
| | - Haichun Yang
- Vanderbilt University Medical Center, Department of Pathology, Microbiology and Immunology, Nashville, Tennessee, United States
| | - Agnes B Fogo
- Vanderbilt University Medical Center, Department of Pathology, Microbiology and Immunology, Nashville, Tennessee, United States
| | - Yuankai Huo
- Vanderbilt University, Department of Computer Science, Nashville, Tennessee, United States
| |
Collapse
|
11
|
Noguchi T, Ando T, Emoto S, Nozawa H, Kawai K, Sasaki K, Murono K, Kishikawa J, Ishi H, Yokoyama Y, Abe S, Nagai Y, Anzai H, Sonoda H, Hata K, Sasaki T, Ishihara S. Artificial Intelligence Program to Predict p53 Mutations in Ulcerative Colitis-Associated Cancer or Dysplasia. Inflamm Bowel Dis 2022; 28:1072-1080. [PMID: 35278081 DOI: 10.1093/ibd/izab350] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/08/2021] [Indexed: 12/26/2022]
Abstract
BACKGROUND The diagnosis of colitis-associated cancer or dysplasia is important in the treatment of ulcerative colitis. Immunohistochemistry of p53 along with hematoxylin and eosin (H&E) staining is conventionally used to accurately diagnose the pathological conditions. However, evaluation of p53 immunohistochemistry in all biopsied specimens is expensive and time-consuming for pathologists. In this study, we aimed to develop an artificial intelligence program using a deep learning algorithm to investigate and predict p53 immunohistochemical staining from H&E-stained slides. METHODS We cropped 25 849 patches from whole-slide images of H&E-stained slides with the corresponding p53-stained slides. These slides were prepared from samples of 12 patients with colitis-associated neoplasia who underwent total colectomy. We annotated all glands in the whole-slide images of the H&E-stained slides and grouped them into 3 classes: p53 positive, p53 negative, and p53 null. We used 80% of the patches for training a convolutional neural network (CNN), 10% for validation, and 10% for final testing. RESULTS The trained CNN glands were classified into 2 or 3 classes according to p53 positivity, with a mean average precision of 0.731 to 0.754. The accuracy, sensitivity (recall), specificity, positive predictive value (precision), and F-measure of the prediction of p53 immunohistochemical staining of the glands detected by the trained CNN were 0.86 to 0.91, 0.73 to 0.83, 0.91 to 0.92, 0.82 to 0.89, and 0.77 to 0.86, respectively. CONCLUSIONS Our trained CNN can be used as a reasonable alternative to conventional p53 immunohistochemical staining in the pathological diagnosis of colitis-associated neoplasia, which is accurate, saves time, and is cost-effective.
Collapse
Affiliation(s)
- Tatsuki Noguchi
- Department of Surgical Oncology, University of Tokyo, Tokyo, Japan
| | - Takumi Ando
- Department of Pathology, University of Tokyo, Tokyo, Japan.,Department of Next-Generation Pathology Information Networking, University of Tokyo, Tokyo, Japan
| | - Shigenobu Emoto
- Department of Surgical Oncology, University of Tokyo, Tokyo, Japan
| | - Hiroaki Nozawa
- Department of Surgical Oncology, University of Tokyo, Tokyo, Japan
| | - Kazushige Kawai
- Department of Surgical Oncology, University of Tokyo, Tokyo, Japan
| | - Kazuhito Sasaki
- Department of Surgical Oncology, University of Tokyo, Tokyo, Japan
| | - Koji Murono
- Department of Surgical Oncology, University of Tokyo, Tokyo, Japan
| | - Junko Kishikawa
- Department of Surgical Oncology, University of Tokyo, Tokyo, Japan
| | - Hiroaki Ishi
- Department of Surgical Oncology, University of Tokyo, Tokyo, Japan
| | | | - Shinya Abe
- Department of Surgical Oncology, University of Tokyo, Tokyo, Japan
| | - Yuzo Nagai
- Department of Surgical Oncology, University of Tokyo, Tokyo, Japan
| | - Hiroyuki Anzai
- Department of Surgical Oncology, University of Tokyo, Tokyo, Japan
| | - Hirofumi Sonoda
- Department of Surgical Oncology, University of Tokyo, Tokyo, Japan
| | - Keisuke Hata
- Department of Surgical Oncology, University of Tokyo, Tokyo, Japan
| | - Takeshi Sasaki
- Department of Pathology, University of Tokyo, Tokyo, Japan.,Department of Next-Generation Pathology Information Networking, University of Tokyo, Tokyo, Japan
| | | |
Collapse
|
12
|
Girolami I, Pantanowitz L, Marletta S, Hermsen M, van der Laak J, Munari E, Furian L, Vistoli F, Zaza G, Cardillo M, Gesualdo L, Gambaro G, Eccher A. Artificial intelligence applications for pre-implantation kidney biopsy pathology practice: a systematic review. J Nephrol 2022; 35:1801-1808. [PMID: 35441256 PMCID: PMC9458558 DOI: 10.1007/s40620-022-01327-8] [Citation(s) in RCA: 25] [Impact Index Per Article: 12.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2022] [Accepted: 03/28/2022] [Indexed: 10/29/2022]
Abstract
BACKGROUND Transplant nephropathology is a highly specialized field of pathology comprising both the evaluation of organ donor biopsy for organ allocation and post-transplant graft biopsy for assessment of rejection or graft damage. The introduction of digital pathology with whole-slide imaging (WSI) in clinical research, trials and practice has catalyzed the application of artificial intelligence (AI) for histopathology, with development of novel machine-learning models for tissue interrogation and discovery. We aimed to review the literature for studies specifically applying AI algorithms to WSI-digitized pre-implantation kidney biopsy. METHODS A systematic search was carried out in the electronic databases PubMed-MEDLINE and Embase until 25th September, 2021 with a combination of the key terms "kidney", "biopsy", "transplantation" and "artificial intelligence" and their aliases. Studies dealing with the application of AI algorithms coupled with WSI in pre-implantation kidney biopsies were included. The main theme addressed was detection and quantification of tissue components. Extracted data were: author, year and country of the study, type of biopsy features investigated, number of cases, type of algorithm deployed, main results of the study in terms of diagnostic outcome, and the main limitations of the study. RESULTS Of 5761 retrieved articles, 7 met our inclusion criteria. All studies focused largely on AI-based detection and classification of glomerular structures and to a lesser extent on tubular and vascular structures. Performance of AI algorithms was excellent and promising. CONCLUSION All studies highlighted the importance of expert pathologist annotation to reliably train models and the need to acknowledge clinical nuances of the pre-implantation setting. Close cooperation between computer scientists and practicing as well as expert renal pathologists is needed, helping to refine the performance of AI-based models for routine pre-implantation kidney biopsy clinical practice.
Collapse
Affiliation(s)
- Ilaria Girolami
- Division of Pathology, Central Hospital Bolzano, Bolzano, Italy
| | - Liron Pantanowitz
- Department of Pathology and Clinical Labs, University of Michigan, Ann Arbor, MI, USA
| | - Stefano Marletta
- Department of Diagnostics and Public Health, University and Hospital Trust of Verona, Verona, Italy
| | - Meyke Hermsen
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Enrico Munari
- Pathology Unit, Department of Molecular and Translational Medicine, Spedali Civili-University of Brescia, Brescia, Italy
| | - Lucrezia Furian
- Department of Surgical, Oncological and Gastroenterological Sciences, Unit of Kidney and Pancreas Transplantation, University of Padua, Padua, Italy
| | - Fabio Vistoli
- Division of General and Transplant Surgery, University of Pisa, Pisa, Italy
| | - Gianluigi Zaza
- Department of Nephro-Urology, Nephrology, Dialysis and Transplant Unit, University of Foggia, Foggia, Italy
| | | | - Loreto Gesualdo
- Nephrology, Dialysis, and Transplantation Unit, Department of Emergency and Organ Transplantation, University of Bari Aldo Moro, Bari, Italy
| | - Giovanni Gambaro
- Department of General Medicine, Renal Unit, University and Hospital Trust of Verona, Verona, Italy
| | - Albino Eccher
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, P.le Stefani n. 1, 37126, Verona, Italy.
| |
Collapse
|
13
|
Aljabri M, AlAmir M, AlGhamdi M, Abdel-Mottaleb M, Collado-Mesa F. Towards a better understanding of annotation tools for medical imaging: a survey. MULTIMEDIA TOOLS AND APPLICATIONS 2022; 81:25877-25911. [PMID: 35350630 PMCID: PMC8948453 DOI: 10.1007/s11042-022-12100-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/06/2021] [Revised: 08/04/2021] [Accepted: 01/03/2022] [Indexed: 05/07/2023]
Abstract
Medical imaging refers to several different technologies that are used to view the human body to diagnose, monitor, or treat medical conditions. It requires significant expertise to efficiently and correctly interpret the images generated by each of these technologies, which among others include radiography, ultrasound, and magnetic resonance imaging. Deep learning and machine learning techniques provide different solutions for medical image interpretation including those associated with detection and diagnosis. Despite the huge success of deep learning algorithms in image analysis, training algorithms to reach human-level performance in these tasks depends on the availability of large amounts of high-quality training data, including high-quality annotations to serve as ground-truth. Different annotation tools have been developed to assist with the annotation process. In this survey, we present the currently available annotation tools for medical imaging, including descriptions of graphical user interfaces (GUI) and supporting instruments. The main contribution of this study is to provide an intensive review of the popular annotation tools and show their successful usage in annotating medical imaging dataset to guide researchers in this area.
Collapse
Affiliation(s)
- Manar Aljabri
- Department of Computer Science, Umm Al-Qura University, Mecca, Saudi Arabia
| | - Manal AlAmir
- Department of Computer Science, Umm Al-Qura University, Mecca, Saudi Arabia
| | - Manal AlGhamdi
- Department of Computer Science, Umm Al-Qura University, Mecca, Saudi Arabia
| | | | - Fernando Collado-Mesa
- Department of Radiology, University of Miami Miller School of Medicine, Florida, FL USA
| |
Collapse
|
14
|
Nguyen EH, Yang H, Deng R, Lu Y, Zhu Z, Roland JT, Lu L, Landman BA, Fogo AB, Huo Y. Circle Representation for Medical Object Detection. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:746-754. [PMID: 34699352 PMCID: PMC8963364 DOI: 10.1109/tmi.2021.3122835] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Box representation has been extensively used for object detection in computer vision. Such representation is efficacious but not necessarily optimized for biomedical objects (e.g., glomeruli), which play an essential role in renal pathology. In this paper, we propose a simple circle representation for medical object detection and introduce CircleNet, an anchor-free detection framework. Compared with the conventional bounding box representation, the proposed bounding circle representation innovates in three-fold: (1) it is optimized for ball-shaped biomedical objects; (2) The circle representation reduced the degree of freedom compared with box representation; (3) It is naturally more rotation invariant. When detecting glomeruli and nuclei on pathological images, the proposed circle representation achieved superior detection performance and be more rotation-invariant, compared with the bounding box. The code has been made publicly available: https://github.com/hrlblab/CircleNet.
Collapse
|
15
|
Weis CA, Bindzus JN, Voigt J, Runz M, Hertjens S, Gaida MM, Popovic ZV, Porubsky S. Assessment of glomerular morphological patterns by deep learning algorithms. J Nephrol 2022; 35:417-427. [PMID: 34982414 PMCID: PMC8927010 DOI: 10.1007/s40620-021-01221-9] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/30/2021] [Indexed: 12/11/2022]
Abstract
Background Compilation of different morphological lesion signatures is characteristic of renal pathology. Previous studies have documented the potential value of artificial intelligence (AI) in recognizing relatively clear-cut glomerular structures and patterns, such as segmental or global sclerosis or mesangial hypercellularity. This study aimed to test the capacity of deep learning algorithms to recognize complex glomerular structural changes that reflect common diagnostic dilemmas in nephropathology. Methods For this purpose, we defined nine classes of glomerular morphological patterns and trained twelve convolutional neuronal network (CNN) models on these. The two-step training process was done on a first dataset defined by an expert nephropathologist (12,253 images) and a second consensus dataset (11,142 images) defined by three experts in the field. Results The efficacy of CNN training was evaluated using another set with 180 consensus images, showing convincingly good classification results (kappa-values 0.838–0.938). Furthermore, we elucidated the image areas decisive for CNN-based decision making by class activation maps. Finally, we demonstrated that the algorithm could decipher glomerular disease patterns coinciding in a single glomerulus (e.g. necrosis along with mesangial and endocapillary hypercellularity). Conclusions In summary, our model, focusing on glomerular lesions detectable by conventional microscopy, is the first sui generis to deploy deep learning as a reliable and promising tool in recognition of even discrete and/or overlapping morphological changes. Our results provide a stimulus for ongoing projects that integrate further input levels next to morphology (such as immunohistochemistry, electron microscopy, and clinical information) to develop a novel tool applicable for routine diagnostic nephropathology. Supplementary Information The online version contains supplementary material available at 10.1007/s40620-021-01221-9.
Collapse
Affiliation(s)
- Cleo-Aron Weis
- Institute of Pathology, University Medical Centre Mannheim, University of Heidelberg, 68167, Mannheim, Germany.
| | - Jan Niklas Bindzus
- Institute of Pathology, University Medical Centre Mannheim, University of Heidelberg, 68167, Mannheim, Germany
| | - Jonas Voigt
- Institute of Pathology, University Medical Centre Mannheim, University of Heidelberg, 68167, Mannheim, Germany
| | - Marlen Runz
- Institute of Pathology, University Medical Centre Mannheim, University of Heidelberg, 68167, Mannheim, Germany.,Mannheim Institute for Intelligent Systems in Medicine, University Medical Centre Mannheim, University of Heidelberg, Mannheim, Germany
| | - Svetlana Hertjens
- Institute of Medical Statistics and Biometry, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| | - Matthias M Gaida
- Institute of Pathology, University Medical Center of the Johannes Gutenberg University Mainz, Langenbeckstrasse 1, 55131, Mainz, Germany
| | - Zoran V Popovic
- Institute of Pathology, University Medical Centre Mannheim, University of Heidelberg, 68167, Mannheim, Germany
| | - Stefan Porubsky
- Institute of Pathology, University Medical Center of the Johannes Gutenberg University Mainz, Langenbeckstrasse 1, 55131, Mainz, Germany.
| |
Collapse
|
16
|
OUP accepted manuscript. Nephrol Dial Transplant 2022; 37:2093-2101. [DOI: 10.1093/ndt/gfac143] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Indexed: 11/12/2022] Open
|
17
|
Zheng Z, Zhang X, Ding J, Zhang D, Cui J, Fu X, Han J, Zhu P. Deep Learning-Based Artificial Intelligence System for Automatic Assessment of Glomerular Pathological Findings in Lupus Nephritis. Diagnostics (Basel) 2021; 11:diagnostics11111983. [PMID: 34829330 PMCID: PMC8621095 DOI: 10.3390/diagnostics11111983] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Revised: 10/19/2021] [Accepted: 10/21/2021] [Indexed: 12/14/2022] Open
Abstract
Accurate assessment of renal histopathology is crucial for the clinical management of patients with lupus nephritis (LN). However, the current classification system has poor interpathologist agreement. This paper proposes a deep convolutional neural network (CNN)-based system that detects and classifies glomerular pathological findings in LN. A dataset of 349 renal biopsy whole-slide images (WSIs) (163 patients with LN, periodic acid-Schiff stain, 3906 glomeruli) annotated by three expert nephropathologists was used. The CNN models YOLOv4 and VGG16 were employed to localise the glomeruli and classify glomerular lesions (slight/severe impairments or sclerotic lesions). An additional 321 unannotated WSIs from 161 patients were used for performance evaluation at the per-patient kidney level. The proposed model achieved an accuracy of 0.951 and Cohen's kappa of 0.932 (95% CI 0.915-0.949) for the entire test set for classifying the glomerular lesions. For multiclass detection at the glomerular level, the mean average precision of the CNN was 0.807, with 'slight' and 'severe' glomerular lesions being easily identified (F1: 0.924 and 0.952, respectively). At the per-patient kidney level, the model achieved a high agreement with nephropathologist (linear weighted kappa: 0.855, 95% CI: 0.795-0.916, p < 0.001; quadratic weighted kappa: 0.906, 95% CI: 0.873-0.938, p < 0.001). The results suggest that deep learning is a feasible assistive tool for the objective and automatic assessment of pathological LN lesions.
Collapse
Affiliation(s)
- Zhaohui Zheng
- Department of Clinical Immunology, Xijing Hospital, Fourth Military Medical University, Xi’an 710032, China; (Z.Z.); (J.D.); (X.F.)
| | - Xiangsen Zhang
- School of Automation, Northwestern Polytechnical University, Xi’an 710072, China; (X.Z.); (D.Z.)
| | - Jin Ding
- Department of Clinical Immunology, Xijing Hospital, Fourth Military Medical University, Xi’an 710032, China; (Z.Z.); (J.D.); (X.F.)
| | - Dingwen Zhang
- School of Automation, Northwestern Polytechnical University, Xi’an 710072, China; (X.Z.); (D.Z.)
| | - Jihong Cui
- Lab of Tissue Engineering, College of Life Sciences, Northwest University, Xi’an 710069, China;
| | - Xianghui Fu
- Department of Clinical Immunology, Xijing Hospital, Fourth Military Medical University, Xi’an 710032, China; (Z.Z.); (J.D.); (X.F.)
| | - Junwei Han
- School of Automation, Northwestern Polytechnical University, Xi’an 710072, China; (X.Z.); (D.Z.)
- Correspondence: (J.H.); (P.Z.)
| | - Ping Zhu
- Department of Clinical Immunology, Xijing Hospital, Fourth Military Medical University, Xi’an 710032, China; (Z.Z.); (J.D.); (X.F.)
- Correspondence: (J.H.); (P.Z.)
| |
Collapse
|
18
|
Yang CK, Lee CY, Wang HS, Huang SC, Liang PI, Chen JS, Kuo CF, Tu KH, Yeh CY, Chen TD. Glomerular Disease Classification and Lesion Identification by Machine Learning. Biomed J 2021; 45:675-685. [PMID: 34506971 PMCID: PMC9486238 DOI: 10.1016/j.bj.2021.08.011] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2020] [Revised: 06/21/2021] [Accepted: 08/31/2021] [Indexed: 12/13/2022] Open
Abstract
Background Classification of glomerular diseases and identification of glomerular lesions require careful morphological examination by experienced nephropathologists, which is labor-intensive, time-consuming, and prone to interobserver variability. In this regard, recent advance in machine learning-based image analysis is promising. Methods We combined Mask Region-based Convolutional Neural Networks (Mask R–CNN) with an additional classification step to build a glomerulus detection model using human kidney biopsy samples. A Long Short-Term Memory (LSTM) recurrent neural network was applied for glomerular disease classification, and another two-stage model using ResNeXt-101 was constructed for glomerular lesion identification in cases of lupus nephritis. Results The detection model showed state-of-the-art performance on variedly stained slides with F1 scores up to 0.944. The disease classification model showed good accuracies up to 0.940 on recognizing different glomerular diseases based on H&E whole slide images. The lesion identification model demonstrated high discriminating power with area under the receiver operating characteristic curve up to 0.947 for various glomerular lesions. Models showed good generalization on external testing datasets. Conclusion This study is the first-of-its-kind showing how each step of kidney biopsy interpretation carried out by nephropathologists can be captured and simulated by machine learning models. The models were integrated into a whole slide image viewing and annotating platform to enable nephropathologists to review, correct, and confirm the inference results. Further improvement on model performances and incorporating inputs from immunofluorescence, electron microscopy, and clinical data might realize actual clinical use.
Collapse
Affiliation(s)
- Cheng-Kun Yang
- aetherAI, Co., Ltd., 9F., No.3-2, Park St., Nangang Dist., Taipei City 115, Taiwan.
| | - Ching-Yi Lee
- aetherAI, Co., Ltd., 9F., No.3-2, Park St., Nangang Dist., Taipei City 115, Taiwan.
| | - Hsiang-Sheng Wang
- Department of Anatomic Pathology, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Shun-Chen Huang
- Department of Anatomic Pathology, Chang Gung Memorial Hospital Kaohsiung Branch, No. 123, Dapi Rd., Niaosong Dist., Kaohsiung City 833, Taiwan.
| | - Peir-In Liang
- Department of Pathology, Kaohsiung Medical University Hospital, No. 100, Ziyou 1st Rd., Sanmin Dist., Kaohsiung City 807, Taiwan.
| | - Jung-Sheng Chen
- Center for Artificial Intelligence in Medicine, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Chang-Fu Kuo
- Center for Artificial Intelligence in Medicine, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Kun-Hua Tu
- Department of Nephrology, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Chao-Yuan Yeh
- aetherAI, Co., Ltd., 9F., No.3-2, Park St., Nangang Dist., Taipei City 115, Taiwan.
| | - Tai-Di Chen
- Department of Anatomic Pathology, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| |
Collapse
|
19
|
Liu X, Li M, Wu Y, Chen Y, Hao F, Zhou D, Wang C, Ma C, Shi G, Zhou X. An efficient glomerular object locator for renal whole slide images using proposal-free network and dynamic scale evaluation method. AI COMMUN 2021. [DOI: 10.3233/aic-210073] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
In the diagnosis of chronic kidney disease, glomerulus as the blood filter provides important information for an accurate disease diagnosis. Thus automatic localization of the glomeruli is the necessary groundwork for future auxiliary kidney disease diagnosis, such as glomerular classification and area measurement. In this paper, we propose an efficient glomerular object locator in kidney whole slide image(WSI) based on proposal-free network and dynamic scale evaluation method. In the training phase, we construct an intensive proposal-free network which can learn efficiently the fine-grained features of the glomerulus. In the evaluation phase, a dynamic scale evaluation method is utilized to help the well-trained model find the most appropriate evaluation scale for each high-resolution WSI. We collect and digitalize 1204 renal biopsy microscope slides containing more than 41000 annotated glomeruli, which is the largest number of dataset to our best knowledge. We validate the each component of the proposed locator via the ablation study. Experimental results confirm that the proposed locator outperforms recently proposed approaches and pathologists by comparing F 1 and run time in localizing glomeruli from WSIs at a resolution of 0.25 μm/pixel and thus achieves state-of-the-art performance. Particularly, the proposed locator can be embedded into the renal intelligent auxiliary diagnosis system for renal clinical diagnosis by localizing glomeruli in high-resolution WSIs effectively.
Collapse
Affiliation(s)
- Xueyu Liu
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
| | - Ming Li
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
| | - Yongfei Wu
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
- Faculty of Science and Technology, University of Macau, Taipa, Macau, China. E-mail:
| | - Yilin Chen
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
| | - Fang Hao
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
| | - Daoxiang Zhou
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
| | - Chen Wang
- Department of Pathology, Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, China. E-mail:
| | - Chuanfeng Ma
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
| | - Guangze Shi
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
| | - Xiaoshuang Zhou
- Department of Nephrology, Shanxi Provincial People’s Hospital, Taiyuan, Shanxi, China. E-mail:
| |
Collapse
|
20
|
Salvi M, Mogetta A, Gambella A, Molinaro L, Barreca A, Papotti M, Molinari F. Automated assessment of glomerulosclerosis and tubular atrophy using deep learning. Comput Med Imaging Graph 2021; 90:101930. [PMID: 33964790 DOI: 10.1016/j.compmedimag.2021.101930] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2020] [Revised: 02/20/2021] [Accepted: 04/21/2021] [Indexed: 01/16/2023]
Abstract
In kidney transplantations, pathologists evaluate the architecture of both glomeruli, interstitium and tubules to assess the nephron status. An accurate assessment of glomerulosclerosis and tubular atrophy is crucial for determining kidney acceptance, which is currently based on the pathologists' histological evaluations on renal biopsies in addition to clinical data. In this work, we present an automated algorithm, called RENTAG (Robust EvaluatioN of Tubular Atrophy & Glomerulosclerosis), for the segmentation and classification of glomerular and tubular structures in histopathological images. The proposed novel strategy combines the accuracy of a level-set with the semantic segmentation of convolutional neural networks to detect the glomeruli and tubules contours. In the TEST set, our method exhibited excellent performance in both glomeruli (dice score: 0.9529) and tubule (dice score: 0.9174) detection and outperformed all the compared methods. To the best of our knowledge, the RENTAG algorithm is the first fully automated method capable of quantifying glomerulosclerosis and tubular atrophy in digital histological images. The developed software can be employed for the analysis of pre-transplantation biopsies to support the pathologists' diagnostic activity.
Collapse
Affiliation(s)
- Massimo Salvi
- Politecnico di Torino, PoliToBIOMed Lab, Biolab, Department of Electronics and Telecommunications, Corso Duca degli Abruzzi 24, Turin, 10129, Italy.
| | - Alessandro Mogetta
- Politecnico di Torino, PoliToBIOMed Lab, Biolab, Department of Electronics and Telecommunications, Corso Duca degli Abruzzi 24, Turin, 10129, Italy
| | - Alessandro Gambella
- Pathology Unit, Department of Medical Sciences, University of Turin, Via Santena 7, Turin, 10126, Italy
| | - Luca Molinaro
- A.O.U. Città della Salute e della Scienza Hospital, Division of Pathology, Corso Bramante 88, Turin, 10126, Italy
| | - Antonella Barreca
- A.O.U. Città della Salute e della Scienza Hospital, Division of Pathology, Corso Bramante 88, Turin, 10126, Italy
| | - Mauro Papotti
- University of Turin, Division of Pathology, Department of Oncology, Via Santena 7, Turin, 10126, Italy
| | - Filippo Molinari
- Politecnico di Torino, PoliToBIOMed Lab, Biolab, Department of Electronics and Telecommunications, Corso Duca degli Abruzzi 24, Turin, 10129, Italy
| |
Collapse
|
21
|
Lo YC, Chung IF, Guo SN, Wen MC, Juang CF. Cycle-consistent GAN-based stain translation of renal pathology images with glomerulus detection application. Appl Soft Comput 2021. [DOI: 10.1016/j.asoc.2020.106822] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
22
|
A Deep Learning Instance Segmentation Approach for Global Glomerulosclerosis Assessment in Donor Kidney Biopsies. ELECTRONICS 2020. [DOI: 10.3390/electronics9111768] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
The histological assessment of glomeruli is fundamental for determining if a kidney is suitable for transplantation. The Karpinski score is essential to evaluate the need for a single or dual kidney transplant and includes the ratio between the number of sclerotic glomeruli and the overall number of glomeruli in a kidney section. The manual evaluation of kidney biopsies performed by pathologists is time-consuming and error-prone, so an automatic framework to delineate all the glomeruli present in a kidney section can be very useful. Our experiments have been conducted on a dataset provided by the Department of Emergency and Organ Transplantations (DETO) of Bari University Hospital. This dataset is composed of 26 kidney biopsies coming from 19 donors. The rise of Convolutional Neural Networks (CNNs) has led to a realm of methods which are widely applied in Medical Imaging. Deep learning techniques are also very promising for the segmentation of glomeruli, with a variety of existing approaches. Many methods only focus on semantic segmentation—which consists in segmentation of individual pixels—or ignore the problem of discriminating between non-sclerotic and sclerotic glomeruli, so these approaches are not optimal or inadequate for transplantation assessment. In this work, we employed an end-to-end fully automatic approach based on Mask R-CNN for instance segmentation and classification of glomeruli. We also compared the results obtained with a baseline based on Faster R-CNN, which only allows detection at bounding boxes level. With respect to the existing literature, we improved the Mask R-CNN approach in sliding window contexts, by employing a variant of the Non-Maximum Suppression (NMS) algorithm, which we called Non-Maximum-Area Suppression (NMAS). The obtained results are very promising, leading to improvements over existing literature. The baseline Faster R-CNN-based approach obtained an F-Measure of 0.904 and 0.667 for non-sclerotic and sclerotic glomeruli, respectively. The Mask R-CNN approach has a significant improvement over the baseline, obtaining an F-Measure of 0.925 and 0.777 for non-sclerotic and sclerotic glomeruli, respectively. The proposed method is very promising for the instance segmentation and classification of glomeruli, and allows to make a robust evaluation of global glomerulosclerosis. We also compared Karpinski score obtained with our algorithm to that obtained with pathologists’ annotations to show the soundness of the proposed workflow from a clinical point of view.
Collapse
|
23
|
Yang H, Deng R, Lu Y, Zhu Z, Chen Y, Roland JT, Lu L, Landman BA, Fogo AB, Huo Y. CircleNet: Anchor-free Glomerulus Detection with Circle Representation. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION : MICCAI ... INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION 2020; 2020:35-44. [PMID: 34414404 PMCID: PMC8372751 DOI: 10.1007/978-3-030-59719-1_4] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/03/2022]
Abstract
Object detection networks are powerful in computer vision, but not necessarily optimized for biomedical object detection. In this work, we propose CircleNet, a simple anchor-free detection method with circle representation for detection of the ball-shaped glomerulus. Different from the traditional bounding box based detection method, the bounding circle (1) reduces the degrees of freedom of detection representation, (2) is naturally rotation invariant, (3) and optimized for ball-shaped objects. The key innovation to enable this representation is the anchor-free framework with the circle detection head. We evaluate CircleNet in the context of detection of glomerulus. CircleNet increases average precision of the glomerulus detection from 0.598 to 0.647. Another key advantage is that CircleNet achieves better rotation consistency compared with bounding box representations.
Collapse
Affiliation(s)
- Haichun Yang
- Vanderbilt University Medical Center, Nashville TN 37215, USA
| | | | - Yuzhe Lu
- Vanderbilt University, Nashville TN 37215, USA
| | - Zheyu Zhu
- Vanderbilt University, Nashville TN 37215, USA
| | - Ye Chen
- Vanderbilt University, Nashville TN 37215, USA
| | - Joseph T Roland
- Vanderbilt University Medical Center, Nashville TN 37215, USA
| | - Le Lu
- PAII Inc., Bethesda MD 20817, USA
| | | | - Agnes B Fogo
- Vanderbilt University Medical Center, Nashville TN 37215, USA
| | - Yuankai Huo
- Vanderbilt University, Nashville TN 37215, USA
| |
Collapse
|
24
|
Uchino E, Suzuki K, Sato N, Kojima R, Tamada Y, Hiragi S, Yokoi H, Yugami N, Minamiguchi S, Haga H, Yanagita M, Okuno Y. Classification of glomerular pathological findings using deep learning and nephrologist-AI collective intelligence approach. Int J Med Inform 2020; 141:104231. [PMID: 32682317 DOI: 10.1016/j.ijmedinf.2020.104231] [Citation(s) in RCA: 39] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2019] [Revised: 05/30/2020] [Accepted: 07/06/2020] [Indexed: 12/20/2022]
Abstract
BACKGROUND Automated classification of glomerular pathological findings is potentially beneficial in establishing an efficient and objective diagnosis in renal pathology. While previous studies have verified the artificial intelligence (AI) models for the classification of global sclerosis and glomerular cell proliferation, there are several other glomerular pathological findings required for diagnosis, and the comprehensive models for the classification of these major findings have not yet been reported. Whether the cooperation between these AI models and clinicians improves diagnostic performance also remains unknown. Here, we developed AI models to classify glomerular images for major findings required for pathological diagnosis and investigated whether those models could improve the diagnostic performance of nephrologists. METHODS We used a dataset of 283 kidney biopsy cases comprising 15,888 glomerular images that were annotated by a total of 25 nephrologists. AI models to classify seven pathological findings: global sclerosis, segmental sclerosis, endocapillary proliferation, mesangial matrix accumulation, mesangial cell proliferation, crescent, and basement membrane structural changes, were constructed using deep learning by fine-tuning of InceptionV3 convolutional neural network. Subsequently, we compared the agreement to truth labels between majority decision among nephrologists with or without the AI model as a voter. RESULTS Our model for global sclerosis showed high performance (area under the curve: periodic acid-Schiff, 0.986; periodic acid methenamine silver, 0.983); the models for the other findings also showed performance close to those of nephrologists. By adding the AI model output to majority decision among nephrologists, out of the 14 constructed models, the results of the majority decision showed improvement in sensitivity for 10 models (four of them were statistically significant) and specificity for eight models (five significant). CONCLUSION Our study showed a proof-of-concept for the classification of multiple glomerular findings in a comprehensive method of deep learning and suggested its potential effectiveness in improving diagnostic accuracy of clinicians.
Collapse
Affiliation(s)
- Eiichiro Uchino
- Department of Medical Intelligent Systems, Graduate School of Medicine, Kyoto University, Kyoto, Japan; Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | | | - Noriaki Sato
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan; Department of Biomedical Data Intelligence, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Ryosuke Kojima
- Department of Biomedical Data Intelligence, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Yoshinori Tamada
- Department of Medical Intelligent Systems, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Shusuke Hiragi
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan; Division of Medical Informatics and Administration Planning, Kyoto University Hospital, Kyoto, Japan
| | - Hideki Yokoi
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | | | - Sachiko Minamiguchi
- Department of Diagnostic Pathology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Hironori Haga
- Department of Diagnostic Pathology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Motoko Yanagita
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan; Institute for the Advanced Study of Human Biology (ASHBi), Kyoto University, Kyoto, Japan.
| | - Yasushi Okuno
- Department of Biomedical Data Intelligence, Graduate School of Medicine, Kyoto University, Kyoto, Japan; RIKEN, The Drug Development Data Intelligence Platform Group, Yokohama, Japan.
| |
Collapse
|
25
|
Inaba A, Hori K, Yoda Y, Ikematsu H, Takano H, Matsuzaki H, Watanabe Y, Takeshita N, Tomioka T, Ishii G, Fujii S, Hayashi R, Yano T. Artificial intelligence system for detecting superficial laryngopharyngeal cancer with high efficiency of deep learning. Head Neck 2020; 42:2581-2592. [PMID: 32542892 DOI: 10.1002/hed.26313] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2019] [Revised: 04/18/2020] [Accepted: 05/15/2020] [Indexed: 12/29/2022] Open
Abstract
BACKGROUND There are no published reports evaluating the ability of artificial intelligence (AI) in the endoscopic diagnosis of superficial laryngopharyngeal cancer (SLPC). We presented our newly developed diagnostic AI model for SLPC detection. METHODS We used RetinaNet for object detection. SLPC and normal laryngopharyngeal mucosal images obtained from narrow-band imaging were used for the learning and validation data sets. Each independent data set comprised 400 SLPC and 800 normal mucosal images. The diagnostic AI model was constructed stage-wise and evaluated at each learning stage using validation data sets. RESULTS In the validation data sets (100 SLPC cases), the median tumor size was 13.2 mm; flat/elevated/depressed types were found in 77/21/2 cases. Sensitivity, specificity, and accuracy improved each time a learning image was added and were 95.5%, 98.4%, and 97.3%, respectively, after learning all SLPC and normal mucosal images. CONCLUSIONS The novel AI model is helpful for detection of laryngopharyngeal cancer at an early stage.
Collapse
Affiliation(s)
- Atsushi Inaba
- Department of Gastroenterology and Endoscopy, National Cancer Center Hospital East, Kashiwa, Chiba, Japan.,Course of Advanced Clinical Research of Cancer, Juntendo University Graduate School of Medicine, Bunkyo-ku, Tokyo, Japan
| | - Keisuke Hori
- Department of Gastroenterology and Endoscopy, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
| | - Yusuke Yoda
- Department of Gastroenterology and Endoscopy, National Cancer Center Hospital East, Kashiwa, Chiba, Japan.,Medical Device Innovation Center, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
| | - Hiroaki Ikematsu
- Department of Gastroenterology and Endoscopy, National Cancer Center Hospital East, Kashiwa, Chiba, Japan.,Division of Science and Technology for Endoscopy, Exploratory Oncology Research & Clinical Trial Center, National Cancer Center East, Kashiwa, Chiba, Japan
| | - Hiroaki Takano
- Medical Device Innovation Center, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
| | - Hiroki Matsuzaki
- Medical Device Innovation Center, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
| | - Yoshiki Watanabe
- Department of Medical Information, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
| | - Nobuyoshi Takeshita
- Medical Device Innovation Center, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
| | - Toshifumi Tomioka
- Department of Head and Neck Surgery, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
| | - Genichiro Ishii
- Course of Advanced Clinical Research of Cancer, Juntendo University Graduate School of Medicine, Bunkyo-ku, Tokyo, Japan.,Division of Pathology, Exploratory Oncology Research & Clinical Trial Center, National Cancer Center East, Kashiwa, Chiba, Japan
| | - Satoshi Fujii
- Division of Pathology, Exploratory Oncology Research & Clinical Trial Center, National Cancer Center East, Kashiwa, Chiba, Japan
| | - Ryuichi Hayashi
- Department of Head and Neck Surgery, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
| | - Tomonori Yano
- Department of Gastroenterology and Endoscopy, National Cancer Center Hospital East, Kashiwa, Chiba, Japan.,Medical Device Innovation Center, National Cancer Center Hospital East, Kashiwa, Chiba, Japan
| |
Collapse
|
26
|
Semantic Segmentation Framework for Glomeruli Detection and Classification in Kidney Histological Sections. ELECTRONICS 2020. [DOI: 10.3390/electronics9030503] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
The evaluation of kidney biopsies performed by expert pathologists is a crucial process for assessing if a kidney is eligible for transplantation. In this evaluation process, an important step consists of the quantification of global glomerulosclerosis, which is the ratio between sclerotic glomeruli and the overall number of glomeruli. Since there is a shortage of organs available for transplantation, a quick and accurate assessment of global glomerulosclerosis is essential for retaining the largest number of eligible kidneys. In the present paper, the authors introduce a Computer-Aided Diagnosis (CAD) system to assess global glomerulosclerosis. The proposed tool is based on Convolutional Neural Networks (CNNs). In particular, the authors considered approaches based on Semantic Segmentation networks, such as SegNet and DeepLab v3+. The dataset has been provided by the Department of Emergency and Organ Transplantations (DETO) of Bari University Hospital, and it is composed of 26 kidney biopsies coming from 19 donors. The dataset contains 2344 non-sclerotic glomeruli and 428 sclerotic glomeruli. The proposed model consents to achieve promising results in the task of automatically detecting and classifying glomeruli, thus easing the burden of pathologists. We get high performance both at pixel-level, achieving mean F-score higher than 0.81, and Weighted Intersection over Union (IoU) higher than 0.97 for both SegNet and Deeplab v3+ approaches, and at object detection level, achieving 0.924 as best F-score for non-sclerotic glomeruli and 0.730 as best F-score for sclerotic glomeruli.
Collapse
|
27
|
Gallardo-Caballero R, García-Orellana CJ, García-Manso A, González-Velasco HM, Tormo-Molina R, Macías-Macías M. Precise Pollen Grain Detection in Bright Field Microscopy Using Deep Learning Techniques. SENSORS (BASEL, SWITZERLAND) 2019; 19:E3583. [PMID: 31426511 PMCID: PMC6720915 DOI: 10.3390/s19163583] [Citation(s) in RCA: 23] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/27/2019] [Revised: 08/12/2019] [Accepted: 08/15/2019] [Indexed: 12/18/2022]
Abstract
The determination of daily concentrations of atmospheric pollen is important in the medical and biological fields. Obtaining pollen concentrations is a complex and time-consuming task for specialized personnel. The automatic location of pollen grains is a handicap due to the high complexity of the images to be processed, with polymorphic and clumped pollen grains, dust, or debris. The purpose of this study is to analyze the feasibility of implementing a reliable pollen grain detection system based on a convolutional neural network architecture, which will be used later as a critical part of an automated pollen concentration estimation system. We used a training set of 251 videos to train our system. As the videos record the process of focusing the samples, this system makes use of the 3D information presented by several focal planes. Besides, a separate set of 135 videos (containing 1234 pollen grains of 11 pollen types) was used to evaluate detection performance. The results are promising in detection (98.54% of recall and 99.75% of precision) and location accuracy (0.89 IoU as the average value). These results suggest that this technique can provide a reliable basis for the development of an automated pollen counting system.
Collapse
Affiliation(s)
- Ramón Gallardo-Caballero
- Instituto de Computación Científica Avanzada (ICCAEx) and CAPI Research Group, Universidad de Extremadura, E-06006 Badajoz, Spain.
| | - Carlos J García-Orellana
- Instituto de Computación Científica Avanzada (ICCAEx) and CAPI Research Group, Universidad de Extremadura, E-06006 Badajoz, Spain
| | - Antonio García-Manso
- Instituto de Computación Científica Avanzada (ICCAEx) and CAPI Research Group, Universidad de Extremadura, E-06006 Badajoz, Spain
| | - Horacio M González-Velasco
- Instituto de Computación Científica Avanzada (ICCAEx) and CAPI Research Group, Universidad de Extremadura, E-06006 Badajoz, Spain
| | - Rafael Tormo-Molina
- Department of Plant Biology, Ecology and Earth Sciences, Universidad de Extremadura, E-06006 Badajoz, Spain
| | - Miguel Macías-Macías
- Instituto de Computación Científica Avanzada (ICCAEx) and CAPI Research Group, Universidad de Extremadura, E-06006 Badajoz, Spain
| |
Collapse
|