1
|
You Z, Han B, Shi Z, Zhao M, Du S, Liu H, Hei X, Ren X, Yan Y. Vocal Cord Leukoplakia Classification Using Siamese Network Under Small Samples of White Light Endoscopy Images. Otolaryngol Head Neck Surg 2024; 170:1099-1108. [PMID: 38037413 DOI: 10.1002/ohn.591] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Revised: 10/25/2023] [Accepted: 11/02/2023] [Indexed: 12/02/2023]
Abstract
OBJECTIVE Accurate vocal cord leukoplakia classification is instructive for clinical diagnosis and surgical treatment. This article introduces a reliable very deep Siamese network for accurate vocal cord leukoplakia classification. STUDY DESIGN A study of a classification network based on a retrospective database. SETTING Academic university and hospital. METHODS The white light image datasets of vocal cord leukoplakia used in this article were classified into 6 classes: normal tissues, inflammatory keratosis, mild dysplasia, moderate dysplasia, severe dysplasia, and squamous cell carcinoma. The classification performance was assessed by comparing it with 6 classical deep learning models, including AlexNet, VGG Net, Google Inception, ResNet, DenseNet, and Vision Transformer. RESULTS Experiments show the superior classification performance of our proposed network compared to state-of-the-art methods. The overall accuracy is 0.9756. The values of sensitivity and specificity are very high as well. The confusion matrix provides information for the 6-class classification task and demonstrates the superiority of our proposed network. CONCLUSION Our very deep Siamese network can provide accurate classification results of vocal cord leukoplakia, which facilitates early detection, clinical diagnosis, and surgical treatment. The excellent performance obtained in white light images can reduce the cost for patients, especially those living in developing countries.
Collapse
Affiliation(s)
- Zhenzhen You
- Scool of Computer Science and Engineering, Shaanxi Key Laboratory for Network Computing and Security Technology, School of Computer Science and Engineering, Xi'an University of Technology, Xi'an, China
| | - Botao Han
- Scool of Computer Science and Engineering, Shaanxi Key Laboratory for Network Computing and Security Technology, School of Computer Science and Engineering, Xi'an University of Technology, Xi'an, China
| | - Zhenghao Shi
- Scool of Computer Science and Engineering, Shaanxi Key Laboratory for Network Computing and Security Technology, School of Computer Science and Engineering, Xi'an University of Technology, Xi'an, China
| | - Minghua Zhao
- Scool of Computer Science and Engineering, Shaanxi Key Laboratory for Network Computing and Security Technology, School of Computer Science and Engineering, Xi'an University of Technology, Xi'an, China
| | - Shuangli Du
- Scool of Computer Science and Engineering, Shaanxi Key Laboratory for Network Computing and Security Technology, School of Computer Science and Engineering, Xi'an University of Technology, Xi'an, China
| | - Haiqin Liu
- Department of Otorhinolaryngology, Second Affiliated Hospital of Medical College, Xi'an Jiaotong University, Xi'an, China
| | - Xinhong Hei
- Scool of Computer Science and Engineering, Shaanxi Key Laboratory for Network Computing and Security Technology, School of Computer Science and Engineering, Xi'an University of Technology, Xi'an, China
| | - Xiaoyong Ren
- Department of Otorhinolaryngology, Second Affiliated Hospital of Medical College, Xi'an Jiaotong University, Xi'an, China
| | - Yan Yan
- Department of Otorhinolaryngology, Second Affiliated Hospital of Medical College, Xi'an Jiaotong University, Xi'an, China
| |
Collapse
|
2
|
Cazzaniga G, Rossi M, Eccher A, Girolami I, L'Imperio V, Van Nguyen H, Becker JU, Bueno García MG, Sbaraglia M, Dei Tos AP, Gambaro G, Pagni F. Time for a full digital approach in nephropathology: a systematic review of current artificial intelligence applications and future directions. J Nephrol 2024; 37:65-76. [PMID: 37768550 PMCID: PMC10920416 DOI: 10.1007/s40620-023-01775-w] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Accepted: 08/22/2023] [Indexed: 09/29/2023]
Abstract
INTRODUCTION Artificial intelligence (AI) integration in nephropathology has been growing rapidly in recent years, facing several challenges including the wide range of histological techniques used, the low occurrence of certain diseases, and the need for data sharing. This narrative review retraces the history of AI in nephropathology and provides insights into potential future developments. METHODS Electronic searches in PubMed-MEDLINE and Embase were made to extract pertinent articles from the literature. Works about automated image analysis or the application of an AI algorithm on non-neoplastic kidney histological samples were included and analyzed to extract information such as publication year, AI task, and learning type. Prepublication servers and reviews were not included. RESULTS Seventy-six (76) original research articles were selected. Most of the studies were conducted in the United States in the last 7 years. To date, research has been mainly conducted on relatively easy tasks, like single-stain glomerular segmentation. However, there is a trend towards developing more complex tasks such as glomerular multi-stain classification. CONCLUSION Deep learning has been used to identify patterns in complex histopathology data and looks promising for the comprehensive assessment of renal biopsy, through the use of multiple stains and virtual staining techniques. Hybrid and collaborative learning approaches have also been explored to utilize large amounts of unlabeled data. A diverse team of experts, including nephropathologists, computer scientists, and clinicians, is crucial for the development of AI systems for nephropathology. Collaborative efforts among multidisciplinary experts result in clinically relevant and effective AI tools.
Collapse
Affiliation(s)
- Giorgio Cazzaniga
- Department of Medicine and Surgery, Pathology, Fondazione IRCCS San Gerardo dei Tintori, Università di Milano-Bicocca, Monza, Italy.
| | - Mattia Rossi
- Division of Nephrology, Department of Medicine, University of Verona, Piazzale Aristide Stefani, 1, 37126, Verona, Italy
| | - Albino Eccher
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, P.le Stefani n. 1, 37126, Verona, Italy
- Department of Medical and Surgical Sciences for Children and Adults, University of Modena and Reggio Emilia, University Hospital of Modena, Modena, Italy
| | - Ilaria Girolami
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, P.le Stefani n. 1, 37126, Verona, Italy
| | - Vincenzo L'Imperio
- Department of Medicine and Surgery, Pathology, Fondazione IRCCS San Gerardo dei Tintori, Università di Milano-Bicocca, Monza, Italy
| | - Hien Van Nguyen
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, 77004, USA
| | - Jan Ulrich Becker
- Institute of Pathology, University Hospital of Cologne, Cologne, Germany
| | - María Gloria Bueno García
- VISILAB Research Group, E.T.S. Ingenieros Industriales, University of Castilla-La Mancha, Ciudad Real, Spain
| | - Marta Sbaraglia
- Department of Pathology, Azienda Ospedale-Università Padova, Padua, Italy
- Department of Medicine, University of Padua School of Medicine, Padua, Italy
| | - Angelo Paolo Dei Tos
- Department of Pathology, Azienda Ospedale-Università Padova, Padua, Italy
- Department of Medicine, University of Padua School of Medicine, Padua, Italy
| | - Giovanni Gambaro
- Division of Nephrology, Department of Medicine, University of Verona, Piazzale Aristide Stefani, 1, 37126, Verona, Italy
| | - Fabio Pagni
- Department of Medicine and Surgery, Pathology, Fondazione IRCCS San Gerardo dei Tintori, Università di Milano-Bicocca, Monza, Italy
| |
Collapse
|
3
|
Xia P, Lv Z, Wen Y, Zhang B, Zhao X, Zhang B, Wang Y, Cui H, Wang C, Zheng H, Qin Y, Sun L, Ye N, Cheng H, Yao L, Zhou H, Zhen J, Hu Z, Zhu W, Zhang F, Li X, Ren F, Chen L. Development of a multiple convolutional neural network-facilitated diagnostic screening program for immunofluorescence images of IgA nephropathy and idiopathic membranous nephropathy. Clin Kidney J 2023; 16:2503-2513. [PMID: 38046020 PMCID: PMC10689194 DOI: 10.1093/ckj/sfad153] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Indexed: 12/05/2023] Open
Abstract
Background Immunoglobulin A nephropathy (IgAN) and idiopathic membranous nephropathy (IMN) are the most common glomerular diseases. Immunofluorescence (IF) tests of renal tissues are crucial for the diagnosis. We developed a multiple convolutional neural network (CNN)-facilitated diagnostic program to assist the IF diagnosis of IgAN and IMN. Methods The diagnostic program consisted of four parts: a CNN trained as a glomeruli detection module, an IF intensity comparator, dual-CNN (D-CNN) trained as a deposition appearance and location classifier and a post-processing module. A total of 1573 glomerular IF images from 1009 patients with glomerular diseases were used for the training and validation of the diagnostic program. A total of 1610 images of 426 patients from different hospitals were used as test datasets. The performance of the diagnostic program was compared with nephropathologists. Results In >90% of the tested images, the glomerulus location module achieved an intersection over union >0.8. The accuracy of the D-CNN in recognizing irregular granular mesangial deposition and fine granular deposition along the glomerular basement membrane was 96.1% and 93.3%, respectively. As for the diagnostic program, the accuracy, sensitivity and specificity of diagnosing suspected IgAN were 97.6%, 94.4% and 96.0%, respectively. The accuracy, sensitivity and specificity of diagnosing suspected IMN were 91.7%, 88.9% and 95.8%, respectively. The corresponding areas under the curve (AUCs) were 0.983 and 0.935. When tested with images from the outside hospital, the diagnostic program showed stable performance. The AUCs for diagnosing suspected IgAN and IMN were 0.972 and 0.948, respectively. Compared with inexperienced nephropathologists, the program showed better performance. Conclusion The proposed diagnostic program could assist the IF diagnosis of IgAN and IMN.
Collapse
Affiliation(s)
- Peng Xia
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Zhilong Lv
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
- School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China
| | - Yubing Wen
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | | | - Xuesong Zhao
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Boyao Zhang
- Beijing Zhijian Life Technology, Beijing, China
| | - Ying Wang
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Haoyuan Cui
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Chuanpeng Wang
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Hua Zheng
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yan Qin
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Lijun Sun
- Department of Nephrology, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
| | - Nan Ye
- Department of Nephrology, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
| | - Hong Cheng
- Department of Nephrology, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
| | - Li Yao
- Department of Nephrology, First Hospital Affiliated to China Medical University, Shenyang, China
| | - Hua Zhou
- Department of Nephrology, Shengjing Hospital of China Medical University, Shenyang, China
| | - Junhui Zhen
- Department of Pathology, Qilu Hospital of Shandong University, Jinan, China
| | - Zhao Hu
- Department of Nephrology, Qilu Hospital of Shandong University, Jinan, China
| | - Weiguo Zhu
- Department of Information, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Fa Zhang
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
| | - Xuemei Li
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Fei Ren
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
| | - Limeng Chen
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
4
|
Fu Y, Jiang L, Pan S, Chen P, Wang X, Dai N, Chen X, Xu M. Deep multi-task learning for nephropathy diagnosis on immunofluorescence images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 241:107747. [PMID: 37619430 DOI: 10.1016/j.cmpb.2023.107747] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 06/14/2023] [Accepted: 08/03/2023] [Indexed: 08/26/2023]
Abstract
BACKGROUND AND OBJECTIVE As an advanced technique, immunofluorescence (IF) is one of the most widely-used medical image for nephropathy diagnosis, due to its ease of acquisition with low cost. In practice, the clinically collected IF images are commonly corrupted by blurs at different degrees, mainly because of the inaccurate focus at the acquisition stage. Although deep neural network (DNN) methods achieve the great success in nephropathy diagnosis, their performance dramatically drops over the blurred IF images. This significantly limits the potential of leveraging the advanced DNN techniques in real-world nephropathy diagnosis scenarios. METHODS This paper first establishes two IF databases with synthetic blurs (IFVB) and real-world blurs (Real-IF) for nephropathy diagnosis, respectively, including 1,659 patients and 6,521 IF images with various degrees of blurs. According to the analysis on these two databases, we propose a deep hierarchical multi-task learning based nephropathy diagnosis (DeepMT-ND) method to bridge the gap between the low-level vision and high-level medical tasks. Specifically, DeepMT-ND simultaneously handles the main task of automatic nephropathy diagnosis, as well as the auxiliary tasks of image quality assessment (IQA) and de-blurring. RESULTS Extensive experiments show the superiority of our DeepMT-ND in terms of diagnosis accuracy and generalization ability. For instance, our method performs better than nephrologists with at least 15.4% and 6.5% accuracy improvements in IFVB and Real-IF, respectively. Meanwhile, our method also achieves comparable performance in two auxiliary tasks of IQA and de-blurring on blurred IF images. CONCLUSIONS In this paper, we propose a new DeepMT-ND method for nephropathy diagnosis on blurred IF images. The proposed hierarchical multi-task learning framework provides the new scope to narrow the gap between the low-level vision and high-level medical tasks, and will contribute to nephropathy diagnosis in clinical scenarios. The diagnosis accuracy and generalization ability of DeepMT-ND are experimentally verified to be effective over both synthetic and real-world databases.
Collapse
Affiliation(s)
- Yibing Fu
- School of Electronic and Information Engineering, Beihang University, Beijing, China
| | - Lai Jiang
- School of Electronic and Information Engineering, Beihang University, Beijing, China
| | - Sai Pan
- Department of Nephrology, Chinese People's Liberation Army General Hospital, Beijing, China
| | - Pu Chen
- Department of Nephrology, Chinese People's Liberation Army General Hospital, Beijing, China
| | - Xiaofei Wang
- Department of Clinical Neurosciences, University of Cambridge, Cambridge, United Kingdom
| | - Ning Dai
- School of Electronic and Information Engineering, Beihang University, Beijing, China
| | - Xiangmei Chen
- Department of Nephrology, Chinese People's Liberation Army General Hospital, Beijing, China.
| | - Mai Xu
- School of Electronic and Information Engineering, Beihang University, Beijing, China.
| |
Collapse
|
5
|
Hao F, Liu X, Li M, Han W. Accurate Kidney Pathological Image Classification Method Based on Deep Learning and Multi-Modal Fusion Method with Application to Membranous Nephropathy. Life (Basel) 2023; 13:life13020399. [PMID: 36836756 PMCID: PMC9960995 DOI: 10.3390/life13020399] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Revised: 01/25/2023] [Accepted: 01/27/2023] [Indexed: 02/04/2023] Open
Abstract
Membranous nephropathy is one of the most prevalent conditions responsible for nephrotic syndrome in adults. It is clinically nonspecific and mainly diagnosed by kidney biopsy pathology, with three prevalent techniques: light microscopy, electron microscopy, and immunofluorescence microscopy. Manual observation of glomeruli one by one under the microscope is very time-consuming, and there are certain differences in the observation results between physicians. This study makes use of whole-slide images scanned by a light microscope as well as immunofluorescence images to classify patients with membranous nephropathy. The framework mainly includes a glomerular segmentation module, a confidence coefficient extraction module, and a multi-modal fusion module. This framework first identifies and segments the glomerulus from whole-slide images and immunofluorescence images, and then a glomerular classifier is trained to extract the features of each glomerulus. The results are then combined to produce the final diagnosis. The results of the experiments show that the F1-score of image classification results obtained by combining two kinds of features, which can reach 97.32%, is higher than those obtained by using only light-microscopy-observed images or immunofluorescent images, which reach 92.76% and 93.20%, respectively. Experiments demonstrate that considering both WSIs and immunofluorescence images is effective in improving the diagnosis of membranous nephropathy.
Collapse
Affiliation(s)
- Fang Hao
- College of Data Science, Taiyuan University of Technology, Taiyuan 030024, China
- Correspondence:
| | - Xueyu Liu
- College of Data Science, Taiyuan University of Technology, Taiyuan 030024, China
| | - Ming Li
- College of Data Science, Taiyuan University of Technology, Taiyuan 030024, China
| | - Weixia Han
- Department of Pathology, Second Hospital of Shanxi Medical University, Taiyuan 030001, China
| |
Collapse
|
6
|
Artificial Intelligence-Assisted Renal Pathology: Advances and Prospects. J Clin Med 2022; 11:jcm11164918. [PMID: 36013157 PMCID: PMC9410196 DOI: 10.3390/jcm11164918] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2022] [Revised: 07/30/2022] [Accepted: 08/11/2022] [Indexed: 11/17/2022] Open
Abstract
Digital imaging and advanced microscopy play a pivotal role in the diagnosis of kidney diseases. In recent years, great achievements have been made in digital imaging, providing novel approaches for precise quantitative assessments of nephropathology and relieving burdens of renal pathologists. Developing novel methods of artificial intelligence (AI)-assisted technology through multidisciplinary interaction among computer engineers, renal specialists, and nephropathologists could prove beneficial for renal pathology diagnoses. An increasing number of publications has demonstrated the rapid growth of AI-based technology in nephrology. In this review, we offer an overview of AI-assisted renal pathology, including AI concepts and the workflow of processing digital image data, focusing on the impressive advances of AI application in disease-specific backgrounds. In particular, this review describes the applied computer vision algorithms for the segmentation of kidney structures, diagnosis of specific pathological changes, and prognosis prediction based on images. Lastly, we discuss challenges and prospects to provide an objective view of this topic.
Collapse
|