1
|
Cazzaniga G, Rossi M, Eccher A, Girolami I, L'Imperio V, Van Nguyen H, Becker JU, Bueno García MG, Sbaraglia M, Dei Tos AP, Gambaro G, Pagni F. Time for a full digital approach in nephropathology: a systematic review of current artificial intelligence applications and future directions. J Nephrol 2024; 37:65-76. [PMID: 37768550 PMCID: PMC10920416 DOI: 10.1007/s40620-023-01775-w] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Accepted: 08/22/2023] [Indexed: 09/29/2023]
Abstract
INTRODUCTION Artificial intelligence (AI) integration in nephropathology has been growing rapidly in recent years, facing several challenges including the wide range of histological techniques used, the low occurrence of certain diseases, and the need for data sharing. This narrative review retraces the history of AI in nephropathology and provides insights into potential future developments. METHODS Electronic searches in PubMed-MEDLINE and Embase were made to extract pertinent articles from the literature. Works about automated image analysis or the application of an AI algorithm on non-neoplastic kidney histological samples were included and analyzed to extract information such as publication year, AI task, and learning type. Prepublication servers and reviews were not included. RESULTS Seventy-six (76) original research articles were selected. Most of the studies were conducted in the United States in the last 7 years. To date, research has been mainly conducted on relatively easy tasks, like single-stain glomerular segmentation. However, there is a trend towards developing more complex tasks such as glomerular multi-stain classification. CONCLUSION Deep learning has been used to identify patterns in complex histopathology data and looks promising for the comprehensive assessment of renal biopsy, through the use of multiple stains and virtual staining techniques. Hybrid and collaborative learning approaches have also been explored to utilize large amounts of unlabeled data. A diverse team of experts, including nephropathologists, computer scientists, and clinicians, is crucial for the development of AI systems for nephropathology. Collaborative efforts among multidisciplinary experts result in clinically relevant and effective AI tools.
Collapse
Affiliation(s)
- Giorgio Cazzaniga
- Department of Medicine and Surgery, Pathology, Fondazione IRCCS San Gerardo dei Tintori, Università di Milano-Bicocca, Monza, Italy.
| | - Mattia Rossi
- Division of Nephrology, Department of Medicine, University of Verona, Piazzale Aristide Stefani, 1, 37126, Verona, Italy
| | - Albino Eccher
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, P.le Stefani n. 1, 37126, Verona, Italy
- Department of Medical and Surgical Sciences for Children and Adults, University of Modena and Reggio Emilia, University Hospital of Modena, Modena, Italy
| | - Ilaria Girolami
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, P.le Stefani n. 1, 37126, Verona, Italy
| | - Vincenzo L'Imperio
- Department of Medicine and Surgery, Pathology, Fondazione IRCCS San Gerardo dei Tintori, Università di Milano-Bicocca, Monza, Italy
| | - Hien Van Nguyen
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, 77004, USA
| | - Jan Ulrich Becker
- Institute of Pathology, University Hospital of Cologne, Cologne, Germany
| | - María Gloria Bueno García
- VISILAB Research Group, E.T.S. Ingenieros Industriales, University of Castilla-La Mancha, Ciudad Real, Spain
| | - Marta Sbaraglia
- Department of Pathology, Azienda Ospedale-Università Padova, Padua, Italy
- Department of Medicine, University of Padua School of Medicine, Padua, Italy
| | - Angelo Paolo Dei Tos
- Department of Pathology, Azienda Ospedale-Università Padova, Padua, Italy
- Department of Medicine, University of Padua School of Medicine, Padua, Italy
| | - Giovanni Gambaro
- Division of Nephrology, Department of Medicine, University of Verona, Piazzale Aristide Stefani, 1, 37126, Verona, Italy
| | - Fabio Pagni
- Department of Medicine and Surgery, Pathology, Fondazione IRCCS San Gerardo dei Tintori, Università di Milano-Bicocca, Monza, Italy
| |
Collapse
|
2
|
Altini N, Rossini M, Turkevi-Nagy S, Pesce F, Pontrelli P, Prencipe B, Berloco F, Seshan S, Gibier JB, Pedraza Dorado A, Bueno G, Peruzzi L, Rossi M, Eccher A, Li F, Koumpis A, Beyan O, Barratt J, Vo HQ, Mohan C, Nguyen HV, Cicalese PA, Ernst A, Gesualdo L, Bevilacqua V, Becker JU. Performance and limitations of a supervised deep learning approach for the histopathological Oxford Classification of glomeruli with IgA nephropathy. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 242:107814. [PMID: 37722311 DOI: 10.1016/j.cmpb.2023.107814] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Revised: 09/11/2023] [Accepted: 09/12/2023] [Indexed: 09/20/2023]
Abstract
BACKGROUND AND OBJECTIVE The Oxford Classification for IgA nephropathy is the most successful example of an evidence-based nephropathology classification system. The aim of our study was to replicate the glomerular components of Oxford scoring with an end-to-end deep learning pipeline that involves automatic glomerular segmentation followed by classification for mesangial hypercellularity (M), endocapillary hypercellularity (E), segmental sclerosis (S) and active crescents (C). METHODS A total number of 1056 periodic acid-Schiff (PAS) whole slide images (WSIs), coming from 386 kidney biopsies, were annotated. Several detection models for glomeruli, based on the Mask R-CNN architecture, were trained on 587 WSIs, validated on 161 WSIs, and tested on 127 WSIs. For the development of segmentation models, 20,529 glomeruli were annotated, of which 16,571 as training and 3958 as validation set. The test set of the segmentation module comprised of 2948 glomeruli. For the Oxford classification, 6206 expert-annotated glomeruli from 308 PAS WSIs were labelled for M, E, S, C and split into a training set of 4298 glomeruli from 207 WSIs, and a test set of 1908 glomeruli. We chose the best-performing models to construct an end-to-end pipeline, which we named MESCnn (MESC classification by neural network), for the glomerular Oxford classification of WSIs. RESULTS Instance segmentation yielded excellent results with an AP50 ranging between 78.2-80.1 % (79.4 ± 0.7 %) on the validation and 75.1-77.7 % (76.5 ± 0.9 %) on the test set. The aggregated Jaccard Index was between 73.4-75.9 % (75.0 ± 0.8 %) on the validation and 69.1-73.4 % (72.2 ± 1.4 %) on the test set. At granular glomerular level, Oxford Classification was best replicated for M with EfficientNetV2-L with a mean ROC-AUC of 90.2 % and a mean precision/recall area under the curve (PR-AUC) of 81.8 %, best for E with MobileNetV2 (ROC-AUC 94.7 %) and ResNet50 (PR-AUC 75.8 %), best for S with EfficientNetV2-M (mean ROC-AUC 92.7 %, mean PR-AUC 87.7 %), best for C with EfficientNetV2-L (ROC-AUC 92.3 %) and EfficientNetV2-S (PR-AUC 54.7 %). At biopsy-level, correlation between expert and deep learning labels fulfilled the demands of the Oxford Classification. CONCLUSION We designed an end-to-end pipeline for glomerular Oxford Classification on both a granular glomerular and an entire biopsy level. Both the glomerular segmentation and the classification modules are freely available for further development to the renal medicine community.
Collapse
Affiliation(s)
- Nicola Altini
- Department of Electrical and Information Engineering (DEI), Polytechnic University of Bari, Via Edoardo Orabona n.4, Bari 70126, Italy
| | - Michele Rossini
- Nephrology Dialysis and Transplantation Unit, Department of Emergency and Organ Transplantation, University of Bari, Bari, Italy
| | - Sándor Turkevi-Nagy
- Department of Pathology, Albert Szent-Györgyi Health Center, University of Szeged, Szeged, Hungary
| | - Francesco Pesce
- Nephrology Dialysis and Transplantation Unit, Department of Emergency and Organ Transplantation, University of Bari, Bari, Italy; Division of Renal Medicine, "Fatebenefratelli Isola Tiberina - Gemelli Isola", Rome, Italy
| | - Paola Pontrelli
- Nephrology Dialysis and Transplantation Unit, Department of Emergency and Organ Transplantation, University of Bari, Bari, Italy
| | - Berardino Prencipe
- Department of Electrical and Information Engineering (DEI), Polytechnic University of Bari, Via Edoardo Orabona n.4, Bari 70126, Italy
| | - Francesco Berloco
- Department of Electrical and Information Engineering (DEI), Polytechnic University of Bari, Via Edoardo Orabona n.4, Bari 70126, Italy
| | - Surya Seshan
- Department of Pathology, Weill-Cornell Medical Center/New York Presbyterian Hospital, New York, NY, USA
| | - Jean-Baptiste Gibier
- Department of Pathology, Pathology Institute, Lille University Hospital (CHU), Lille, France
| | | | - Gloria Bueno
- VISILAB Research Group, University of Castilla-La Mancha, Ciudad Real, Spain
| | - Licia Peruzzi
- AOU Città della Salute e della Scienza di Torino, Regina Margherita Children's Hospital, Turin, Italy
| | - Mattia Rossi
- Division of Nephrology, Department of Medicine, University of Verona, Verona, Italy
| | - Albino Eccher
- Section of Pathology, Department of Medical and Surgical Sciences for Children and Adults, University of Modena and Reggio Emilia, University Hospital of Modena, Modena, Italy
| | - Feifei Li
- Faculty of Medicine, University Hospital Cologne, University of Cologne, Institute for Medical Informatics, Cologne, Germany
| | - Adamantios Koumpis
- Faculty of Medicine, University Hospital Cologne, University of Cologne, Institute for Medical Informatics, Cologne, Germany
| | - Oya Beyan
- Faculty of Medicine, University Hospital Cologne, University of Cologne, Institute for Medical Informatics, Cologne, Germany
| | - Jonathan Barratt
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| | - Huy Quoc Vo
- Department of Biomedical Engineering, University of Houston, Houston, USA
| | - Chandra Mohan
- Department of Biomedical Engineering, University of Houston, Houston, USA
| | - Hien Van Nguyen
- Department of Biomedical Engineering, University of Houston, Houston, USA
| | | | - Angela Ernst
- Institute of Medical Statistics and Computational Biology, University of Cologne, Cologne, Germany
| | - Loreto Gesualdo
- Nephrology Dialysis and Transplantation Unit, Department of Emergency and Organ Transplantation, University of Bari, Bari, Italy
| | - Vitoantonio Bevilacqua
- Department of Electrical and Information Engineering (DEI), Polytechnic University of Bari, Via Edoardo Orabona n.4, Bari 70126, Italy; Apulian Bioengineering s.r.l., Via delle Violette n.14, Modugno 70026, Italy.
| | - Jan Ulrich Becker
- Institute of Pathology, University Hospital of Cologne, Cologne, Germany
| |
Collapse
|
3
|
Jacq A, Tarris G, Jaugey A, Paindavoine M, Maréchal E, Bard P, Rebibou JM, Ansart M, Calmo D, Bamoulid J, Tinel C, Ducloux D, Crepin T, Chabannes M, Funes de la Vega M, Felix S, Martin L, Legendre M. Automated evaluation with deep learning of total interstitial inflammation and peritubular capillaritis on kidney biopsies. Nephrol Dial Transplant 2023; 38:2786-2798. [PMID: 37197910 DOI: 10.1093/ndt/gfad094] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Indexed: 05/19/2023] Open
Abstract
BACKGROUND Interstitial inflammation and peritubular capillaritis are observed in many diseases on native and transplant kidney biopsies. A precise and automated evaluation of these histological criteria could help stratify patients' kidney prognoses and facilitate therapeutic management. METHODS We used a convolutional neural network to evaluate those criteria on kidney biopsies. A total of 423 kidney samples from various diseases were included; 83 kidney samples were used for the neural network training, 106 for comparing manual annotations on limited areas to automated predictions, and 234 to compare automated and visual gradings. RESULTS The precision, recall and F-score for leukocyte detection were, respectively, 81%, 71% and 76%. Regarding peritubular capillaries detection the precision, recall and F-score were, respectively, 82%, 83% and 82%. There was a strong correlation between the predicted and observed grading of total inflammation, as for the grading of capillaritis (r = 0.89 and r = 0.82, respectively, all P < .0001). The areas under the receiver operating characteristics curves for the prediction of pathologists' Banff total inflammation (ti) and peritubular capillaritis (ptc) scores were respectively all above 0.94 and 0.86. The kappa coefficients between the visual and the neural networks' scores were respectively 0.74, 0.78 and 0.68 for ti ≥1, ti ≥2 and ti ≥3, and 0.62, 0.64 and 0.79 for ptc ≥1, ptc ≥2 and ptc ≥3. In a subgroup of patients with immunoglobulin A nephropathy, the inflammation severity was highly correlated to kidney function at biopsy on univariate and multivariate analyses. CONCLUSION We developed a tool using deep learning that scores the total inflammation and capillaritis, demonstrating the potential of artificial intelligence in kidney pathology.
Collapse
Affiliation(s)
- Amélie Jacq
- Department of Nephrology, CHU Dijon, Dijon, France
| | | | - Adrien Jaugey
- ESIREM School, Dijon, France
- LEAD, Laboratoire de l'étude de l'apprentissage et du Développement, Dijon, France
| | - Michel Paindavoine
- LEAD, Laboratoire de l'étude de l'apprentissage et du Développement, Dijon, France
| | | | - Patrick Bard
- ESIREM School, Dijon, France
- LEAD, Laboratoire de l'étude de l'apprentissage et du Développement, Dijon, France
| | - Jean-Michel Rebibou
- Department of Nephrology, CHU Dijon, Dijon, France
- UMR 1098, INCREASE, Besançon, France
| | - Manon Ansart
- ESIREM School, Dijon, France
- LEAD, Laboratoire de l'étude de l'apprentissage et du Développement, Dijon, France
| | - Doris Calmo
- Department of Nephrology, CHU Besançon, Besançon, France
| | - Jamal Bamoulid
- UMR 1098, INCREASE, Besançon, France
- Department of Nephrology, CHU Besançon, Besançon, France
| | - Claire Tinel
- Department of Nephrology, CHU Dijon, Dijon, France
| | - Didier Ducloux
- UMR 1098, INCREASE, Besançon, France
- Department of Nephrology, CHU Besançon, Besançon, France
| | - Thomas Crepin
- UMR 1098, INCREASE, Besançon, France
- Department of Nephrology, CHU Besançon, Besançon, France
| | - Melchior Chabannes
- UMR 1098, INCREASE, Besançon, France
- Department of Nephrology, CHU Besançon, Besançon, France
| | | | - Sophie Felix
- Department of Pathology, CHU Besançon, Besançon, France
| | | | - Mathieu Legendre
- Department of Nephrology, CHU Dijon, Dijon, France
- UMR 1098, INCREASE, Besançon, France
| |
Collapse
|
4
|
Gaupp C, Schmid B, Tripal P, Edwards A, Daniel C, Zimmermann S, Goppelt-Struebe M, Willam C, Rosen S, Schley G. Reconfiguration and loss of peritubular capillaries in chronic kidney disease. Sci Rep 2023; 13:19660. [PMID: 37952029 PMCID: PMC10640592 DOI: 10.1038/s41598-023-46146-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 10/27/2023] [Indexed: 11/14/2023] Open
Abstract
Functional and structural alterations of peritubular capillaries (PTCs) are a major determinant of chronic kidney disease (CKD). Using a software-based algorithm for semiautomatic segmentation and morphometric quantification, this study analyzes alterations of PTC shape associated with chronic tubulointerstitial injury in three mouse models and in human biopsies. In normal kidney tissue PTC shape was predominantly elongated, whereas the majority of PTCs associated with chronic tubulointerstitial injury had a rounder shape. This was reflected by significantly reduced PTC luminal area, perimeter and diameters as well as by significantly increased circularity and roundness. These morphological alterations were consistent in all mouse models and human kidney biopsies. The mean circularity of PTCs correlated significantly with categorized glomerular filtration rates and the degree of interstitial fibrosis and tubular atrophy (IFTA) and classified the presence of CKD or IFTA. 3D reconstruction of renal capillaries revealed not only a significant reduction, but more importantly a substantial simplification and reconfiguration of the renal microvasculature in mice with chronic tubulointerstitial injury. Computational modelling predicted that round PTCs can deliver oxygen more homogeneously to the surrounding tissue. Our findings indicate that alterations of PTC shape represent a common and uniform reaction to chronic tubulointerstitial injury independent of the underlying kidney disease.
Collapse
Affiliation(s)
- Charlotte Gaupp
- Department of Nephrology and Hypertension, Friedrich-Alexander University Erlangen-Nürnberg (FAU) and University Hospital Erlangen, Ulmenweg 18, 91054, Erlangen, Germany
| | - Benjamin Schmid
- Optical Imaging Center Erlangen, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Philipp Tripal
- Optical Imaging Center Erlangen, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Aurélie Edwards
- Department of Biomedical Engineering, Boston University, Boston, MA, USA
| | - Christoph Daniel
- Department of Nephropathology, Friedrich-Alexander University Erlangen-Nürnberg (FAU) and University Hospital Erlangen, Erlangen, Germany
| | - Stefan Zimmermann
- Department of Computer Science, University of Applied Sciences Worms, Worms, Germany
| | - Margarete Goppelt-Struebe
- Department of Nephrology and Hypertension, Friedrich-Alexander University Erlangen-Nürnberg (FAU) and University Hospital Erlangen, Ulmenweg 18, 91054, Erlangen, Germany
| | - Carsten Willam
- Department of Nephrology and Hypertension, Friedrich-Alexander University Erlangen-Nürnberg (FAU) and University Hospital Erlangen, Ulmenweg 18, 91054, Erlangen, Germany
| | - Seymour Rosen
- Department of Pathology, Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, MA, USA
| | - Gunnar Schley
- Department of Nephrology and Hypertension, Friedrich-Alexander University Erlangen-Nürnberg (FAU) and University Hospital Erlangen, Ulmenweg 18, 91054, Erlangen, Germany.
| |
Collapse
|
5
|
Liu X, Wu Y, Chen Y, Hui D, Zhang J, Hao F, Lu Y, Cheng H, Zeng Y, Han W, Wang C, Li M, Zhou X, Zheng W. Diagnosis of diabetic kidney disease in whole slide images via AI-driven quantification of pathological indicators. Comput Biol Med 2023; 166:107470. [PMID: 37722173 DOI: 10.1016/j.compbiomed.2023.107470] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Revised: 08/29/2023] [Accepted: 09/04/2023] [Indexed: 09/20/2023]
Abstract
Diagnosis of diabetic kidney disease (DKD) mainly relies on screening the morphological variations and internal lesions of glomeruli from pathological kidney biopsy. The prominent pathological alterations of glomeruli for DKD include glomerular hypertrophy and nodular mesangial sclerosis. However, the qualitative judgment of these alterations is inaccurate and inconstant due to the intra- and inter-subject variability of pathologists. It is necessary to design artificial intelligence (AI) methods for accurate quantification of these pathological alterations and outcome prediction of DKD. In this work, we present an AI-driven framework to quantify the volume of glomeruli and degree of nodular mesangial sclerosis, respectively, based on an instance segmentation module and a novel weakly supervised Macro-Micro Aggregation (MMA) module. Subsequently, we construct classic machine learning models to predict the degree of DKD based on three selected pathological indicators via factor analysis. These corresponding modules are trained and tested on a total of 281 whole slide images (WSIs) digitized from two hospitals with different scanners. Our designed AI framework achieved inspiring results with 0.926 mIoU for glomerulus segmentation, and 0.899 F1 score for glomerulus classification in the external testing dataset. Meantime, the visualized results of the MMA module could reflect the location of the lesions. The performance of predicting disease achieved the F1 score of 0.917, which further proved the effectiveness of our AI-driven quantification of pathological indicators. Additionally, the interpretation of the machine learning model with the SHAP method showed similar accordance with the development of DKD in pathology. In conclusion, the proposed auxiliary diagnostic technologies have the feasibility for quantitative analysis of glomerular pathological tissues and alterations in DKD. Pathological quantitative indicators will also make it more convenient to provide doctors with assistance in clinical practice.
Collapse
Affiliation(s)
- Xueyu Liu
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Yongfei Wu
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China.
| | - Yilin Chen
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Dongna Hui
- Department of Nephrology, Shanxi Provincial People's Hospital, Taiyuan, Shanxi, China
| | - Jianan Zhang
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Fang Hao
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Yuanyue Lu
- Department of Nephrology, Shanxi Provincial People's Hospital, Taiyuan, Shanxi, China
| | - Hangbei Cheng
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Yue Zeng
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Weixia Han
- Department of Pathology, Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
| | - Chen Wang
- Department of Pathology, Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
| | - Ming Li
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Xiaoshuang Zhou
- Department of Nephrology, Shanxi Provincial People's Hospital, Taiyuan, Shanxi, China.
| | - Wen Zheng
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| |
Collapse
|
6
|
Besusparis J, Morkunas M, Laurinavicius A. A Spatially Guided Machine-Learning Method to Classify and Quantify Glomerular Patterns of Injury in Histology Images. J Imaging 2023; 9:220. [PMID: 37888327 PMCID: PMC10607091 DOI: 10.3390/jimaging9100220] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Revised: 09/26/2023] [Accepted: 10/09/2023] [Indexed: 10/28/2023] Open
Abstract
Introduction The diagnosis of glomerular diseases is primarily based on visual assessment of histologic patterns. Semi-quantitative scoring of active and chronic lesions is often required to assess individual characteristics of the disease. Reproducibility of the visual scoring systems remains debatable, while digital and machine-learning technologies present opportunities to detect, classify and quantify glomerular lesions, also considering their inter- and intraglomerular heterogeneity. MATERIALS AND METHODS We performed a cross-validated comparison of three modifications of a convolutional neural network (CNN)-based approach for recognition and intraglomerular quantification of nine main glomerular patterns of injury. Reference values provided by two nephropathologists were used for validation. For each glomerular image, visual attention heatmaps were generated with a probability of class attribution for further intraglomerular quantification. The quality of classifier-produced heatmaps was evaluated by intersection over union metrics (IoU) between predicted and ground truth localization heatmaps. RESULTS A proposed spatially guided modification of the CNN classifier achieved the highest glomerular pattern classification accuracies, with area under curve (AUC) values up to 0.981. With regards to heatmap overlap area and intraglomerular pattern quantification, the spatially guided classifier achieved a significantly higher generalized mean IoU value compared to single-multiclass and multiple-binary classifiers. CONCLUSIONS We propose a spatially guided CNN classifier that in our experiments reveals the potential to achieve high accuracy for the localization of intraglomerular patterns.
Collapse
Affiliation(s)
- Justinas Besusparis
- Faculty of Medicine, Vilnius University, M.K.Ciurlionio 21, LT-03101 Vilnius, Lithuania; (M.M.); (A.L.)
- National Center of Pathology, Affiliate of Vilnius University Hospital Santaros Clinics, P. Baublio 5, LT-08406 Vilnius, Lithuania
| | - Mindaugas Morkunas
- Faculty of Medicine, Vilnius University, M.K.Ciurlionio 21, LT-03101 Vilnius, Lithuania; (M.M.); (A.L.)
- National Center of Pathology, Affiliate of Vilnius University Hospital Santaros Clinics, P. Baublio 5, LT-08406 Vilnius, Lithuania
| | - Arvydas Laurinavicius
- Faculty of Medicine, Vilnius University, M.K.Ciurlionio 21, LT-03101 Vilnius, Lithuania; (M.M.); (A.L.)
- National Center of Pathology, Affiliate of Vilnius University Hospital Santaros Clinics, P. Baublio 5, LT-08406 Vilnius, Lithuania
| |
Collapse
|
7
|
Zhang H, Botler M, Kooman JP. Deep Learning for Image Analysis in Kidney Care. ADVANCES IN KIDNEY DISEASE AND HEALTH 2023; 30:25-32. [PMID: 36723278 DOI: 10.1053/j.akdh.2022.11.003] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/15/2022] [Revised: 08/23/2022] [Accepted: 11/07/2022] [Indexed: 12/24/2022]
Abstract
Analysis of medical images, such as radiological or tissue specimens, is an indispensable part of medical diagnostics. Conventionally done manually, the process may sometimes be time-consuming and prone to interobserver variability. Image classification and segmentation by deep learning strategies, predominantly convolutional neural networks, may provide a significant advance in the diagnostic process. In renal medicine, most evidence has been generated around the radiological assessment of renal abnormalities and histological analysis of renal biopsy specimens' segmentation. In this article, the basic principles of image analysis by convolutional neural networks, brief descriptions of convolutional neural networks, and their system architecture for image analysis are discussed, in combination with examples regarding their use in image analysis in nephrology.
Collapse
Affiliation(s)
| | | | - Jeroen P Kooman
- Division of Nephrology, Department of Internal Medicine, Maastricht University Medical Center, Maastricht, The Netherlands
| |
Collapse
|
8
|
Artificial Intelligence-Assisted Renal Pathology: Advances and Prospects. J Clin Med 2022; 11:jcm11164918. [PMID: 36013157 PMCID: PMC9410196 DOI: 10.3390/jcm11164918] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2022] [Revised: 07/30/2022] [Accepted: 08/11/2022] [Indexed: 11/17/2022] Open
Abstract
Digital imaging and advanced microscopy play a pivotal role in the diagnosis of kidney diseases. In recent years, great achievements have been made in digital imaging, providing novel approaches for precise quantitative assessments of nephropathology and relieving burdens of renal pathologists. Developing novel methods of artificial intelligence (AI)-assisted technology through multidisciplinary interaction among computer engineers, renal specialists, and nephropathologists could prove beneficial for renal pathology diagnoses. An increasing number of publications has demonstrated the rapid growth of AI-based technology in nephrology. In this review, we offer an overview of AI-assisted renal pathology, including AI concepts and the workflow of processing digital image data, focusing on the impressive advances of AI application in disease-specific backgrounds. In particular, this review describes the applied computer vision algorithms for the segmentation of kidney structures, diagnosis of specific pathological changes, and prognosis prediction based on images. Lastly, we discuss challenges and prospects to provide an objective view of this topic.
Collapse
|
9
|
Bülow RD, Hölscher DL, Boor P. Automatische Bildanalyse und künstliche Intelligenz in der Nephropathologie. DIE NEPHROLOGIE 2022. [PMCID: PMC9360682 DOI: 10.1007/s11560-022-00598-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Hintergrund Fragestellung Material und Methoden Ergebnisse Diskussion
Collapse
Affiliation(s)
- Roman D. Bülow
- Institut für Pathologie, Uniklinik RWTH Aachen, Pauwelsstr. 30, 52074 Aachen, Deutschland
| | - David L. Hölscher
- Institut für Pathologie, Uniklinik RWTH Aachen, Pauwelsstr. 30, 52074 Aachen, Deutschland
| | - Peter Boor
- Institut für Pathologie, Uniklinik RWTH Aachen, Pauwelsstr. 30, 52074 Aachen, Deutschland
- Medizinische Klinik II, Uniklinik RWTH Aachen, Pauwelsstr. 30, 52074 Aachen, Deutschland
| |
Collapse
|
10
|
Hara S, Haneda E, Kawakami M, Morita K, Nishioka R, Zoshima T, Kometani M, Yoneda T, Kawano M, Karashima S, Nambo H. Evaluating tubulointerstitial compartments in renal biopsy specimens using a deep learning-based approach for classifying normal and abnormal tubules. PLoS One 2022; 17:e0271161. [PMID: 35816495 PMCID: PMC9273082 DOI: 10.1371/journal.pone.0271161] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2022] [Accepted: 06/27/2022] [Indexed: 12/02/2022] Open
Abstract
Renal pathology is essential for diagnosing and assessing the severity and prognosis of kidney diseases. Deep learning-based approaches have developed rapidly and have been applied in renal pathology. However, methods for the automated classification of normal and abnormal renal tubules remain scarce. Using a deep learning-based method, we aimed to classify normal and abnormal renal tubules, thereby assisting renal pathologists in the evaluation of renal biopsy specimens. Consequently, we developed a U-Net-based segmentation model using randomly selected regions obtained from 21 renal biopsy specimens. Further, we verified its performance in multiclass segmentation by calculating the Dice coefficients (DCs). We used 15 cases of tubulointerstitial nephritis to assess its applicability in aiding routine diagnoses conducted by renal pathologists and calculated the agreement ratio between diagnoses conducted by two renal pathologists and the time taken for evaluation. We also determined whether such diagnoses were improved when the output of segmentation was considered. The glomeruli and interstitium had the highest DCs, whereas the normal and abnormal renal tubules had intermediate DCs. Following the detailed evaluation of the tubulointerstitial compartments, the proximal, distal, atrophied, and degenerated tubules had intermediate DCs, whereas the arteries and inflamed tubules had low DCs. The annotation and output areas involving normal and abnormal tubules were strongly correlated in each class. The pathological concordance for the glomerular count, t, ct, and ci scores of the Banff classification of renal allograft pathology remained high with or without the segmented images. However, in terms of time consumption, the quantitative assessment of tubulitis, tubular atrophy, degenerated tubules, and the interstitium was improved significantly when renal pathologists considered the segmentation output. Deep learning algorithms can assist renal pathologists in the classification of normal and abnormal tubules in renal biopsy specimens, thereby facilitating the enhancement of renal pathology and ensuring appropriate clinical decisions.
Collapse
Affiliation(s)
- Satoshi Hara
- Medical Education Research Center, Graduate School of Medical Sciences, Kanazawa University, Kanazawa, Japan
- Department of Rheumatology, Kanazawa University Graduate School of Medicine, Kanazawa, Japan
| | - Emi Haneda
- School of Electrical Information Communication Engineering, College of Science and Engineering, Kanazawa University, Kanazawa, Japan
| | - Masaki Kawakami
- School of Electrical Information Communication Engineering, College of Science and Engineering, Kanazawa University, Kanazawa, Japan
| | - Kento Morita
- School of Electrical Information Communication Engineering, College of Science and Engineering, Kanazawa University, Kanazawa, Japan
| | - Ryo Nishioka
- Department of Rheumatology, Kanazawa University Graduate School of Medicine, Kanazawa, Japan
| | - Takeshi Zoshima
- Department of Rheumatology, Kanazawa University Graduate School of Medicine, Kanazawa, Japan
| | - Mitsuhiro Kometani
- Department of Endocrinology and Metabolism, Kanazawa University Graduate School of Medicine, Kanazawa, Japan
| | - Takashi Yoneda
- Department of Endocrinology and Metabolism, Kanazawa University Graduate School of Medicine, Kanazawa, Japan
- Department of Health Promotion and Medicine of the Future, Kanazawa University Graduate School of Medicine, Kanazawa, Japan
- Faculty of Transdisciplinary Sciences, Institute of Transdisciplinary Sciences, Kanazawa University, Kanazawa, Japan
| | - Mitsuhiro Kawano
- Department of Rheumatology, Kanazawa University Graduate School of Medicine, Kanazawa, Japan
- * E-mail: (MK); (HN)
| | | | - Hidetaka Nambo
- School of Electrical Information Communication Engineering, College of Science and Engineering, Kanazawa University, Kanazawa, Japan
- * E-mail: (MK); (HN)
| |
Collapse
|
11
|
Bülow RD, Marsh JN, Swamidass SJ, Gaut JP, Boor P. The potential of artificial intelligence-based applications in kidney pathology. Curr Opin Nephrol Hypertens 2022; 31:251-257. [PMID: 35165248 PMCID: PMC9035059 DOI: 10.1097/mnh.0000000000000784] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
PURPOSE OF REVIEW The field of pathology is currently undergoing a significant transformation from traditional glass slides to a digital format dependent on whole slide imaging. Transitioning from glass to digital has opened the field to development and application of image analysis technology, commonly deep learning methods (artificial intelligence [AI]) to assist pathologists with tissue examination. Nephropathology is poised to leverage this technology to improve precision, accuracy, and efficiency in clinical practice. RECENT FINDINGS Through a multidisciplinary approach, nephropathologists, and computer scientists have made significant recent advances in developing AI technology to identify histological structures within whole slide images (segmentation), quantification of histologic structures, prediction of clinical outcomes, and classifying disease. Virtual staining of tissue and automation of electron microscopy imaging are emerging applications with particular significance for nephropathology. SUMMARY AI applied to image analysis in nephropathology has potential to transform the field by improving diagnostic accuracy and reproducibility, efficiency, and prognostic power. Reimbursement, demonstration of clinical utility, and seamless workflow integration are essential to widespread adoption.
Collapse
Affiliation(s)
- Roman D. Bülow
- Institute of Pathology, RWTH Aachen University Hospital, Aachen, Germany
| | - Jon N. Marsh
- Washington University School of Medicine in St. Louis, Department of Pathology and Immunology
| | - S. Joshua Swamidass
- Washington University School of Medicine in St. Louis, Department of Pathology and Immunology
| | - Joseph P. Gaut
- Washington University School of Medicine in St. Louis, Department of Pathology and Immunology
| | - Peter Boor
- Institute of Pathology, RWTH Aachen University Hospital, Aachen, Germany
- Department of Nephrology and Immunology, RWTH Aachen University Hospital, Aachen, Germany
- Electron Microscopy Facility, RWTH Aachen University Hospital, Aachen, Germany
| |
Collapse
|