Assessment of glomerular morphological patterns by deep learning algorithms

Deep learning-based glomerulus detection and classification with generative morphology augmentation in renal pathology images

Glomerulus morphology on renal pathology images provides valuable diagnosis and outcome prediction information. To provide better care, an efficient, standardized, and scalable method is urgently needed to optimize the time-consuming and labor-intensive interpretation process by renal pathologists. This paper proposes a deep convolutional neural network (CNN)-based approach to automatically detect and classify glomeruli with different stains in renal pathology images. In the glomerulus detection stage, this paper proposes a flattened Xception with a feature pyramid network (FX-FPN). The FX-FPN is employed as a backbone in the framework of faster region-based CNN to improve glomerulus detection performance. In the classification stage, this paper considers classifications of five glomerulus morphologies using a flattened Xception classifier. To endow the classifier with higher discriminability, this paper proposes a generative data augmentation approach for patch-based glomerulus morphology augmentation. New glomerulus patches of different morphologies are generated for data augmentation through the cycle-consistent generative adversarial network (CycleGAN). The single detection model shows the F1 score up to 0.9524 in H&E and PAS stains. The classification result shows that the average sensitivity and specificity are 0.7077 and 0.9316, respectively, by using the flattened Xception with the original training data. The sensitivity and specificity increase to 0.7623 and 0.9443, respectively, by using the generative data augmentation. Comparisons with different deep CNN models show the effectiveness and superiority of the proposed approach.

Global attention based GNN with Bayesian collaborative learning for glomerular lesion recognition

BACKGROUND

Glomerular lesions reflect the onset and progression of renal disease. Pathological diagnoses are widely regarded as the definitive method for recognizing these lesions, as the deviations in histopathological structures closely correlate with impairments in renal function.

METHODS

Deep learning plays a crucial role in streamlining the laborious, challenging, and subjective task of recognizing glomerular lesions by pathologists. However, the current methods treat pathology images as data in regular Euclidean space, limiting their ability to efficiently represent the complex local features and global connections. In response to this challenge, this paper proposes a graph neural network (GNN) that utilizes global attention pooling (GAP) to more effectively extract high-level semantic features from glomerular images. The model incorporates Bayesian collaborative learning (BCL), enhancing node feature fine-tuning and fusion during training. In addition, this paper adds a soft classification head to mitigate the semantic ambiguity associated with a purely hard classification.

RESULTS

This paper conducted extensive experiments on four glomerular datasets, comprising a total of 491 whole slide images (WSIs) and 9030 images. The results demonstrate that the proposed model achieves impressive F1 scores of 81.37%, 90.12%, 87.72%, and 98.68% on four private datasets for glomerular lesion recognition. These scores surpass the performance of the other models used for comparison. Furthermore, this paper employed a publicly available BReAst Carcinoma Subtyping (BRACS) dataset with an 85.61% F1 score to further prove the superiority of the proposed model.

CONCLUSION

The proposed model not only facilitates precise recognition of glomerular lesions but also serves as a potent tool for diagnosing kidney diseases effectively. Furthermore, the framework and training methodology of the GNN can be adeptly applied to address various pathology image classification challenges.

Deep learning applications for kidney histology analysis

PURPOSE OF REVIEW

Nephropathology is increasingly incorporating computational methods to enhance research and diagnostic accuracy. The widespread adoption of digital pathology, coupled with advancements in deep learning, will likely transform our pathology practices. Here, we discuss basic concepts of deep learning, recent applications in nephropathology, current challenges in implementation and future perspectives.

RECENT FINDINGS

Deep learning models have been developed and tested in various areas of nephropathology, for example, predicting kidney disease progression or diagnosing diseases based on imaging and clinical data. Despite their promising potential, challenges remain that hinder a wider adoption, for example, the lack of prospective evidence and testing in real-world scenarios.

SUMMARY

Deep learning offers great opportunities to improve quantitative and qualitative kidney histology analysis for research and clinical nephropathology diagnostics. Although exciting approaches already exist, the potential of deep learning in nephropathology is only at its beginning and we can expect much more to come.

[Advances in computational quantitative nephropathology]

BACKGROUND

Semiquantitative histological scoring systems are frequently used in nephropathology. In computational nephropathology, the focus is on generating quantitative data from histology (so-called pathomics). Several recent studies have collected such data using next-generation morphometry (NGM) based on segmentations by artificial neural networks and investigated their usability for various clinical or diagnostic purposes.

AIM

To present an overview of the current state of studies regarding renal pathomics and to identify current challenges and potential solutions.

MATERIALS AND METHODS

Due to the literature restriction (maximum of 30 references), studies were selected based on a database search that processed as much data as possible, used innovative methodologies, and/or were ideally multicentric in design.

RESULTS AND DISCUSSION

Pathomics studies in the kidney have impressively demonstrated that morphometric data are useful clinically (for example, for prognosis assessment) and translationally. Further development of NGM requires overcoming some challenges, including better standardization and generation of prospective evidence.

Recognition of intraglomerular histological features with deep learning in protocol transplant biopsies and their association with kidney function and prognosis

Background

The Banff Classification may not adequately address protocol transplant biopsies categorized as normal in patients experiencing unexplained graft function deterioration. This study seeks to employ convolutional neural networks to automate the segmentation of glomerular cells and capillaries and assess their correlation with transplant function.

Methods

A total of 215 patients were categorized into three groups. In the Training cohort, glomerular cells and capillaries from 37 patients were manually annotated to train the networks. The Test cohort (24 patients) compared manual annotations vs automated predictions, while the Application cohort (154 protocol transplant biopsies) examined predicted factors in relation to kidney function and prognosis.

Results

In the Test cohort, the networks recognized histological structures with Precision, Recall, F-score and Intersection Over Union exceeding 0.92, 0.85, 0.89 and 0.74, respectively. Univariate analysis revealed associations between the estimated glomerular filtration rate (eGFR) at biopsy and relative endothelial area (r = 0.19, P = .027), endothelial cell density (r = 0.20, P = .017), mean parietal epithelial cell area (r = -0.38, P < .001), parietal epithelial cell density (r = 0.29, P < .001) and mesangial cell density (r = 0.22, P = .010). Multivariate analysis retained only endothelial cell density as associated with eGFR (Beta = 0.13, P = .040). Endothelial cell density (r = -0.22, P = .010) and mean podocyte area (r = 0.21, P = .016) were linked to proteinuria at biopsy. Over 44 ± 29 months, 25 patients (16%) reached the primary composite endpoint (dialysis initiation, or 30% eGFR sustained decline), with relative endothelial area, mean endothelial cell area and parietal epithelial cell density below medians linked to this endpoint [hazard ratios, respectively, of 2.63 (P = .048), 2.60 (P = .039) and 3.23 (P = .019)].

Conclusion

This study automated the measurement of intraglomerular cells and capillaries. Our results suggest that the precise segmentation of endothelial and epithelial cells may serve as a potential future marker for the risk of graft loss.

Performance and limitations of a supervised deep learning approach for the histopathological Oxford Classification of glomeruli with IgA nephropathy

BACKGROUND AND OBJECTIVE

The Oxford Classification for IgA nephropathy is the most successful example of an evidence-based nephropathology classification system. The aim of our study was to replicate the glomerular components of Oxford scoring with an end-to-end deep learning pipeline that involves automatic glomerular segmentation followed by classification for mesangial hypercellularity (M), endocapillary hypercellularity (E), segmental sclerosis (S) and active crescents (C).

METHODS

A total number of 1056 periodic acid-Schiff (PAS) whole slide images (WSIs), coming from 386 kidney biopsies, were annotated. Several detection models for glomeruli, based on the Mask R-CNN architecture, were trained on 587 WSIs, validated on 161 WSIs, and tested on 127 WSIs. For the development of segmentation models, 20,529 glomeruli were annotated, of which 16,571 as training and 3958 as validation set. The test set of the segmentation module comprised of 2948 glomeruli. For the Oxford classification, 6206 expert-annotated glomeruli from 308 PAS WSIs were labelled for M, E, S, C and split into a training set of 4298 glomeruli from 207 WSIs, and a test set of 1908 glomeruli. We chose the best-performing models to construct an end-to-end pipeline, which we named MESCnn (MESC classification by neural network), for the glomerular Oxford classification of WSIs.

RESULTS

Instance segmentation yielded excellent results with an AP50 ranging between 78.2-80.1 % (79.4 ± 0.7 %) on the validation and 75.1-77.7 % (76.5 ± 0.9 %) on the test set. The aggregated Jaccard Index was between 73.4-75.9 % (75.0 ± 0.8 %) on the validation and 69.1-73.4 % (72.2 ± 1.4 %) on the test set. At granular glomerular level, Oxford Classification was best replicated for M with EfficientNetV2-L with a mean ROC-AUC of 90.2 % and a mean precision/recall area under the curve (PR-AUC) of 81.8 %, best for E with MobileNetV2 (ROC-AUC 94.7 %) and ResNet50 (PR-AUC 75.8 %), best for S with EfficientNetV2-M (mean ROC-AUC 92.7 %, mean PR-AUC 87.7 %), best for C with EfficientNetV2-L (ROC-AUC 92.3 %) and EfficientNetV2-S (PR-AUC 54.7 %). At biopsy-level, correlation between expert and deep learning labels fulfilled the demands of the Oxford Classification.

CONCLUSION

We designed an end-to-end pipeline for glomerular Oxford Classification on both a granular glomerular and an entire biopsy level. Both the glomerular segmentation and the classification modules are freely available for further development to the renal medicine community.

Classification of Glomerular Pathology Images in Children Using Convolutional Neural Networks with Improved SE-ResNet Module

Classification of glomerular pathology based on histology sections is the key to diagnose the type and degree of kidney diseases. To address problems in the classification of glomerular lesions in children, a deep learning-based complete glomerular classification framework was designed to detect and classify glomerular pathology. A neural network integrating Resnet and Senet (RS-INet) was proposed and a glomerular classification algorithm implemented to achieve high-precision classification of glomerular pathology. SE-Resnet was applied with improvement by transforming the convolutional layer of the original Resnet residual block into a convolutional block with smaller parameters as well as reduced network parameters on the premise of ensuring network performance. Experimental results showed that our algorithm had the best performance in differentiating mesangial proliferative glomerulonephritis (MsPGN), crescent glomerulonephritis (CGN), and glomerulosclerosis (GS) from normal glomerulus (Normal) compared with other classification algorithms. The accuracy rates were 0.960, 0.940, 0.937, and 0.968, respectively. This suggests that the classification algorithm proposed in the present study is able to identify glomerular lesions with a higher precision, and distinguish similar glomerular pathologies from each other.

A Spatially Guided Machine-Learning Method to Classify and Quantify Glomerular Patterns of Injury in Histology Images

Introduction The diagnosis of glomerular diseases is primarily based on visual assessment of histologic patterns. Semi-quantitative scoring of active and chronic lesions is often required to assess individual characteristics of the disease. Reproducibility of the visual scoring systems remains debatable, while digital and machine-learning technologies present opportunities to detect, classify and quantify glomerular lesions, also considering their inter- and intraglomerular heterogeneity.

MATERIALS AND METHODS

We performed a cross-validated comparison of three modifications of a convolutional neural network (CNN)-based approach for recognition and intraglomerular quantification of nine main glomerular patterns of injury. Reference values provided by two nephropathologists were used for validation. For each glomerular image, visual attention heatmaps were generated with a probability of class attribution for further intraglomerular quantification. The quality of classifier-produced heatmaps was evaluated by intersection over union metrics (IoU) between predicted and ground truth localization heatmaps.

RESULTS

A proposed spatially guided modification of the CNN classifier achieved the highest glomerular pattern classification accuracies, with area under curve (AUC) values up to 0.981. With regards to heatmap overlap area and intraglomerular pattern quantification, the spatially guided classifier achieved a significantly higher generalized mean IoU value compared to single-multiclass and multiple-binary classifiers.

CONCLUSIONS

We propose a spatially guided CNN classifier that in our experiments reveals the potential to achieve high accuracy for the localization of intraglomerular patterns.

Prediction of stability coefficient of open-pit mine slope based on artificial intelligence deep learning algorithm

The mining of open pit mines is widespread in China, and there are many cases of landslide accidents. Therefore, the problem of slope stability is highlighted. The stability of the slope is a factor that directly affects the mining efficiency and the safety of the entire mining process. According to the statistics, there is a 15 percent chance of finding landslide risk in China's large-scale mines. And due to the expansion of the mining scale of the enterprise, the problem of slope stability has become increasingly obvious, which has become a major subject in the study of open-pit mine engineering. In order to better predict the slope stability coefficient, this study takes a mine in China as a case to deeply discuss the accuracy of different algorithms in the stability calculation, and then uses a deep learning algorithm to study the stability under rainfall conditions. The change of the coefficient and the change of the stability coefficient before and after the slope treatment are experimentally studied with the displacement of the monitoring point. The result shows that the safety coefficient calculated by the algorithm in this paper is about 7% lower than that of the traditional algorithm. In the slope stability analysis before treatment, the safety factor calculated by the algorithm in this paper is 1.086, and the algorithm in this paper is closer to reality. In the stability analysis of the slope after treatment, the safety factor calculated by the algorithm in this paper is 1.227, and the stability factor meets the requirements of the specification. It also shows that the deep learning algorithm effectively improves the efficiency of the slope stability factor prediction and improves security during project development.

Current progress in artificial intelligence-assisted medical image analysis for chronic kidney disease: A literature review

Chronic kidney disease (CKD) causes irreversible damage to kidney structure and function. Arising from various etiologies, risk factors for CKD include hypertension and diabetes. With a progressively increasing global prevalence, CKD is an important public health problem worldwide. Medical imaging has become an important diagnostic tool for CKD through the non-invasive identification of macroscopic renal structural abnormalities. Artificial intelligence (AI)-assisted medical imaging techniques aid clinicians in the analysis of characteristics that cannot be easily discriminated by the naked eye, providing valuable information for the identification and management of CKD. Recent studies have demonstrated the effectiveness of AI-assisted medical image analysis as a clinical support tool using radiomics- and deep learning-based AI algorithms for improving the early detection, pathological assessment, and prognostic evaluation of various forms of CKD, including autosomal dominant polycystic kidney disease. Herein, we provide an overview of the potential roles of AI-assisted medical image analysis for the diagnosis and management of CKD.

Artificial Intelligence-Assisted Renal Pathology: Advances and Prospects

Digital imaging and advanced microscopy play a pivotal role in the diagnosis of kidney diseases. In recent years, great achievements have been made in digital imaging, providing novel approaches for precise quantitative assessments of nephropathology and relieving burdens of renal pathologists. Developing novel methods of artificial intelligence (AI)-assisted technology through multidisciplinary interaction among computer engineers, renal specialists, and nephropathologists could prove beneficial for renal pathology diagnoses. An increasing number of publications has demonstrated the rapid growth of AI-based technology in nephrology. In this review, we offer an overview of AI-assisted renal pathology, including AI concepts and the workflow of processing digital image data, focusing on the impressive advances of AI application in disease-specific backgrounds. In particular, this review describes the applied computer vision algorithms for the segmentation of kidney structures, diagnosis of specific pathological changes, and prognosis prediction based on images. Lastly, we discuss challenges and prospects to provide an objective view of this topic.

Natural Language Processing in Diagnostic Texts from Nephropathology

Introduction: This study investigates whether it is possible to predict a final diagnosis based on a written nephropathological description—as a surrogate for image analysis—using various NLP methods. Methods: For this work, 1107 unlabelled nephropathological reports were included. (i) First, after separating each report into its microscopic description and diagnosis section, the diagnosis sections were clustered unsupervised to less than 20 diagnostic groups using different clustering techniques. (ii) Second, different text classification methods were used to predict the diagnostic group based on the microscopic description section. Results: The best clustering results (i) could be achieved with HDBSCAN, using BoW-based feature extraction methods. Based on keywords, these clusters can be mapped to certain diagnostic groups. A transformer encoder-based approach as well as an SVM worked best regarding diagnosis prediction based on the histomorphological description (ii). Certain diagnosis groups reached F1-scores of up to 0.892 while others achieved weak classification metrics. Conclusion: While textual morphological description alone enables retrieving the correct diagnosis for some entities, it does not work sufficiently for other entities. This is in accordance with a previous image analysis study on glomerular change patterns, where some diagnoses are associated with one pattern, but for others, there exists a complex pattern combination.