A user-friendly tool for cloud-based whole slide image segmentation with examples from renal histopathology

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Investigating quantitative histological characteristics in renal pathology using HistoLens

HistoLens is an open-source graphical user interface developed using MATLAB AppDesigner for visual and quantitative analysis of histological datasets. HistoLens enables users to interrogate sets of digitally annotated whole slide images to efficiently characterize histological differences between disease and experimental groups. Users can dynamically visualize the distribution of 448 hand-engineered features quantifying color, texture, morphology, and distribution across microanatomic sub-compartments. Additionally, users can map differentially detected image features within the images by highlighting affected regions. We demonstrate the utility of HistoLens to identify hand-engineered features that correlate with pathognomonic renal glomerular characteristics distinguishing diabetic nephropathy and amyloid nephropathy from the histologically unremarkable glomeruli in minimal change disease. Additionally, we examine the use of HistoLens for glomerular feature discovery in the Tg26 mouse model of HIV-associated nephropathy. We identify numerous quantitative glomerular features distinguishing Tg26 transgenic mice from wild-type mice, corresponding to a progressive renal disease phenotype. Thus, we demonstrate an off-the-shelf and ready-to-use toolkit for quantitative renal pathology applications.

Histopathology language-image representation learning for fine-grained digital pathology cross-modal retrieval

Large-scale digital whole slide image (WSI) datasets analysis have gained significant attention in computer-aided cancer diagnosis. Content-based histopathological image retrieval (CBHIR) is a technique that searches a large database for data samples matching input objects in both details and semantics, offering relevant diagnostic information to pathologists. However, the current methods are limited by the difficulty of gigapixels, the variable size of WSIs, and the dependence on manual annotations. In this work, we propose a novel histopathology language-image representation learning framework for fine-grained digital pathology cross-modal retrieval, which utilizes paired diagnosis reports to learn fine-grained semantics from the WSI. An anchor-based WSI encoder is built to extract hierarchical region features and a prompt-based text encoder is introduced to learn fine-grained semantics from the diagnosis reports. The proposed framework is trained with a multivariate cross-modal loss function to learn semantic information from the diagnosis report at both the instance level and region level. After training, it can perform four types of retrieval tasks based on the multi-modal database to support diagnostic requirements. We conducted experiments on an in-house dataset and a public dataset to evaluate the proposed method. Extensive experiments have demonstrated the effectiveness of the proposed method and its advantages to the present histopathology retrieval methods. The code is available at https://github.com/hudingyi/FGCR.

Mesangial Expansion by Morphometry at 5 y After Kidney Transplantation: Incidence, Risk Factors, and Association With Graft Loss

Background

Mesangial expansion (ME) is an understudied histologic lesion in renal allografts. The current Banff mm score is not reproducible and may miss important ME features. The study aimed to improve the quantification of ME using morphometry, assess changes over time, and determine its association with allograft loss.

Methods

We studied ME in 1-y and 5-y surveillance biopsies in 835 kidney transplants performed between January 2000 and December 2013. ME was assessed using the Banff mm score by a central pathologist and by morphometry. We derived 3 different morphometric measures: (1) %ME mm (%glomeruli with ME in ≥2 lobules, like Banff mm); (2) %MEany (%glomeruli with any ME lesion); and (3) %ME area (sum of all ME areas/all glomerular tuft areas). Unadjusted and adjusted Cox models assessed the risk of death-censored allograft loss.

Results

From 1- to 5-y biopsies, the mean Banff mm score increased from 0.18 to 0.34, whereas %ME mm increased from 2.5% to 13.3%. Banff mm score had modest correlations with morphometric ME measures. Moderate-severe %ME mm was present in 20.1% of 5-y biopsies, whereas only 6.6% of Banff mm scores were. In general, higher ME on both 1- and 5-y biopsies was associated with a deceased donor, older recipient age, recipient diabetes/obesity (present in >50% of severely affected biopsies), higher hemoglobin A1c at 5 y posttransplant, and recurrent kidney disease. Higher ME on 5-y biopsies was associated with delayed graft function. A higher Banff mm score at 1-y biopsy and morphometry ME measures at 5-y biopsy were associated with rejection during the first year posttransplant. Morphometric ME measures were associated with allograft loss independent of Banff scores and all clinical characteristics, including kidney function and recurrent disease. The model with %MEany had the highest c-statistic (0.872).

Conclusions

Banff mm score underestimates the pervasiveness of ME in 5-y biopsies. ME is common and associated with alloimmune and nonalloimmune causes of graft loss.

Precision Medicine in Nephrology: An Integrative Framework of Multidimensional Data in the Kidney Precision Medicine Project

Chronic kidney disease (CKD) and acute kidney injury (AKI) are heterogeneous syndromes defined clinically by serial measures of kidney function. Each condition possesses strong histopathologic associations, including glomerular obsolescence or acute tubular necrosis, respectively. Despite such characterization, there remains wide variation in patient outcomes and treatment responses. Precision medicine efforts, as exemplified by the Kidney Precision Medicine Project (KPMP), have begun to establish evolving, spatially anchored, cellular and molecular atlases of the cell types, states, and niches of the kidney in health and disease. The KPMP atlas provides molecular context for CKD and AKI disease drivers and will help define subtypes of disease that are not readily apparent from canonical functional or histopathologic characterization but instead are appreciable through advanced clinical phenotyping, pathomic, transcriptomic, proteomic, epigenomic, and metabolomic interrogation of kidney biopsy samples. This perspective outlines the structure of the KPMP, its approach to the integration of these diverse datasets, and its major outputs relevant to future patient care.

DEPICTER: Deep representation clustering for histology annotation

Automatic segmentation of histopathology whole-slide images (WSI) usually involves supervised training of deep learning models with pixel-level labels to classify each pixel of the WSI into tissue regions such as benign or cancerous. However, fully supervised segmentation requires large-scale data manually annotated by experts, which can be expensive and time-consuming to obtain. Non-fully supervised methods, ranging from semi-supervised to unsupervised, have been proposed to address this issue and have been successful in WSI segmentation tasks. But these methods have mainly been focused on technical advancements in algorithmic performance rather than on the development of practical tools that could be used by pathologists or researchers in real-world scenarios. In contrast, we present DEPICTER (Deep rEPresentatIon ClusTERing), an interactive segmentation tool for histopathology annotation that produces a patch-wise dense segmentation map at WSI level. The interactive nature of DEPICTER leverages self- and semi-supervised learning approaches to allow the user to participate in the segmentation producing reliable results while reducing the workload. DEPICTER consists of three steps: first, a pretrained model is used to compute embeddings from image patches. Next, the user selects a number of benign and cancerous patches from the multi-resolution image. Finally, guided by the deep representations, label propagation is achieved using our novel seeded iterative clustering method or by directly interacting with the embedding space via feature space gating. We report both real-time interaction results with three pathologists and evaluate the performance on three public cancer classification dataset benchmarks through simulations. The code and demos of DEPICTER are publicly available at https://github.com/eduardchelebian/depicter.

ComPRePS: An Automated Cloud-based Image Analysis tool to democratize AI in Digital Pathology

Artificial intelligence (AI) has extensive applications in a wide range of disciplines including healthcare and clinical practice. Advances in high-resolution whole-slide brightfield microscopy allow for the digitization of histologically stained tissue sections, producing gigapixel-scale whole-slide images (WSI). The significant improvement in computing and revolution of deep neural network (DNN)-based AI technologies over the last decade allow us to integrate massively parallelized computational power, cutting-edge AI algorithms, and big data storage, management, and processing. Applied to WSIs, AI has created opportunities for improved disease diagnostics and prognostics with the ultimate goal of enhancing precision medicine and resulting patient care. The National Institutes of Health (NIH) has recognized the importance of developing standardized principles for data management and discovery for the advancement of science and proposed the Findable, Accessible, Interoperable, Reusable, (FAIR) Data Principles1 with the goal of building a modernized biomedical data resource ecosystem to establish collaborative research communities. In line with this mission and to democratize AI-based image analysis in digital pathology, we propose ComPRePS: an end-to-end automated Computational Renal Pathology Suite which combines massive scalability, on-demand cloud computing, and an easy-to-use web-based user interface for data upload, storage, management, slide-level visualization, and domain expert interaction. Moreover, our platform is equipped with both in-house and collaborator developed sophisticated AI algorithms in the back-end server for image analysis to identify clinically relevant micro-anatomic functional tissue units (FTU) and to extract image features.

Accelerating pharmaceutical R&D with a user-friendly AI system for histopathology image analysis

A system for analysis of histopathology data within a pharmaceutical R&D environment has been developed with the intention of enabling interdisciplinary collaboration. State-of-the-art AI tools have been deployed as easy-to-use self-service modules within an open-source whole slide image viewing platform, so that non-data scientist users (e.g., clinicians) can utilize and evaluate pre-trained algorithms and retrieve quantitative results. The outputs of analysis are automatically cataloged in the database to track data provenance and can be viewed interactively on the slide as annotations or heatmaps. Commonly used models for analysis of whole slide images including segmentation, extraction of hand-engineered features for segmented regions, and slide-level classification using multi-instance learning are included and new models can be added as needed. The source code that supports running inference with these models internally is backed up by a robust CI/CD pipeline to ensure model versioning, robust testing, and seamless deployment of the latest models. Examples of the use of this system in a pharmaceutical development workflow include glomeruli segmentation, enumeration of podocyte count from WT-1 immuno-histochemistry, measurement of beta-1 integrin target engagement from immunofluorescence, digital glomerular phenotyping from periodic acid-Schiff histology, PD-L1 score prediction using multi-instance learning, and the deployment of the open-source Segment Anything model to speed up annotation.

Tubular and Glomerular Size by Cortex Depth as Predictor of Progressive CKD after Radical Nephrectomy for Tumor

SIGNIFICANCE STATEMENT

Glomerular size differs by cortex depth. Larger nephrons are prognostic of progressive kidney disease, but it is unknown whether this risk differs by cortex depth or by glomeruli versus proximal or distal tubule size. We studied the average minor axis diameter in oval proximal and distal tubules separately and by cortex depth in patients who had radical nephrectomy to remove a tumor from 2019 to 2020. In adjusted analyses, larger glomerular volume in the middle and deep cortex predicted progressive kidney disease. Wider proximal tubular diameter did not predict progressive kidney disease independent of glomerular volume. Wider distal tubular diameter showed a gradient of strength of prediction of progressive kidney disease in the more superficial cortex than in the deep cortex.

BACKGROUND

Larger nephrons are prognostic of progressive kidney disease, but whether this risk differs by nephron segments or by depth in the cortex is unclear.

METHODS

We studied patients who underwent radical nephrectomy for a tumor between 2000 and 2019. Large wedge kidney sections were scanned into digital images. We estimated the diameters of proximal and distal tubules by the minor axis of oval tubular profiles and estimated glomerular volume with the Weibel-Gomez stereological model. Analyses were performed separately in the superficial, middle, and deep cortex. Cox proportional hazard models assessed the risk of progressive CKD (dialysis, kidney transplantation, sustained eGFR <10 ml/min per 1.73 m 2 , or a sustained 40% decline from the postnephrectomy baseline eGFR) with glomerular volume or tubule diameters. At each cortical depth, models were unadjusted, adjusted for glomerular volume or tubular diameter, and further adjusted for clinical characteristics (age, sex, body mass index, hypertension, diabetes, postnephrectomy baseline eGFR, and proteinuria).

RESULTS

Among 1367 patients were 62 progressive CKD events during a median follow-up of 4.5 years. Glomerular volume predicted CKD outcomes at all depths, but only in the middle and deep cortex after adjusted analyses. Proximal tubular diameter also predicted progressive CKD at any depth but not after adjusted analyses. Distal tubular diameter showed a gradient of more strongly predicting progressive CKD in the superficial than deep cortex, even in adjusted analysis.

CONCLUSIONS

Larger glomeruli are independent predictors of progressive CKD in the deeper cortex, whereas in the superficial cortex, wider distal tubular diameters are an independent predictor of progressive CKD.

Advances and prospects for the Human BioMolecular Atlas Program (HuBMAP)

The Human BioMolecular Atlas Program (HuBMAP) aims to create a multi-scale spatial atlas of the healthy human body at single-cell resolution by applying advanced technologies and disseminating resources to the community. As the HuBMAP moves past its first phase, creating ontologies, protocols and pipelines, this Perspective introduces the production phase: the generation of reference spatial maps of functional tissue units across many organs from diverse populations and the creation of mapping tools and infrastructure to advance biomedical research.

Unleashing the potential of AI for pathology: challenges and recommendations

Computational pathology is currently witnessing a surge in the development of AI techniques, offering promise for achieving breakthroughs and significantly impacting the practices of pathology and oncology. These AI methods bring with them the potential to revolutionize diagnostic pipelines as well as treatment planning and overall patient care. Numerous peer-reviewed studies reporting remarkable performance across diverse tasks serve as a testimony to the potential of AI in the field. However, widespread adoption of these methods in clinical and pre-clinical settings still remains a challenge. In this review article, we present a detailed analysis of the major obstacles encountered during the development of effective models and their deployment in practice. We aim to provide readers with an overview of the latest developments, assist them with insights into identifying some specific challenges that may require resolution, and suggest recommendations and potential future research directions. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Segmentation of human functional tissue units in support of a Human Reference Atlas

The Human BioMolecular Atlas Program (HuBMAP) aims to compile a Human Reference Atlas (HRA) for the healthy adult body at the cellular level. Functional tissue units (FTUs), relevant for HRA construction, are of pathobiological significance. Manual segmentation of FTUs does not scale; highly accurate and performant, open-source machine-learning algorithms are needed. We designed and hosted a Kaggle competition that focused on development of such algorithms and 1200 teams from 60 countries participated. We present the competition outcomes and an expanded analysis of the winning algorithms on additional kidney and colon tissue data, and conduct a pilot study to understand spatial location and density of FTUs across the kidney. The top algorithm from the competition, Tom, outperforms other algorithms in the expanded study, while using fewer computational resources. Tom was added to the HuBMAP infrastructure to run kidney FTU segmentation at scale-showcasing the value of Kaggle competitions for advancing research.

High-throughput image analysis with deep learning captures heterogeneity and spatial relationships after kidney injury

Recovery from acute kidney injury can vary widely in patients and in animal models. Immunofluorescence staining can provide spatial information about heterogeneous injury responses, but often only a fraction of stained tissue is analyzed. Deep learning can expand analysis to larger areas and sample numbers by substituting for time-intensive manual or semi-automated quantification techniques. Here we report one approach to leverage deep learning tools to quantify heterogenous responses to kidney injury that can be deployed without specialized equipment or programming expertise. We first demonstrated that deep learning models generated from small training sets accurately identified a range of stains and structures with performance similar to that of trained human observers. We then showed this approach accurately tracks the evolution of folic acid induced kidney injury in mice and highlights spatially clustered tubules that fail to repair. We then demonstrated that this approach captures the variation in recovery across a robust sample of kidneys after ischemic injury. Finally, we showed markers of failed repair after ischemic injury were correlated both spatially within and between animals and that failed repair was inversely correlated with peritubular capillary density. Combined, we demonstrate the utility and versatility of our approach to capture spatially heterogenous responses to kidney injury.

Spatially Aware Transformer Networks for Contextual Prediction of Diabetic Nephropathy Progression from Whole Slide Images

Diabetic nephropathy (DN) in the context of type 2 diabetes is the leading cause of end-stage renal disease (ESRD) in the United States. DN is graded based on glomerular morphology and has a spatially heterogeneous presentation in kidney biopsies that complicates pathologists' predictions of disease progression. Artificial intelligence and deep learning methods for pathology have shown promise for quantitative pathological evaluation and clinical trajectory estimation; but, they often fail to capture large-scale spatial anatomy and relationships found in whole slide images (WSIs). In this study, we present a transformer-based, multi-stage ESRD prediction framework built upon nonlinear dimensionality reduction, relative Euclidean pixel distance embeddings between every pair of observable glomeruli, and a corresponding spatial self-attention mechanism for a robust contextual representation. We developed a deep transformer network for encoding WSI and predicting future ESRD using a dataset of 56 kidney biopsy WSIs from DN patients at Seoul National University Hospital. Using a leave-one-out cross-validation scheme, our modified transformer framework outperformed RNNs, XGBoost, and logistic regression baseline models, and resulted in an area under the receiver operating characteristic curve (AUC) of 0.97 (95% CI: 0.90-1.00) for predicting two-year ESRD, compared with an AUC of 0.86 (95% CI: 0.66-0.99) without our relative distance embedding, and an AUC of 0.76 (95% CI: 0.59-0.92) without a denoising autoencoder module. While the variability and generalizability induced by smaller sample sizes are challenging, our distance-based embedding approach and overfitting mitigation techniques yielded results that suggest opportunities for future spatially aware WSI research using limited pathology datasets.