Image analysis based grading of bladder carcinoma. Comparison of object, texture and graph based methods and their reproducibility

Clinical applications of graph neural networks in computational histopathology: A review

Pathological examination is the optimal approach for diagnosing cancer, and with the advancement of digital imaging technologies, it has spurred the emergence of computational histopathology. The objective of computational histopathology is to assist in clinical tasks through image processing and analysis techniques. In the early stages, the technique involved analyzing histopathology images by extracting mathematical features, but the performance of these models was unsatisfactory. With the development of artificial intelligence (AI) technologies, traditional machine learning methods were applied in this field. Although the performance of the models improved, there were issues such as poor model generalization and tedious manual feature extraction. Subsequently, the introduction of deep learning techniques effectively addressed these problems. However, models based on traditional convolutional architectures could not adequately capture the contextual information and deep biological features in histopathology images. Due to the special structure of graphs, they are highly suitable for feature extraction in tissue histopathology images and have achieved promising performance in numerous studies. In this article, we review existing graph-based methods in computational histopathology and propose a novel and more comprehensive graph construction approach. Additionally, we categorize the methods and techniques in computational histopathology according to different learning paradigms. We summarize the common clinical applications of graph-based methods in computational histopathology. Furthermore, we discuss the core concepts in this field and highlight the current challenges and future research directions.

Machine learning in computational histopathology: Challenges and opportunities

Digital histopathological images, high-resolution images of stained tissue samples, are a vital tool for clinicians to diagnose and stage cancers. The visual analysis of patient state based on these images are an important part of oncology workflow. Although pathology workflows have historically been conducted in laboratories under a microscope, the increasing digitization of histopathological images has led to their analysis on computers in the clinic. The last decade has seen the emergence of machine learning, and deep learning in particular, a powerful set of tools for the analysis of histopathological images. Machine learning models trained on large datasets of digitized histopathology slides have resulted in automated models for prediction and stratification of patient risk. In this review, we provide context for the rise of such models in computational histopathology, highlight the clinical tasks they have found success in automating, discuss the various machine learning techniques that have been applied to this domain, and underscore open problems and opportunities.

An update on computational pathology tools for genitourinary pathology practice: A review paper from the Genitourinary Pathology Society (GUPS)

Machine learning has been leveraged for image analysis applications throughout a multitude of subspecialties. This position paper provides a perspective on the evolutionary trajectory of practical deep learning tools for genitourinary pathology through evaluating the most recent iterations of such algorithmic devices. Deep learning tools for genitourinary pathology demonstrate potential to enhance prognostic and predictive capacity for tumor assessment including grading, staging, and subtype identification, yet limitations in data availability, regulation, and standardization have stymied their implementation.

Applications of Digital and Computational Pathology and Artificial Intelligence in Genitourinary Pathology Diagnostics

As machine learning (ML) solutions for genitourinary pathology image analysis are fostered by a progressively digitized laboratory landscape, these integrable modalities usher in a revolution in histopathological diagnosis. As technology advances, limitations stymying clinical artificial intelligence (AI) will not be extinguished without thorough validation and interrogation of ML tools by pathologists and regulatory bodies alike. ML solutions deployed in clinical settings for applications in prostate pathology yield promising results. Recent breakthroughs in clinical artificial intelligence for genitourinary pathology demonstrate unprecedented generalizability, heralding prospects for a future in which AI-driven assistive solutions may be seen as laboratory faculty, rather than novelty.

Application of Artificial Intelligence in Pathology: Trends and Challenges

Given the recent success of artificial intelligence (AI) in computer vision applications, many pathologists anticipate that AI will be able to assist them in a variety of digital pathology tasks. Simultaneously, tremendous advancements in deep learning have enabled a synergy with artificial intelligence (AI), allowing for image-based diagnosis on the background of digital pathology. There are efforts for developing AI-based tools to save pathologists time and eliminate errors. Here, we describe the elements in the development of computational pathology (CPATH), its applicability to AI development, and the challenges it faces, such as algorithm validation and interpretability, computing systems, reimbursement, ethics, and regulations. Furthermore, we present an overview of novel AI-based approaches that could be integrated into pathology laboratory workflows.

The Genitourinary Pathology Society Update on Classification and Grading of Flat and Papillary Urothelial Neoplasia With New Reporting Recommendations and Approach to Lesions With Mixed and Early Patterns of Neoplasia

The Genitourinary Pathology Society (GUPS) undertook a critical review of the recent advances in bladder neoplasia with a focus on issues relevant to the practicing surgical pathologist for the understanding and effective reporting of bladder cancer, emphasizing particularly on the newly accumulated evidence post-2016 World Health Organization (WHO) classification. The work is presented in 2 manuscripts. Here, in the first, we revisit the nomenclature and classification system used for grading flat and papillary urothelial lesions centering on clinical relevance, and on dilemmas related to application in routine reporting. As patients of noninvasive bladder cancer frequently undergo cystoscopy and biopsy in their typically prolonged clinical course and for surveillance of disease, we discuss morphologies presented in these scenarios which may not have readily applicable diagnostic terms in the WHO classification. The topic of inverted patterns in urothelial neoplasia, particularly when prominent or exclusive, and beyond inverted papilloma has not been addressed formally in the WHO classification. Herein we provide a through review and suggest guidelines for when and how to report such lesions. In promulgating these GUPS recommendations, we aim to provide clarity on the clinical application of these not so uncommon diagnostically challenging situations encountered in routine practice, while also importantly advocating consistent terminology which would inform future work.

Deep learning in digital pathology image analysis: a survey

Deep learning (DL) has achieved state-of-the-art performance in many digital pathology analysis tasks. Traditional methods usually require hand-crafted domain-specific features, and DL methods can learn representations without manually designed features. In terms of feature extraction, DL approaches are less labor intensive compared with conventional machine learning methods. In this paper, we comprehensively summarize recent DL-based image analysis studies in histopathology, including different tasks (e.g., classification, semantic segmentation, detection, and instance segmentation) and various applications (e.g., stain normalization, cell/gland/region structure analysis). DL methods can provide consistent and accurate outcomes. DL is a promising tool to assist pathologists in clinical diagnosis.

Automated Detection and Grading of Non-Muscle-Invasive Urothelial Cell Carcinoma of the Bladder

Accurate grading of non-muscle-invasive urothelial cell carcinoma is of major importance; however, high interobserver variability exists. A fully automated detection and grading network based on deep learning is proposed to enhance reproducibility. A total of 328 transurethral resection specimens from 232 patients were included, and a consensus reading by three specialized pathologists was used. The slides were digitized, and the urothelium was annotated by expert observers. The U-Net-based segmentation network was trained to automatically detect urothelium. This detection was used as input for the classification network. The classification network aimed to grade the tumors according to the World Health Organization grading system adopted in 2004. The automated grading was compared with the consensus and individual grading. The segmentation network resulted in an accurate detection of urothelium. The automated grading shows moderate agreement (κ = 0.48 ± 0.14 SEM) with the consensus reading. The agreement among pathologists ranges between fair (κ = 0.35 ± 0.13 SEM and κ = 0.38 ± 0.11 SEM) and moderate (κ = 0.52 ± 0.13 SEM). The automated classification correctly graded 76% of the low-grade cancers and 71% of the high-grade cancers according to the consensus reading. These results indicate that deep learning can be used for the fully automated detection and grading of urothelial cell carcinoma.

Computed tomography-based texture analysis of bladder cancer: differentiating urothelial carcinoma from micropapillary carcinoma

PURPOSE

The purpose of the study is to determine the feasibility of using computed tomography-based texture analysis (CTTA) in differentiating between urothelial carcinomas (UC) of the bladder from micropapillary carcinomas (MPC) of the bladder.

METHODS

Regions of interests (ROIs) of computerized tomography (CT) images of 33 MPCs and 33 UCs were manually segmented and saved. Custom MATLAB code was used to extract voxel information corresponding to the ROI. The segmented tumors were input to a pre-existing radiomics platform with a CTTA panel. A total of 58 texture metrics were extracted using four different texture extraction techniques and statistically analyzed using a Wilcoxon rank-sum test to determine the differences between UCs and MPCs.

RESULTS

Of the 58 texture metrics extracted using the gray level co-occurrence matrix (GLCM) and gray level difference matrix (GLDM), 28 texture metrics were statistically significant (p < 0.05) for differences in tumor textures and 27 texture metrics were statistically significant (p < 0.05) for peritumoral fat textures. The remaining nine metrics extracted using histogram and fast Fourier transform analyses did not show significant differences between the textures of the tumors and their peritumoral fat.

CONCLUSIONS

CTTA shows that MPC have a more heterogeneous texture compared to UC. As visual discrimination of MPC from UC from clinical CT scans are difficult, results from this study suggest that tumor heterogeneity extracted using GLCM and GLDM may be a good imaging aid in segregating MPC from UC. This tool can aid clinicians in further sub-classifying bladder cancers on routine imaging, a process which has potential to alter treatment and patient care.

ARTIFICIAL INTELLIGENT SYSTEMS APPLICATION IN CERVICAL CANCER PATHOLOGICAL CELL IMAGE CLASSIFICATION SYSTEMS — A REVIEW

Cervical cancer cell images play an important part in diagnosing the cancer among the females worldwide. Existing noises, overlapping cells, mucus, blood and air artifacts in cervical cancer cell images makes their classification a hard task. It makes it difficult for both pathologists and intelligent systems to segment and classify them into normal, pre-cancerous and cancerous cells. However, true cell segmentation is needed for pathologists to make for accurate diagnosis. In this paper, a review of algorithms used for cervical cancer cell image classification is presented. This includes pre-processing steps (noise reduction and cell segmentation/without segmentation), feature extraction, and intelligent diagnosis systems and their evaluations. Finally, future research trends on cervical cell classification to achieve complete accuracy are described.

New morphological features for grading pancreatic ductal adenocarcinomas

Pathological diagnosis is influenced by subjective factors such as the individual experience and knowledge of doctors. Therefore, it may be interpreted in different ways for the same symptoms. The appearance of digital pathology has created good foundation for objective diagnoses based on quantitative feature analysis. Recently, numerous studies are being done to develop automated diagnosis based on the digital pathology. But there are as of yet no general automated methods for pathological diagnosis due to its specific nature. Therefore, specific methods according to a type of disease and a lesion could be designed. This study proposes quantitative features that are designed to diagnose pancreatic ductal adenocarcinomas. In the diagnosis of pancreatic ductal adenocarcinomas, the region of interest is a duct that consists of lumen and epithelium. Therefore, we first segment the lumen and epithelial nuclei from a tissue image. Then, we extract the specific features to diagnose the pancreatic ductal adenocarcinoma from the segmented objects. The experiment evaluated the classification performance of the SVM learned by the proposed features. The results showed an accuracy of 94.38% in the experiment distinguishing between pancreatic ductal adenocarcinomas and normal tissue and a classification accuracy of 77.03% distinguishing between the stages of pancreatic ductal adenocarcinomas.

Automatic differential diagnosis of pancreatic serous and mucinous cystadenomas based on morphological features

Generally, pathological diagnosis using an electron microscope is time-consuming and likely to result in a subjective judgment, because pathologists perform manual screening of tissue slides at high magnifications. Recently, the advent of digital pathology technology has provided the basis for convenient screening and quantitative analysis by digitizing tissue slides through a computer system. However, a screening process with high magnification still takes quite a long time. To solve these problems, recently the use of computer-aided design techniques for performing pathologic diagnosis has been increasing in digital pathology. For pathological diagnosis, we need different diagnostic methods for different regions with different characteristics. Therefore, in order to effectively diagnose different lesions and types of diseases, a quantitative method for extracting specific features is required in computerized pathologic diagnosis. This study is about an automated differential diagnosis system to differentiate between benign serous cystadenoma and possibly-malignant mucinous cystadenoma. In order to diagnose cystic tumors, the first step is identifying a cystic region and inspecting its epithelial cells. First, we identify the lumen boundary of a cyst using the Direction Cumulative Map considering 8-ways. Then, the Epithelial Nuclei Identification algorithm is used to discern epithelial nuclei. After that, three morphological features for the differential diagnosis of mucinous and serous cystadenomas are extracted. To demonstrate the superiority of the proposed features, the experiments compared performance of the classifiers learned by using the proposed morphological features and the classical morphological features based on nuclei. The classifiers in the simulations are as follows; Bayesian Classifier, k-Nearest Neighbors, Support Vector Machine, and Artificial Neural Network. The results show that all classifiers using the proposed features have the best classification performance.

Coupled analysis of in vitro and histology tissue samples to quantify structure-function relationship

The structure/function relationship is fundamental to our understanding of biological systems at all levels, and drives most, if not all, techniques for detecting, diagnosing, and treating disease. However, at the tissue level of biological complexity we encounter a gap in the structure/function relationship: having accumulated an extraordinary amount of detailed information about biological tissues at the cellular and subcellular level, we cannot assemble it in a way that explains the correspondingly complex biological functions these structures perform. To help close this information gap we define here several quantitative temperospatial features that link tissue structure to its corresponding biological function. Both histological images of human tissue samples and fluorescence images of three-dimensional cultures of human cells are used to compare the accuracy of in vitro culture models with their corresponding human tissues. To the best of our knowledge, there is no prior work on a quantitative comparison of histology and in vitro samples. Features are calculated from graph theoretical representations of tissue structures and the data are analyzed in the form of matrices and higher-order tensors using matrix and tensor factorization methods, with a goal of differentiating between cancerous and healthy states of brain, breast, and bone tissues. We also show that our techniques can differentiate between the structural organization of native tissues and their corresponding in vitro engineered cell culture models.

A resampling-based Markovian model for automated colon cancer diagnosis

In recent years, there has been a great effort in the research of implementing automated diagnostic systems for tissue images. One major challenge in this implementation is to design systems that are robust to image variations. In order to meet this challenge, it is important to learn the systems on a large number of labeled images from a different range of variation. However, acquiring labeled images is quite difficult in this domain, and hence, the labeled training data are typically very limited. Although the issue of having limited labeled data is acknowledged by many researchers, it has rarely been considered in the system design. This paper successfully addresses this issue, introducing a new resampling framework to simulate variations in tissue images. This framework generates multiple sequences from an image for its representation and models them using a Markov process. Working with colon tissue images, our experiments show that this framework increases the generalization capacity of a learner by increasing the size and variation of the training data and improves the classification performance of a given image by combining the decisions obtained on its sequences.

Detection and classification of thyroid follicular lesions based on nuclear structure from histopathology images

Follicular lesions of the thyroid are traditionally difficult and tedious challenges in diagnostic surgical pathology in part due to lack of obvious discriminatory cytological and microarchitectural features. We describe a computerized method to detect and classify follicular adenoma of the thyroid, follicular carcinoma of the thyroid, and normal thyroid based on the nuclear chromatin distribution from digital images of tissue obtained by routine histological methods. Our method is based on determining whether a set of nuclei, obtained from histological images using automated image segmentation, is most similar to sets of nuclei obtained from normal or diseased tissues. This comparison is performed utilizing numerical features, a support vector machine, and a simple voting strategy. We also describe novel methods to identify unique and defining chromatin patterns pertaining to each class. Unlike previous attempts in detecting and classifying these thyroid lesions using computational imaging, our results show that our method can automatically classify the data pertaining to 10 different human cases with 100% accuracy after blind cross validation using at most 43 nuclei randomly selected from each patient. We conclude that nuclear structure alone contains enough information to automatically classify the normal thyroid, follicular carcinoma, and follicular adenoma, as long as groups of nuclei (instead of individual ones) are used. We also conclude that the distribution of nuclear size and chromatin concentration (how tightly packed it is) seem to be discriminating features between nuclei of follicular adenoma, follicular carcinoma, and normal thyroid.

Cancer Cell Image Analysis and Visualization

We review computerized cancer cell image analysis and visualization research over the past 30 years. Image acquisition, feature extraction, classification, and visualization from two-dimensional to three-dimensional image algorithms are introduced with case studies of bladder, prostate, breast, and renal carcinomas.

Quantification of spatial parameters in 3D cellular constructs using graph theory

Multispectral three-dimensional (3D) imaging provides spatial information for biological structures that cannot be measured by traditional methods. This work presents a method of tracking 3D biological structures to quantify changes over time using graph theory. Cell-graphs were generated based on the pairwise distances, in 3D-Euclidean space, between nuclei during collagen I gel compaction. From these graphs quantitative features are extracted that measure both the global topography and the frequently occurring local structures of the “tissue constructs.” The feature trends can be controlled by manipulating compaction through cell density and are significant when compared to random graphs. This work presents a novel methodology to track a simple 3D biological event and quantitatively analyze the underlying structural change. Further application of this method will allow for the study of complex biological problems that require the quantification of temporal-spatial information in 3D and establish a new paradigm in understanding structure-function relationships.

ECM-Aware Cell-Graph Mining for Bone Tissue Modeling and Classification

Pathological examination of a biopsy is the most reliable and widely used technique to diagnose bone cancer. However, it suffers from both inter- and intra- observer subjectivity. Techniques for automated tissue modeling and classification can reduce this subjectivity and increases the accuracy of bone cancer diagnosis. This paper presents a graph theoretical method, called extracellular matrix (ECM)-aware cell-graph mining, that combines the ECM formation with the distribution of cells in hematoxylin and eosin (H&E) stained histopathological images of bone tissues samples. This method can identify different types of cells that coexist in the same tissue as a result of its functional state. Thus, it models the structure-function relationships more precisely and classifies bone tissue samples accurately for cancer diagnosis. The tissue images are segmented, using the eigenvalues of the Hessian matrix, to compute spatial coordinates of cell nuclei as the nodes of corresponding cell-graph. Upon segmentation a color code is assigned to each node based on the composition of its surrounding ECM. An edge is hypothesized (and established) between a pair of nodes if the corresponding cell membranes are in physical contact and if they share the same color. Hence, multiple colored-cell-graphs coexist in a tissue each modeling a different cell-type organization. Both topological and spectral features of ECM-aware cell-graphs are computed to quantify the structural properties of tissue samples and classify their different functional states as healthy, fractured, or cancerous using support vector machines. Classification accuracy comparison to related work shows that ECM-aware cell-graph approach yields 90.0% whereas Delaunay triangulation and simple cell-graph approach achieves 75.0% and 81.1% accuracy, respectively.

Color graphs for automated cancer diagnosis and grading

This paper reports a new structural method to mathematically represent and quantify a tissue for the purpose of automated and objective cancer diagnosis and grading. Unlike the previous structural methods, which quantify a tissue considering the spatial distributions of its cell nuclei, the proposed method relies on the use of distributions of multiple tissue components for the representation. To this end, it constructs a graph on multiple tissue components and colors its edges depending on the component types of their endpoints. Subsequently, it extracts a new set of structural features from these color graphs and uses these features in the classification of tissues. Working with the images of colon tissues, our experiments demonstrate that the color-graph approach leads to 82.65% test accuracy and that it significantly improves the performance of its counterparts.

Cell-graph mining for breast tissue modeling and classification

We consider the problem of automated cancer diagnosis in the context of breast tissues. We present graph theoretical techniques that identify and compute quantitative metrics for tissue characterization and classification. We segment digital images of histopatological tissue samples using k-means algorithm. For each segmented image we generate different cell-graphs using positional coordinates of cells and surrounding matrix components. These cell-graphs have 500-2000 cells(nodes) with 1000-10000 links depending on the tissue and the type of cell-graph being used. We calculate a set of global metrics from cell-graphs and use them as the feature set for learning. We compare our technique, hierarchical cell graphs, with other techniques based on intensity values of images, Delaunay triangulation of the cells, the previous technique we proposed for brain tissue images and with the hybrid approach that we introduce in this paper. Among the compared techniques, hierarchical-graph approach gives 81.8% accuracy whereas we obtain 61.0%, 54.1% and 75.9% accuracy with intensity-based features, Delaunay triangulation and our previous technique, respectively.

Grading of renal cell carcinoma by 3D morphological analysis of cell nuclei

This study attempted to develop a method for 3D visualization and quantitative analysis of cell nuclei for renal cell carcinoma (RCC) grading and evaluated the feasibility of such quantitative analysis. We compared the correct classification rate (CCR) for each of the classifiers based on the 2D features of cell nuclei (diameter, area, perimeter, and circularity) and the 3D features of cell nuclei (volume, surface area, and spherical shape factor). The results showed that the classifier using the 3D features provided better results for grading. Our method could overcome the limitations inherent in 2D analysis and could improve the accuracy and reproducibility of quantification of cell nuclei.

Prediction of radiosensitivity in human bladder cell lines using nuclear chromatin phenotype

BACKGROUND

Nuclear texture analysis measures phenotypic changes in chromatin distribution within a cell nucleus, while the alkaline Comet assay is a sensitive method for measuring the extent of DNA breakage in individual cells. The authors aim to use both methods to provide information about the sensitivity of cells to ionizing radiation.

METHODS

The alkaline Comet assay was performed on six human bladder carcinoma cell lines and one human urothelial cell line exposed to gamma-radiation doses from 0 to 10 Gy. Nuclear chromatin texture analysis of 40 features was then performed in the same cell lines exposed to 0, 2, and 6 Gy to explore if nuclear phenotype was related to radiation sensitivity.

RESULTS

Comet assay results demonstrated that the cell lines exhibited different levels of radiosensitivity and could be divided into a radiosensitive and a radioresistant group at >6 Gy. Using stepwise discriminant analysis, a subset of important nuclear texture features that best discriminated between sensitive and resistant cell lines were identified A classification function, defined using these features, correctly classified 81.75% of all cells into their radiosensitive or radioresistant groups based on their pretreatment chromatin phenotype. Posttreatment chromatin changes also varied between cell lines, with sensitive cell lines showing a relaxed chromatin conformation following radiation, whereas resistant cell lines exhibited chromatin condensation.

CONCLUSIONS

The authors conclude that the alkaline Comet assay and nuclear texture methodologies may prove to be valuable aids in predicting the response of tumor cells to radiotherapy.

Grading the invasive component of urothelial carcinoma of the bladder and its relationship with progression-free survival

Although grading is valuable prognostically in pTa and pT1 papillary urothelial carcinoma, it is unclear whether it provides any prognostic information when applied to the invasive component in muscle-invasive carcinoma. The authors analyzed 93 cases of muscle-invasive urothelial carcinoma of the bladder treated with radical cystectomy for which follow-up information was available. Each case was graded using the Malmström grading system for urothelial carcinoma, applied to the invasive component. Pathologic stage, lymph node status, and histologic invasion pattern were also recorded and correlated with progression-free survival. Thirty-four cases (37%) were pT2, 40 (43%) were pT3, and 19 (20%) were pT4. Of the 77 patients who had a lymph node dissection at the time of cystectomy, 34 (44%) had metastatic carcinoma to one or more lymph nodes. The median survival for pT2, pT3, and pT4 stages was 85, 24, and 29 months, respectively (p = 0.0001). Lymph node-negative and lymph node-positive patients had a median survival of 63 and 23 months, respectively (p = 0.0001). Fifteen patients (16%) were graded as 2b and 78 patients (84%) were graded as 3. Median survival of patients graded as 2b was 34 months compared with 31 months for patients graded as 3 (p value not significant). Three invasive patterns were recognized: nodular (n = 13, 14%), trabecular (n = 39, 42%), and infiltrative (n = 41, 44%). The presence of any infiltrative pattern in the tumor was associated with a median survival of 29 months, compared with 85 months in tumors without an infiltrative pattern (p = 0.06). Pathologic T stage and lymph node status remain the most powerful predictors of progression in muscle-invasive urothelial carcinoma. In this group of patients histologic grade, as defined by the Malmström system and as applied to the invasive component, provided no additional prognostic information. An infiltrative growth pattern may be associated with a more dismal prognosis.

Automatic quantification of microvessel density in urinary bladder carcinoma

Seventy-three TUR-T biopsies from bladder carcinoma were evaluated regarding microvessel density, defined as microvessel number (nMVD) and cross-section endothelial cell area (aMVD). A semi-automatic and a newly developed, automatic image analysis technique were applied in immunostainings, performed according to an optimized staining protocol. In 12 cases a comparison of biopsy material and the corresponding cystectomy specimen were tested, showing a good correlation in 11 of 12 cases (92%). The techniques proved reproducible for both nMVD and aMVD quantifications related to total tumour area. However, the automatic method was dependent on high immunostaining quality. Simultaneous, semi-automatic quantification of microvessels, stroma and epithelial fraction resulted in a decreased reproducibility. Quantification in ten images, selected in a descending order of MVD by subjective visual judgement, showed a poor observer capacity to estimate and rank MVD. Based on our results we propose quantification of MVD related to one tissue compartment. When staining quality is of high standard, automatic quantification is applicable, which facilitates quantification of multiple areas and thus, should minimize selection variability.