Histological image classification using biologically interpretable shape-based features

HSA-net with a novel CAD pipeline boosts both clinical brain tumor MR image classification and segmentation

Brain tumors are among the most prevalent neoplasms in current medical studies. Accurately distinguishing and classifying brain tumor types accurately is crucial for patient treatment and survival in clinical practice. However, existing computer-aided diagnostic pipelines are inadequate for practical medical use due to tumor complexity. In this study, we curated a multi-centre brain tumor dataset that includes various clinical brain tumor data types, including segmentation and classification annotations, surpassing previous efforts. To enhance brain tumor segmentation accuracy, we propose a new segmentation method: HSA-Net. This method utilizes the Shared Weight Dilated Convolution module (SWDC) and Hybrid Dense Dilated Convolution module (HDense) to capture multi-scale information while minimizing parameter count. The Effective Multi-Dimensional Attention (EMA) and Important Feature Attention (IFA) modules effectively aggregate task-related information. We introduce a novel clinical brain tumor computer-aided diagnosis pipeline (CAD) that combines HSA-Net with pipeline modification. This approach not only improves segmentation accuracy but also utilizes the segmentation mask as an additional channel feature to enhance brain tumor classification results. Our experimental evaluation of 3327 real clinical data demonstrates the effectiveness of the proposed method, achieving an average Dice coefficient of 86.85 % for segmentation and a classification accuracy of 95.35 %. We also validated the effectiveness of our proposed method using the publicly available BraTS dataset.

Quantile Index Biomarkers Based on Single-Cell Expression Data

Current histocytometry methods enable single-cell quantification of biomolecules in tumor tissue sections by multiple detection technologies, including multiplex fluorescence-based immunohistochemistry or in situ hybridization. Quantitative pathology platforms can provide distributions of cellular signal intensity (CSI) levels of biomolecules across the entire cell populations of interest within the sampled tumor tissue. However, the heterogeneity of CSI levels is usually ignored, and the simple mean signal intensity value is considered a cancer biomarker. Here we consider the entire distribution of CSI expression levels of a given biomolecule in the cancer cell population as a predictor of clinical outcome. The proposed quantile index (QI) biomarker is defined as the weighted average of CSI distribution quantiles in individual tumors. The weight for each quantile is determined by fitting a functional regression model for a clinical outcome. That is, the weights are optimized so that the resulting QI has the highest power to predict a relevant clinical outcome. The proposed QI biomarkers were derived for proteins expressed in cancer cells of malignant breast tumors and demonstrated improved prognostic value compared with the standard mean signal intensity predictors. The R package Qindex implementing QI biomarkers has been developed. The proposed approach is not limited to immunohistochemistry data and can be based on any cell-level expressions of proteins or nucleic acids.

HETEROGENEITY ANALYSIS VIA INTEGRATING MULTI-SOURCES HIGH-DIMENSIONAL DATA WITH APPLICATIONS TO CANCER STUDIES

This study has been motivated by cancer research, in which heterogeneity analysis plays an important role and can be roughly classified as unsupervised or supervised. In supervised heterogeneity analysis, the finite mixture of regression (FMR) technique is used extensively, under which the covariates affect the response differently in subgroups. High-dimensional molecular and, very recently, histopathological imaging features have been analyzed separately and shown to be effective for heterogeneity analysis. For simpler analysis, they have been shown to contain overlapping, but also independent information. In this article, our goal is to conduct the first and more effective FMR-based cancer heterogeneity analysis by integrating high-dimensional molecular and histopathological imaging features. A penalization approach is developed to regularize estimation, select relevant variables, and, equally importantly, promote the identification of independent information. Consistency properties are rigorously established. An effective computational algorithm is developed. A simulation and an analysis of The Cancer Genome Atlas (TCGA) lung cancer data demonstrate the practical effectiveness of the proposed approach. Overall, this study provides a practical and useful new way of conducting supervised cancer heterogeneity analysis.

On The Potential of Image Moments for Medical Diagnosis

Medical imaging is widely used for diagnosis and postoperative or post-therapy monitoring. The ever-increasing number of images produced has encouraged the introduction of automated methods to assist doctors or pathologists. In recent years, especially after the advent of convolutional neural networks, many researchers have focused on this approach, considering it to be the only method for diagnosis since it can perform a direct classification of images. However, many diagnostic systems still rely on handcrafted features to improve interpretability and limit resource consumption. In this work, we focused our efforts on orthogonal moments, first by providing an overview and taxonomy of their macrocategories and then by analysing their classification performance on very different medical tasks represented by four public benchmark data sets. The results confirmed that convolutional neural networks achieved excellent performance on all tasks. Despite being composed of much fewer features than those extracted by the networks, orthogonal moments proved to be competitive with them, showing comparable and, in some cases, better performance. In addition, Cartesian and harmonic categories provided a very low standard deviation, proving their robustness in medical diagnostic tasks. We strongly believe that the integration of the studied orthogonal moments can lead to more robust and reliable diagnostic systems, considering the performance obtained and the low variation of the results. Finally, since they have been shown to be effective on both magnetic resonance and computed tomography images, they can be easily extended to other imaging techniques.

Transforming Diagnosis and Therapeutics Using Cancer Genomics

In past quarter of the century, much has been understood about the genetic variation and abnormal genes that activate cancer in humans. All the cancers somehow possess alterations in the DNA sequence of cancer cell's genome. In present, we are heading toward the era where it is possible to obtain complete genome of the cancer cells for their better diagnosis, categorization and to explore treatment options.

DeepNoise: Signal and Noise Disentanglement Based on Classifying Fluorescent Microscopy Images via Deep Learning

The high-content image-based assay is commonly leveraged for identifying the phenotypic impact of genetic perturbations in biology field. However, a persistent issue remains unsolved during experiments: the interferential technical noises caused by systematic errors (e.g., temperature, reagent concentration, and well location) are always mixed up with the real biological signals, leading to misinterpretation of any conclusion drawn. Here, we reported a mean teacher-based deep learning model (DeepNoise) that can disentangle biological signals from the experimental noises. Specifically, we aimed to classify the phenotypic impact of 1108 different genetic perturbations screened from 125,510 fluorescent microscopy images, which were totally unrecognizable by the human eye. We validated our model by participating in the Recursion Cellular Image Classification Challenge, and DeepNoise achieved an extremely high classification score (accuracy: 99.596%), ranking the 2nd place among 866 participating groups. This promising result indicates the successful separation of biological and technical factors, which might help decrease the cost of treatment development and expedite the drug discovery process. The source code of DeepNoise is available at https://github.com/Scu-sen/Recursion-Cellular-Image-Classification-Challenge.

PCformer: an MVI recognition method via classification of the MVI boundary according to histopathological images of liver cancer

Liver cancer is one of the most common cancers leading to death in the world. Microvascular invasion (MVI) is a principal reason for the poor long-term survival rate after liver cancer surgery. Early detection and treatment are very important for improving the survival rate. Manual examination of MVI based on histopathological images is very inefficient and time consuming. MVI automatic diagnosis based on deep learning methods can effectively deal with this problem, reduce examination time, and improve detection efficiency. In recent years, deep learning-based methods have been widely used in histopathological image analysis because of their impressive performance. However, it is very challenging to identify MVI directly using deep learning methods, especially under the interference of hepatocellular carcinoma (HCC) because there is no obvious difference in the histopathological level between HCC and MVI. To cope with this problem, we adopt a method of classifying the MVI boundary to avoid interference from HCC. Nonetheless, due to the specificity of the histopathological tissue structure with the MVI boundary, the effect of transfer learning using the existing models is not obvious. Therefore, in this paper, according to the features of the MVI boundary histopathological tissue structure, we propose a new classification model, i.e., the PCformer, which combines the convolutional neural network (CNN) method with a visual transformer and improves the recognition performance of the MVI boundary histopathological image. Experimental results show that our method has better performance than other models based on a CNN or a transformer.

A weakly supervised deep learning-based method for glioma subtype classification using WSI and mpMRIs

Accurate glioma subtype classification is critical for the treatment management of patients with brain tumors. Developing an automatically computer-aided algorithm for glioma subtype classification is challenging due to many factors. One of the difficulties is the label constraint. Specifically, each case is simply labeled the glioma subtype without precise annotations of lesion regions information. In this paper, we propose a novel hybrid fully convolutional neural network (CNN)-based method for glioma subtype classification using both whole slide imaging (WSI) and multiparametric magnetic resonance imagings (mpMRIs). It is comprised of two methods: a WSI-based method and a mpMRIs-based method. For the WSI-based method, we categorize the glioma subtype using a 2D CNN on WSIs. To overcome the label constraint issue, we extract the truly representative patches for the glioma subtype classification in a weakly supervised fashion. For the mpMRIs-based method, we develop a 3D CNN-based method by analyzing the mpMRIs. The mpMRIs-based method consists of brain tumor segmentation and classification. Finally, to enhance the robustness of the predictions, we fuse the WSI-based and mpMRIs-based results guided by a confidence index. The experimental results on the validation dataset in the competition of CPM-RadPath 2020 show the comprehensive judgments from both two modalities can achieve better performance than the ones by solely using WSI or mpMRIs. Furthermore, our result using the proposed method ranks the third place in the CPM-RadPath 2020 in the testing phase. The proposed method demonstrates a competitive performance, which is creditable to the success of weakly supervised approach and the strategy of label agreement from multi-modality data.

Quantification of spatial tumor heterogeneity in immunohistochemistry staining images

MOTIVATION

Quantitative immunofluorescence is often used for immunohistochemistry quantification of proteins that serve as cancer biomarkers. Advanced image analysis systems for pathology allow capturing expression levels in each individual cell or subcellular compartment. However, only the mean signal intensity within the cancer tissue region of interest is usually considered as biomarker completely ignoring the issue of tumor heterogeneity.

RESULTS

We propose using immunohistochemistry image-derived information on the spatial distribution of cellular signal intensity (CSI) of protein expression within the cancer cell population to quantify both mean expression level and tumor heterogeneity of CSI levels. We view CSI levels as marks in a marked point process of cancer cells in the tissue and define spatial indices based on conditional mean and conditional variance of the marked point process. The proposed methodology provides objective metrics of cell-to-cell heterogeneity in protein expressions that allow discriminating between different patterns of heterogeneity. The prognostic utility of new spatial indices is investigated and compared to the standard mean signal intensity biomarkers using the protein expressions in tissue microarrays incorporating tumor tissues from 1000+ breast cancer patients.

AVAILABILITY AND IMPLEMENTATION: THE R CODE FOR COMPUTING THE PROPOSED SPATIAL INDICES IS INCLUDED AS SUPPLEMENTARY MATERIAL

.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Two Ensemble-CNN Approaches for Colorectal Cancer Tissue Type Classification

In recent years, automatic tissue phenotyping has attracted increasing interest in the Digital Pathology (DP) field. For Colorectal Cancer (CRC), tissue phenotyping can diagnose the cancer and differentiate between different cancer grades. The development of Whole Slide Images (WSIs) has provided the required data for creating automatic tissue phenotyping systems. In this paper, we study different hand-crafted feature-based and deep learning methods using two popular multi-classes CRC-tissue-type databases: Kather-CRC-2016 and CRC-TP. For the hand-crafted features, we use two texture descriptors (LPQ and BSIF) and their combination. In addition, two classifiers are used (SVM and NN) to classify the texture features into distinct CRC tissue types. For the deep learning methods, we evaluate four Convolutional Neural Network (CNN) architectures (ResNet-101, ResNeXt-50, Inception-v3, and DenseNet-161). Moreover, we propose two Ensemble CNN approaches: Mean-Ensemble-CNN and NN-Ensemble-CNN. The experimental results show that the proposed approaches outperformed the hand-crafted feature-based methods, CNN architectures and the state-of-the-art methods in both databases.

Feature-driven local cell graph (FLocK): New computational pathology-based descriptors for prognosis of lung cancer and HPV status of oropharyngeal cancers

Local spatial arrangement of nuclei in histopathology images of different cancer subtypes has been shown to have prognostic value. In order to capture localized nuclear architectural information, local cell cluster graph-based measurements have been proposed. However, conventional ways of cell graph construction only utilize nuclear spatial proximity, and do not differentiate between different cell types while constructing the graph. In this paper, we present feature-driven local cell cluster graph (FLocK), a new approach to constructing local cell graphs by simultaneously considering spatial proximity and attributes of the individual nuclei (e.g. shape, size, texture). In addition, we have designed a new set of quantitative graph-derived metrics to be extracted from FLocKs, in turn capturing the interplay between different proximally located clusters of nuclei. We have evaluated the efficacy of FLocK features extracted from H&E stained tissue images in two clinical applications: to classify short-term vs. long-term survival among patients of early stage non-small cell lung cancer (ES-NSCLC), and also to predict human papillomavirus (HPV) status of oropharyngeal squamous cell carcinoma (OP-SCCs). In the classification of long-term vs. short-term survival among patients of ES-NSCLC (training cohort, n = 434), the top 10 discriminative FLocK features related to the variation of FLocK size and intersected FLocK distance were identified, via Minimum Redundancy and Maximum Relevance (MRMR) selection, in 100 runs of 10-fold cross-validation, and in conjunction with a linear discriminant classifier yielded a mean AUC of 0.68 for predicting survival in the training cohort. This is better than other state-of-art histomorphometric and deep learning classifiers (cell cluster graphs (AUC = 0.62), global cell graph (AUC = 0.56), nuclear shape (AUC = 0.54), nuclear orientation (AUC = 0.61), AlexNet (AUC = 0.55), ResNet (AUC = 0.56)). The FLocK-based classifier yielded an AUC of 0.70 in an independent testing cohort (n = 150). The patients identified as "high-risk" had significantly poorer overall survival in the testing cohort, with a hazard ratio (95% confidence interval) of 2.24 (1.24-4.05), p = 0.01144). In the classification of HPV status of OP-SCC, the top three FLocK features pertaining to the portion of intersected FLocKs were used to construct a classifier, which yielded an AUC of 0.80 in the training cohort (n = 50), and an accuracy of 0.78 in an independent testing cohort (n = 35). The combination of FLocK measurements with cell cluster graphs, nuclear orientation, and nuclear shape improved the training AUC to 0.87, 0.91 and 0.85, respectively. Deep learning approaches yielded marginally better performance than the FLocK-based classifier in this application, with AUC = 0.78 for AlexNet, AUC = 0.81 for ResNet, and AUC = 0.76 for FLocK-based classifier in the testing cohort. However, the combination of two hand-crafted features: FLocK and nuclear orientation yielded a better performance (AUC = 0.84). FLocK provides a unique and quantitative way to analyze histology images of solid tumors and interrogate tumor morphology from a different aspect than existing histomorphometrics. The source code can be accessed at https://github.com/hacylu/FLocK.

Multiplex Cellular Communities in Multi-Gigapixel Colorectal Cancer Histology Images for Tissue Phenotyping

In computational pathology, automated tissue phenotyping in cancer histology images is a fundamental tool for profiling tumor microenvironments. Current tissue phenotyping methods use features derived from image patches which may not carry biological significance. In this work, we propose a novel multiplex cellular community-based algorithm for tissue phenotyping integrating cell-level features within a graph-based hierarchical framework. We demonstrate that such integration offers better performance compared to prior deep learning and texture-based methods as well as to cellular community based methods using uniplex networks. To this end, we construct celllevel graphs using texture, alpha diversity and multi-resolution deep features. Using these graphs, we compute cellular connectivity features which are then employed for the construction of a patch-level multiplex network. Over this network, we compute multiplex cellular communities using a novel objective function. The proposed objective function computes a low-dimensional subspace from each cellular network and subsequently seeks a common low-dimensional subspace using the Grassmann manifold. We evaluate our proposed algorithm on three publicly available datasets for tissue phenotyping, demonstrating a significant improvement over existing state-of-the-art methods.

Cellular community detection for tissue phenotyping in colorectal cancer histology images

Classification of various types of tissue in cancer histology images based on the cellular compositions is an important step towards the development of computational pathology tools for systematic digital profiling of the spatial tumor microenvironment. Most existing methods for tissue phenotyping are limited to the classification of tumor and stroma and require large amount of annotated histology images which are often not available. In the current work, we pose the problem of identifying distinct tissue phenotypes as finding communities in cellular graphs or networks. First, we train a deep neural network for cell detection and classification into five distinct cellular components. Considering the detected nuclei as nodes, potential cell-cell connections are assigned using Delaunay triangulation resulting in a cell-level graph. Based on this cell graph, a feature vector capturing potential cell-cell connection of different types of cells is computed. These feature vectors are used to construct a patch-level graph based on chi-square distance. We map patch-level nodes to the geometric space by representing each node as a vector of geodesic distances from other nodes in the network and iteratively drifting the patch nodes in the direction of positive density gradients towards maximum density regions. The proposed algorithm is evaluated on a publicly available dataset and another new large-scale dataset consisting of 280K patches of seven tissue phenotypes. The estimated communities have significant biological meanings as verified by the expert pathologists. A comparison with current state-of-the-art methods reveals significant performance improvement in tissue phenotyping.

Cell Image Classification: A Comparative Overview

Cell image classification methods are currently being used in numerous applications in cell biology and medicine. Applications include understanding the effects of genes and drugs in screening experiments, understanding the role and subcellular localization of different proteins, as well as diagnosis and prognosis of cancer from images acquired using cytological and histological techniques. The article also reviews three main approaches for cell image classification most often used: numerical feature extraction, end-to-end classification with neural networks (NNs), and transport-based morphometry (TBM). In addition, we provide comparisons on four different cell imaging datasets to highlight the relative strength of each method. The results computed using four publicly available datasets show that numerical features tend to carry the best discriminative information for most of the classification tasks. Results also show that NN-based methods produce state-of-the-art results in the dataset that contains a relatively large number of training samples. Data augmentation or the choice of a more recently reported architecture does not necessarily improve the classification performance of NNs in the datasets with limited number of training samples. If understanding and visualization are desired aspects, TBM methods can offer the ability to invert classification functions, and thus can aid in the interpretation of results. These and other comparison outcomes are discussed with the aim of clarifying the advantages and disadvantages of each method. © 2020 International Society for Advancement of Cytometry.

Bioinformatics analysis of whole slide images reveals significant neighborhood preferences of tumor cells in Hodgkin lymphoma

In pathology, tissue images are evaluated using a light microscope, relying on the expertise and experience of pathologists. There is a great need for computational methods to quantify and standardize histological observations. Computational quantification methods become more and more essential to evaluate tissue images. In particular, the distribution of tumor cells and their microenvironment are of special interest. Here, we systematically investigated tumor cell properties and their spatial neighborhood relations by a new application of statistical analysis to whole slide images of Hodgkin lymphoma, a tumor arising in lymph nodes, and inflammation of lymph nodes called lymphadenitis. We considered properties of more than 400, 000 immunohistochemically stained, CD30-positive cells in 35 whole slide images of tissue sections from subtypes of the classical Hodgkin lymphoma, nodular sclerosis and mixed cellularity, as well as from lymphadenitis. We found that cells of specific morphology exhibited significantly favored and unfavored spatial neighborhood relations of cells in dependence of their morphology. This information is important to evaluate differences between Hodgkin lymph nodes infiltrated by tumor cells (Hodgkin lymphoma) and inflamed lymph nodes, concerning the neighborhood relations of cells and the sizes of cells. The quantification of neighborhood relations revealed new insights of relations of CD30-positive cells in different diagnosis cases. The approach is general and can easily be applied to whole slide image analysis of other tumor types.

In pathology, histological diagnosis is still challenging, in particular, for tumor diseases. Pathologists diagnose the disease and its stage of development on the basis of evaluation and interpretation of images of tissue sections. The quantification of experimental data to support decisions of diagnosis and prognosis, applying bioinformatics methods, is an important issue. Here, we introduce a new, general approach to analyze tissue images of tumor and non-tumor patients and to evaluate the distribution of tumor cells in the tissue. Moreover, we consider neighborhood relations between immunostained cells of different cell morphology. We focus on a special type of lymph node tumor, the Hodgkin lymphoma, exploring the two main types of the classical Hodgkin lymphoma, the nodular sclerosis and the mixed cellularity, and the non-tumor case, the lymphadenitis, representing an inflammation of the lymph node. We considered more than 400, 000 cells immunohistochemically stained with CD30 in 35 whole slide images of tissue sections. We found that cells of specific morphology exhibited significant relations to cells of certain morphology as spatial nearest neighbor. We could show different neighborhood patterns of CD30-positive cells between tumor and non-tumor. The approach is general and can easily be applied to other tumor types.

Improved support vector machine classification algorithm based on adaptive feature weight updating in the Hadoop cluster environment

An image classification algorithm based on adaptive feature weight updating is proposed to address the low classification accuracy of the current single-feature classification algorithms and simple multifeature fusion algorithms. The MapReduce parallel programming model on the Hadoop platform is used to perform an adaptive fusion of hue, local binary pattern (LBP) and scale-invariant feature transform (SIFT) features extracted from images to derive optimal combinations of weights. The support vector machine (SVM) classifier is then used to perform parallel training to obtain the optimal SVM classification model, which is then tested. The Pascal VOC 2012, Caltech 256 and SUN databases are adopted to build a massive image library. The speedup, classification accuracy and training time are tested in the experiment, and the results show that a linear growth tendency is present in the speedup of the system in a cluster environment. In consideration of the hardware costs, time, performance and accuracy, the algorithm is superior to mainstream classification algorithms, such as the power mean SVM and convolutional neural network (CNN). As the number and types of images both increase, the classification accuracy rate exceeds 95%. When the number of images reaches 80,000, the training time of the proposed algorithm is only 1/5 that of traditional single-node architecture algorithms. This result reflects the effectiveness of the algorithm, which provides a basis for the effective analysis and processing of image big data.

Accurate label-free 3-part leukocyte recognition with single cell lens-free imaging flow cytometry

Three-part white blood cell differentials which are key to routine blood workups are typically performed in centralized laboratories on conventional hematology analyzers operated by highly trained staff. With the trend of developing miniaturized blood analysis tool for point-of-need in order to accelerate turnaround times and move routine blood testing away from centralized facilities on the rise, our group has developed a highly miniaturized holographic imaging system for generating lens-free images of white blood cells in suspension. Analysis and classification of its output data, constitutes the final crucial step ensuring appropriate accuracy of the system. In this work, we implement reference holographic images of single white blood cells in suspension, in order to establish an accurate ground truth to increase classification accuracy. We also automate the entire workflow for analyzing the output and demonstrate clear improvement in the accuracy of the 3-part classification. High-dimensional optical and morphological features are extracted from reconstructed digital holograms of single cells using the ground-truth images and advanced machine learning algorithms are investigated and implemented to obtain 99% classification accuracy. Representative features of the three white blood cell subtypes are selected and give comparable results, with a focus on rapid cell recognition and decreased computational cost.

Predicting cancer outcomes from histology and genomics using convolutional networks

Predicting the expected outcome of patients diagnosed with cancer is a critical step in treatment. Advances in genomic and imaging technologies provide physicians with vast amounts of data, yet prognostication remains largely subjective, leading to suboptimal clinical management. We developed a computational approach based on deep learning to predict the overall survival of patients diagnosed with brain tumors from microscopic images of tissue biopsies and genomic biomarkers. This method uses adaptive feedback to simultaneously learn the visual patterns and molecular biomarkers associated with patient outcomes. Our approach surpasses the prognostic accuracy of human experts using the current clinical standard for classifying brain tumors and presents an innovative approach for objective, accurate, and integrated prediction of patient outcomes.

Cancer histology reflects underlying molecular processes and disease progression and contains rich phenotypic information that is predictive of patient outcomes. In this study, we show a computational approach for learning patient outcomes from digital pathology images using deep learning to combine the power of adaptive machine learning algorithms with traditional survival models. We illustrate how these survival convolutional neural networks (SCNNs) can integrate information from both histology images and genomic biomarkers into a single unified framework to predict time-to-event outcomes and show prediction accuracy that surpasses the current clinical paradigm for predicting the overall survival of patients diagnosed with glioma. We use statistical sampling techniques to address challenges in learning survival from histology images, including tumor heterogeneity and the need for large training cohorts. We also provide insights into the prediction mechanisms of SCNNs, using heat map visualization to show that SCNNs recognize important structures, like microvascular proliferation, that are related to prognosis and that are used by pathologists in grading. These results highlight the emerging role of deep learning in precision medicine and suggest an expanding utility for computational analysis of histology in the future practice of pathology.

Detection and delineation of squamous neoplasia with hyperspectral imaging in a mouse model of tongue carcinogenesis

Hyperspectral imaging (HSI) holds the potential for the noninvasive detection of cancers. Oral cancers are often diagnosed at a late stage when treatment is less effective and the mortality and morbidity rates are high. Early detection of oral cancer is, therefore, crucial in order to improve the clinical outcomes. To investigate the potential of HSI as a noninvasive diagnostic tool, an animal study was designed to acquire hyperspectral images of in vivo and ex vivo mouse tongues from a chemically induced tongue carcinogenesis model. A variety of machine-learning algorithms, including discriminant analysis, ensemble learning, and support vector machines, were evaluated for tongue neoplasia detection using HSI and were validated by the reconstructed pathological gold-standard maps. The diagnostic performance of HSI, autofluorescence imaging, and fluorescence imaging were compared in this study. Color-coded prediction maps were generated to display the predicted location and distribution of premalignant and malignant lesions. This study suggests that hyperspectral imaging combined with machine-learning techniques can provide a noninvasive tool for the quantitative detection and delineation of squamous neoplasia.

Informatics Approaches to Address New Challenges in the Classification of Lymphoid Malignancies

Purpose

Lymphoid malignancies are remarkably heterogeneous, with variations in outcomes and clinical, biologic, and histologic presentation complicating classification according to the World Health Organization guidelines. Incorrect classification of lymphoid neoplasms can result in suboptimal therapeutic strategies for individual patients and confound the interpretation of clinical trials involving personalized, class-based treatments. This review discusses the potential role of pathology informatics in improving the classification accuracy and objectivity for lymphoid malignancies.

Design

We identified peer-reviewed publications examining pathology informatics approaches for the classification of lymphoid malignancies, reviewed developments in the lymphoma classification systems, and summarized computational methods for pathologic assessment that can impact practice.

Results

Computer-assisted pathology image analysis algorithms in lymphoma most commonly have been applied to follicular lymphoma to address biologic heterogeneity and subjectivity in the process of classification.

Conclusion

Objective methods are available to assist pathologists in lymphoma classification and grading, and have been demonstrated to provide measurable benefits in specific contexts. Future validation and extension of these approaches will require datasets that link high resolution pathology images available for image analysis algorithms with clinical variables and follow up outcomes.

Interactive phenotyping of large-scale histology imaging data with HistomicsML

Whole-slide imaging of histologic sections captures tissue microenvironments and cytologic details in expansive high-resolution images. These images can be mined to extract quantitative features that describe tissues, yielding measurements for hundreds of millions of histologic objects. A central challenge in utilizing this data is enabling investigators to train and evaluate classification rules for identifying objects related to processes like angiogenesis or immune response. In this paper we describe HistomicsML, an interactive machine-learning system for digital pathology imaging datasets. This framework uses active learning to direct user feedback, making classifier training efficient and scalable in datasets containing 10⁸+ histologic objects. We demonstrate how this system can be used to phenotype microvascular structures in gliomas to predict survival, and to explore the molecular pathways associated with these phenotypes. Our approach enables researchers to unlock phenotypic information from digital pathology datasets to investigate prognostic image biomarkers and genotype-phenotype associations.

A structured latent model for ovarian carcinoma subtyping from histopathology slides

Accurate subtyping of ovarian carcinomas is an increasingly critical and often challenging diagnostic process. This work focuses on the development of an automatic classification model for ovarian carcinoma subtyping. Specifically, we present a novel clinically inspired contextual model for histopathology image subtyping of ovarian carcinomas. A whole slide image is modelled using a collection of tissue patches extracted at multiple magnifications. An efficient and effective feature learning strategy is used for feature representation of a tissue patch. The locations of salient, discriminative tissue regions are treated as latent variables allowing the model to explicitly ignore portions of the large tissue section that are unimportant for classification. These latent variables are considered in a structured formulation to model the contextual information represented from the multi-magnification analysis of tissues. A novel, structured latent support vector machine formulation is defined and used to combine information from multiple magnifications while simultaneously operating within the latent variable framework. The structural and contextual nature of our method addresses the challenges of intra-class variation and pathologists' workload, which are prevalent in histopathology image classification. Extensive experiments on a dataset of 133 patients demonstrate the efficacy and accuracy of the proposed method against state-of-the-art approaches for histopathology image classification. We achieve an average multi-class classification accuracy of 90%, outperforming existing works while obtaining substantial agreement with six clinicians tested on the same dataset.

PathoSpotter-K: A computational tool for the automatic identification of glomerular lesions in histological images of kidneys

PathoSpotter is a computational system designed to assist pathologists in teaching about and researching kidney diseases. PathoSpotter-K is the version that was developed to detect nephrological lesions in digital images of kidneys. Here, we present the results obtained using the first version of PathoSpotter-K, which uses classical image processing and pattern recognition methods to detect proliferative glomerular lesions with an accuracy of 88.3 ± 3.6%. Such performance is only achieved by similar systems if they use images of cell in contexts that are much less complex than the glomerular structure. The results indicate that the approach can be applied to the development of systems designed to train pathology students and to assist pathologists in determining large-scale clinicopathological correlations in morphological research.

Predicting Heart Rejection Using Histopathological Whole-Slide Imaging and Deep Neural Network with Dropout

Cardiac allograft rejection is one major limitation for long-term survival for patients with heart transplants. The endomyocardial biopsy is one gold standard to screen heart rejection for patients that have heart transplantation. However, manual identification of heart rejection is expensive and time-consuming. With the development of imaging processing techniques and machine learning tools, automatic prediction of heart rejection using whole-slide images is one promising approach to improve the care of patients with heart transplants. In this paper, we first develop a histopathological whole-slide image processing pipeline to extract features automatically. Then, we construct deep neural networks with and without regularization and dropout to classify the patients into nonrejection and rejection respectively. Our results show that neural networks with regularization and dropout can significantly reduce overfitting and achieve more stable accuracies.

Triaging Diagnostically Relevant Regions from Pathology Whole Slides of Breast Cancer: A Texture Based Approach

PURPOSE

Pathologists often look at whole slide images (WSIs) at low magnification to find potentially important regions and then zoom in to higher magnification to perform more sophisticated analysis of the tissue structures. Many automated methods of WSI analysis attempt to preprocess the down-sampled image in order to select salient regions which are then further analyzed by a more computationally intensive step at full magnification. Although it can greatly reduce processing times, this process may lead to small potentially important regions being overlooked at low magnification. We propose a texture analysis technique to ease the processing of H&E stained WSIs by triaging clinically important regions.

METHOD

Image patches randomly selected from the whole tissue area were divided into smaller tiles and Gaussian-like texture filters were applied to them. Texture filter responses from each tile were combined together and statistical measures were derived from their histograms of responses. Bag of visual words pipeline was then employed to combine extracted features from tiles to form one histogram of words per every image patch. A support vector machine classifier was trained using the calculated histograms of words to be able to distinguish between clinically relevant and irrelevant patches.

RESULT

Experimental analysis on 5151 image patches from 10 patient cases (65 tissue slides) indicated that our proposed texture technique out-performed two previously proposed colour and intensity based methods with an area under the ROC curve of 0.87.

CONCLUSION

Texture features can be employed to triage clinically important areas within large WSIs.

Semantic interpretation of robust imaging features for Fuhrman grading of renal carcinoma

Pattern recognition in tissue biopsy images can assist in clinical diagnosis and identify relevant image characteristics linked with various biological characteristics. Although previous work suggests several informative imaging features for pattern recognition, there exists a semantic gap between characteristics of these features and pathologists' interpretation of histopathological images. To address this challenge, we develop a clinical decision support system for automated Fuhrman grading of renal carcinoma biopsy images. We extract 1316 color, shape, texture and topology features and develop one vs. all models for four Fuhrman grades. Our models are highly accurate with 90.4% accuracy in a four-class prediction. Predictivity analysis suggests good generalization of the model development methodology through robustness to dataset sampling in cross-validation. We provide a semantic interpretation for the imaging features used in these models by linking features to pathologists' grading criteria. Our study identifies novel imaging features that are semantically linked to Fuhrman grading criteria.

Supremacy of modern morphometry in typing renal oncocytoma and malignant look-alikes

In the era of tumour type-specific therapies, the correct typing of renal tumours is of prime importance. As immunotyping and genotyping approaches are laborious and fall short of standardization, we used whole-scale computer-assisted morphometry instead. Three different types of renal tumours with different prognoses and therapies, notoriously prone to mistyping, were analysed . The sample of 335 tumours included clear cell renal cell carcinoma, chromophobe renal cell carcinoma and renal oncocytoma. The sample was analysed using H&E stains of tissue microarrrays in combination with an image-scanning software. Nuclear and cytoplasmic features were registered with the aid of computer-assisted morphometry. Features included shape, texture, colour and colour intensity for different cell compartments, e.g. nuclei and cytoplasm. The software passed several training steps for final validation. Using morphometry, we were able to classify the three renal tumour types correctly, with a 100 % specificity compared to the WHO typing. Nuclear features dominated the typing of chromophobe renal cell carcinoma, whereas cytoplasmic features were the leading classificators for renal oncocytoma. The grading of clear cell renal cell carcinoma attained a specificity of 80 %. In conclusion, modern morphometry may serve as a tool for typing renal epithelial tumours and additionally draws the attention to future nuclear research in chromophobe renal cell carcinoma.

Pathology imaging informatics for quantitative analysis of whole-slide images

Objectives

With the objective of bringing clinical decision support systems to reality, this article reviews histopathological whole-slide imaging informatics methods, associated challenges, and future research opportunities.

Target audience

This review targets pathologists and informaticians who have a limited understanding of the key aspects of whole-slide image (WSI) analysis and/or a limited knowledge of state-of-the-art technologies and analysis methods.

Scope

First, we discuss the importance of imaging informatics in pathology and highlight the challenges posed by histopathological WSI. Next, we provide a thorough review of current methods for: quality control of histopathological images; feature extraction that captures image properties at the pixel, object, and semantic levels; predictive modeling that utilizes image features for diagnostic or prognostic applications; and data and information visualization that explores WSI for de novo discovery. In addition, we highlight future research directions and discuss the impact of large public repositories of histopathological data, such as the Cancer Genome Atlas, on the field of pathology informatics. Following the review, we present a case study to illustrate a clinical decision support system that begins with quality control and ends with predictive modeling for several cancer endpoints. Currently, state-of-the-art software tools only provide limited image processing capabilities instead of complete data analysis for clinical decision-making. We aim to inspire researchers to conduct more research in pathology imaging informatics so that clinical decision support can become a reality.