Computer aided diagnostic tools aim to empower rather than replace pathologists: Lessons learned from computational chess

Network analysis of histopathological image features and genomics data improving prognosis performance in clear cell renal cell carcinoma

INTRODUCTION

Clear cell renal cell carcinoma is the most common type of kidney cancer, but the prediction of prognosis remains a challenge.

METHODS

We collected whole-slide histopathological images, corresponding clinical and genetic information from the The Cancer Imaging Archive and The Cancer Genome Atlas databases and randomly divided patients into training (n = 197) and validation (n = 84) cohorts. After feature extraction by CellProfiler, we used 2 different machine learning techniques (Least Absolute Shrinkage and Selector Operation-regularized Cox and Support Vector Machine-Recursive Feature Elimination) and weighted gene co-expression network analysis to select prognosis-related image features and genes, respectively. These features and genes were integrated into a joint model using random forest and used to create a nomogram that combines other predictive indicators.

RESULTS

A total of 4 overlapped features were identified, represented by the computed histopathological risk score in the random forest model, and showed predictive value for overall survival (test set: 1-year area under the curves (AUC) = 0.726, 3-year AUC = 0.727, and 5-year AUC = 0.764). The histopathological-genetic risk score (HGRS) integrating the genetic information computed performed better than the model that used image features only (test set: 1-year AUC = 0.682, 3-year AUC = 0.734, and 5-year AUC = 0.78). The nomogram (gender, stage, and HGRS) achieved the highest net benefit according to decision curve analysis compared to HGRS or clinical model.

CONCLUSION

This study developed a histopathological-genetic-related nomogram by combining histopathological features and clinical predictors, providing a more comprehensive prognostic assessment for clear cell renal cell carcinoma patients.

Assessment of AURKA expression and prognosis prediction in lung adenocarcinoma using machine learning-based pathomics signature

Objective

This study aimed to develop quantitative feature-based models from histopathological images to assess aurora kinase A (AURKA) expression and predict the prognosis of patients with lung adenocarcinoma (LUAD).

Methods

A dataset of patients with LUAD was derived from the cancer genome atlas (TCGA) with information on clinical characteristics, RNA sequencing and histopathological images. The TCGA-LUAD cohort was randomly divided into training (n = 229) and testing (n = 98) sets. We extracted quantitative image features from histopathological slides of patients with LUAD using computational approaches, constructed a predictive model for AURKA expression in the training set, and estimated their predictive performance in the test set. A Cox proportional hazards model was used to assess whether the pathomic scores (PS) generated by the model independently predicted LUAD survival.

Results

High AURKA expression was an independent risk factor for overall survival (OS) in patients with LUAD (hazard ratio = 1.816, 95 % confidence intervals = 1.257-2.623, P = 0.001). The model based on histopathological image features had significant predictive value for AURKA expression: the area under the curve of the receiver operating characteristic curve in the training set and validation set was 0.809 and 0.739, respectively. Decision curve analysis showed that the model had clinical utility. Patients with high PS and low PS had different survival rates (P = 0.019). Multivariate analysis suggested that PS was an independent prognostic factor for LUAD (hazard ratio = 1.615, 95 % confidence intervals = 1.071-2.438, P = 0.022).

Conclusion

Pathomics models based on machine learning can accurately predict AURKA expression and the PS generated by the model can predict LUAD prognosis.

Heterogeneity of computational pathomic signature predicts drug resistance and intra-tumor heterogeneity of ovarian cancer

BACKGROUND

Chemotherapy resistance is the main cause of ovarian cancer progression and even death. However, there are no clear indicators for predicting the risk of drug resistance in patients. Intra-tumor heterogeneity (ITH) is one of the characteristics of malignant tumors, which is associated with the treatment and prognosis of tumors. Accordingly, our study aims to investigate the correlation between the image features of intra-tumor heterogeneity and drug resistance of ovarian cancer based on artificial intelligence.

METHODS

We obtained hematoxylin and eosin staining frozen histopathological images of ovarian cancer and paracarcinoma tissues from the Cancer Genome Atlas. We extracted quantitative image features of whole-slide images based on the automatic image nuclear segmentation processing technology. After that, we used bioinformatics analysis to find the relationship between image features of intra-tumor heterogeneity and drug resistance.

RESULTS

Our results show that our automatic image processing process based on computer artificial intelligence can extract image features effectively, and the key image features extracted are closely related to ITH. Among them, the Perimeter.sd image feature with the most prominent ITH feature can accurately predict the risk of platinum-based chemotherapy drug resistance in ovarian cancer patients.

CONCLUSION

Automatic image processing and feature extraction based on artificial intelligence have excellent results. Perimeter.sd can be used as a useful image feature indicator for evaluating ITH. ITH is associated with drug resistance of ovarian cancer, so ITH characteristics can be used as an effective indicator to evaluate drug resistance in patients with ovarian cancer.

Boosting microscopic object detection via feature activation map guided poisson blending

Microscopic examination of visible components based on micrographs is the gold standard for testing in biomedical research and clinical diagnosis. The application of object detection technology in bioimages not only improves the efficiency of the analyst but also provides decision support to ensure the objectivity and consistency of diagnosis. However, the lack of large annotated datasets is a significant impediment in rapidly deploying object detection models for microscopic formed elements detection. Standard augmentation methods used in object detection are not appropriate because they are prone to destroy the original micro-morphological information to produce counterintuitive micrographs, which is not conducive to build the trust of analysts in the intelligent system. Here, we propose a feature activation map-guided boosting mechanism dedicated to microscopic object detection to improve data efficiency. Our results show that the boosting mechanism provides solid gains in the object detection model deployed for microscopic formed elements detection. After image augmentation, the mean Average Precision (mAP) of baseline and strong baseline of the Chinese herbal medicine micrograph dataset are increased by 16.3% and 5.8% respectively. Similarly, on the urine sediment dataset, the boosting mechanism resulted in an improvement of 8.0% and 2.6% in mAP of the baseline and strong baseline maps respectively. Moreover, the method shows strong generalizability and can be easily integrated into any main-stream object detection model. The performance enhancement is interpretable, making it more suitable for microscopic biomedical applications.

Computerized tertiary lymphoid structures density on H&E-images is a prognostic biomarker in resectable lung adenocarcinoma

The increased amount of tertiary lymphoid structures (TLSs) is associated with a favorable prognosis in patients with lung adenocarcinoma (LUAD). However, evaluating TLSs manually is an experience-dependent and time-consuming process, which limits its clinical application. In this multi-center study, we developed an automated computational workflow for quantifying the TLS density in the tumor region of routine hematoxylin and eosin (H&E)-stained whole-slide images (WSIs). The association between the computerized TLS density and disease-free survival (DFS) was further explored in 802 patients with resectable LUAD of three cohorts. Additionally, a Cox proportional hazard regression model, incorporating clinicopathological variables and the TLS density, was established to assess its prognostic ability. The computerized TLS density was an independent prognostic biomarker in patients with resectable LUAD. The integration of the TLS density with clinicopathological variables could support individualized clinical decision-making by improving prognostic stratification.

A Weakly Supervised Deep Learning Model and Human-Machine Fusion for Accurate Grading of Renal Cell Carcinoma from Histopathology Slides

(1) Background: The Fuhrman grading (FG) system is widely used in the management of clear cell renal cell carcinoma (ccRCC). However, it is affected by observer variability and irreproducibility in clinical practice. We aimed to use a deep learning multi-class model called SSL-CLAM to assist in diagnosing the FG status of ccRCC patients using digitized whole slide images (WSIs). (2) Methods: We recruited 504 eligible ccRCC patients from The Cancer Genome Atlas (TCGA) cohort and obtained 708 hematoxylin and eosin-stained WSIs for the development and internal validation of the SSL-CLAM model. Additionally, we obtained 445 WSIs from 188 ccRCC eligible patients in the Clinical Proteomic Tumor Analysis Consortium (CPTAC) cohort as an independent external validation set. A human-machine fusion approach was used to validate the added value of the SSL-CLAM model for pathologists. (3) Results: The SSL-CLAM model successfully diagnosed the five FG statuses (Grade-0, 1, 2, 3, and 4) of ccRCC, and achieved AUCs of 0.917 and 0.887 on the internal and external validation sets, respectively, outperforming a junior pathologist. For the normal/tumor classification (Grade-0, Grade-1/2/3/4) task, the SSL-CLAM model yielded AUCs close to 1 on both the internal and external validation sets. The SSL-CLAM model achieved a better performance for the two-tiered FG (Grade-0, Grade-1/2, and Grade-3/4) task, with AUCs of 0.936 and 0.915 on the internal and external validation sets, respectively. The human-machine diagnostic performance was superior to that of the SSL-CLAM model, showing promising prospects. In addition, the high-attention regions of the SSL-CLAM model showed that with an increasing FG status, the cell nuclei in the tumor region become larger, with irregular contours and increased cellular pleomorphism. (4) Conclusions: Our findings support the feasibility of using deep learning and human-machine fusion methods for FG classification on WSIs from ccRCC patients, which may assist pathologists in making diagnostic decisions.

Multi-scale pathology image texture signature is a prognostic factor for resectable lung adenocarcinoma: a multi-center, retrospective study

BACKGROUND

Tumor histomorphology analysis plays a crucial role in predicting the prognosis of resectable lung adenocarcinoma (LUAD). Computer-extracted image texture features have been previously shown to be correlated with outcome. However, a comprehensive, quantitative, and interpretable predictor remains to be developed.

METHODS

In this multi-center study, we included patients with resectable LUAD from four independent cohorts. An automated pipeline was designed for extracting texture features from the tumor region in hematoxylin and eosin (H&E)-stained whole slide images (WSIs) at multiple magnifications. A multi-scale pathology image texture signature (MPIS) was constructed with the discriminative texture features in terms of overall survival (OS) selected by the LASSO method. The prognostic value of MPIS for OS was evaluated through univariable and multivariable analysis in the discovery set (n = 111) and the three external validation sets (V₁, n = 115; V₂, n = 116; and V₃, n = 246). We constructed a Cox proportional hazards model incorporating clinicopathological variables and MPIS to assess whether MPIS could improve prognostic stratification. We also performed histo-genomics analysis to explore the associations between texture features and biological pathways.

RESULTS

A set of eight texture features was selected to construct MPIS. In multivariable analysis, a higher MPIS was associated with significantly worse OS in the discovery set (HR 5.32, 95%CI 1.72-16.44; P = 0.0037) and the three external validation sets (V₁: HR 2.63, 95%CI 1.10-6.29, P = 0.0292; V₂: HR 2.99, 95%CI 1.34-6.66, P = 0.0075; V₃: HR 1.93, 95%CI 1.15-3.23, P = 0.0125). The model that integrated clinicopathological variables and MPIS had better discrimination for OS compared to the clinicopathological variables-based model in the discovery set (C-index, 0.837 vs. 0.798) and the three external validation sets (V₁: 0.704 vs. 0.679; V₂: 0.728 vs. 0.666; V₃: 0.696 vs. 0.669). Furthermore, the identified texture features were associated with biological pathways, such as cytokine activity, structural constituent of cytoskeleton, and extracellular matrix structural constituent.

CONCLUSIONS

MPIS was an independent prognostic biomarker that was robust and interpretable. Integration of MPIS with clinicopathological variables improved prognostic stratification in resectable LUAD and might help enhance the quality of individualized postoperative care.

Integrative Analysis of Histopathological Images and Genomic Data in Colon Adenocarcinoma

Background

Colon adenocarcinoma (COAD) is one of the most common malignant tumors in the world. The histopathological features are crucial for the diagnosis, prognosis, and therapy of COAD.

Methods

We downloaded 719 whole-slide histopathological images from TCIA, and 459 corresponding HTSeq-counts mRNA expression and clinical data were obtained from TCGA. Histopathological image features were extracted by CellProfiler. Prognostic image features were selected by the least absolute shrinkage and selection operator (LASSO) and support vector machine (SVM) algorithms. The co-expression gene module correlated with prognostic image features was identified by weighted gene co-expression network analysis (WGCNA). Random forest was employed to construct an integrative prognostic model and calculate the histopathological-genomic prognosis factor (HGPF).

Results

There were five prognostic image features and one co-expression gene module involved in the model construction. The time-dependent receiver operating curve showed that the prognostic model had a significant prognostic value. Patients were divided into high-risk group and low-risk group based on the HGPF. Kaplan-Meier analysis indicated that the overall survival of the low-risk group was significantly better than the high-risk group.

Conclusions

These results suggested that the histopathological image features had a certain ability to predict the survival of COAD patients. The integrative prognostic model based on the histopathological images and genomic features could further improve the prognosis prediction in COAD, which may assist the clinical decision in the future.

Clinical Decision Support for Ovarian Carcinoma Subtype Classification: A Pilot Observer Study With Pathology Trainees

CONTEXT.—

Clinical decision support (CDS) systems could assist less experienced pathologists with certain diagnostic tasks for which subspecialty training or extensive experience is typically needed. The effect of decision support on pathologist performance for such diagnostic tasks has not been examined.

OBJECTIVE.—

To examine the impact of a CDS tool for the classification of ovarian carcinoma subtypes by pathology trainees in a pilot observer study using digital pathology.

DESIGN.—

Histologic review on 90 whole slide images from 75 ovarian cancer patients was conducted by 6 pathology residents using: (1) unaided review of whole slide images, and (2) aided review, where in addition to whole slide images observers used a CDS tool that provided information about the presence of 8 histologic features important for subtype classification that were identified previously by an expert in gynecologic pathology. The reference standard of ovarian subtype consisted of majority consensus from a panel of 3 gynecologic pathology experts.

RESULTS.—

Aided review improved pairwise concordance with the reference standard for 5 of 6 observers by 3.3% to 17.8% (for 2 observers, increase was statistically significant) and mean interobserver agreement by 9.2% (not statistically significant). Observers benefited the most when the CDS tool prompted them to look for missed histologic features that were definitive for a certain subtype. Observer performance varied widely across cases with unanimous and nonunanimous reference classification, supporting the need for balancing data sets in terms of case difficulty.

CONCLUSIONS.—

Findings showed the potential of CDS systems to close the knowledge gap between pathologists for complex diagnostic tasks.

Development of a Histopathology Informatics Pipeline for Classification and Prediction of Clinical Outcomes in Subtypes of Renal Cell Carcinoma

PURPOSE

Histopathology evaluation is the gold standard for diagnosing clear cell (ccRCC), papillary, and chromophobe renal cell carcinoma (RCC). However, interrater variability has been reported, and the whole-slide histopathology images likely contain underutilized biological signals predictive of genomic profiles.

EXPERIMENTAL DESIGN

To address this knowledge gap, we obtained whole-slide histopathology images and demographic, genomic, and clinical data from The Cancer Genome Atlas, the Clinical Proteomic Tumor Analysis Consortium, and Brigham and Women's Hospital (Boston, MA) to develop computational methods for integrating data analyses. Leveraging these large and diverse datasets, we developed fully automated convolutional neural networks to diagnose renal cancers and connect quantitative pathology patterns with patients' genomic profiles and prognoses.

RESULTS

Our deep convolutional neural networks successfully detected malignancy (AUC in the independent validation cohort: 0.964-0.985), diagnosed RCC histologic subtypes (independent validation AUCs of the best models: 0.953-0.993), and predicted stage I ccRCC patients' survival outcomes (log-rank test P = 0.02). Our machine learning approaches further identified histopathology image features indicative of copy-number alterations (AUC > 0.7 in multiple genes in patients with ccRCC) and tumor mutation burden.

CONCLUSIONS

Our results suggest that convolutional neural networks can extract histologic signals predictive of patients' diagnoses, prognoses, and genomic variations of clinical importance. Our approaches can systematically identify previously unknown relations among diverse data modalities.

Deep Convolutional Neural Network-Based Lymph Node Metastasis Prediction for Colon Cancer Using Histopathological Images

Background

Human evaluation of pathological slides cannot accurately predict lymph node metastasis (LNM), although accurate prediction is essential to determine treatment and follow-up strategies for colon cancer. We aimed to develop accurate histopathological features for LNM in colon cancer.

Methods

We developed a deep convolutional neural network model to distinguish the cancer tissue component of colon cancer using data from the tissue bank of the National Center for Tumor Diseases and the pathology archive at the University Medical Center Mannheim, Germany. This model was applied to whole-slide pathological images of colon cancer patients from The Cancer Genome Atlas (TCGA). The predictive value of the peri-tumoral stroma (PTS) score for LNM was assessed.

Results

A total of 164 patients with stages I, II, and III colon cancer from TCGA were analyzed. The mean PTS score was 0.380 (± SD = 0.285), and significantly higher PTS scores were observed in patients in the LNM-positive group than those in the LNM-negative group (P < 0.001). In the univariate analyses, the PTS scores for the LNM-positive group were significantly higher than those for the LNM-negative group (P < 0.001). Further, the PTS scores in lymphatic invasion and any one of perineural, lymphatic, or venous invasion were significantly increased in the LNM-positive group (P < 0.001 and P < 0.001).

Conclusion

We established the PTS score, a simplified reproducible parameter, for predicting LNM in colon cancer using computer-based analysis that could be used to guide treatment decisions. These findings warrant further confirmation through large-scale prospective clinical trials.

Standardized Method for Defining a 1-mm2 Region of Interest for Calculation of Mitotic Rate on Melanoma Whole Slide Images

CONTEXT.—

Mitotic rate counting is essential in pathologic evaluations in melanoma. The American Joint Committee on Cancer recommends reporting the number of mitotic figures (MFs) in a 1-mm2 area encompassing the "hot spot." There is currently no standard procedure for delineating a 1-mm2 region of interest for MF counting on a digital whole slide image (WSI) of melanoma.

OBJECTIVE.—

To establish a standardized method to enclose a 1-mm2 region of interest for MF counting in melanoma based on WSIs and assess the method's effectiveness.

DESIGN.—

Whole slide images were visualized using the ImageScope viewer (Aperio). Different monitors and viewing magnifications were explored and the annotation tools provided by ImageScope were evaluated. For validation, we compared mitotic rates obtained from WSIs with our method and those from glass slides with traditional microscopy with 30 melanoma cases.

RESULTS.—

Of the monitors we examined, a 32-inch monitor with 3840 × 2160 resolution was optimal for counting MFs within a 1-mm2 region of interest in melanoma. When WSIs were viewed in the ImageScope viewer, ×10 to ×20 magnification during screening could efficiently locate a hot spot and ×20 to ×40 magnification during counting could accurately identify MFs. Fixed-shape annotations with 500 × 500-μm squares or circles can precisely and efficiently enclose a 1-mm2 region of interest. Our method on WSIs was able to produce a higher mitotic rate than with glass slides.

CONCLUSIONS.—

Whole slide images may be used to efficiently count MFs. We recommend fixed-shape annotation with 500 × 500-μm squares or circles for routine practice in counting MFs for melanoma.

Derivation of a nuclear heterogeneity image index to grade DCIS

Abnormalities in cell nuclear morphology are a hallmark of cancer. Histological assessment of cell nuclear morphology is frequently used by pathologists to grade ductal carcinoma in situ (DCIS). Objective methods that allow standardization and reproducibility of cell nuclear morphology assessment have potential to improve the criteria needed to predict DCIS progression and recurrence. Aggressive cancers are highly heterogeneous. We asked whether cell nuclear morphology heterogeneity could be incorporated into a metric to classify DCIS. We developed a nuclear heterogeneity image index to objectively, and quantitatively grade DCIS. A whole-tissue cell nuclear morphological analysis, that classified tumors by the worst ten percent in a duct-by-duct manner, identified nuclear size ranges associated with each DCIS grade. Digital image analysis further revealed increasing heterogeneity within ducts or between ducts in tissues of worsening DCIS grade. The findings illustrate how digital image analysis comprises a supplemental tool for pathologists to objectively classify DCIS and in the future, may provide a method to predict patient outcome through analysis of nuclear heterogeneity.

Interactive thyroid whole slide image diagnostic system using deep representation

BACKGROUND AND OBJECTIVES

The vast size of the histopathology whole slide image poses formidable challenges to its automatic diagnosis. With the goal of computer-aided diagnosis and the insights that suspicious regions are generally easy to identify in thyroid whole slide images (WSIs), we develop an interactive whole slide diagnostic system for thyroid frozen sections based on the suspicious regions preselected by pathologists.

METHODS

We propose to generate feature representations for the suspicious regions via extracting and fusing patch features using deep neural networks. We then evaluate region classification and retrieval on four classifiers and three supervised hashing methods based on the feature representations. The code is released at https://github.com/PingjunChen/ThyroidInteractive.

RESULTS

We evaluate the proposed system on 345 thyroid frozen sections and achieve 96.1% cross-validated classification accuracy, and retrieval mean average precision (MAP) of 0.972.

CONCLUSIONS

With the participation of pathologists, the system possesses the following four notable advantages compared to directly handling whole slide images: 1) Reduced interference of irrelevant regions; 2) Alleviated computation and memory cost. 3) Fine-grained and precise suspicious region retrieval. 4) Cooperative relationship between pathologists and the diagnostic system. Additionally, experimental results demonstrate the potential of the proposed system on the practical thyroid frozen section diagnosis.

Clinical use of a machine learning histopathological image signature in diagnosis and survival prediction of clear cell renal cell carcinoma

Due to the complicated histopathological characteristics of renal neoplasms, traditional distinguishing of clear cell renal cell carcinoma (ccRCC) by naked eyes of experienced pathologist remains labor intensive and time consuming. Here, we extracted quantitative features of hematoxylin-eosin-stained images using CellProfiler and performed machine learning method to develop and verify a novel computational recognition of digital pathology for diagnosis and prognosis of ccRCC patients in the training, test and external validation cohort. The diagnostic model based on digital pathology could accurately distinguish ccRCC from normal renal tissues, with area under the curve (AUC) of 96.0%, 94.5% and 87.6% in the training, test and external validation cohorts, respectively. It could also accurately distinguish ccRCC from other pathological types of renal cancer, with AUC values of 97.0% and 81.4% in the Cancer Genome Atlas (TCGA) cohort and General cohort. We next developed and verified a computational recognition prognosis model with risk score. There was a significant difference in disease-free survival comparing patients with high vs low risk score in training cohort (hazard ratio = 2.72, P < .0001) and validation cohort (hazard ratio = 9.50, P = .0091). The integrated nomogram based on our computational recognition risk score and clinicopathologic factors demonstrated excellent survival prediction for ccRCC patients, with increased accuracy by 6.6% in patients from Shanghai General Hospital and by 2.5% in patients from TCGA cohort when compared to current tumor stages/grade systems. These results indicate the potential clinical use of our machine learning histopathological image signature in diagnosis and survival prediction of ccRCC.

Computational Cytology: Lessons Learned from Pap Test Computer-Assisted Screening

BACKGROUND

In the face of rapid technological advances in computational cytology including artificial intelligence (AI), optimization of its application to clinical practice would benefit from reflection on the lessons learned from the decades-long journey in the development of computer-assisted Pap test screening.

SUMMARY

The initial driving force for automated screening in cytology was the overwhelming number of Pap tests requiring manual screening, leading to workflow backlogs and incorrect diagnoses. Several companies invested resources to address these concerns utilizing different specimen processing techniques and imaging systems. However, not all companies were commercially prosperous. Successful implementation of this new technology required viable use cases, improved clinical outcomes, and an acceptable means of integration into the daily workflow of cytopathology laboratories. Several factors including supply and demand, Food and Drug Administration (FDA) oversight, reimbursement, overcoming learning curves and workflow changes associated with the adoption of new technology, and cytologist apprehension, played a significant role in either promoting or preventing the widespread adoption of automated screening technologies. Key Messages: Any change in health care, particularly those involving new technology that impacts clinical workflow, is bound to have its successes and failures. However, perseverance through learning curves, optimizing workflow processes, improvements in diagnostic accuracy, and regulatory and financial approval can facilitate widespread adoption of these technologies. Given their history with successfully implementing automated Pap test screening, cytologists are uniquely positioned to not only help with the development of AI technology for other areas of pathology, but also to guide how they are utilized, regulated, and managed.

Histopathological distinction of non-invasive and invasive bladder cancers using machine learning approaches

Background

One of the most challenging tasks for bladder cancer diagnosis is to histologically differentiate two early stages, non-invasive Ta and superficially invasive T1, the latter of which is associated with a significantly higher risk of disease progression. Indeed, in a considerable number of cases, Ta and T1 tumors look very similar under microscope, making the distinction very difficult even for experienced pathologists. Thus, there is an urgent need for a favoring system based on machine learning (ML) to distinguish between the two stages of bladder cancer.

Methods

A total of 1177 images of bladder tumor tissues stained by hematoxylin and eosin were collected by pathologists at University of Rochester Medical Center, which included 460 non-invasive (stage Ta) and 717 invasive (stage T1) tumors. Automatic pipelines were developed to extract features for three invasive patterns characteristic to the T1 stage bladder cancer (i.e., desmoplastic reaction, retraction artifact, and abundant pinker cytoplasm), using imaging processing software ImageJ and CellProfiler. Features extracted from the images were analyzed by a suite of machine learning approaches.

Results

We extracted nearly 700 features from the Ta and T1 tumor images. Unsupervised clustering analysis failed to distinguish hematoxylin and eosin images of Ta vs. T1 tumors. With a reduced set of features, we successfully distinguished 1177 Ta or T1 images with an accuracy of 91–96% by six supervised learning methods. By contrast, convolutional neural network (CNN) models that automatically extract features from images produced an accuracy of 84%, indicating that feature extraction driven by domain knowledge outperforms CNN-based automatic feature extraction. Further analysis revealed that desmoplastic reaction was more important than the other two patterns, and the number and size of nuclei of tumor cells were the most predictive features.

Conclusions

We provide a ML-empowered, feature-centered, and interpretable diagnostic system to facilitate the accurate staging of Ta and T1 diseases, which has a potential to apply to other types of cancer.

Report on computational assessment of Tumor Infiltrating Lymphocytes from the International Immuno-Oncology Biomarker Working Group

Assessment of tumor-infiltrating lymphocytes (TILs) is increasingly recognized as an integral part of the prognostic workflow in triple-negative (TNBC) and HER2-positive breast cancer, as well as many other solid tumors. This recognition has come about thanks to standardized visual reporting guidelines, which helped to reduce inter-reader variability. Now, there are ripe opportunities to employ computational methods that extract spatio-morphologic predictive features, enabling computer-aided diagnostics. We detail the benefits of computational TILs assessment, the readiness of TILs scoring for computational assessment, and outline considerations for overcoming key barriers to clinical translation in this arena. Specifically, we discuss: 1. ensuring computational workflows closely capture visual guidelines and standards; 2. challenges and thoughts standards for assessment of algorithms including training, preanalytical, analytical, and clinical validation; 3. perspectives on how to realize the potential of machine learning models and to overcome the perceptual and practical limits of visual scoring.

Classifying non-small cell lung cancer types and transcriptomic subtypes using convolutional neural networks

OBJECTIVE

Non-small cell lung cancer is a leading cause of cancer death worldwide, and histopathological evaluation plays the primary role in its diagnosis. However, the morphological patterns associated with the molecular subtypes have not been systematically studied. To bridge this gap, we developed a quantitative histopathology analytic framework to identify the types and gene expression subtypes of non-small cell lung cancer objectively.

MATERIALS AND METHODS

We processed whole-slide histopathology images of lung adenocarcinoma (n = 427) and lung squamous cell carcinoma patients (n = 457) in the Cancer Genome Atlas. We built convolutional neural networks to classify histopathology images, evaluated their performance by the areas under the receiver-operating characteristic curves (AUCs), and validated the results in an independent cohort (n = 125).

RESULTS

To establish neural networks for quantitative image analyses, we first built convolutional neural network models to identify tumor regions from adjacent dense benign tissues (AUCs > 0.935) and recapitulated expert pathologists' diagnosis (AUCs > 0.877), with the results validated in an independent cohort (AUCs = 0.726-0.864). We further demonstrated that quantitative histopathology morphology features identified the major transcriptomic subtypes of both adenocarcinoma and squamous cell carcinoma (P < .01).

DISCUSSION

Our study is the first to classify the transcriptomic subtypes of non-small cell lung cancer using fully automated machine learning methods. Our approach does not rely on prior pathology knowledge and can discover novel clinically relevant histopathology patterns objectively. The developed procedure is generalizable to other tumor types or diseases.

An End-to-end System for Automatic Characterization of Iba1 Immunopositive Microglia in Whole Slide Imaging

Traumatic brain injury (TBI) is one of the leading causes of death and disability worldwide. Detailed studies of the microglial response after TBI require high throughput quantification of changes in microglial count and morphology in histological sections throughout the brain. In this paper, we present a fully automated end-to-end system that is capable of assessing microglial activation in white matter regions on whole slide images of Iba1 stained sections. Our approach involves the division of the full brain slides into smaller image patches that are subsequently automatically classified into white and grey matter sections. On the patches classified as white matter, we jointly apply functional minimization methods and deep learning classification to identify Iba1-immunopositive microglia. Detected cells are then automatically traced to preserve their complex branching structure after which fractal analysis is applied to determine the activation states of the cells. The resulting system detects white matter regions with 84% accuracy, detects microglia with a performance level of 0.70 (F1 score, the harmonic mean of precision and sensitivity) and performs binary microglia morphology classification with a 70% accuracy. This automated pipeline performs these analyses at a 20-fold increase in speed when compared to a human pathologist. Moreover, we have demonstrated robustness to variations in stain intensity common for Iba1 immunostaining. A preliminary analysis was conducted that indicated that this pipeline can identify differences in microglia response due to TBI. An automated solution to microglia cell analysis can greatly increase standardized analysis of brain slides, allowing pathologists and neuroscientists to focus on characterizing the associated underlying diseases and injuries.

Classification and Prognosis Prediction from Histopathological Images of Hepatocellular Carcinoma by a Fully Automated Pipeline Based on Machine Learning

OBJECTIVE

The aim of this study was to develop quantitative feature-based models from histopathological images to distinguish hepatocellular carcinoma (HCC) from adjacent normal tissue and predict the prognosis of HCC patients after surgical resection.

METHODS

A fully automated pipeline was constructed using computational approaches to analyze the quantitative features of histopathological slides of HCC patients, in which the features were extracted from the hematoxylin and eosin (H&E)-stained whole-slide images of HCC patients from The Cancer Genome Atlas and tissue microarray images from West China Hospital. The extracted features were used to train the statistical models that classify tissue slides and predict patients' survival outcomes by machine-learning methods.

RESULTS

A total of 1733 quantitative image features were extracted from each histopathological slide. The diagnostic classifier based on 31 features was able to successfully distinguish HCC from adjacent normal tissues in both the test [area under the receiver operating characteristic curve (AUC) 0.988] and external validation sets (AUC 0.886). The random-forest prognostic model using 46 features was able to significantly stratify patients in each set into longer- or shorter-term survival groups according to their assigned risk scores. Moreover, the prognostic model we constructed showed comparable predicting accuracy as TNM staging systems in predicting patients' survival at different time points after surgery.

CONCLUSIONS

Our findings suggest that machine-learning models derived from image features can assist clinicians in HCC diagnosis and its prognosis prediction after hepatectomy.

Automated Osteosclerosis Grading of Clinical Biopsies Using Infrared Spectroscopic Imaging

Osteosclerosis and myefibrosis are complications of myeloproliferative neoplasms. These disorders result in excess growth of trabecular bone and collagen fibers that replace hematopoietic cells, resulting in abnormal bone marrow function. Treatments using imatinib and JAK2 pathway inhibitors can be effective on osteosclerosis and fibrosis; therefore, accurate grading is critical for tracking treatment effectiveness. Current grading standards use a four-class system based on analysis of biopsies stained with three histological stains: hematoxylin and eosin (H&E), Masson's trichrome, and reticulin. However, conventional grading can be subjective and imprecise, impacting the effectiveness of treatment. In this Article, we demonstrate that mid-infrared spectroscopic imaging may serve as a quantitative diagnostic tool for quantitatively tracking disease progression and response to treatment. The proposed approach is label-free and provides automated quantitative analysis of osteosclerosis and collagen fibrosis.

Complete Digital Pathology for Routine Histopathology Diagnosis in a Multicenter Hospital Network

CONTEXT.—

Complete digital pathology and whole slide imaging for routine histopathology diagnosis is currently in use in few laboratories worldwide. Granada University Hospitals, Spain, which comprises 4 hospitals, adopted full digital pathology for primary histopathology diagnosis in 2016.

OBJECTIVE.—

To describe the methodology adopted and the resulting experience at Granada University Hospitals in transitioning to full digital diagnosis.

DESIGN.—

All histopathology glass slides generated for routine diagnosis were digitized at ×40 using the Philips IntelliSite Pathology Solution, which includes an ultrafast scanner and an image management system. All hematoxylin-eosin-stained preparations and immunohistochemistry and histochemistry slides were digitized. The existing sample-tracking software and image management system were integrated to allow data interchange through the Health Level 7 protocol.

RESULTS.—

Circa 160 000 specimens have been signed out using digital pathology for primary diagnosis. This comprises more than 800 000 digitized glass slides. The scanning error rate during the implementation phase was below 1.5%, and subsequent workflow optimization rendered this rate negligible. Since implementation, Granada University Hospitals pathologists have signed out 21% more cases per year on average.

CONCLUSIONS.—

Digital pathology is an adequate medium for primary histopathology diagnosis. Successful digitization relies on existing sample tracking and integration of the information technology infrastructure. Rapid and reliable scanning at ×40 equivalent was key to the transition to a fully digital workflow. Digital pathology resulted in efficiency gains in the preanalytical and analytical phases, and created the basis for computational pathology: the use of computer-assisted tools to aid diagnosis.

Computer-Aided Laser Dissection: A Microdissection Workflow Leveraging Image Analysis Tools

Introduction

The development and application of new molecular diagnostic assays based on next-generation sequencing and proteomics require improved methodologies for procurement of target cells from histological sections. Laser microdissection can successfully isolate distinct cells from tissue specimens based on visual selection for many research and clinical applications. However, this can be a daunting task when a large number of cells are required for molecular analysis or when a sizeable number of specimens need to be evaluated.

Materials and Methods

To improve the efficiency of the cellular identification process, we describe a microdissection workflow that leverages recently developed and open source image analysis algorithms referred to as computer-aided laser dissection (CALD). CALD permits a computer algorithm to identify the cells of interest and drive the dissection process.

Results

We describe several "use cases" that demonstrate the integration of image analytic tools probabilistic pairwise Markov model, ImageJ, spatially invariant vector quantization (SIVQ), and eSeg onto the ThermoFisher Scientific ArcturusXT and Leica LMD7000 microdissection platforms.

Conclusions

The CALD methodology demonstrates the integration of image analysis tools with the microdissection workflow and shows the potential impact to clinical and life science applications.

Convolutional Deep Belief Network with Feature Encoding for Classification of Neuroblastoma Histological Images

Background:

Neuroblastoma is the most common extracranial solid tumor in children younger than 5 years old. Optimal management of neuroblastic tumors depends on many factors including histopathological classification. The gold standard for classification of neuroblastoma histological images is visual microscopic assessment. In this study, we propose and evaluate a deep learning approach to classify high-resolution digital images of neuroblastoma histology into five different classes determined by the Shimada classification.

Subjects and Methods:

We apply a combination of convolutional deep belief network (CDBN) with feature encoding algorithm that automatically classifies digital images of neuroblastoma histology into five different classes. We design a three-layer CDBN to extract high-level features from neuroblastoma histological images and combine with a feature encoding model to extract features that are highly discriminative in the classification task. The extracted features are classified into five different classes using a support vector machine classifier.

Data:

We constructed a dataset of 1043 neuroblastoma histological images derived from Aperio scanner from 125 patients representing different classes of neuroblastoma tumors.

Results:

The weighted average F-measure of 86.01% was obtained from the selected high-level features, outperforming state-of-the-art methods.

Conclusion:

The proposed computer-aided classification system, which uses the combination of deep architecture and feature encoding to learn high-level features, is highly effective in the classification of neuroblastoma histological images.

Integrative Analysis of Histopathological Images and Genomic Data Predicts Clear Cell Renal Cell Carcinoma Prognosis

In cancer, both histopathologic images and genomic signatures are used for diagnosis, prognosis, and subtyping. However, combining histopathologic images with genomic data for predicting prognosis, as well as the relationships between them, has rarely been explored. In this study, we present an integrative genomics framework for constructing a prognostic model for clear cell renal cell carcinoma. We used patient data from The Cancer Genome Atlas (n = 410), extracting hundreds of cellular morphologic features from digitized whole-slide images and eigengenes from functional genomics data to predict patient outcome. The risk index generated by our model correlated strongly with survival, outperforming predictions based on considering morphologic features or eigengenes separately. The predicted risk index also effectively stratified patients in early-stage (stage I and stage II) tumors, whereas no significant survival difference was observed using staging alone. The prognostic value of our model was independent of other known clinical and molecular prognostic factors for patients with clear cell renal cell carcinoma. Overall, this workflow and the shared software code provide building blocks for applying similar approaches in other cancers. Cancer Res; 77(21); e91-100. ©2017 AACR.

A structured latent model for ovarian carcinoma subtyping from histopathology slides

Accurate subtyping of ovarian carcinomas is an increasingly critical and often challenging diagnostic process. This work focuses on the development of an automatic classification model for ovarian carcinoma subtyping. Specifically, we present a novel clinically inspired contextual model for histopathology image subtyping of ovarian carcinomas. A whole slide image is modelled using a collection of tissue patches extracted at multiple magnifications. An efficient and effective feature learning strategy is used for feature representation of a tissue patch. The locations of salient, discriminative tissue regions are treated as latent variables allowing the model to explicitly ignore portions of the large tissue section that are unimportant for classification. These latent variables are considered in a structured formulation to model the contextual information represented from the multi-magnification analysis of tissues. A novel, structured latent support vector machine formulation is defined and used to combine information from multiple magnifications while simultaneously operating within the latent variable framework. The structural and contextual nature of our method addresses the challenges of intra-class variation and pathologists' workload, which are prevalent in histopathology image classification. Extensive experiments on a dataset of 133 patients demonstrate the efficacy and accuracy of the proposed method against state-of-the-art approaches for histopathology image classification. We achieve an average multi-class classification accuracy of 90%, outperforming existing works while obtaining substantial agreement with six clinicians tested on the same dataset.

Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features

Lung cancer is the most prevalent cancer worldwide, and histopathological assessment is indispensable for its diagnosis. However, human evaluation of pathology slides cannot accurately predict patients' prognoses. In this study, we obtain 2,186 haematoxylin and eosin stained histopathology whole-slide images of lung adenocarcinoma and squamous cell carcinoma patients from The Cancer Genome Atlas (TCGA), and 294 additional images from Stanford Tissue Microarray (TMA) Database. We extract 9,879 quantitative image features and use regularized machine-learning methods to select the top features and to distinguish shorter-term survivors from longer-term survivors with stage I adenocarcinoma (P<0.003) or squamous cell carcinoma (P=0.023) in the TCGA data set. We validate the survival prediction framework with the TMA cohort (P<0.036 for both tumour types). Our results suggest that automatically derived image features can predict the prognosis of lung cancer patients and thereby contribute to precision oncology. Our methods are extensible to histopathology images of other organs.

Diagnosis of lung cancer through manual histopathology evaluation is insufficient to predict patient survival. Here, the authors use computerized image processing to identify diagnostically relevant image features and use these features to distinguish lung cancer patients with different prognoses.

Improving the creation and reporting of structured findings during digital pathology review

Background:

Today, pathology reporting consists of many separate tasks, carried out by multiple people. Common tasks include dictation during case review, transcription, verification of the transcription, report distribution, and report the key findings to follow-up registries. Introduction of digital workstations makes it possible to remove some of these tasks and simplify others. This study describes the work presented at the Nordic Symposium on Digital Pathology 2015, in Linköping, Sweden.

Methods:

We explored the possibility to have a digital tool that simplifies image review by assisting note-taking, and with minimal extra effort, populates a structured report. Thus, our prototype sees reporting as an activity interleaved with image review rather than a separate final step. We created an interface to collect, sort, and display findings for the most common reporting needs, such as tumor size, grading, and scoring.

Results:

The interface was designed to reduce the need to retain partial findings in the head or on paper, while at the same time be structured enough to support automatic extraction of key findings for follow-up registry reporting. The final prototype was evaluated with two pathologists, diagnosing complicated partial mastectomy cases. The pathologists experienced that the prototype aided them during the review and that it created a better overall workflow.

Conclusions:

These results show that it is feasible to simplify the reporting tasks in a way that is not distracting, while at the same time being able to automatically extract the key findings. This simplification is possible due to the realization that the structured format needed for automatic extraction of data can be used to offload the pathologists’ working memory during the diagnostic review.

A robust nonlinear tissue-component discrimination method for computational pathology

Advances in digital pathology, specifically imaging instrumentation and data management, have allowed for the development of computational pathology tools with the potential for better, faster, and cheaper diagnosis, prognosis, and prediction of disease. Images of tissue sections frequently vary in color appearance across research laboratories and medical facilities because of differences in tissue fixation, staining protocols, and imaging instrumentation, leading to difficulty in the development of robust computational tools. To address this challenge, we propose a novel nonlinear tissue-component discrimination (NLTD) method to register automatically the color space of histopathology images and visualize individual tissue components, independent of color differences between images. Our results show that the NLTD method could effectively discriminate different tissue components from different types of tissues prepared at different institutions. Further, we demonstrate that NLTD can improve the accuracy of nuclear detection and segmentation algorithms, compared with using conventional color deconvolution methods, and can quantitatively analyze immunohistochemistry images. Together, the NLTD method is objective, robust, and effective, and can be easily implemented in the emerging field of computational pathology.

Wide-field computational imaging of pathology slides using lens-free on-chip microscopy

Optical examination of microscale features in pathology slides is one of the gold standards to diagnose disease. However, the use of conventional light microscopes is partially limited owing to their relatively high cost, bulkiness of lens-based optics, small field of view (FOV), and requirements for lateral scanning and three-dimensional (3D) focus adjustment. We illustrate the performance of a computational lens-free, holographic on-chip microscope that uses the transport-of-intensity equation, multi-height iterative phase retrieval, and rotational field transformations to perform wide-FOV imaging of pathology samples with comparable image quality to a traditional transmission lens-based microscope. The holographically reconstructed image can be digitally focused at any depth within the object FOV (after image capture) without the need for mechanical focus adjustment and is also digitally corrected for artifacts arising from uncontrolled tilting and height variations between the sample and sensor planes. Using this lens-free on-chip microscope, we successfully imaged invasive carcinoma cells within human breast sections, Papanicolaou smears revealing a high-grade squamous intraepithelial lesion, and sickle cell anemia blood smears over a FOV of 20.5 mm(2). The resulting wide-field lens-free images had sufficient image resolution and contrast for clinical evaluation, as demonstrated by a pathologist's blinded diagnosis of breast cancer tissue samples, achieving an overall accuracy of ~99%. By providing high-resolution images of large-area pathology samples with 3D digital focus adjustment, lens-free on-chip microscopy can be useful in resource-limited and point-of-care settings.

Structure- and ligand-based virtual screening identifies new scaffolds for inhibitors of the oncoprotein MDM2

A major challenge in the field of ligand discovery is to identify chemically useful fragments that can be developed into inhibitors of specific protein-protein interactions. Low molecular weight fragments (with molecular weight less than 250 Da) are likely to bind weakly to a protein’s surface. Here we use a new virtual screening procedure which uses a combination of similarity searching and docking to identify chemically tractable scaffolds that bind to the p53-interaction site of MDM2. The binding has been verified using capillary electrophoresis which has proven to be an excellent screening method for such small, weakly binding ligands.

Validating whole slide digital morphometric analysis as a microscopy tool

Whole slide imaging (WSI) can be used to quantify multiple responses within tissue sections during histological analysis. Feature Analysis on Consecutive Tissue Sections (FACTS®) allows the investigator to perform digital morphometric analysis (DMA) within specified regions of interest (ROI) across multiple serial sections at faster rates when compared with manual morphometry methods. Using FACTS® in conjunction with WSI is a powerful analysis tool, which allows DMA to target specific ROI across multiple tissue sections stained for different biomarkers. DMA may serve as an appropriate alternative to classic, manual, histologic morphometric measures, which have historically relied on the selection of high-powered fields of views and manual scoring (e.g., a gold standard). In the current study, existing preserved samples were used to determine if DMA would provide similar results to manual counting methods. Rodent hearts (n=14, left ventricles) were stained with Masson's trichrome, and reacted for cluster of differentiation 68 (CD-68). This study found no statistical significant difference between a classic, manual method and the use of digital algorithms to perform the similar counts (p=0.38). DMA offers researchers the ability to accurately evaluate morphological characteristics in a reproducible fashion without investigator bias and with higher throughput.

Selective invocation of shape priors for deformable segmentation and morphologic classification of prostate cancer tissue microarrays

Shape based active contours have emerged as a natural solution to overlap resolution. However, most of these shape-based methods are computationally expensive. There are instances in an image where no overlapping objects are present and applying these schemes results in significant computational overhead without any accompanying, additional benefit. In this paper we present a novel adaptive active contour scheme (AdACM) that combines boundary and region based energy terms with a shape prior in a multi level set formulation. To reduce the computational overhead, the shape prior term in the variational formulation is only invoked for those instances in the image where overlaps between objects are identified; these overlaps being identified via a contour concavity detection scheme. By not having to invoke all three terms (shape, boundary, region) for segmenting every object in the scene, the computational expense of the integrated active contour model is dramatically reduced, a particularly relevant consideration when multiple objects have to be segmented on very large histopathological images. The AdACM was employed for the task of segmenting nuclei on 80 prostate cancer tissue microarray images from 40 patient studies. Nuclear shape based, architectural and textural features extracted from these segmentations were extracted and found to able to discriminate different Gleason grade patterns with a classification accuracy of 86% via a quadratic discriminant analysis (QDA) classifier. On average the AdACM model provided 60% savings in computational times compared to a non-optimized hybrid active contour model involving a shape prior.

The history of pathology informatics: A global perspective

Pathology informatics has evolved to varying levels around the world. The history of pathology informatics in different countries is a tale with many dimensions. At first glance, it is the familiar story of individuals solving problems that arise in their clinical practice to enhance efficiency, better manage (e.g., digitize) laboratory information, as well as exploit emerging information technologies. Under the surface, however, lie powerful resource, regulatory, and societal forces that helped shape our discipline into what it is today. In this monograph, for the first time in the history of our discipline, we collectively perform a global review of the field of pathology informatics. In doing so, we illustrate how general far-reaching trends such as the advent of computers, the Internet and digital imaging have affected pathology informatics in the world at large. Major drivers in the field included the need for pathologists to comply with national standards for health information technology and telepathology applications to meet the scarcity of pathology services and trained people in certain countries. Following trials by a multitude of investigators, not all of them successful, it is apparent that innovation alone did not assure the success of many informatics tools and solutions. Common, ongoing barriers to the widespread adoption of informatics devices include poor information technology infrastructure in undeveloped areas, the cost of technology, and regulatory issues. This review offers a deeper understanding of how pathology informatics historically developed and provides insights into what the promising future might hold.

Prognostic value of automatically extracted nuclear morphometric features in whole slide images of male breast cancer

Numerous studies have shown the prognostic significance of nuclear morphometry in breast cancer patients. Wide acceptance of morphometric methods has, however, been hampered by the tedious and time consuming nature of the manual segmentation of nuclei and the lack of equipment for high throughput digitization of slides. Recently, whole slide imaging became more affordable and widely available, making fully digital pathology archives feasible. In this study, we employ an automatic nuclei segmentation algorithm to extract nuclear morphometry features related to size and we analyze their prognostic value in male breast cancer. The study population comprised 101 male breast cancer patients for whom survival data was available (median follow-up of 5.7 years). Automatic segmentation was performed on digitized tissue microarray slides, and for each patient, the mean nuclear area and the standard deviation of the nuclear area were calculated. In univariate survival analysis, a significant difference was found between patients with low and high mean nuclear area (P=0.022), while nuclear atypia score did not provide prognostic value. In Cox regression, mean nuclear area had independent additional prognostic value (P=0.032) to tumor size and tubule formation. In conclusion, we present an automatic method for nuclear morphometry and its application in male breast cancer prognosis. The automatically extracted mean nuclear area proved to be a significant prognostic indicator. With the increasing availability of slide scanning equipment in pathology labs, these kinds of quantitative approaches can be easily integrated in the workflow of routine pathology practice.

Cascaded discrimination of normal, abnormal, and confounder classes in histopathology: Gleason grading of prostate cancer

BACKGROUND

Automated classification of histopathology involves identification of multiple classes, including benign, cancerous, and confounder categories. The confounder tissue classes can often mimic and share attributes with both the diseased and normal tissue classes, and can be particularly difficult to identify, both manually and by automated classifiers. In the case of prostate cancer, they may be several confounding tissue types present in a biopsy sample, posing as major sources of diagnostic error for pathologists. Two common multi-class approaches are one-shot classification (OSC), where all classes are identified simultaneously, and one-versus-all (OVA), where a "target" class is distinguished from all "non-target" classes. OSC is typically unable to handle discrimination of classes of varying similarity (e.g. with images of prostate atrophy and high grade cancer), while OVA forces several heterogeneous classes into a single "non-target" class. In this work, we present a cascaded (CAS) approach to classifying prostate biopsy tissue samples, where images from different classes are grouped to maximize intra-group homogeneity while maximizing inter-group heterogeneity.

RESULTS

We apply the CAS approach to categorize 2000 tissue samples taken from 214 patient studies into seven classes: epithelium, stroma, atrophy, prostatic intraepithelial neoplasia (PIN), and prostate cancer Gleason grades 3, 4, and 5. A series of increasingly granular binary classifiers are used to split the different tissue classes until the images have been categorized into a single unique class. Our automatically-extracted image feature set includes architectural features based on location of the nuclei within the tissue sample as well as texture features extracted on a per-pixel level. The CAS strategy yields a positive predictive value (PPV) of 0.86 in classifying the 2000 tissue images into one of 7 classes, compared with the OVA (0.77 PPV) and OSC approaches (0.76 PPV).

CONCLUSIONS

Use of the CAS strategy increases the PPV for a multi-category classification system over two common alternative strategies. In classification problems such as histopathology, where multiple class groups exist with varying degrees of heterogeneity, the CAS system can intelligently assign class labels to objects by performing multiple binary classifications according to domain knowledge.

Pathology Imaging Informatics for Clinical Practice and Investigative and Translational Research

Pathologists routinely interpret gross and microscopic specimens to render diagnoses and to engage in a broad spectrum of investigative research. Multiple studies have demonstrated that imaging technologies have progressed to a level at which properly digitized specimens provide sufficient quality comparable to the traditional glass slides examinations. Continued advancements in this area will have a profound impact on the manner in which pathology is conducted from this point on. Several leading institutions have already undertaken ambitious projects directed toward digitally imaging, archiving, and sharing pathology specimens. As a result of these advances, the use of informatics in diagnostic and investigative pathology applications is expanding rapidly. In addition, the advent of novel technologies such as multispectral imaging makes it possible to visualize and analyze imaged specimens using multiple wavelengths simultaneously. As these powerful technologies become increasingly accepted and adopted, the opportunities for gaining new insight into the underlying mechanisms of diseases as well as the potential for discriminating among subtypes of pathologies are growing accordingly.

Image microarrays (IMA): Digital pathology's missing tool

Introduction:

The increasing availability of whole slide imaging (WSI) data sets (digital slides) from glass slides offers new opportunities for the development of computer-aided diagnostic (CAD) algorithms. With the all-digital pathology workflow that these data sets will enable in the near future, literally millions of digital slides will be generated and stored. Consequently, the field in general and pathologists, specifically, will need tools to help extract actionable information from this new and vast collective repository.

Methods:

To address this limitation, we designed and implemented a tool (dCORE) to enable the systematic capture of image tiles with constrained size and resolution that contain desired histopathologic features.

Results:

In this communication, we describe a user-friendly tool that will enable pathologists to mine digital slides archives to create image microarrays (IMAs). IMAs are to digital slides as tissue microarrays (TMAs) are to cell blocks. Thus, a single digital slide could be transformed into an array of hundreds to thousands of high quality digital images, with each containing key diagnostic morphologies and appropriate controls. Current manual digital image cut-and-paste methods that allow for the creation of a grid of images (such as an IMA) of matching resolutions are tedious.

Conclusion:

The ability to create IMAs representing hundreds to thousands of vetted morphologic features has numerous applications in education, proficiency testing, consensus case review, and research. Lastly, in a manner analogous to the way conventional TMA technology has significantly accelerated in situ studies of tissue specimens use of IMAs has similar potential to significantly accelerate CAD algorithm development.

Review and cross-validation of gene expression signatures and melanoma prognosis

In melanoma, there is an urgent need to identify novel biomarkers with prognostic performance superior to traditional clinical and histological parameters. Gene expression-based prognostic signatures offer promise, but studies have been challenged by sample scarcity, cohort heterogeneity, and doubts about the efficacy of such signatures relative to current clinical practices. Motivated by new studies that have begun to address these challenges, we reviewed prognostic signatures derived from gene expression microarray analysis of human melanoma tissue. We used REMARK-based criteria to select the most relevant studies and directly compared their signature gene lists. Through functional ontology enrichment analysis, we observed that these independent data sets converge in part upon immune response processes and the G-protein signaling NRAS-regulation pathway, both important in melanoma development and progression. The signatures correctly predicted patient outcome in independent gene expression data sets with some notably low misclassification rates, particularly among studies involving more advanced-stage tumors. This successful cross-validation indicates that gene expression analysis-based signatures are becoming translationally relevant to care of melanoma patients, as well as improving understanding of the aspects of melanoma biology that determine patient outcome.