Machine-based morphologic analysis of glioblastoma using whole-slide pathology images uncovers clinically relevant molecular correlates

Assessment of AURKA expression and prognosis prediction in lung adenocarcinoma using machine learning-based pathomics signature

Objective

This study aimed to develop quantitative feature-based models from histopathological images to assess aurora kinase A (AURKA) expression and predict the prognosis of patients with lung adenocarcinoma (LUAD).

Methods

A dataset of patients with LUAD was derived from the cancer genome atlas (TCGA) with information on clinical characteristics, RNA sequencing and histopathological images. The TCGA-LUAD cohort was randomly divided into training (n = 229) and testing (n = 98) sets. We extracted quantitative image features from histopathological slides of patients with LUAD using computational approaches, constructed a predictive model for AURKA expression in the training set, and estimated their predictive performance in the test set. A Cox proportional hazards model was used to assess whether the pathomic scores (PS) generated by the model independently predicted LUAD survival.

Results

High AURKA expression was an independent risk factor for overall survival (OS) in patients with LUAD (hazard ratio = 1.816, 95 % confidence intervals = 1.257-2.623, P = 0.001). The model based on histopathological image features had significant predictive value for AURKA expression: the area under the curve of the receiver operating characteristic curve in the training set and validation set was 0.809 and 0.739, respectively. Decision curve analysis showed that the model had clinical utility. Patients with high PS and low PS had different survival rates (P = 0.019). Multivariate analysis suggested that PS was an independent prognostic factor for LUAD (hazard ratio = 1.615, 95 % confidence intervals = 1.071-2.438, P = 0.022).

Conclusion

Pathomics models based on machine learning can accurately predict AURKA expression and the PS generated by the model can predict LUAD prognosis.

AI powered quantification of nuclear morphology in cancers enables prediction of genome instability and prognosis

While alterations in nucleus size, shape, and color are ubiquitous in cancer, comprehensive quantification of nuclear morphology across a whole-slide histologic image remains a challenge. Here, we describe the development of a pan-tissue, deep learning-based digital pathology pipeline for exhaustive nucleus detection, segmentation, and classification and the utility of this pipeline for nuclear morphologic biomarker discovery. Manually-collected nucleus annotations were used to train an object detection and segmentation model for identifying nuclei, which was deployed to segment nuclei in H&E-stained slides from the BRCA, LUAD, and PRAD TCGA cohorts. Interpretable features describing the shape, size, color, and texture of each nucleus were extracted from segmented nuclei and compared to measurements of genomic instability, gene expression, and prognosis. The nuclear segmentation and classification model trained herein performed comparably to previously reported models. Features extracted from the model revealed differences sufficient to distinguish between BRCA, LUAD, and PRAD. Furthermore, cancer cell nuclear area was associated with increased aneuploidy score and homologous recombination deficiency. In BRCA, increased fibroblast nuclear area was indicative of poor progression-free and overall survival and was associated with gene expression signatures related to extracellular matrix remodeling and anti-tumor immunity. Thus, we developed a powerful pan-tissue approach for nucleus segmentation and featurization, enabling the construction of predictive models and the identification of features linking nuclear morphology with clinically-relevant prognostic biomarkers across multiple cancer types.

AI Deployment on GBM Diagnosis: A Novel Approach to Analyze Histopathological Images Using Image Feature-Based Analysis

BACKGROUND

Glioblastoma (GBM) is one of the most common malignant primary brain tumors, which accounts for 60-70% of all gliomas. Conventional diagnosis and the decision of post-operation treatment plan for glioblastoma is mainly based on the feature-based qualitative analysis of hematoxylin and eosin-stained (H&E) histopathological slides by both an experienced medical technologist and a pathologist. The recent development of digital whole slide scanners makes AI-based histopathological image analysis feasible and helps to diagnose cancer by accurately counting cell types and/or quantitative analysis. However, the technology available for digital slide image analysis is still very limited. This study aimed to build an image feature-based computer model using histopathology whole slide images to differentiate patients with glioblastoma (GBM) from healthy control (HC).

METHOD

Two independent cohorts of patients were used. The first cohort was composed of 262 GBM patients of the Cancer Genome Atlas Glioblastoma Multiform Collection (TCGA-GBM) dataset from the cancer imaging archive (TCIA) database. The second cohort was composed of 60 GBM patients collected from a local hospital. Also, a group of 60 participants with no known brain disease were collected. All the H&E slides were collected. Thirty-three image features (22 GLCM and 11 GLRLM) were retrieved from the tumor volume delineated by medical technologist on H&E slides. Five machine-learning algorithms including decision-tree (DT), extreme-boost (EB), support vector machine (SVM), random forest (RF), and linear model (LM) were used to build five models using the image features extracted from the first cohort of patients. Models built were deployed using the selected key image features for GBM diagnosis from the second cohort (local patients) as model testing, to identify and verify key image features for GBM diagnosis.

RESULTS

All five machine learning algorithms demonstrated excellent performance in GBM diagnosis and achieved an overall accuracy of 100% in the training and validation stage. A total of 12 GLCM and 3 GLRLM image features were identified and they showed a significant difference between the normal and the GBM image. However, only the SVM model maintained its excellent performance in the deployment of the models using the independent local cohort, with an accuracy of 93.5%, sensitivity of 86.95%, and specificity of 99.73%.

CONCLUSION

In this study, we have identified 12 GLCM and 3 GLRLM image features which can aid the GBM diagnosis. Among the five models built, the SVM model proposed in this study demonstrated excellent accuracy with very good sensitivity and specificity. It could potentially be used for GBM diagnosis and future clinical application.

Exploring the Vital Link Between Glioma, Neuron, and Neural Activity in the Context of Invasion

Because of their ability to infiltrate normal brain tissue, gliomas frequently evade microscopic surgical excision. The histologic infiltrative property of human glioma has been previously characterized as Scherer secondary structures, of which the perivascular satellitosis is a prospective target for anti-angiogenic treatment in high-grade gliomas. However, the mechanisms underlying perineuronal satellitosis remain unclear, and therapy remains lacking. Our knowledge of the mechanism underlying Scherer secondary structures has improved over time. New techniques, such as laser capture microdissection and optogenetic stimulation, have advanced our understanding of glioma invasion mechanisms. Although laser capture microdissection is a useful tool for studying gliomas that infiltrate the normal brain microenvironment, optogenetics and mouse xenograft glioma models have been extensively used in studies demonstrating the unique role of synaptogenesis in glioma proliferation and identification of potential therapeutic targets. Moreover, a rare glioma cell line is established that, when transplanted in the mouse brain, can replicate and recapitulate the human diffuse invasion phenotype. This review discusses the primary molecular causes of glioma, its histopathology-based invasive mechanisms, and the importance of neuronal activity and interactions between glioma cells and neurons in the brain microenvironment. It also explores current methods and models of gliomas.

Knowledge-Based Representation Learning for Nucleus Instance Classification From Histopathological Images

The classification of nuclei in H&E-stained histopathological images is a fundamental step in the quantitative analysis of digital pathology. Most existing methods employ multi-class classification on the detected nucleus instances, while the annotation scale greatly limits their performance. Moreover, they often downplay the contextual information surrounding nucleus instances that is critical for classification. To explicitly provide contextual information to the classification model, we design a new structured input consisting of a content-rich image patch and a target instance mask. The image patch provides rich contextual information, while the target instance mask indicates the location of the instance to be classified and emphasizes its shape. Benefiting from our structured input format, we propose Structured Triplet for representation learning, a triplet learning framework on unlabelled nucleus instances with customized positive and negative sampling strategies. We pre-train a feature extraction model based on this framework with a large-scale unlabeled dataset, making it possible to train an effective classification model with limited annotated data. We also add two auxiliary branches, namely the attribute learning branch and the conventional self-supervised learning branch, to further improve its performance. As part of this work, we will release a new dataset of H&E-stained pathology images with nucleus instance masks, containing 20,187 patches of size 1024 ×1024 , where each patch comes from a different whole-slide image. The model pre-trained on this dataset with our framework significantly reduces the burden of extensive labeling. We show a substantial improvement in nucleus classification accuracy compared with the state-of-the-art methods.

Artificial intelligence for precision medicine in autoimmune liver disease

Autoimmune liver diseases (AiLDs) are rare autoimmune conditions of the liver and the biliary tree with unknown etiology and limited treatment options. AiLDs are inherently characterized by a high degree of complexity, which poses great challenges in understanding their etiopathogenesis, developing novel biomarkers and risk-stratification tools, and, eventually, generating new drugs. Artificial intelligence (AI) is considered one of the best candidates to support researchers and clinicians in making sense of biological complexity. In this review, we offer a primer on AI and machine learning for clinicians, and discuss recent available literature on its applications in medicine and more specifically how it can help to tackle major unmet needs in AiLDs.

An integrative web-based software tool for multi-dimensional pathology whole-slide image analytics

Objective.In the era of precision medicine, human tumor atlas-oriented studies have been significantly facilitated by high-resolution, multi-modal tissue based microscopic pathology image analytics. To better support such tissue-based investigations, we have developed Digital Pathology Laboratory (DPLab), a publicly available web-based platform, to assist biomedical research groups, non-technical end users, and clinicians for pathology whole-slide image visualization, annotation, analysis, and sharing via web browsers.Approach.A major advancement of this work is the easy-to-follow methods to reconstruct three-dimension (3D) tissue image volumes by registering two-dimension (2D) whole-slide pathology images of serial tissue sections stained by hematoxylin and eosin (H&E), and immunohistochemistry (IHC). The integration of these serial slides stained by different methods provides cellular phenotype and pathophysiologic states in the context of a 3D tissue micro-environment. DPLab is hosted on a publicly accessible server and connected to a backend computational cluster for intensive image analysis computations, with results visualized, downloaded, and shared via a web interface.Main results.Equipped with an analysis toolbox of numerous image processing algorithms, DPLab supports continued integration of community-contributed algorithms and presents an effective solution to improve the accessibility and dissemination of image analysis algorithms by research communities.Significance.DPLab represents the first step in making next generation tissue investigation tools widely available to the research community, enabling and facilitating discovery of clinically relevant disease mechanisms in a digital 3D tissue space.

The relationship between radiomics and pathomics in Glioblastoma patients: Preliminary results from a cross-scale association study

Glioblastoma multiforme (GBM) typically exhibits substantial intratumoral heterogeneity at both microscopic and radiological resolution scales. Diffusion Weighted Imaging (DWI) and dynamic contrast-enhanced (DCE) magnetic resonance imaging (MRI) are two functional MRI techniques that are commonly employed in clinic for the assessment of GBM tumor characteristics. This work presents initial results aiming at determining if radiomics features extracted from preoperative ADC maps and post-contrast T1 (T1C) images are associated with pathomic features arising from H&E digitized pathology images. 48 patients from the public available CPTAC-GBM database, for which both radiology and pathology images were available, were involved in the study. 91 radiomics features were extracted from ADC maps and post-contrast T1 images using PyRadiomics. 65 pathomic features were extracted from cell detection measurements from H&E images. Moreover, 91 features were extracted from cell density maps of H&E images at four different resolutions. Radiopathomic associations were evaluated by means of Spearman's correlation (ρ) and factor analysis. p values were adjusted for multiple correlations by using a false discovery rate adjustment. Significant cross-scale associations were identified between pathomics and ADC, both considering features (n = 186, 0.45 < ρ < 0.74 in absolute value) and factors (n = 5, 0.48 < ρ < 0.54 in absolute value). Significant but fewer ρ values were found concerning the association between pathomics and radiomics features (n = 53, 0.5 < ρ < 0.65 in absolute value) and factors (n = 2, ρ = 0.63 and ρ = 0.53 in absolute value). The results of this study suggest that cross-scale associations may exist between digital pathology and ADC and T1C imaging. This can be useful not only to improve the knowledge concerning GBM intratumoral heterogeneity, but also to strengthen the role of radiomics approach and its validation in clinical practice as "virtual biopsy", introducing new insights for omics integration toward a personalized medicine approach.

Large-scale extraction of interpretable features provides new insights into kidney histopathology – a proof-of-concept study

Whole slide images contain a magnitude of quantitative information that may not be fully explored in qualitative visual assessments. We propose: (1) a novel pipeline for extracting a comprehensive set of visual features, which are detectable by a pathologist, as well as sub-visual features, which are not discernible by human experts and (2) perform detailed analyses on renal images from mice with experimental unilateral ureteral obstruction. An important criterion for these features is that they are easy to interpret, as opposed to features obtained from neural networks. We extract and compare features from pathological and healthy control kidneys to learn how the compartments (glomerulus, Bowman's capsule, tubule, interstitium, artery, and arterial lumen) are affected by the pathology. We define feature selection methods to extract the most informative and discriminative features. We perform statistical analyses to understand the relation of the extracted features, both individually, and in combinations, with tissue morphology and pathology. Particularly for the presented case-study, we highlight features that are affected in each compartment. With this, prior biological knowledge, such as the increase in interstitial nuclei, is confirmed and presented in a quantitative way, alongside with novel findings, like color and intensity changes in glomeruli and Bowman's capsule. The proposed approach is therefore an important step towards quantitative, reproducible, and rater-independent analysis in histopathology.

Multi-Scale Attention Convolutional Network for Masson Stained Bile Duct Segmentation from Liver Pathology Images

In clinical practice, the Ishak Score system would be adopted to perform the evaluation of the grading and staging of hepatitis according to whether portal areas have fibrous expansion, bridging with other portal areas, or bridging with central veins. Based on these staging criteria, it is necessary to identify portal areas and central veins when performing the Ishak Score staging. The bile ducts have variant types and are very difficult to be detected under a single magnification, hence pathologists must observe bile ducts at different magnifications to obtain sufficient information. This pathologic examinations in routine clinical practice, however, would result in the labor intensive and expensive examination process. Therefore, the automatic quantitative analysis for pathologic examinations has had an increased demand and attracted significant attention recently. A multi-scale inputs of attention convolutional network is proposed in this study to simulate pathologists' examination procedure for observing bile ducts under different magnifications in liver biopsy. The proposed multi-scale attention network integrates cell-level information and adjacent structural feature information for bile duct segmentation. In addition, the attention mechanism of proposed model enables the network to focus the segmentation task on the input of high magnification, reducing the influence from low magnification input, but still helps to provide wider field of surrounding information. In comparison with existing models, including FCN, U-Net, SegNet, DeepLabv3 and DeepLabv3-plus, the experimental results demonstrated that the proposed model improved the segmentation performance on Masson bile duct segmentation task with 72.5% IOU and 84.1% F1-score.

Harnessing multimodal data integration to advance precision oncology

Advances in quantitative biomarker development have accelerated new forms of data-driven insights for patients with cancer. However, most approaches are limited to a single mode of data, leaving integrated approaches across modalities relatively underdeveloped. Multimodal integration of advanced molecular diagnostics, radiological and histological imaging, and codified clinical data presents opportunities to advance precision oncology beyond genomics and standard molecular techniques. However, most medical datasets are still too sparse to be useful for the training of modern machine learning techniques, and significant challenges remain before this is remedied. Combined efforts of data engineering, computational methods for analysis of heterogeneous data and instantiation of synergistic data models in biomedical research are required for success. In this Perspective, we offer our opinions on synthesizing complementary modalities of data with emerging multimodal artificial intelligence methods. Advancing along this direction will result in a reimagined class of multimodal biomarkers to propel the field of precision oncology in the coming decade.

Computer extracted features of nuclear morphology in hematoxylin and eosin images distinguish Stage II and IV colon tumors

We assessed the utility of quantitative features of colon cancer nuclei, extracted from digitized hematoxylin and eosin-stained whole slide images (WSIs), to distinguish between Stage II from Stage IV colon cancers. Our discovery cohort comprised 100 Stage II and Stage IV colon cancer cases sourced from the University Hospitals Cleveland Medical Center (UHCMC). We performed initial (independent) model validation on 51 (143) Stage II and 79 (54) Stage IV colon cancer cases from UHCMC (The Cancer Genome Atlas's Colon Adenocarcinoma, TCGA-COAD, cohort). Our approach comprised the following steps, (1) a fully convolutional deep neural network with VGG-18 architecture was trained to locate cancer on WSIs, (2) another deep-learning model based on Mask-RCNN with Resnet-50 architecture was used to segment all nuclei from within the identified cancer region, (3) a total of 26,641 quantitative morphometric features pertaining to nuclear shape, size, and texture were extracted from within and outside tumor nuclei, (4) a random forest classifier was trained to distinguish between Stage II and Stage IV colon cancers using the 5 most discriminatory features selected by the Wilcoxon rank-sum test. Our trained classifier using these top 5 features yielded an AUC of 0.81 and 0.78, respectively, on the held-out cases in UHCMC and TCGA validation sets. For 197 TCGA-COAD cases, the Cox-proportional hazards model yielded a hazard ratio of 2.20 (95% CI: 1.24-3.88) with a concordance index of 0.71 using only top-five features for risk stratification of overall survival. The Kaplan-Meier estimate also showed statistically significant separation between the low-risk and high-risk patients with a log-rank p-value of 0.0097. Finally, unsupervised clustering of the top-five features revealed that Stage IV colon cancers with peritoneal spread were morphologically more similar to Stage II colon cancers with no long-term metastases than Stage IV colon cancers with hematogenous spread. This article is protected by copyright. All rights reserved.

Artificial Intelligence in Brain Tumour Surgery-An Emerging Paradigm

Artificial intelligence (AI) platforms have the potential to cause a paradigm shift in brain tumour surgery. Brain tumour surgery augmented with AI can result in safer and more effective treatment. In this review article, we explore the current and future role of AI in patients undergoing brain tumour surgery, including aiding diagnosis, optimising the surgical plan, providing support during the operation, and better predicting the prognosis. Finally, we discuss barriers to the successful clinical implementation, the ethical concerns, and we provide our perspective on how the field could be advanced.

Visual Analytics for Hypothesis-Driven Exploration in Computational Pathology

Recent advances in computational and algorithmic power are evolving the field of medical imaging rapidly. In cancer research, many new directions are sought to characterize patients with additional imaging features derived from radiology and pathology images. The emerging field of Computational Pathology targets the high-throughput extraction and analysis of the spatial distribution of cells from digital histopathology images. The associated morphological and architectural features allow researchers to quantify and characterize new imaging biomarkers for cancer diagnosis, prognosis, and treatment decisions. However, while the image feature space grows, exploration and analysis become more difficult and ineffective. There is a need for dedicated interfaces for interactive data manipulation and visual analysis of computational pathology and clinical data. For this purpose, we present IIComPath, a visual analytics approach that enables clinical researchers to formulate hypotheses and create computational pathology pipelines involving cohort construction, spatial analysis of image-derived features, and cohort analysis. We demonstrate our approach through use cases that investigate the prognostic value of current diagnostic features and new computational pathology biomarkers.

Building robust pathology image analyses with uncertainty quantification

BACKGROUND AND OBJECTIVE

Computerized pathology image analysis is an important tool in research and clinical settings, which enables quantitative tissue characterization and can assist a pathologist's evaluation. The aim of our study is to systematically quantify and minimize uncertainty in output of computer based pathology image analysis.

METHODS

Uncertainty quantification (UQ) and sensitivity analysis (SA) methods, such as Variance-Based Decomposition (VBD) and Morris One-At-a-Time (MOAT), are employed to track and quantify uncertainty in a real-world application with large Whole Slide Imaging datasets - 943 Breast Invasive Carcinoma (BRCA) and 381 Lung Squamous Cell Carcinoma (LUSC) patients. Because these studies are compute intensive, high-performance computing systems and efficient UQ/SA methods were combined to provide efficient execution. UQ/SA has been able to highlight parameters of the application that impact the results, as well as nuclear features that carry most of the uncertainty. Using this information, we built a method for selecting stable features that minimize application output uncertainty.

RESULTS

The results show that input parameter variations significantly impact all stages (segmentation, feature computation, and survival analysis) of the use case application. We then identified and classified features according to their robustness to parameter variation, and using the proposed features selection strategy, for instance, patient grouping stability in survival analysis has been improved from in 17% and 34% for BRCA and LUSC, respectively.

CONCLUSIONS

This strategy created more robust analyses, demonstrating that SA and UQ are important methods that may increase confidence digital pathology.

Neuroblastoma GD2 Expression and Computational Analysis of Aptamer-Based Bioaffinity Targeting

Neuroblastoma (NB) is a neuroectodermal embryonic cancer that originates from primordial neural crest cells, and amongst pediatric cancers with high mortality rates. NB is categorized into high-, intermediate-, and low-risk cases. A significant proportion of high-risk patients who achieve remission have a minimal residual disease (MRD) that causes relapse. Whilst there exists a myriad of advanced treatment options for NB, it is still characterized by a high relapse rate, resulting in a reduced chance of survival. Disialoganglioside (GD2) is a lipo-ganglioside containing a fatty acid derivative of sphingosine that is coupled to a monosaccharide and a sialic acid. Amongst pediatric solid tumors, NB tumor cells are known to express GD2; hence, it represents a unique antigen for subclinical NB MRD detection and analysis with implications in determining a response for treatment. This article discusses NB MRD expression and analytical assays for GD2 detection and quantification as well as computational approaches for GD2 characterization based on high-throughput image processing and genomic data analysis.

Histopathological image and gene expression pattern analysis for predicting molecular features and prognosis of head and neck squamous cell carcinoma

BACKGROUND

Histopathological image features offer a quantitative measurement of cellular morphology, and probably help for better diagnosis and prognosis in head and neck squamous cell carcinoma (HNSCC).

METHODS

We first used histopathological image features and machine-learning algorithms to predict molecular features of 212 HNSCC patients from The Cancer Genome Atlas (TCGA). Next, we divided TCGA-HNSCC cohort into training set (n = 149) and test set (n = 63), and obtained tissue microarrays as an external validation set (n = 126). We identified the gene expression profile correlated to image features by bioinformatics analysis.

RESULTS

Histopathological image features combined with random forest may predict five somatic mutations, transcriptional subtypes, and methylation subtypes, with area under curve (AUC) ranging from 0.828 to 0.968. The prediction model based on image features could predict overall survival, with 5-year AUC of 0.831, 0.782, and 0.751 in training, test, and validation sets. We next established an integrative prognostic model of image features and gene expressions, which obtained better performance in training set (5-year AUC = 0.860) and test set (5-year AUC = 0.826). According to histopathological transcriptomics risk score (HTRS) generated by the model, high-risk and low-risk patients had different survival in training set (HR = 4.09, p < 0.001) and test set (HR=3.08, p = 0.019). Multivariate analysis suggested that HTRS was an independent predictor in training set (HR = 5.17, p < 0.001). The nomogram combining HTRS and clinical factors had higher net benefit than conventional clinical evaluation.

CONCLUSIONS

Histopathological image features provided a promising approach to predict mutations, molecular subtypes, and prognosis of HNSCC. The integration of image features and gene expression data had potential for improving prognosis prediction in HNSCC.

Integrative Models of Histopathological Image Features and Omics Data Predict Survival in Head and Neck Squamous Cell Carcinoma

Background

Both histopathological image features and genomics data were associated with survival outcome of cancer patients. However, integrating features of histopathological images, genomics and other omics for improving prognosis prediction has not been reported in head and neck squamous cell carcinoma (HNSCC).

Methods

A dataset of 216 HNSCC patients was derived from the Cancer Genome Atlas (TCGA) with information of clinical characteristics, genetic mutation, RNA sequencing, protein expression and histopathological images. Patients were randomly assigned into training (n = 108) or validation (n = 108) sets. We extracted 593 quantitative image features, and used random forest algorithm with 10-fold cross-validation to build prognostic models for overall survival (OS) in training set, then compared the area under the time-dependent receiver operating characteristic curve (AUC) in validation set.

Results

In validation set, histopathological image features had significant predictive value for OS (5-year AUC = 0.784). The histopathology + omics models showed better predictive performance than genomics, transcriptomics or proteomics alone. Moreover, the multi-omics model incorporating image features, genomics, transcriptomics and proteomics reached the maximal 1-, 3-, and 5-year AUC of 0.871, 0.908, and 0.929, with most significant survival difference (HR = 10.66, 95%CI: 5.06–26.8, p < 0.001). Decision curve analysis also revealed a better net benefit of multi-omics model.

Conclusion

The histopathological images could provide complementary features to improve prognostic performance for HNSCC patients. The integrative model of histopathological image features and omics data might serve as an effective tool for survival prediction and risk stratification in clinical practice.

AI in Medical Imaging Informatics: Current Challenges and Future Directions

This paper reviews state-of-the-art research solutions across the spectrum of medical imaging informatics, discusses clinical translation, and provides future directions for advancing clinical practice. More specifically, it summarizes advances in medical imaging acquisition technologies for different modalities, highlighting the necessity for efficient medical data management strategies in the context of AI in big healthcare data analytics. It then provides a synopsis of contemporary and emerging algorithmic methods for disease classification and organ/ tissue segmentation, focusing on AI and deep learning architectures that have already become the de facto approach. The clinical benefits of in-silico modelling advances linked with evolving 3D reconstruction and visualization applications are further documented. Concluding, integrative analytics approaches driven by associate research branches highlighted in this study promise to revolutionize imaging informatics as known today across the healthcare continuum for both radiology and digital pathology applications. The latter, is projected to enable informed, more accurate diagnosis, timely prognosis, and effective treatment planning, underpinning precision medicine.

Histo-genomics: digital pathology at the forefront of precision medicine

The toughest challenge OMICs face is that they provide extremely high molecular resolution but poor spatial information. Understanding the cellular/histological context of the overwhelming genetic data is critical for a full understanding of the clinical behavior of a malignant tumor. Digital pathology can add an extra layer of information to help visualize in a spatial and microenvironmental context the molecular information of cancer. Thus, histo-genomics provide a unique chance for data integration. In the era of a precision medicine, a four-dimensional (4D) (temporal/spatial) analysis of cancer aided by digital pathology can be a critical step to understand the evolution/progression of different cancers and consequently tailor individual treatment plans. For instance, the integration of molecular biomarkers expression into a three-dimensional (3D) image of a digitally scanned tumor can offer a better understanding of its subtype, behavior, host immune response and prognosis. Using advanced digital image analysis, a larger spectrum of parameters can be analyzed as potential predictors of clinical behavior. Correlation between morphological features and host immune response can be also performed with therapeutic implications. Radio-histomics, or the interface of radiological images and histology is another emerging exciting field which encompasses the integration of radiological imaging with digital pathological images, genomics, and clinical data to portray a more holistic approach to understating and treating disease. These advances in digital slide scanning are not without technical challenges, which will be addressed carefully in this review with quick peek at its future.

Optimizing parameter sensitivity analysis of large-scale microscopy image analysis workflows with multilevel computation reuse

Parameter sensitivity analysis (SA) is an effective tool to gain knowledge about complex analysis applications and assess the variability in their analysis results. However, it is an expensive process as it requires the execution of the target application multiple times with a large number of different input parameter values. In this work, we propose optimizations to reduce the overall computation cost of SA in the context of analysis applications that segment high-resolution slide tissue images, ie, images with resolutions of 100k × 100k pixels. Two cost-cutting techniques are combined to efficiently execute SA: use of distributed hybrid systems for parallel execution and computation reuse at multiple levels of an analysis pipeline to reduce the amount of computation. These techniques were evaluated using a cancer image analysis workflow on a hybrid cluster with 256 nodes, each with an Intel Phi and a dual socket CPU. Our parallel execution method attained an efficiency of over 90% on 256 nodes. The hybrid execution on the CPU and Intel Phi improved the performance by 2×. Multilevel computation reuse led to performance gains of over 2.9×.

Artificial intelligence in digital pathology - new tools for diagnosis and precision oncology

In the past decade, advances in precision oncology have resulted in an increased demand for predictive assays that enable the selection and stratification of patients for treatment. The enormous divergence of signalling and transcriptional networks mediating the crosstalk between cancer, stromal and immune cells complicates the development of functionally relevant biomarkers based on a single gene or protein. However, the result of these complex processes can be uniquely captured in the morphometric features of stained tissue specimens. The possibility of digitizing whole-slide images of tissue has led to the advent of artificial intelligence (AI) and machine learning tools in digital pathology, which enable mining of subvisual morphometric phenotypes and might, ultimately, improve patient management. In this Perspective, we critically evaluate various AI-based computational approaches for digital pathology, focusing on deep neural networks and 'hand-crafted' feature-based methodologies. We aim to provide a broad framework for incorporating AI and machine learning tools into clinical oncology, with an emphasis on biomarker development. We discuss some of the challenges relating to the use of AI, including the need for well-curated validation datasets, regulatory approval and fair reimbursement strategies. Finally, we present potential future opportunities for precision oncology.

High gene expression levels of VEGFA and CXCL8 in the peritumoral brain zone are associated with the recurrence of glioblastoma: A bioinformatics analysis

The present study aimed to identify differentially regulated genes between the peritumoral brain zone (PBZ) and tumor core (TC) of glioblastoma (GBM), to elucidate the underlying molecular mechanisms and provide a target for the treatment of tumors. The GSE13276 and GSE116520 datasets were downloaded from the Gene Expression Omnibus (GEO) database. Differentially expressed genes (DEGs) for the PBZ and TC were obtained using the GEO2R tool. The bioinformatics and evolutionary genomics online tool Venn was used to identify common DEGs between the two datasets. The Database for Annotation, Visualization, and Integrated Discovery online tool was used to analyze enriched pathways of the Gene Ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) databases. The Search Tool for the Retrieval of Interacting Genes/Proteins online tool was used to construct a protein-protein interaction (PPI) network of DEGs. Hub genes were identified using Cytohubba, a plug-in for Cytoscape. The Gene Expression Profiling Interactive Analysis (GEPIA) database was utilized to perform survival analysis. In total, 75 DEGs, including 12 upregulated and 63 downregulated genes, were identified. In the GO term analysis, these DEGs were mainly enriched in ‘regulation of angiogenesis’ and ‘central nervous system development’. Furthermore, in the KEGG pathway analysis, the DEGs were mainly enriched in ‘bladder cancer’ and ‘endocytosis’. When filtering the results of the PPI network analysis using Cytohubba, a total of 10 hub genes, including proteolipid protein 1, myelin associated oligodendrocyte basic protein, contactin 2, myelin oligodendrocyte glycoprotein, myelin basic protein, myelin associated glycoprotein, SRY-box transcription factor 10, C-X-C motif chemokine ligand 8 (CXCL8), vascular endothelial growth factor A (VEGFA) and plasmolipin, were identified. These hub genes were further subjected to GO term and KEGG pathway analysis, and were revealed to be enriched in ‘central nervous system development’, ‘bladder cancer’ and ‘rheumatoid arthritis’. These hub genes were used to perform survival analysis using the GEPIA database, and it was determined that VEGFA and CXCL8 were significantly associated with a reduction in the overall survival of patients with GBM. In conclusion, the results suggest that the recurrence of GBM is associated with high gene expression levels VEGFA and CXCL8, and the development of the central nervous system.

Microvascularity detection and quantification in glioma: a novel deep-learning-based framework

Microvascularity is highly correlated with the grading and subtyping of gliomas, making this one of its most important histological features. Accurate quantitative analysis of microvessels is helpful for the development of a targeted therapy for antiangiogenesis. The deep-learning algorithm is by far the most effective segmentation and detection model and enables location and recognition of complex microvascular networks in large images obtained from hematoxylin and eosin (H&E) stained specimens. We proposed an automated deep-learning-based method to detect and quantify the microvascularity in glioma and applied it to comprehensive clinical analyses. A total of 350 glioma patients were enrolled in our study, for which digitalized imaging of H&E stained slides were reviewed, molecular diagnosis was performed and follow-up was investigated. The microvascular features were compared according to their histologic types, molecular types, and patients' prognosis. The results show that the proposed method can quantify microvascular characteristics automatically and effectively. Significant increases of microvascular density and microvascular area were observed in glioblastomas (95% p < 0.001 in density, 170% p < 0.001 in area) in comparison with other histologic types; increases were also observed in cases with TERT-mut only (68% p < 0.001 in density, 54% p < 0.001 in area) compared with other molecular types. Survival analysis showed that microvascular features can be used to cluster cases into two groups with different survival periods (hazard ratio [HR] 2.843, log-rank <0.001), which indicates the quantified microvascular features may potentially be alternative signatures for revealing patients' prognosis. This deep-learning-based method may be a useful tool in routine clinical practice for precise diagnosis and antiangiogenic treatment.

The invasive proteome of glioblastoma revealed by laser-capture microdissection

Background

Glioblastomas are heterogeneous tumors composed of a necrotic and tumor core and an invasive periphery.

Methods

Here, we performed a proteomics analysis of laser-capture micro-dissected glioblastoma core and invasive areas of patient-derived xenografts.

Results

Bioinformatics analysis identified enriched proteins in central and invasive tumor areas. Novel markers of invasion were identified, the genes proteolipid protein 1 (PLP1) and Dynamin-1 (DNM1), which were subsequently validated in tumors and by functional assays.

Conclusions

In summary, our results identify new networks and molecules that may play an important role in glioblastoma development and may constitute potential novel therapeutic targets.

Innovative approaches for cancer treatment: current perspectives and new challenges

Every year, cancer is responsible for millions of deaths worldwide and, even though much progress has been achieved in medicine, there are still many issues that must be addressed in order to improve cancer therapy. For this reason, oncological research is putting a lot of effort towards finding new and efficient therapies which can alleviate critical side effects caused by conventional treatments. Different technologies are currently under evaluation in clinical trials or have been already introduced into clinical practice. While nanomedicine is contributing to the development of biocompatible materials both for diagnostic and therapeutic purposes, bioengineering of extracellular vesicles and cells derived from patients has allowed designing ad hoc systems and univocal targeting strategies. In this review, we will provide an in-depth analysis of the most innovative advances in basic and applied cancer research.

The Promise of Digital Biopsy for the Prediction of Tumor Molecular Features and Clinical Outcomes Associated With Immunotherapy

Immunotherapy by immune checkpoint inhibitors has emerged as an effective treatment for a slight proportion of patients with aggressive tumors. Currently, some molecular determinants, such as the expression of the programmed cell death ligand-1 (PD-L1) or the tumor mutational burden (TMB) have been used in the clinical practice as predictive biomarkers, although they fail in consistency, applicability, or reliability to precisely identify the responding patients mainly because of their spatial intratumoral heterogeneity. Therefore, new biomarkers for early prediction of patient response to immunotherapy, that could integrate several approaches, are eagerly sought. Novel methods of quantitative image analysis (such as radiomics or pathomics) might offer a comprehensive approach providing spatial and temporal information from macroscopic imaging features potentially predictive of underlying molecular drivers, tumor-immune microenvironment, tumor-related prognosis, and clinical outcome (in terms of response or toxicity) following immunotherapy. Preliminary results from radiomics and pathomics analysis have demonstrated their ability to correlate image features with PD-L1 tumor expression, high CD3 cell infiltration or CD8 cell expression, or to produce an image signature concordant with gene expression. Furthermore, the predictive power of radiomics and pathomics can be improved by combining information from other modalities, such as blood values or molecular features, leading to increase the accuracy of these models. Thus, “digital biopsy,” which could be defined by non-invasive and non-consuming digital techniques provided by radiomics and pathomics, may have the potential to allow for personalized approach for cancer patients treated with immunotherapy.

Deep Learning for Semantic Segmentation vs. Classification in Computational Pathology: Application to Mitosis Analysis in Breast Cancer Grading

Existing computational approaches have not yet resulted in effective and efficient computer-aided tools that are used in pathologists' daily practice. Focusing on a computer-based qualification for breast cancer diagnosis, the present study proposes two deep learning architectures to efficiently and effectively detect and classify mitosis in a histopathological tissue sample. The first method consists of two parts, entailing a preprocessing of the digital histological image and a free-handcrafted-feature Convolutional Neural Network (CNN) used for binary classification. Results show that the methodology proposed can achieve 95% accuracy in testing, with an F1-score of 94.35%. This result is higher than the results using classical image processing techniques and also higher than the approaches combining CCNs with handcrafted features. The second approach is an end-to-end methodology using semantic segmentation. Results showed that this algorithm can achieve an accuracy higher than 95% in testing and an average Dice index of 0.6, higher than the existing results using CNNs (0.9 F1-score). Additionally, due to the semantic properties of the deep learning approach, an end-to-end deep learning framework is viable to perform both tasks: detection and classification of mitosis. The results show the potential of deep learning in the analysis of Whole Slide Images (WSI) and its integration to computer-aided systems. The extension of this work to whole slide images is also addressed in the last sections; as well as, some computational key points that are useful when constructing a computer-aided-system inspired by the proposed technology.

Multi-objective Parameter Auto-tuning for Tissue Image Segmentation Workflows

We propose a software platform that integrates methods and tools for multi-objective parameter auto-tuning in tissue image segmentation workflows. The goal of our work is to provide an approach for improving the accuracy of nucleus/cell segmentation pipelines by tuning their input parameters. The shape, size, and texture features of nuclei in tissue are important biomarkers for disease prognosis, and accurate computation of these features depends on accurate delineation of boundaries of nuclei. Input parameters in many nucleus segmentation workflows affect segmentation accuracy and have to be tuned for optimal performance. This is a time-consuming and computationally expensive process; automating this step facilitates more robust image segmentation workflows and enables more efficient application of image analysis in large image datasets. Our software platform adjusts the parameters of a nuclear segmentation algorithm to maximize the quality of image segmentation results while minimizing the execution time. It implements several optimization methods to search the parameter space efficiently. In addition, the methodology is developed to execute on high-performance computing systems to reduce the execution time of the parameter tuning phase. These capabilities are packaged in a Docker container for easy deployment and can be used through a friendly interface extension in 3D Slicer. Our results using three real-world image segmentation workflows demonstrate that the proposed solution is able to (1) search a small fraction (about 100 points) of the parameter space, which contains billions to trillions of points, and improve the quality of segmentation output by × 1.20, × 1.29, and × 1.29, on average; (2) decrease the execution time of a segmentation workflow by up to 11.79× while improving output quality; and (3) effectively use parallel systems to accelerate parameter tuning and segmentation phases.

MaReIA: A Cloud MapReduce Based High Performance Whole Slide Image Analysis Framework

Recent advancements in systematic analysis of high resolution whole slide images have increase efficiency of diagnosis, prognosis and prediction of cancer and important diseases. Due to the enormous sizes and dimensions of whole slide images, the analysis requires extensive computing resources which are not commonly available. Images have to be tiled for processing due to computer memory limitations, which lead to inaccurate results due to the ignorance of boundary crossing objects. Thus, we propose a generic and highly scalable cloud-based image analysis framework for whole slide images. The framework enables parallelized integration of image analysis steps, such as segmentation and aggregation of micro-structures in a single pipeline, and generation of final objects manageable by databases. The core concept relies on the abstraction of objects in whole slide images as different classes of spatial geometries, which in turn can be handled as text based records in MapReduce. The framework applies an overlapping partitioning scheme on images, and provides parallelization of tiling and image segmentation based on MapReduce architecture. It further provides robust object normalization, graceful handling of boundary objects with an efficient spatial indexing based matching method to generate accurate results. Our experiments on Amazon EMR show that MaReIA is highly scalable, generic and extremely cost effective by benchmark tests.

Integrated analysis of 34 microarray datasets reveals CBX3 as a diagnostic and prognostic biomarker in glioblastoma

BACKGROUND

Glioblastomas have a high degree of malignancy, high recurrence rate, high mortality rate, and low cure rate. Searching for new markers of glioblastomas is of great significance for improving the diagnosis, prognosis and treatment of glioma.

METHODS

Using the GEO public database, we combined 34 glioma microarray datasets containing 1893 glioma samples and conducted genetic data mining through statistical analysis, bioclustering, and pathway analysis. The results were validated in TCGA, CGGA, and internal cohorts. We further selected a gene for subsequent experiments and conducted cell proliferation and cell cycle analyses to verify the biological function of this gene.

RESULTS

Eight glioblastoma-specific differentially expressed genes were screened using GEO. In the TCGA and CGGA cohorts, patients with high CBX3, BARD1, EGFR, or IFRD1 expression had significantly shorter survival but patients with high GUCY1A3 or MOBP expression had significantly longer survival than patients with lower expression of these genes. After reviewing the literature, we selected the CBX3 gene for further experiments. We confirmed that CBX3 was overexpressed in glioblastoma by immunohistochemical analysis of tissue microarrays and qPCR analysis of surgical specimens. The functional assay results showed that silencing CBX3 arrests the cell cycle in the G2/M phase, thereby weakening the cell proliferation ability.

CONCLUSIONS

We used a multidisciplinary approach to analyze glioblastoma samples in 34 microarray datasets, revealing novel diagnostic and prognostic biomarkers in patients with glioblastoma and providing a new direction for screening tumor markers.

Sensitivity analysis in digital pathology: Handling large number of parameters with compute expensive workflows

Digital pathology imaging enables valuable quantitative characterizations of tissue state at the sub-cellular level. While there is a growing set of methods for analysis of whole slide tissue images, many of them are sensitive to changes in input parameters. Evaluating how analysis results are affected by variations in input parameters is important for the development of robust methods. Executing algorithm sensitivity analyses by systematically varying input parameters is an expensive task because a single evaluation run with a moderate number of tissue images may take hours or days. Our work investigates the use of Surrogate Models (SMs) along with parallel execution to speed up parameter sensitivity analysis (SA). This approach significantly reduces the SA cost, because the SM execution is inexpensive. The evaluation of several SM strategies with two image segmentation workflows demonstrates that a SA study with SMs attains results close to a SA with real application runs (mean absolute error lower than 0.022), while the SM accelerates the SA execution by 51 × . We also show that, although the number of parameters in the example workflows is high, most of the uncertainty can be associated with a few parameters. In order to identify the impact of variations in segmentation results to downstream analyses, we carried out a survival analysis with 387 Lung Squamous Cell Carcinoma cases. This analysis was repeated using 3 values for the most significant parameters identified by the SA for the two segmentation algorithms; about 600 million cell nuclei were segmented per run. The results show that significance of the survival correlations of patient groups, assessed by a logrank test, are strongly affected by the segmentation parameter changes. This indicates that sensitivity analysis is an important tool for evaluating the stability of conclusions from image analyses.

The Transient Receptor Potential Vanilloid Type-2(TRPV2) Ion Channels in Neurogenesis andGliomagenesis: Cross-Talk between TranscriptionFactors and Signaling Molecules

Recently, the finding of cancer stem cells in brain tumors has increased the possibilities for advancing new therapeutic approaches with the aim to overcome the limits of current available treatments. In addition, a role for ion channels, particularly of TRP channels, in developing neurons as well as in brain cancer development and progression have been demonstrated. Herein, we focus on the latest advancements in understanding the role of TRPV2, a Ca²⁺ permeable channel belonging to the TRPV subfamily in neurogenesis and gliomagenesis. TRPV2 has been found to be expressed in both neural progenitor cells and glioblastoma stem/progenitor-like cells (GSCs). In developing neurons, post-translational modifications of TRPV2 (e.g., phosphorylation by ERK2) are required to stimulate Ca²⁺ signaling and nerve growth factor-mediated neurite outgrowth. TRPV2 overexpression also promotes GSC differentiation and reduces gliomagenesis in vitro and in vivo. In glioblastoma, TRPV2 inhibits survival and proliferation, and induces Fas/CD95-dependent apoptosis. Furthermore, by proteomic analysis, the identification of a TRPV2 interactome-based signature and its relation to glioblastoma progression/recurrence, high or low overall survival and drug resistance strongly suggest an important role of the TRPV2 channel as a potential biomarker in glioblastoma prognosis and therapy.

Machine Learning Models for Multiparametric Glioma Grading With Quantitative Result Interpretations

Gliomas are the most common primary malignant brain tumors in adults. Accurate grading is crucial as therapeutic strategies are often disparate for different grades and may influence patient prognosis. This study aims to provide an automated glioma grading platform on the basis of machine learning models. In this paper, we investigate contributions of multi-parameters from multimodal data including imaging parameters or features from the Whole Slide images (WSI) and the proliferation marker Ki-67 for automated brain tumor grading. For each WSI, we extract both visual parameters such as morphology parameters and sub-visual parameters including first-order and second-order features. On the basis of machine learning models, our platform classifies gliomas into grades II, III, and IV. Furthermore, we quantitatively interpret and reveal the important parameters contributing to grading with the Local Interpretable Model-Agnostic Explanations (LIME) algorithm. The quantitative analysis and explanation may assist clinicians to better understand the disease and accordingly to choose optimal treatments for improving clinical outcomes. The performance of our grading model was evaluated with cross-validation, which randomly divided the patients into non-overlapping training and testing sets and repeatedly validated the model on the different testing sets. The primary results indicated that this modular platform approach achieved the highest grading accuracy of 0.90 ± 0.04 with support vector machine (SVM) algorithm, with grading accuracies of 0.91 ± 0.08, 0.90 ± 0.08, and 0.90 ± 0.07 for grade II, III, and IV gliomas, respectively.

A Practical Guide to Whole Slide Imaging: A White Paper From the Digital Pathology Association

Context.—

Whole slide imaging (WSI) represents a paradigm shift in pathology, serving as a necessary first step for a wide array of digital tools to enter the field. Its basic function is to digitize glass slides, but its impact on pathology workflows, reproducibility, dissemination of educational material, expansion of service to underprivileged areas, and intrainstitutional and interinstitutional collaboration exemplifies a significant innovative movement with far-reaching effects. Although the benefits of WSI to pathology practices, academic centers, and research institutions are many, the complexities of implementation remain an obstacle to widespread adoption. In the wake of the first regulatory clearance of WSI for primary diagnosis in the United States, some barriers to adoption have fallen. Nevertheless, implementation of WSI remains a difficult prospect for many institutions, especially those with stakeholders unfamiliar with the technologies necessary to implement a system or who cannot effectively communicate to executive leadership and sponsors the benefits of a technology that may lack clear and immediate reimbursement opportunity.

Objectives.—

To present an overview of WSI technology—present and future—and to demonstrate several immediate applications of WSI that support pathology practice, medical education, research, and collaboration.

Data Sources.—

Peer-reviewed literature was reviewed by pathologists, scientists, and technologists who have practical knowledge of and experience with WSI.

Conclusions.—

Implementation of WSI is a multifaceted and inherently multidisciplinary endeavor requiring contributions from pathologists, technologists, and executive leadership. Improved understanding of the current challenges to implementation, as well as the benefits and successes of the technology, can help prospective users identify the best path for success.

Cooperative and out-of-core execution of the irregular wavefront propagation pattern on hybrid machines with IntelⓇ Xeon Phi™

The Irregular Wavefront Propagation Pattern (IWPP) is a core computing structure in several image analysis operations. Efficient implementation of IWPP on the Intel Xeon Phi is difficult because of the irregular data access and computation characteristics. The traditional IWPP algorithm relies on atomic instructions, which are not available in the SIMD set of the Intel Phi. To overcome this limitation, we have proposed a new IWPP algorithm that can take advantage of non-atomic SIMD instructions supported on the Intel Xeon Phi. We have also developed and evaluated methods to use CPU and Intel Phi cooperatively for parallel execution of the IWPP algorithms. Our new cooperative IWPP version is also able to handle large out-of-core images that would not fit into the memory of the accelerator. The new IWPP algorithm is used to implement the Morphological Reconstruction and Fill Holes operations, which are operations commonly found in image analysis applications. The vectorization implemented with the new IWPP has attained improvements of up to about 5× on top of the original IWPP and significant gains as compared to state-of-the-art the CPU and GPU versions. The new version running on an Intel Phi is 6.21× and 3.14× faster than running on a 16-core CPU and on a GPU, respectively. Finally, the cooperative execution using two Intel Phi devices and a multi-core CPU has reached performance gains of 2.14× as compared to the execution using a single Intel Xeon Phi.

Predicting cancer outcomes from histology and genomics using convolutional networks

Predicting the expected outcome of patients diagnosed with cancer is a critical step in treatment. Advances in genomic and imaging technologies provide physicians with vast amounts of data, yet prognostication remains largely subjective, leading to suboptimal clinical management. We developed a computational approach based on deep learning to predict the overall survival of patients diagnosed with brain tumors from microscopic images of tissue biopsies and genomic biomarkers. This method uses adaptive feedback to simultaneously learn the visual patterns and molecular biomarkers associated with patient outcomes. Our approach surpasses the prognostic accuracy of human experts using the current clinical standard for classifying brain tumors and presents an innovative approach for objective, accurate, and integrated prediction of patient outcomes.

Cancer histology reflects underlying molecular processes and disease progression and contains rich phenotypic information that is predictive of patient outcomes. In this study, we show a computational approach for learning patient outcomes from digital pathology images using deep learning to combine the power of adaptive machine learning algorithms with traditional survival models. We illustrate how these survival convolutional neural networks (SCNNs) can integrate information from both histology images and genomic biomarkers into a single unified framework to predict time-to-event outcomes and show prediction accuracy that surpasses the current clinical paradigm for predicting the overall survival of patients diagnosed with glioma. We use statistical sampling techniques to address challenges in learning survival from histology images, including tumor heterogeneity and the need for large training cohorts. We also provide insights into the prediction mechanisms of SCNNs, using heat map visualization to show that SCNNs recognize important structures, like microvascular proliferation, that are related to prognosis and that are used by pathologists in grading. These results highlight the emerging role of deep learning in precision medicine and suggest an expanding utility for computational analysis of histology in the future practice of pathology.

Localization of Diagnostically Relevant Regions of Interest in Whole Slide Images: a Comparative Study

Whole slide digital imaging technology enables researchers to study pathologists' interpretive behavior as they view digital slides and gain new understanding of the diagnostic medical decision-making process. In this study, we propose a simple yet important analysis to extract diagnostically relevant regions of interest (ROIs) from tracking records using only pathologists' actions as they viewed biopsy specimens in the whole slide digital imaging format (zooming, panning, and fixating). We use these extracted regions in a visual bag-of-words model based on color and texture features to predict diagnostically relevant ROIs on whole slide images. Using a logistic regression classifier in a cross-validation setting on 240 digital breast biopsy slides and viewport tracking logs of three expert pathologists, we produce probability maps that show 74 % overlap with the actual regions at which pathologists looked. We compare different bag-of-words models by changing dictionary size, visual word definition (patches vs. superpixels), and training data (automatically extracted ROIs vs. manually marked ROIs). This study is a first step in understanding the scanning behaviors of pathologists and the underlying reasons for diagnostic errors.

Informatics Approaches to Address New Challenges in the Classification of Lymphoid Malignancies

Purpose

Lymphoid malignancies are remarkably heterogeneous, with variations in outcomes and clinical, biologic, and histologic presentation complicating classification according to the World Health Organization guidelines. Incorrect classification of lymphoid neoplasms can result in suboptimal therapeutic strategies for individual patients and confound the interpretation of clinical trials involving personalized, class-based treatments. This review discusses the potential role of pathology informatics in improving the classification accuracy and objectivity for lymphoid malignancies.

Design

We identified peer-reviewed publications examining pathology informatics approaches for the classification of lymphoid malignancies, reviewed developments in the lymphoma classification systems, and summarized computational methods for pathologic assessment that can impact practice.

Results

Computer-assisted pathology image analysis algorithms in lymphoma most commonly have been applied to follicular lymphoma to address biologic heterogeneity and subjectivity in the process of classification.

Conclusion

Objective methods are available to assist pathologists in lymphoma classification and grading, and have been demonstrated to provide measurable benefits in specific contexts. Future validation and extension of these approaches will require datasets that link high resolution pathology images available for image analysis algorithms with clinical variables and follow up outcomes.

Integrative Analysis of Histopathological Images and Genomic Data Predicts Clear Cell Renal Cell Carcinoma Prognosis

In cancer, both histopathologic images and genomic signatures are used for diagnosis, prognosis, and subtyping. However, combining histopathologic images with genomic data for predicting prognosis, as well as the relationships between them, has rarely been explored. In this study, we present an integrative genomics framework for constructing a prognostic model for clear cell renal cell carcinoma. We used patient data from The Cancer Genome Atlas (n = 410), extracting hundreds of cellular morphologic features from digitized whole-slide images and eigengenes from functional genomics data to predict patient outcome. The risk index generated by our model correlated strongly with survival, outperforming predictions based on considering morphologic features or eigengenes separately. The predicted risk index also effectively stratified patients in early-stage (stage I and stage II) tumors, whereas no significant survival difference was observed using staging alone. The prognostic value of our model was independent of other known clinical and molecular prognostic factors for patients with clear cell renal cell carcinoma. Overall, this workflow and the shared software code provide building blocks for applying similar approaches in other cancers. Cancer Res; 77(21); e91-100. ©2017 AACR.

The Digital Slide Archive: A Software Platform for Management, Integration, and Analysis of Histology for Cancer Research

Tissue-based cancer studies can generate large amounts of histology data in the form of glass slides. These slides contain important diagnostic, prognostic, and biological information and can be digitized into expansive and high-resolution whole-slide images using slide-scanning devices. Effectively utilizing digital pathology data in cancer research requires the ability to manage, visualize, share, and perform quantitative analysis on these large amounts of image data, tasks that are often complex and difficult for investigators with the current state of commercial digital pathology software. In this article, we describe the Digital Slide Archive (DSA), an open-source web-based platform for digital pathology. DSA allows investigators to manage large collections of histologic images and integrate them with clinical and genomic metadata. The open-source model enables DSA to be extended to provide additional capabilities. Cancer Res; 77(21); e75-78. ©2017 AACR.

Interactive phenotyping of large-scale histology imaging data with HistomicsML

Whole-slide imaging of histologic sections captures tissue microenvironments and cytologic details in expansive high-resolution images. These images can be mined to extract quantitative features that describe tissues, yielding measurements for hundreds of millions of histologic objects. A central challenge in utilizing this data is enabling investigators to train and evaluate classification rules for identifying objects related to processes like angiogenesis or immune response. In this paper we describe HistomicsML, an interactive machine-learning system for digital pathology imaging datasets. This framework uses active learning to direct user feedback, making classifier training efficient and scalable in datasets containing 10⁸+ histologic objects. We demonstrate how this system can be used to phenotype microvascular structures in gliomas to predict survival, and to explore the molecular pathways associated with these phenotypes. Our approach enables researchers to unlock phenotypic information from digital pathology datasets to investigate prognostic image biomarkers and genotype-phenotype associations.

Parallel and Efficient Sensitivity Analysis of Microscopy Image Segmentation Workflows in Hybrid Systems

We investigate efficient sensitivity analysis (SA) of algorithms that segment and classify image features in a large dataset of high-resolution images. Algorithm SA is the process of evaluating variations of methods and parameter values to quantify differences in the output. A SA can be very compute demanding because it requires re-processing the input dataset several times with different parameters to assess variations in output. In this work, we introduce strategies to efficiently speed up SA via runtime optimizations targeting distributed hybrid systems and reuse of computations from runs with different parameters. We evaluate our approach using a cancer image analysis workflow on a hybrid cluster with 256 nodes, each with an Intel Phi and a dual socket CPU. The SA attained a parallel efficiency of over 90% on 256 nodes. The cooperative execution using the CPUs and the Phi available in each node with smart task assignment strategies resulted in an additional speedup of about 2×. Finally, multi-level computation reuse lead to an additional speedup of up to 2.46× on the parallel version. The level of performance attained with the proposed optimizations will allow the use of SA in large-scale studies.

A novel machine learning approach reveals latent vascular phenotypes predictive of renal cancer outcome

Gene expression signatures are commonly used as predictive biomarkers, but do not capture structural features within the tissue architecture. Here we apply a 2-step machine learning framework for quantitative imaging of tumor vasculature to derive a spatially informed, prognostic gene signature. The trained algorithms classify endothelial cells and generate a vascular area mask (VAM) in H&E micrographs of clear cell renal cell carcinoma (ccRCC) cases from The Cancer Genome Atlas (TCGA). Quantification of VAMs led to the discovery of 9 vascular features (9VF) that predicted disease-free-survival in a discovery cohort (n = 64, HR = 2.3). Correlation analysis and information gain identified a 14 gene expression signature related to the 9VF's. Two generalized linear models with elastic net regularization (14VF and 14GT), based on the 14 genes, separated independent cohorts of up to 301 cases into good and poor disease-free survival groups (14VF HR = 2.4, 14GT HR = 3.33). For the first time, we successfully applied digital image analysis and targeted machine learning to develop prognostic, morphology-based, gene expression signatures from the vascular architecture. This novel morphogenomic approach has the potential to improve previous methods for biomarker development.

Three-dimensional Imaging and Scanning: Current and Future Applications for Pathology

Imaging is vital for the assessment of physiologic and phenotypic details. In the past, biomedical imaging was heavily reliant on analog, low-throughput methods, which would produce two-dimensional images. However, newer, digital, and high-throughput three-dimensional (3D) imaging methods, which rely on computer vision and computer graphics, are transforming the way biomedical professionals practice. 3D imaging has been useful in diagnostic, prognostic, and therapeutic decision-making for the medical and biomedical professions. Herein, we summarize current imaging methods that enable optimal 3D histopathologic reconstruction: Scanning, 3D scanning, and whole slide imaging. Briefly mentioned are emerging platforms, which combine robotics, sectioning, and imaging in their pursuit to digitize and automate the entire microscopy workflow. Finally, both current and emerging 3D imaging methods are discussed in relation to current and future applications within the context of pathology.

Development of a Framework for Large Scale Three-Dimensional Pathology and Biomarker Imaging and Spatial Analytics

With the rapid advancement in large-throughput scanning technologies, digital pathology has emerged as platform with promise for diagnostic approaches, but also for high-throughput quantitative data extraction and analysis for translational research. Digital pathology and biomarker images are rich sources of information on tissue architecture, cell diversity and morphology, and molecular pathway activation. However, the understanding of disease in three-dimension (3D) has been hampered by their traditional two-dimension (2D) representations on histologic slides. In this paper, we propose a scalable image processing framework to quantitatively investigate 3D phenotypic and cell-specific molecular features from digital pathology and biomarker images in information- lossless 3D tissue space. We also develop a generalized 3D spatial data management framework with multi-level parallelism and provide a sustainable infrastructure for rapid spatial queries through scalable and efficient spatial data processing. The developed framework can facilitate biomedical research by efficiently processing large-scale, 3D pathology and in-situ biomarker imaging data.

Towards Generation, Management, and Exploration of Combined Radiomics and Pathomics Datasets for Cancer Research

Cancer is a complex multifactorial disease state and the ability to anticipate and steer treatment results will require information synthesis across multiple scales from the host to the molecular level. Radiomics and Pathomics, where image features are extracted from routine diagnostic Radiology and Pathology studies, are also evolving as valuable diagnostic and prognostic indicators in cancer. This information explosion provides new opportunities for integrated, multi-scale investigation of cancer, but also mandates a need to build systematic and integrated approaches to manage, query and mine combined Radiomics and Pathomics data. In this paper, we describe a suite of tools and web-based applications towards building a comprehensive framework to support the generation, management and interrogation of large volumes of Radiomics and Pathomics feature sets and the investigation of correlations between image features, molecular data, and clinical outcome.

AUTOMATED LEVEL SET SEGMENTATION OF HISTOPATHOLOGIC CELLS WITH SPARSE SHAPE PRIOR SUPPORT AND DYNAMIC OCCLUSION CONSTRAINT

In this paper, we propose a novel segmentation method for cells in histopathologic images based on a sparse shape prior guided variational level set framework. We automate the cell contour initialization by detecting seeds and deform contours by minimizing a new energy functional that incorporates a shape term involving sparse shape priors, an adaptive contour occlusion penalty term, and a boundary term encouraging contours to converge to strong edges. As a result, our approach is able to accommodate mutual occlusions and detect contours of multiple intersected cells. We apply our algorithm to a set of whole-slide histopathologic images of brain tumor sections. The proposed method is compared with other popular methods, and demonstrates good accuracy for cell segmentation by quantitative measures, suggesting its promise to support biomedical image-based investigations.