Automatic nuclei segmentation in H&E stained breast cancer histopathology images

Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study

BACKGROUND

For virtually every patient with colorectal cancer (CRC), hematoxylin-eosin (HE)-stained tissue slides are available. These images contain quantitative information, which is not routinely used to objectively extract prognostic biomarkers. In the present study, we investigated whether deep convolutional neural networks (CNNs) can extract prognosticators directly from these widely available images.

METHODS AND FINDINGS

We hand-delineated single-tissue regions in 86 CRC tissue slides, yielding more than 100,000 HE image patches, and used these to train a CNN by transfer learning, reaching a nine-class accuracy of >94% in an independent data set of 7,180 images from 25 CRC patients. With this tool, we performed automated tissue decomposition of representative multitissue HE images from 862 HE slides in 500 stage I-IV CRC patients in the The Cancer Genome Atlas (TCGA) cohort, a large international multicenter collection of CRC tissue. Based on the output neuron activations in the CNN, we calculated a "deep stroma score," which was an independent prognostic factor for overall survival (OS) in a multivariable Cox proportional hazard model (hazard ratio [HR] with 95% confidence interval [CI]: 1.99 [1.27-3.12], p = 0.0028), while in the same cohort, manual quantification of stromal areas and a gene expression signature of cancer-associated fibroblasts (CAFs) were only prognostic in specific tumor stages. We validated these findings in an independent cohort of 409 stage I-IV CRC patients from the "Darmkrebs: Chancen der Verhütung durch Screening" (DACHS) study who were recruited between 2003 and 2007 in multiple institutions in Germany. Again, the score was an independent prognostic factor for OS (HR 1.63 [1.14-2.33], p = 0.008), CRC-specific OS (HR 2.29 [1.5-3.48], p = 0.0004), and relapse-free survival (RFS; HR 1.92 [1.34-2.76], p = 0.0004). A prospective validation is required before this biomarker can be implemented in clinical workflows.

CONCLUSIONS

In our retrospective study, we show that a CNN can assess the human tumor microenvironment and predict prognosis directly from histopathological images.

Muscle specific kinase (MuSK) activation preserves neuromuscular junctions in the diaphragm but is not sufficient to provide a functional benefit in the SOD1G93A mouse model of ALS

Amyotrophic lateral sclerosis (ALS), a neurodegenerative disease affecting motor neurons, is characterized by rapid decline of motor function and ultimately respiratory failure. As motor neuron death occurs late in the disease, therapeutics that prevent the initial disassembly of the neuromuscular junction may offer optimal functional benefit and delay disease progression. To test this hypothesis, we treated the SOD1^G93A mouse model of ALS with an agonist antibody to muscle specific kinase (MuSK), a receptor tyrosine kinase required for the formation and maintenance of the neuromuscular junction. Chronic MuSK antibody treatment fully preserved innervation of the neuromuscular junction when compared with control-treated mice; however, no preservation of diaphragm function, motor neurons, or survival benefit was detected. These data show that anatomical preservation of neuromuscular junctions in the diaphragm via MuSK activation does not correlate with functional benefit in SOD1^G93A mice, suggesting caution in employing MuSK activation as a therapeutic strategy for ALS patients.

AlphaE Integrin Expression Is Increased in the Ileum Relative to the Colon and Unaffected by Inflammation

BACKGROUND

Recent findings suggest that αE expression is enriched on effector T cells and that intestinal αE+ T cells have increased expression of inflammatory cytokines. αE integrin expression is a potential predictive biomarker for response to etrolizumab, a monoclonal antibody against β7 integrin that targets both α4β7 and αEβ7. We evaluated the prevalence and localization of αE+ cells as well as total αE gene expression in healthy and inflammatory bowel disease patients.

METHODS

αE+ cells were identified in ileal and colonic biopsies by immunohistochemistry and counted using an automated algorithm. Gene expression was assessed by quantitative reverse-transcriptase polymerase chain reaction.

RESULTS

In both healthy and inflammatory bowel disease patients, significantly more αE+ cells were present in the epithelium and lamina propria of ileal compared with colonic biopsies. αE gene expression levels were also significantly higher in ileal compared with colonic biopsies. Paired biopsies from the same patient showed moderate correlation of αE expression between the ileum and colon. Inflammation did not affect αE expression, and neither endoscopy nor histology scores correlated with αE gene expression. αE expression was not different between patients based on concomitant medication use except 5-aminosalicylic acid.

CONCLUSION

αE+ cells, which have been shown to have inflammatory potential, are increased in the ileum in comparison with the colon in both Crohn's disease and ulcerative colitis, as well as in healthy subjects. In inflammatory bowel disease patients, αE levels are stable, regardless of inflammatory status or most concomitant medications, which could support its use as a biomarker for etrolizumab.

Nuclear shape and orientation features from H&E images predict survival in early-stage estrogen receptor-positive breast cancers

Early-stage estrogen receptor-positive (ER+) breast cancer (BCa) is the most common type of BCa in the United States. One critical question with these tumors is identifying which patients will receive added benefit from adjuvant chemotherapy. Nuclear pleomorphism (variance in nuclear shape and morphology) is an important constituent of breast grading schemes, and in ER+ cases, the grade is highly correlated with disease outcome. This study aimed to investigate whether quantitative computer-extracted image features of nuclear shape and orientation on digitized images of hematoxylin-stained and eosin-stained tissue of lymph node-negative (LN-), ER+ BCa could help stratify patients into discrete (<10 years short-term vs. >10 years long-term survival) outcome groups independent of standard clinical and pathological parameters. We considered a tissue microarray (TMA) cohort of 276 ER+, LN- patients comprising 150 patients with long-term and 126 patients with short-term overall survival, wherein 177 randomly chosen cases formed the modeling set, and 99 remaining cases the test set. Segmentation of individual nuclei was performed using multiresolution watershed; subsequently, 615 features relating to nuclear shape/texture and orientation disorder were extracted from each TMA spot. The Wilcoxon's rank-sum test identified the 15 most prognostic quantitative histomorphometric features within the modeling set. These features were then subsequently combined via a linear discriminant analysis classifier and evaluated on the test set to assign a probability of long-term vs. short-term disease-specific survival. In univariate survival analysis, patients identified by the image classifier as high risk had significantly poorer survival outcome: hazard ratio (95% confident interval) = 2.91(1.23-6.92), p = 0.02786. Multivariate analysis controlling for T-stage, histology grade, and nuclear grade showed the classifier to be independently predictive of poorer survival: hazard ratio (95% confident interval) = 3.17(0.33-30.46), p = 0.01039. Our results suggest that quantitative histomorphometric features of nuclear shape and orientation are strongly and independently predictive of patient survival in ER+, LN- BCa.

Fast unsupervised nuclear segmentation and classification scheme for automatic allred cancer scoring in immunohistochemical breast tissue images

BACKGROUND AND OBJECTIVE

This paper presents an improved scheme able to perform accurate segmentation and classification of cancer nuclei in immunohistochemical (IHC) breast tissue images in order to provide quantitative evaluation of estrogen or progesterone (ER/PR) receptor status that will assist pathologists in cancer diagnostic process.

METHODS

The proposed segmentation method is based on adaptive local thresholding and an enhanced morphological procedure, which are applied to extract all stained nuclei regions and to split overlapping nuclei. In fact, a new segmentation approach is presented here for cell nuclei detection from the IHC image using a modified Laplacian filter and an improved watershed algorithm. Stromal cells are then removed from the segmented image using an adaptive criterion in order to get fast tumor nuclei recognition. Finally, unsupervised classification of cancer nuclei is obtained by the combination of four common color separation techniques for a subsequent Allred cancer scoring.

RESULTS

Experimental results on various IHC tissue images of different cancer affected patients, demonstrate the effectiveness of the proposed scheme when compared to the manual scoring of pathological experts. A statistical analysis is performed on the whole image database between immuno-score of manual and automatic method, and compared with the scores that have reached using other state-of-art segmentation and classification strategies. According to the performance evaluation, we recorded more than 98% for both accuracy of detected nuclei and image cancer scoring over the truths provided by experienced pathologists which shows the best correlation with the expert's score (Pearson's correlation coefficient = 0.993, p-value < 0.005) and the lowest computational total time of 72.3 s/image (±1.9) compared to recent studied methods.

CONCLUSIONS

The proposed scheme can be easily applied for any histopathological diagnostic process that needs stained nuclear quantification and cancer grading. Moreover, the reduced processing time and manual interactions of our procedure can facilitate its implementation in a real-time device to construct a fully online evaluation system of IHC tissue images.

Comparative Study on Automated Cell Nuclei Segmentation Methods for Cytology Pleural Effusion Images

Automated cell nuclei segmentation is the most crucial step toward the implementation of a computer-aided diagnosis system for cancer cells. Studies on the automated analysis of cytology pleural effusion images are few because of the lack of reliable cell nuclei segmentation methods. Therefore, this paper presents a comparative study of twelve nuclei segmentation methods for cytology pleural effusion images. Each method involves three main steps: preprocessing, segmentation, and postprocessing. The preprocessing and segmentation stages help enhancing the image quality and extracting the nuclei regions from the rest of the image, respectively. The postprocessing stage helps in refining the segmented nuclei and removing false findings. The segmentation methods are quantitatively evaluated for 35 cytology images of pleural effusion by computing five performance metrics. The evaluation results show that the segmentation performances of the Otsu, k-means, mean shift, Chan-Vese, and graph cut methods are 94, 94, 95, 94, and 93%, respectively, with high abnormal nuclei detection rates. The average computational times per image are 1.08, 36.62, 50.18, 330, and 44.03 seconds, respectively. The findings of this study will be useful for current and potential future studies on cytology images of pleural effusion.

Fusion of encoder-decoder deep networks improves delineation of multiple nuclear phenotypes

BACKGROUND

Nuclear segmentation is an important step for profiling aberrant regions of histology sections. If nuclear segmentation can be resolved, then new biomarkers of nuclear phenotypes and their organization can be predicted for the application of precision medicine. However, segmentation is a complex problem as a result of variations in nuclear geometry (e.g., size, shape), nuclear type (e.g., epithelial, fibroblast), nuclear phenotypes (e.g., vesicular, aneuploidy), and overlapping nuclei. The problem is further complicated as a result of variations in sample preparation (e.g., fixation, staining). Our hypothesis is that (i) deep learning techniques can learn complex phenotypic signatures that rise in tumor sections, and (ii) fusion of different representations (e.g., regions, boundaries) contributes to improved nuclear segmentation.

RESULTS

We have demonstrated that training of deep encoder-decoder convolutional networks overcomes complexities associated with multiple nuclear phenotypes, where we evaluate alternative architecture of deep learning for an improved performance against the simplicity of the design. In addition, improved nuclear segmentation is achieved by color decomposition and combining region- and boundary-based features through a fusion network. The trained models have been evaluated against approximately 19,000 manually annotated nuclei, and object-level Precision, Recall, F1-score and Standard Error are reported with the best F1-score being 0.91. Raw training images, annotated images, processed images, and source codes are released as a part of the Additional file 1.

CONCLUSIONS

There are two intrinsic barriers in nuclear segmentation to H&E stained images, which correspond to the diversity of nuclear phenotypes and perceptual boundaries between adjacent cells. We demonstrate that (i) the encoder-decoder architecture can learn complex phenotypes that include the vesicular type; (ii) delineation of overlapping nuclei is enhanced by fusion of region- and edge-based networks; (iii) fusion of ENets produces an improved result over the fusion of UNets; and (iv) fusion of networks is better than multitask learning. We suggest that our protocol enables processing a large cohort of whole slide images for applications in precision medicine.

A Kinect-Based Segmentation of Touching-Pigs for Real-Time Monitoring

Segmenting touching-pigs in real-time is an important issue for surveillance cameras intended for the 24-h tracking of individual pigs. However, methods to do so have not yet been reported. We particularly focus on the segmentation of touching-pigs in a crowded pig room with low-contrast images obtained using a Kinect depth sensor. We reduce the execution time by combining object detection techniques based on a convolutional neural network (CNN) with image processing techniques instead of applying time-consuming operations, such as optimization-based segmentation. We first apply the fastest CNN-based object detection technique (i.e., You Only Look Once, YOLO) to solve the separation problem for touching-pigs. If the quality of the YOLO output is not satisfied, then we try to find the possible boundary line between the touching-pigs by analyzing the shape. Our experimental results show that this method is effective to separate touching-pigs in terms of both accuracy (i.e., 91.96%) and execution time (i.e., real-time execution), even with low-contrast images obtained using a Kinect depth sensor.

Emergence of Convolutional Neural Network in Future Medicine: Why and How. A Review on Brain Tumor Segmentation

Manual analysis of brain tumors magnetic resonance images is usually accompanied by some problem. Several techniques have been proposed for the brain tumor segmentation. This study will be focused on searching popular databases for related studies, theoretical and practical aspects of Convolutional Neural Network surveyed in brain tumor segmentation. Based on our findings, details about related studies including the datasets used, evaluation parameters, preferred architectures and complementary steps analyzed. Deep learning as a revolutionary idea in image processing, achieved brilliant results in brain tumor segmentation too. This can be continuing until the next revolutionary idea emerging.

CD8+ T cell infiltration in breast and colon cancer: A histologic and statistical analysis

The prevalence of cytotoxic tumor infiltrating lymphocytes (TILs) has demonstrated prognostic value in multiple tumor types. In particular, CD8 counts (in combination with CD3 and CD45RO) have been shown to be superior to traditional UICC staging in colon cancer patients and higher total CD8 counts have been associated with better survival in breast cancer patients. However, immune infiltrate heterogeneity can lead to potentially significant misrepresentations of marker prevalence in routine histologic sections. We examined step sections of breast and colorectal cancer samples for CD8+ T cell prevalence by standard chromogenic immunohistochemistry to determine marker variability and inform practice of T cell biomarker assessment in formalin-fixed, paraffin-embedded (FFPE) tissue samples. Stained sections were digitally imaged and CD8+ lymphocytes within defined regions of interest (ROI) including the tumor and surrounding stroma were enumerated. Statistical analyses of CD8+ cell count variability using a linear model/ANOVA framework between patients as well as between levels within a patient sample were performed. Our results show that CD8+ T-cell distribution is highly homogeneous within a standard tissue sample in both colorectal and breast carcinomas. As such, cytotoxic T cell prevalence by immunohistochemistry on a single level or even from a subsample of biopsy fragments taken from that level can be considered representative of cytotoxic T cell infiltration for the entire tumor section within the block. These findings support the technical validity of biomarker strategies relying on CD8 immunohistochemistry.

Tumor Cell Load and Heterogeneity Estimation From Diffusion-Weighted MRI Calibrated With Histological Data: an Example From Lung Cancer

Diffusion-weighted magnetic resonance imaging (DWI) is a key non-invasive imaging technique for cancer diagnosis and tumor treatment assessment, reflecting Brownian movement of water molecules in tissues. Since densely packed cells restrict molecule mobility, tumor tissues produce usually higher signal (a.k.a. less attenuated signal) on isotropic maps compared with normal tissues. However, no general quantitative relation between DWI data and the cell density has been established. In order to link low-resolution clinical cross-sectional data with high-resolution histological information, we developed an image processing and analysis chain, which was used to study the correlation between the diffusion coefficient (D value) estimated from DWI and tumor cellularity from serial histological slides of a resected non-small cell lung cancer tumor. Color deconvolution followed by cell nuclei segmentation was performed on digitized histological images to determine local and cell-type specific 2d (two-dimensional) densities. From these, the 3d cell density was inferred by a model-based sampling technique, which is necessary for the calculation of local and global 3d tumor cell count. Next, DWI sequence information was overlaid with high-resolution CT data and the resected histology using prominent anatomical hallmarks for co-registration of histology tissue blocks and non-invasive imaging modalities' data. The integration of cell numbers information and DWI data derived from different tumor areas revealed a clear negative correlation between cell density and D value. Importantly, spatial tumor cell density can be calculated based on DWI data. In summary, our results demonstrate that tumor cell count and heterogeneity can be predicted from DWI data, which may open new opportunities for personalized diagnosis and therapy optimization.

Segmentation of clustered cells in negative phase contrast images with integrated light intensity and cell shape information

Automated cell segmentation plays a key role in characterisations of cell behaviours for both biology research and clinical practices. Currently, the segmentation of clustered cells still remains as a challenge and is the main reason for false segmentation. In this study, the emphasis was put on the segmentation of clustered cells in negative phase contrast images. A new method was proposed to combine both light intensity and cell shape information through the construction of grey-weighted distance transform (GWDT) within preliminarily segmented areas. With the constructed GWDT, the clustered cells can be detected and then separated with a modified region skeleton-based method. Moreover, a contour expansion operation was applied to get optimised detection of cell boundaries. In this paper, the working principle and detailed procedure of the proposed method are described, followed by the evaluation of the method on clustered cell segmentation. Results show that the proposed method achieves an improved performance in clustered cell segmentation compared with other methods, with 85.8% and 97.16% accuracy rate for clustered cells and all cells, respectively.

An oral cavity squamous cell carcinoma quantitative histomorphometric-based image classifier of nuclear morphology can risk stratify patients for disease-specific survival

Oral cavity squamous cell carcinoma is the most common type of head and neck carcinoma. Its incidence is increasing worldwide, and it is associated with major morbidity and mortality. It is often unclear which patients have aggressive, treatment refractory tumors vs those whose tumors will be more responsive to treatment. Better identification of patients with high- vs low-risk cancers could help provide more tailored treatment approaches and could improve survival rates while decreasing treatment-related morbidity. This study investigates computer-extracted image features of nuclear shape and texture on digitized images of H&E-stained tissue sections for risk stratification of oral cavity squamous cell carcinoma patients compared with standard clinical and pathologic parameters. With a tissue microarray cohort of 115 retrospectively identified oral cavity squamous cell carcinoma patients, 50 were randomly chosen as the modeling set, and the remaining 65 constituted the test set. Following nuclear segmentation and feature extraction, the Wilcoxon rank sum test was used to identify the five most prognostic quantitative histomorphometric features from the modeling set. These top ranked features were then combined via a machine learning classifier to construct the oral cavity histomorphometric-based image classifier (OHbIC). The classifier was then validated for its ability to risk stratify patients for disease-specific outcomes on the test set. On the test set, the classifier yielded an area under the receiver operating characteristic curve of 0.72 in distinguishing disease-specific outcomes. In univariate survival analysis, high-risk patients predicted by the classifier had significantly poorer disease-specific survival (P=0.0335). In multivariate analysis controlling for T/N-stage, resection margins, and smoking status, positive classifier results were independently predictive of poorer disease-specific survival: hazard ratio (95% confidence interval)=11.023 (2.62-46.38) and P=0.001. Our results suggest that quantitative histomorphometric features of local nuclear architecture derived from digitized H&E slides of oral cavity squamous cell carcinomas are independently predictive of patient survival.

A novel overlapped nuclei splitting algorithm for histopathological images

BACKGROUND AND OBJECTIVE

Nuclei segmentation is a common process for quantitative analysis of histopathological images. However, this process generally results in overlapping of nuclei due to the nature of images, the sample preparation and staining, and image acquisition processes as well as insufficiency of 2D histopathological images to represent 3D characteristics of tissues. We present a novel algorithm to split overlapped nuclei.

METHODS

The histopathological images are initially segmented by K-Means segmentation algorithm. Then, nuclei cluster are converted to binary image. The overlapping is detected by applying threshold area value to nuclei in the binary image. The splitting algorithm is applied to the overlapped nuclei. In first stage of splitting, circles are drawn on overlapped nuclei. The radius of the circles is calculated by using circle area formula, and each pixel's coordinates of overlapped nuclei are selected as center coordinates for each circle. The pixels in the circle that contains maximum number of intersected pixels in both the circle and the overlapped nuclei are removed from the overlapped nuclei, and the filled circle labeled as a nucleus.

RESULTS

The algorithm has been tested on histopathological images of healthy and damaged kidney tissues and compared with the results provided by an expert and three related studies. The results demonstrated that the proposed splitting algorithm can segment the overlapping nuclei with accuracy of 84%.

CONCLUSIONS

The study presents a novel algorithm splitting the overlapped nuclei in histopathological images and provides more accurate cell counting in histopathological analysis. Furthermore, the proposed splitting algorithm has the potential to be used in different fields to split any overlapped circular patterns.

Prediction of recurrence in early stage non-small cell lung cancer using computer extracted nuclear features from digital H&E images

Identification of patients with early stage non-small cell lung cancer (NSCLC) with high risk of recurrence could help identify patients who would receive additional benefit from adjuvant therapy. In this work, we present a computational histomorphometric image classifier using nuclear orientation, texture, shape, and tumor architecture to predict disease recurrence in early stage NSCLC from digitized H&E tissue microarray (TMA) slides. Using a retrospective cohort of early stage NSCLC patients (Cohort #1, n = 70), we constructed a supervised classification model involving the most predictive features associated with disease recurrence. This model was then validated on two independent sets of early stage NSCLC patients, Cohort #2 (n = 119) and Cohort #3 (n = 116). The model yielded an accuracy of 81% for prediction of recurrence in the training Cohort #1, 82% and 75% in the validation Cohorts #2 and #3 respectively. A multivariable Cox proportional hazard model of Cohort #2, incorporating gender and traditional prognostic variables such as nodal status and stage indicated that the computer extracted histomorphometric score was an independent prognostic factor (hazard ratio = 20.81, 95% CI: 6.42–67.52, P < 0.001).

Cervical cancer histology image identification method based on texture and lesion area features

The issue of an automated approach for detecting cervical cancer is proposed to improve the accuracy of recognition. Firstly, the cervical cancer histology source images are needed to use image preprocessing for reducing the impact brought by noise of images as well as the impact on subsequent precise feature extraction brought by irrelevant background. Secondly, the images are grouped into ten vertical images and the information of texture feature is extracted by Grey Level Co-occurrence Matrix (GLCM). GLCM is an effective tool to analyze the features of texture. The textures of different diseases in the source image of Cervical Cancer Histology (such as contrast, correlation, entropy, uniformity and energy, etc.) can all be obtained in this way. Thirdly, the image is segmented by using K-means clustering and Marker-controlled watershed Algorithm. And each vertical image is divided into three layers to calculate the areas of different layers. Based on GLCM and lesion area features, the tissues are investigated with segmentation by using Support Vector Machine (SVM) method. Finally, the results show that it is effective and feasible to recognize cervical cancer by automated approach and verified by experiment.

Monitoring and Targeting Anti-VEGF Induced Hypoxia within the Viable Tumor by 19F-MRI and Multispectral Analysis

The effect of anti-angiogenic agents on tumor oxygenation has been in question for a number of years, where both increases and decreases in tumor pO₂ have been observed. This dichotomy in results may be explained by the role of vessel normalization in the response of tumors to anti-angiogenic therapy, where anti-angiogenic therapies may initially improve both the structure and the function of tumor vessels, but more sustained or potent anti-angiogenic treatments will produce an anti-vascular response, producing a more hypoxic environment. The first goal of this study was to employ multispectral (MS) ¹⁹F–MRI to noninvasively quantify viable tumor pO₂ and evaluate the ability of a high dose of an antibody to vascular endothelial growth factor (VEGF) to produce a strong and prolonged anti-vascular response that results in significant tumor hypoxia. The second goal of this study was to target the anti-VEGF induced hypoxic tumor micro-environment with an agent, tirapazamine (TPZ), which has been designed to target hypoxic regions of tumors. These goals have been successfully met, where an antibody that blocks both murine and human VEGF-A (B20.4.1.1) was found by MS ¹⁹F–MRI to produce a strong anti-vascular response and reduce viable tumor pO₂ in an HM-7 xenograft model. TPZ was then employed to target the anti-VEGF-induced hypoxic region. The combination of anti-VEGF and TPZ strongly suppressed HM-7 tumor growth and was superior to control and both monotherapies. This study provides evidence that clinical trials combining anti-vascular agents with hypoxia-activated prodrugs should be considered to improved efficacy in cancer patients.

Combined MEK and ERK inhibition overcomes therapy-mediated pathway reactivation in RAS mutant tumors

Mitogen-activated protein kinase (MAPK) pathway dysregulation is implicated in >30% of all cancers, rationalizing the development of RAF, MEK and ERK inhibitors. While BRAF and MEK inhibitors improve BRAF mutant melanoma patient outcomes, these inhibitors had limited success in other MAPK dysregulated tumors, with insufficient pathway suppression and likely pathway reactivation. In this study we show that inhibition of either MEK or ERK alone only transiently inhibits the MAPK pathway due to feedback reactivation. Simultaneous targeting of both MEK and ERK nodes results in deeper and more durable suppression of MAPK signaling that is not achievable with any dose of single agent, in tumors where feedback reactivation occurs. Strikingly, combined MEK and ERK inhibition is synergistic in RAS mutant models but only additive in BRAF mutant models where the RAF complex is dissociated from RAS and thus feedback productivity is disabled. We discovered that pathway reactivation in RAS mutant models occurs at the level of CRAF with combination treatment resulting in a markedly more active pool of CRAF. However, distinct from single node targeting, combining MEK and ERK inhibitor treatment effectively blocks the downstream signaling as assessed by transcriptional signatures and phospho-p90RSK. Importantly, these data reveal that MAPK pathway inhibitors whose activity is attenuated due to feedback reactivation can be rescued with sufficient inhibition by using a combination of MEK and ERK inhibitors. The MEK and ERK combination significantly suppresses MAPK pathway output and tumor growth in vivo to a greater extent than the maximum tolerated doses of single agents, and results in improved anti-tumor activity in multiple xenografts as well as in two Kras mutant genetically engineered mouse (GEM) models. Collectively, these data demonstrate that combined MEK and ERK inhibition is functionally unique, yielding greater than additive anti-tumor effects and elucidates a highly effective combination strategy in MAPK-dependent cancer, such as KRAS mutant tumors.

Automatic cellularity assessment from post-treated breast surgical specimens

Neoadjuvant treatment (NAT) of breast cancer (BCa) is an option for patients with the locally advanced disease. It has been compared with standard adjuvant therapy with the aim of improving prognosis and surgical outcome. Moreover, the response of the tumor to the therapy provides useful information for patient management. The pathological examination of the tissue sections after surgery is the gold-standard to estimate the residual tumor and the assessment of cellularity is an important component of tumor burden assessment. In the current clinical practice, tumor cellularity is manually estimated by pathologists on hematoxylin and eosin (H&E) stained slides, the quality, and reliability of which might be impaired by inter-observer variability which potentially affects prognostic power assessment in NAT trials. This procedure is also qualitative and time-consuming. In this paper, we describe a method of automatically assessing cellularity. A pipeline to automatically segment nuclei figures and estimate residual cancer cellularity from within patches and whole slide images (WSIs) of BCa was developed. We have compared the performance of our proposed pipeline in estimating residual cancer cellularity with that of two expert pathologists. We found an intra-class agreement coefficient (ICC) of 0.89 (95% CI of [0.70, 0.95]) between pathologists, 0.74 (95% CI of [0.70, 0.77]) between pathologist #1 and proposed method, and 0.75 (95% CI of [0.71, 0.79]) between pathologist #2 and proposed method. We have also successfully applied our proposed technique on a WSI to locate areas with high concentration of residual cancer. The main advantage of our approach is that it is fully automatic and can be used to find areas with high cellularity in WSIs. This provides a first step in developing an automatic technique for post-NAT tumor response assessment from pathology slides. © 2017 International Society for Advancement of Cytometry.

A Dataset and a Technique for Generalized Nuclear Segmentation for Computational Pathology

Nuclear segmentation in digital microscopic tissue images can enable extraction of high-quality features for nuclear morphometrics and other analysis in computational pathology. Conventional image processing techniques, such as Otsu thresholding and watershed segmentation, do not work effectively on challenging cases, such as chromatin-sparse and crowded nuclei. In contrast, machine learning-based segmentation can generalize across various nuclear appearances. However, training machine learning algorithms requires data sets of images, in which a vast number of nuclei have been annotated. Publicly accessible and annotated data sets, along with widely agreed upon metrics to compare techniques, have catalyzed tremendous innovation and progress on other image classification problems, particularly in object recognition. Inspired by their success, we introduce a large publicly accessible data set of hematoxylin and eosin (H&E)-stained tissue images with more than 21000 painstakingly annotated nuclear boundaries, whose quality was validated by a medical doctor. Because our data set is taken from multiple hospitals and includes a diversity of nuclear appearances from several patients, disease states, and organs, techniques trained on it are likely to generalize well and work right out-of-the-box on other H&E-stained images. We also propose a new metric to evaluate nuclear segmentation results that penalizes object- and pixel-level errors in a unified manner, unlike previous metrics that penalize only one type of error. We also propose a segmentation technique based on deep learning that lays a special emphasis on identifying the nuclear boundaries, including those between the touching or overlapping nuclei, and works well on a diverse set of test images.

αEβ7 Integrin Identifies Subsets of Pro-Inflammatory Colonic CD4+ T Lymphocytes in Ulcerative Colitis

BACKGROUND AND AIMS

The αEβ7 integrin is crucial for retention of T lymphocytes at mucosal surfaces through its interaction with E-cadherin. Pathogenic or protective functions of these cells during human intestinal inflammation, such as ulcerative colitis [UC], have not previously been defined, with understanding largely derived from animal model data. Defining this phenotype in human samples is important for understanding UC pathogenesis and is of translational importance for therapeutic targeting of αEβ7-E-cadherin interactions.

METHODS

αEβ7+ and αEβ7- colonic T cell localization, inflammatory cytokine production and expression of regulatory T cell-associated markers were evaluated in cohorts of control subjects and patients with active UC by immunohistochemistry, flow cytometry and real-time PCR of FACS-purified cell populations.

RESULTS

CD4+αEβ7+ T lymphocytes from both healthy controls and UC patients had lower expression of regulatory T cell-associated genes, including FOXP3, IL-10, CTLA-4 and ICOS in comparison with CD4+αEβ7- T lymphocytes. In UC, CD4+αEβ7+ lymphocytes expressed higher levels of IFNγ and TNFα in comparison with CD4+αEβ7- lymphocytes. Additionally the CD4+αEβ7+ subset was enriched for Th17 cells and the recently described Th17/Th1 subset co-expressing both IL-17A and IFNγ, both of which were found at higher frequencies in UC compared to control.

CONCLUSION

αEβ7 integrin expression on human colonic CD4+ T cells was associated with increased production of pro-inflammatory Th1, Th17 and Th17/Th1 cytokines, with reduced expression of regulatory T cell-associated markers. These data suggest colonic CD4+αEβ7+ T cells are pro-inflammatory and may play a role in UC pathobiology.

Dual leucine zipper kinase-dependent PERK activation contributes to neuronal degeneration following insult

The PKR-like endoplasmic reticulum kinase (PERK) arm of the Integrated Stress Response (ISR) is implicated in neurodegenerative disease, although the regulators and consequences of PERK activation following neuronal injury are poorly understood. Here we show that PERK signaling is a component of the mouse MAP kinase neuronal stress response controlled by the Dual Leucine Zipper Kinase (DLK) and contributes to DLK-mediated neurodegeneration. We find that DLK-activating insults ranging from nerve injury to neurotrophin deprivation result in both c-Jun N-terminal Kinase (JNK) signaling and the PERK- and ISR-dependent upregulation of the Activating Transcription Factor 4 (ATF4). Disruption of PERK signaling delays neurodegeneration without reducing JNK signaling. Furthermore, DLK is both sufficient for PERK activation and necessary for engaging the ISR subsequent to JNK-mediated retrograde injury signaling. These findings identify DLK as a central regulator of not only JNK but also PERK stress signaling in neurons, with both pathways contributing to neurodegeneration.

Metastasis detection from whole slide images using local features and random forests

Digital pathology has led to a demand for automated detection of regions of interest, such as cancerous tissue, from scanned whole slide images. With accurate methods using image analysis and machine learning, significant speed-up, and savings in costs through increased throughput in histological assessment could be achieved. This article describes a machine learning approach for detection of cancerous tissue from scanned whole slide images. Our method is based on feature engineering and supervised learning with a random forest model. The features extracted from the whole slide images include several local descriptors related to image texture, spatial structure, and distribution of nuclei. The method was evaluated in breast cancer metastasis detection from lymph node samples. Our results show that the method detects metastatic areas with high accuracy (AUC = 0.97-0.98 for tumor detection within whole image area, AUC = 0.84-0.91 for tumor vs. normal tissue detection) and that the method generalizes well for images from more than one laboratory. Further, the method outputs an interpretable classification model, enabling the linking of individual features to differences between tissue types. © 2017 International Society for Advancement of Cytometry.

Automatic extraction of cell nuclei from H&E-stained histopathological images

Extraction of cell nuclei from hematoxylin and eosin (H&E)-stained histopathological images is an essential preprocessing step in computerized image analysis for disease detection, diagnosis, and prognosis. We present an automated cell nuclei segmentation approach that works with H&E-stained images. A color deconvolution algorithm was first applied to the image to get the hematoxylin channel. Using a morphological operation and thresholding technique on the hematoxylin channel image, candidate target nuclei and background regions were detected, which were then used as markers for a marker-controlled watershed transform segmentation algorithm. Moreover, postprocessing was conducted to split the touching nuclei. For each segmented region from the previous steps, the regional maximum value positions were identified as potential nuclei centers. These maximum values were further grouped into [Formula: see text]-clusters, and the locations within each cluster were connected with the minimum spanning tree technique. Then, these connected positions were utilized as new markers for a watershed segmentation approach. The final number of nuclei at each region was determined by minimizing an objective function that iterated all of the possible [Formula: see text]-values. The proposed method was applied to the pathological images of the tumor tissues from The Cancer Genome Atlas study. Experimental results show that the proposed method can lead to promising results in terms of segmentation accuracy and separation of touching nuclei.

Computer-aided prognosis on breast cancer with hematoxylin and eosin histopathology images: A review

With the advance of digital pathology, image analysis has begun to show its advantages in information analysis of hematoxylin and eosin histopathology images. Generally, histological features in hematoxylin and eosin images are measured to evaluate tumor grade and prognosis for breast cancer. This review summarized recent works in image analysis of hematoxylin and eosin histopathology images for breast cancer prognosis. First, prognostic factors for breast cancer based on hematoxylin and eosin histopathology images were summarized. Then, usual procedures of image analysis for breast cancer prognosis were systematically reviewed, including image acquisition, image preprocessing, image detection and segmentation, and feature extraction. Finally, the prognostic value of image features and image feature–based prognostic models was evaluated. Moreover, we discussed the issues of current analysis, and some directions for future research.

Transcription factor Etv5 is essential for the maintenance of alveolar type II cells

Alveolar type II (AT2) cell dysfunction contributes to a number of significant human pathologies including respiratory distress syndrome, lung adenocarcinoma, and debilitating fibrotic diseases, but the critical transcription factors that maintain AT2 cell identity are unknown. Here we show that the E26 transformation-specific (ETS) family transcription factor Etv5 is essential to maintain AT2 cell identity. Deletion of Etv5 from AT2 cells produced gene and protein signatures characteristic of differentiated alveolar type I (AT1) cells. Consistent with a defect in the AT2 stem cell population, Etv5 deficiency markedly reduced recovery following bleomycin-induced lung injury. Lung tumorigenesis driven by mutant KrasG12D was also compromised by Etv5 deficiency. ERK activation downstream of Ras was found to stabilize Etv5 through inactivation of the cullin-RING ubiquitin ligase CRL4^COP1/DET1 that targets Etv5 for proteasomal degradation. These findings identify Etv5 as a critical output of Ras signaling in AT2 cells, contributing to both lung homeostasis and tumor initiation.

Gland Instance Segmentation Using Deep Multichannel Neural Networks

OBJECTIVE

A new image instance segmentation method is proposed to segment individual glands (instances) in colon histology images. This process is challenging since the glands not only need to be segmented from a complex background, they must also be individually identified.

METHODS

We leverage the idea of image-to-image prediction in recent deep learning by designing an algorithm that automatically exploits and fuses complex multichannel information-regional, location, and boundary cues-in gland histology images. Our proposed algorithm, a deep multichannel framework, alleviates heavy feature design due to the use of convolutional neural networks and is able to meet multifarious requirements by altering channels.

RESULTS

Compared with methods reported in the 2015 MICCAI Gland Segmentation Challenge and other currently prevalent instance segmentation methods, we observe state-of-the-art results based on the evaluation metrics.

CONCLUSION

The proposed deep multichannel algorithm is an effective method for gland instance segmentation.

SIGNIFICANCE

The generalization ability of our model not only enable the algorithm to solve gland instance segmentation problems, but the channel is also alternative that can be replaced for a specific task.

Automatic classification of cancer cells in multispectral microscopic images of lymph node samples

Histopathological analysis is crucial for the diagnosis of a large number of cancer types. A lot of progress has been made in the development of molecular based assays, but many of the cases still require the careful analysis of the stained tissue under a bright-field microscope and its analysis. This procedure is costly and time-consuming. We present a novel method for classification of cancer cells in lymph node images. It is based on the measurement of the spectral image of hematoxylin and eosin stained sample under the microscope and the analysis of the acquired data using state of the art machine learning techniques. The method is based on the analysis of the spectral information of the cells as well as their morphological properties. A large number of descriptors is extracted for each cell location, which are used to train a supervised classifier which discriminates between normal and cancer cells. We show that a reliable analysis can be made with detection rate (recall) of 81%-100% for the cancer class.

Evaluation of nucleus segmentation in digital pathology images through large scale image synthesis

Digital histopathology images with more than 1 Gigapixel are drawing more and more attention in clinical, biomedical research, and computer vision fields. Among the multiple observable features spanning multiple scales in the pathology images, the nuclear morphology is one of the central criteria for diagnosis and grading. As a result it is also the mostly studied target in image computing. Large amount of research papers have devoted to the problem of extracting nuclei from digital pathology images, which is the foundation of any further correlation study. However, the validation and evaluation of nucleus extraction have yet been formulated rigorously and systematically. Some researches report a human verified segmentation with thousands of nuclei, whereas a single whole slide image may contain up to million. The main obstacle lies in the difficulty of obtaining such a large number of validated nuclei, which is essentially an impossible task for pathologist. We propose a systematic validation and evaluation approach based on large scale image synthesis. This could facilitate a more quantitatively validated study for current and future histopathology image analysis field.

Enhancements in localized classification for uterine cervical cancer digital histology image assessment

Background:

In previous research, we introduced an automated, localized, fusion-based approach for classifying uterine cervix squamous epithelium into Normal, CIN1, CIN2, and CIN3 grades of cervical intraepithelial neoplasia (CIN) based on digitized histology image analysis. As part of the CIN assessment process, acellular and atypical cell concentration features were computed from vertical segment partitions of the epithelium region to quantize the relative distribution of nuclei.

Methods:

Feature data was extracted from 610 individual segments from 61 images for epithelium classification into categories of Normal, CIN1, CIN2, and CIN3. The classification results were compared against CIN labels obtained from two pathologists who visually assessed abnormality in the digitized histology images. In this study, individual vertical segment CIN classification accuracy improvement is reported using the logistic regression classifier for an expanded data set of 118 histology images.

Results:

We analyzed the effects on classification using the same pathologist labels for training and testing versus using one pathologist labels for training and the other for testing. Based on a leave-one-out approach for classifier training and testing, exact grade CIN accuracies of 81.29% and 88.98% were achieved for individual vertical segment and epithelium whole-image classification, respectively.

Conclusions:

The Logistic and Random Tree classifiers outperformed the benchmark SVM and LDA classifiers from previous research. The Logistic Regression classifier yielded an improvement of 10.17% in CIN Exact grade classification results based on CIN labels for training-testing for the individual vertical segments and the whole image from the same single expert over the baseline approach using the reduced features. Overall, the CIN classification rates tended to be higher using the training-testing labels for the same expert than for training labels from one expert and testing labels from the other expert. The Exact class fusion- based CIN discrimination results obtained in this study are similar to the Exact class expert agreement rate.

Computer-based image analysis in breast pathology

Whole slide imaging (WSI) has the potential to be utilized in telepathology, teleconsultation, quality assurance, clinical education, and digital image analysis to aid pathologists. In this paper, the potential added benefits of computer-assisted image analysis in breast pathology are reviewed and discussed. One of the major advantages of WSI systems is the possibility of doing computer-based image analysis on the digital slides. The purpose of computer-assisted analysis of breast virtual slides can be (i) segmentation of desired regions or objects such as diagnostically relevant areas, epithelial nuclei, lymphocyte cells, tubules, and mitotic figures, (ii) classification of breast slides based on breast cancer (BCa) grades, the invasive potential of tumors, or cancer subtypes, (iii) prognosis of BCa, or (iv) immunohistochemical quantification. While encouraging results have been achieved in this area, further progress is still required to make computer-based image analysis of breast virtual slides acceptable for clinical practice.

Multi-Pass Adaptive Voting for Nuclei Detection in Histopathological Images

Nuclei detection is often a critical initial step in the development of computer aided diagnosis and prognosis schemes in the context of digital pathology images. While over the last few years, a number of nuclei detection methods have been proposed, most of these approaches make idealistic assumptions about the staining quality of the tissue. In this paper, we present a new Multi-Pass Adaptive Voting (MPAV) for nuclei detection which is specifically geared towards images with poor quality staining and noise on account of tissue preparation artifacts. The MPAV utilizes the symmetric property of nuclear boundary and adaptively selects gradient from edge fragments to perform voting for a potential nucleus location. The MPAV was evaluated in three cohorts with different staining methods: Hematoxylin &Eosin, CD31 &Hematoxylin, and Ki-67 and where most of the nuclei were unevenly and imprecisely stained. Across a total of 47 images and nearly 17,700 manually labeled nuclei serving as the ground truth, MPAV was able to achieve a superior performance, with an area under the precision-recall curve (AUC) of 0.73. Additionally, MPAV also outperformed three state-of-the-art nuclei detection methods, a single pass voting method, a multi-pass voting method, and a deep learning based method.

Automated Segmentation of Nuclei in Breast Cancer Histopathology Images

The process of Nuclei detection in high-grade breast cancer images is quite challenging in the case of image processing techniques due to certain heterogeneous characteristics of cancer nuclei such as enlarged and irregularly shaped nuclei, highly coarse chromatin marginalized to the nuclei periphery and visible nucleoli. Recent reviews state that existing techniques show appreciable segmentation accuracy on breast histopathology images whose nuclei are dispersed and regular in texture and shape; however, typical cancer nuclei are often clustered and have irregular texture and shape properties. This paper proposes a novel segmentation algorithm for detecting individual nuclei from Hematoxylin and Eosin (H&E) stained breast histopathology images. This detection framework estimates a nuclei saliency map using tensor voting followed by boundary extraction of the nuclei on the saliency map using a Loopy Back Propagation (LBP) algorithm on a Markov Random Field (MRF). The method was tested on both whole-slide images and frames of breast cancer histopathology images. Experimental results demonstrate high segmentation performance with efficient precision, recall and dice-coefficient rates, upon testing high-grade breast cancer images containing several thousand nuclei. In addition to the optimal performance on the highly complex images presented in this paper, this method also gave appreciable results in comparison with two recently published methods-Wienert et al. (2012) and Veta et al. (2013), which were tested using their own datasets.

A novel method for morphological pleomorphism and heterogeneity quantitative measurement: Named cell feature level co-occurrence matrix

Background:

Recent developments in molecular pathology and genetic/epigenetic analysis of cancer tissue have resulted in a marked increase in objective and measurable data. In comparison, the traditional morphological analysis approach to pathology diagnosis, which can connect these molecular data and clinical diagnosis, is still mostly subjective. Even though the advent and popularization of digital pathology has provided a boost to computer-aided diagnosis, some important pathological concepts still remain largely non-quantitative and their associated data measurements depend on the pathologist's sense and experience. Such features include pleomorphism and heterogeneity.

Methods and Results:

In this paper, we propose a method for the objective measurement of pleomorphism and heterogeneity, using the cell-level co-occurrence matrix. Our method is based on the widely used Gray-level co-occurrence matrix (GLCM), where relations between neighboring pixel intensity levels are captured into a co-occurrence matrix, followed by the application of analysis functions such as Haralick features. In the pathological tissue image, through image processing techniques, each nucleus can be measured and each nucleus has its own measureable features like nucleus size, roundness, contour length, intra-nucleus texture data (GLCM is one of the methods). In GLCM each nucleus in the tissue image corresponds to one pixel. In this approach the most important point is how to define the neighborhood of each nucleus. We define three types of neighborhoods of a nucleus, then create the co-occurrence matrix and apply Haralick feature functions. In each image pleomorphism and heterogeneity are then determined quantitatively. For our method, one pixel corresponds to one nucleus feature, and we therefore named our method Cell Feature Level Co-occurrence Matrix (CFLCM). We tested this method for several nucleus features.

Conclusion:

CFLCM is showed as a useful quantitative method for pleomorphism and heterogeneity on histopathological image analysis.

Robust Cell Detection of Histopathological Brain Tumor Images Using Sparse Reconstruction and Adaptive Dictionary Selection

Successful diagnostic and prognostic stratification, treatment outcome prediction, and therapy planning depend on reproducible and accurate pathology analysis. Computer aided diagnosis (CAD) is a useful tool to help doctors make better decisions in cancer diagnosis and treatment. Accurate cell detection is often an essential prerequisite for subsequent cellular analysis. The major challenge of robust brain tumor nuclei/cell detection is to handle significant variations in cell appearance and to split touching cells. In this paper, we present an automatic cell detection framework using sparse reconstruction and adaptive dictionary learning. The main contributions of our method are: 1) A sparse reconstruction based approach to split touching cells; 2) An adaptive dictionary learning method used to handle cell appearance variations. The proposed method has been extensively tested on a data set with more than 2000 cells extracted from 32 whole slide scanned images. The automatic cell detection results are compared with the manually annotated ground truth and other state-of-the-art cell detection algorithms. The proposed method achieves the best cell detection accuracy with a F1 score = 0.96.

Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images

Detection and classification of cell nuclei in histopathology images of cancerous tissue stained with the standard hematoxylin and eosin stain is a challenging task due to cellular heterogeneity. Deep learning approaches have been shown to produce encouraging results on histopathology images in various studies. In this paper, we propose a Spatially Constrained Convolutional Neural Network (SC-CNN) to perform nucleus detection. SC-CNN regresses the likelihood of a pixel being the center of a nucleus, where high probability values are spatially constrained to locate in the vicinity of the centers of nuclei. For classification of nuclei, we propose a novel Neighboring Ensemble Predictor (NEP) coupled with CNN to more accurately predict the class label of detected cell nuclei. The proposed approaches for detection and classification do not require segmentation of nuclei. We have evaluated them on a large dataset of colorectal adenocarcinoma images, consisting of more than 20,000 annotated nuclei belonging to four different classes. Our results show that the joint detection and classification of the proposed SC-CNN and NEP produces the highest average F1 score as compared to other recently published approaches. Prospectively, the proposed methods could offer benefit to pathology practice in terms of quantitative analysis of tissue constituents in whole-slide images, and potentially lead to a better understanding of cancer.