DEEP LEARNING-BASED ASSESSMENT OF TUMOR-ASSOCIATED STROMA FOR DIAGNOSING BREAST CANCER IN HISTOPATHOLOGY IMAGES

CNN-based deep learning approach for classification of invasive ductal and metastasis types of breast carcinoma

OBJECTIVE

Breast cancer is one of the leading cancer causes among women worldwide. It can be classified as invasive ductal carcinoma (IDC) or metastatic cancer. Early detection of breast cancer is challenging due to the lack of early warning signs. Generally, a mammogram is recommended by specialists for screening. Existing approaches are not accurate enough for real-time diagnostic applications and thus require better and smarter cancer diagnostic approaches. This study aims to develop a customized machine-learning framework that will give more accurate predictions for IDC and metastasis cancer classification.

METHODS

This work proposes a convolutional neural network (CNN) model for classifying IDC and metastatic breast cancer. The study utilized a large-scale dataset of microscopic histopathological images to automatically perceive a hierarchical manner of learning and understanding.

RESULTS

It is evident that using machine learning techniques significantly (15%-25%) boost the effectiveness of determining cancer vulnerability, malignancy, and demise. The results demonstrate an excellent performance ensuring an average of 95% accuracy in classifying metastatic cells against benign ones and 89% accuracy was obtained in terms of detecting IDC.

CONCLUSIONS

The results suggest that the proposed model improves classification accuracy. Therefore, it could be applied effectively in classifying IDC and metastatic cancer in comparison to other state-of-the-art models.

Biological insights and novel biomarker discovery through deep learning approaches in breast cancer histopathology

Breast cancer remains a highly prevalent disease with considerable inter- and intra-tumoral heterogeneity complicating prognostication and treatment decisions. The utilization and depth of genomic, transcriptomic and proteomic data for cancer has exploded over recent times and the addition of spatial context to this information, by understanding the correlating morphologic and spatial patterns of cells in tissue samples, has created an exciting frontier of research, histo-genomics. At the same time, deep learning (DL), a class of machine learning algorithms employing artificial neural networks, has rapidly progressed in the last decade with a confluence of technical developments - including the advent of modern graphic processing units (GPU), allowing efficient implementation of increasingly complex architectures at scale; advances in the theoretical and practical design of network architectures; and access to larger datasets for training - all leading to sweeping advances in image classification and object detection. In this review, we examine recent developments in the application of DL in breast cancer histology with particular emphasis of those producing biologic insights or novel biomarkers, spanning the extraction of genomic information to the use of stroma to predict cancer recurrence, with the aim of suggesting avenues for further advancing this exciting field.

Applicability analysis of immunotherapy for lung cancer patients based on deep learning

According to global and Chinese cancer statistics, lung cancer is the second most common cancer globally with the highest mortality rate and a severe threat to human life and health. In recent years, immunotherapy has made significant breakthroughs in the treatment of cancer patients. However, only 30% of patients are applicable and may have immune-related adverse events. The traditional immunological inspection methods have limitations and often can not obtain the expected benefits. Deep learning is a typical representation learning method that can spontaneously mine the hidden feature of effective classification from seas of data. In order to alleviate medical resources and reduce costs, this paper proposes a deep learning-based method to predict patients best suited for immune checkpoint blocking therapy from patients CT images. The deep immunotherapy analysis method proposed in this paper is divided into three steps:(1) Using LUNA16 public dataset to develop a deep learning model for nodule detection. (2) Nodule detection was performed on the Anti-PD-1_Lung dataset, and the effectiveness of immunotherapy was determined by comparing the detection results of nodules before and after immunotherapy. (3) After the data set was processed, the deep learning method trained and analyzed the Lung images. According to the experimental results and comparative analysis, the proposed deep immunotherapy analysis method has a good performance in the detection of nodules. It works for the predictions for the applicability of immunotherapy for lung cancer.¹.

A Review of Computer-Aided Expert Systems for Breast Cancer Diagnosis

A computer-aided diagnosis (CAD) expert system is a powerful tool to efficiently assist a pathologist in achieving an early diagnosis of breast cancer. This process identifies the presence of cancer in breast tissue samples and the distinct type of cancer stages. In a standard CAD system, the main process involves image pre-processing, segmentation, feature extraction, feature selection, classification, and performance evaluation. In this review paper, we reviewed the existing state-of-the-art machine learning approaches applied at each stage involving conventional methods and deep learning methods, the comparisons within methods, and we provide technical details with advantages and disadvantages. The aims are to investigate the impact of CAD systems using histopathology images, investigate deep learning methods that outperform conventional methods, and provide a summary for future researchers to analyse and improve the existing techniques used. Lastly, we will discuss the research gaps of existing machine learning approaches for implementation and propose future direction guidelines for upcoming researchers.

Machine Learning Methods for Histopathological Image Analysis: A Review

Histopathological images (HIs) are the gold standard for evaluating some types of tumors for cancer diagnosis. The analysis of such images is time and resource-consuming and very challenging even for experienced pathologists, resulting in inter-observer and intra-observer disagreements. One of the ways of accelerating such an analysis is to use computer-aided diagnosis (CAD) systems. This paper presents a review on machine learning methods for histopathological image analysis, including shallow and deep learning methods. We also cover the most common tasks in HI analysis, such as segmentation and feature extraction. Besides, we present a list of publicly available and private datasets that have been used in HI research.

Deep Learning Image Analysis of Benign Breast Disease to Identify Subsequent Risk of Breast Cancer

Background

New biomarkers of risk may improve breast cancer (BC) risk prediction. We developed a computational pathology method to segment benign breast disease (BBD) whole slide images into epithelium, fibrous stroma, and fat. We applied our method to the BBD BC nested case-control study within the Nurses' Health Studies to assess whether computer-derived tissue composition or a morphometric signature was associated with subsequent risk of BC.

Methods

Tissue segmentation and nuclei detection deep-learning networks were established and applied to 3795 whole slide images from 293 cases who developed BC and 1132 controls who did not. Percentages of each tissue region were calculated, and 615 morphometric features were extracted. Elastic net regression was used to create a BC morphometric signature. Associations between BC risk factors and age-adjusted tissue composition among controls were assessed using analysis of covariance. Unconditional logistic regression, adjusting for the matching factors, BBD histological subtypes, parity, menopausal status, and body mass index evaluated the relationship between tissue composition and BC risk. All statistical tests were 2-sided.

Results

Among controls, direction of associations between BBD subtypes, parity, and number of births with breast composition varied by tissue region; select regions were associated with childhood body size, body mass index, age of menarche, and menopausal status (all P < .05). A higher proportion of epithelial tissue was associated with increased BC risk (odds ratio = 1.39, 95% confidence interval = 0.91 to 2.14, for highest vs lowest quartiles, P trend  = .047). No morphometric signature was associated with BC.

Conclusions

The amount of epithelial tissue may be incorporated into risk assessment models to improve BC risk prediction.

Developing image analysis pipelines of whole-slide images: Pre- and post-processing

Deep learning has pushed the scope of digital pathology beyond simple digitization and telemedicine. The incorporation of these algorithms in routine workflow is on the horizon and maybe a disruptive technology, reducing processing time, and increasing detection of anomalies. While the newest computational methods enjoy much of the press, incorporating deep learning into standard laboratory workflow requires many more steps than simply training and testing a model. Image analysis using deep learning methods often requires substantial pre- and post-processing order to improve interpretation and prediction. Similar to any data processing pipeline, images must be prepared for modeling and the resultant predictions need further processing for interpretation. Examples include artifact detection, color normalization, image subsampling or tiling, removal of errant predictions, etc. Once processed, predictions are complicated by image file size - typically several gigabytes when unpacked. This forces images to be tiled, meaning that a series of subsamples from the whole-slide image (WSI) are used in modeling. Herein, we review many of these methods as they pertain to the analysis of biopsy slides and discuss the multitude of unique issues that are part of the analysis of very large images.

Tabu Search and Machine-Learning Classification of Benign and Malignant Proliferative Breast Lesions

Breast cancer is the most diagnosed cancer among women around the world. The development of computer-aided diagnosis tools is essential to help pathologists to accurately interpret and discriminate between malignant and benign tumors. This paper proposes the development of an automated proliferative breast lesion diagnosis based on machine-learning algorithms. We used Tabu search to select the most significant features. The evaluation of the feature is based on the dependency degree of each attribute in the rough set. The categorization of reduced features was built using five machine-learning algorithms. The proposed models were applied to the BIDMC-MGH and Wisconsin Diagnostic Breast Cancer datasets. The performance measures of the used models were evaluated owing to five criteria. The top performing models were AdaBoost and logistic regression. Comparisons with others works prove the efficiency of the proposed method for superior diagnosis of breast cancer against the reviewed classification techniques.

Deep learning with multimodal representation for pancancer prognosis prediction

MOTIVATION

Estimating the future course of patients with cancer lesions is invaluable to physicians; however, current clinical methods fail to effectively use the vast amount of multimodal data that is available for cancer patients. To tackle this problem, we constructed a multimodal neural network-based model to predict the survival of patients for 20 different cancer types using clinical data, mRNA expression data, microRNA expression data and histopathology whole slide images (WSIs). We developed an unsupervised encoder to compress these four data modalities into a single feature vector for each patient, handling missing data through a resilient, multimodal dropout method. Encoding methods were tailored to each data type-using deep highway networks to extract features from clinical and genomic data, and convolutional neural networks to extract features from WSIs.

RESULTS

We used pancancer data to train these feature encodings and predict single cancer and pancancer overall survival, achieving a C-index of 0.78 overall. This work shows that it is possible to build a pancancer model for prognosis that also predicts prognosis in single cancer sites. Furthermore, our model handles multiple data modalities, efficiently analyzes WSIs and represents patient multimodal data flexibly into an unsupervised, informative representation. We thus present a powerful automated tool to accurately determine prognosis, a key step towards personalized treatment for cancer patients.

AVAILABILITY AND IMPLEMENTATION

https://github.com/gevaertlab/MultimodalPrognosis.

Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer

For prostate cancer patients, the Gleason score is one of the most important prognostic factors, potentially determining treatment independent of the stage. However, Gleason scoring is based on subjective microscopic examination of tumor morphology and suffers from poor reproducibility. Here we present a deep learning system (DLS) for Gleason scoring whole-slide images of prostatectomies. Our system was developed using 112 million pathologist-annotated image patches from 1226 slides, and evaluated on an independent validation dataset of 331 slides. Compared to a reference standard provided by genitourinary pathology experts, the mean accuracy among 29 general pathologists was 0.61 on the validation set. The DLS achieved a significantly higher diagnostic accuracy of 0.70 (p = 0.002) and trended towards better patient risk stratification in correlations to clinical follow-up data. Our approach could improve the accuracy of Gleason scoring and subsequent therapy decisions, particularly where specialist expertise is unavailable. The DLS also goes beyond the current Gleason system to more finely characterize and quantitate tumor morphology, providing opportunities for refinement of the Gleason system itself.

Assessment of Machine Learning Detection of Environmental Enteropathy and Celiac Disease in Children

Importance

Duodenal biopsies from children with enteropathies associated with undernutrition, such as environmental enteropathy (EE) and celiac disease (CD), display significant histopathological overlap.

Objective

To develop a convolutional neural network (CNN) to enhance the detection of pathologic morphological features in diseased vs healthy duodenal tissue.

Design, Setting, and Participants

In this prospective diagnostic study, a CNN consisting of 4 convolutions, 1 fully connected layer, and 1 softmax layer was trained on duodenal biopsy images. Data were provided by 3 sites: Aga Khan University Hospital, Karachi, Pakistan; University Teaching Hospital, Lusaka, Zambia; and University of Virginia, Charlottesville. Duodenal biopsy slides from 102 children (10 with EE from Aga Khan University Hospital, 16 with EE from University Teaching Hospital, 34 with CD from University of Virginia, and 42 with no disease from University of Virginia) were converted into 3118 images. The CNN was designed and analyzed at the University of Virginia. The data were collected, prepared, and analyzed between November 2017 and February 2018.

Main Outcomes and Measures

Classification accuracy of the CNN per image and per case and incorrect classification rate identified by aggregated 10-fold cross-validation confusion/error matrices of CNN models.

Results

Overall, 102 children participated in this study, with a median (interquartile range) age of 31.0 (20.3-75.5) months and a roughly equal sex distribution, with 53 boys (51.9%). The model demonstrated 93.4% case-detection accuracy and had a false-negative rate of 2.4%. Confusion metrics indicated most incorrect classifications were between patients with CD and healthy patients. Feature map activations were visualized and learned distinctive patterns, including microlevel features in duodenal tissues, such as alterations in secretory cell populations.

Conclusions and Relevance

A machine learning-based histopathological analysis model demonstrating 93.4% classification accuracy was developed for identifying and differentiating between duodenal biopsies from children with EE and CD. The combination of the CNN with a deconvolutional network enabled feature recognition and highlighted secretory cells' role in the model's ability to differentiate between these histologically similar diseases.

Development of a cytology-based multivariate analytical risk index for oral cancer

OBJECTIVES

The diagnosis and management of oral cavity cancers are often complicated by the uncertainty of which patients will undergo malignant transformation, obligating close surveillance over time. However, serial biopsies are undesirable, highly invasive, and subject to inherent issues with poor inter-pathologist agreement and unpredictability as a surrogate for malignant transformation and clinical outcomes. The goal of this study was to develop and evaluate a Multivariate Analytical Risk Index for Oral Cancer (MARIO) with potential to provide non-invasive, sensitive, and quantitative risk assessments for monitoring lesion progression.

MATERIALS AND METHODS

A series of predictive models were developed and validated using previously recorded single-cell data from oral cytology samples resulting in a "continuous risk score". Model development consisted of: (1) training base classification models for each diagnostic class pair, (2) pairwise coupling to obtain diagnostic class probabilities, and (3) a weighted aggregation resulting in a continuous MARIO.

RESULTS AND CONCLUSIONS

Diagnostic accuracy based on optimized cut-points for the test dataset ranged from 76.0% for Benign, to 82.4% for Dysplastic, 89.6% for Malignant, and 97.6% for Normal controls for an overall MARIO accuracy of 72.8%. Furthermore, a strong positive relationship with diagnostic severity was demonstrated (Pearson's coefficient = 0.805 for test dataset) as well as the ability of the MARIO to respond to subtle changes in cell composition. The development of a continuous MARIO for PMOL is presented, resulting in a sensitive, accurate, and non-invasive method with potential for enabling monitoring disease progression, recurrence, and the need for therapeutic intervention of these lesions.

Using deep convolutional neural networks to identify and classify tumor-associated stroma in diagnostic breast biopsies

The breast stromal microenvironment is a pivotal factor in breast cancer development, growth and metastases. Although pathologists often detect morphologic changes in stroma by light microscopy, visual classification of such changes is subjective and non-quantitative, limiting its diagnostic utility. To gain insights into stromal changes associated with breast cancer, we applied automated machine learning techniques to digital images of 2387 hematoxylin and eosin stained tissue sections of benign and malignant image-guided breast biopsies performed to investigate mammographic abnormalities among 882 patients, ages 40-65 years, that were enrolled in the Breast Radiology Evaluation and Study of Tissues (BREAST) Stamp Project. Using deep convolutional neural networks, we trained an algorithm to discriminate between stroma surrounding invasive cancer and stroma from benign biopsies. In test sets (928 whole-slide images from 330 patients), this algorithm could distinguish biopsies diagnosed as invasive cancer from benign biopsies solely based on the stromal characteristics (area under the receiver operator characteristics curve = 0.962). Furthermore, without being trained specifically using ductal carcinoma in situ as an outcome, the algorithm detected tumor-associated stroma in greater amounts and at larger distances from grade 3 versus grade 1 ductal carcinoma in situ. Collectively, these results suggest that algorithms based on deep convolutional neural networks that evaluate only stroma may prove useful to classify breast biopsies and aid in understanding and evaluating the biology of breast lesions.

Fully automated detection of breast cancer in screening MRI using convolutional neural networks

Current computer-aided detection (CADe) systems for contrast-enhanced breast MRI rely on both spatial information obtained from the early-phase and temporal information obtained from the late-phase of the contrast enhancement. However, late-phase information might not be available in a screening setting, such as in abbreviated MRI protocols, where acquisition is limited to early-phase scans. We used deep learning to develop a CADe system that exploits the spatial information obtained from the early-phase scans. This system uses three-dimensional (3-D) morphological information in the candidate locations and the symmetry information arising from the enhancement differences of the two breasts. We compared the proposed system to a previously developed system, which uses the full dynamic breast MRI protocol. For training and testing, we used 385 MRI scans, containing 161 malignant lesions. Performance was measured by averaging the sensitivity values between 1/8-eight false positives. In our experiments, the proposed system obtained a significantly ([Formula: see text]) higher average sensitivity ([Formula: see text]) compared with that of the previous CADe system ([Formula: see text]). In conclusion, we developed a CADe system that is able to exploit the spatial information obtained from the early-phase scans and can be used in screening programs where abbreviated MRI protocols are used.

Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images

Currently, histopathological tissue examination by a pathologist represents the gold standard for breast lesion diagnostics. Automated classification of histopathological whole-slide images (WSIs) is challenging owing to the wide range of appearances of benign lesions and the visual similarity of ductal carcinoma in-situ (DCIS) to invasive lesions at the cellular level. Consequently, analysis of tissue at high resolutions with a large contextual area is necessary. We present context-aware stacked convolutional neural networks (CNN) for classification of breast WSIs into normal/benign, DCIS, and invasive ductal carcinoma (IDC). We first train a CNN using high pixel resolution to capture cellular level information. The feature responses generated by this model are then fed as input to a second CNN, stacked on top of the first. Training of this stacked architecture with large input patches enables learning of fine-grained (cellular) details and global tissue structures. Our system is trained and evaluated on a dataset containing 221 WSIs of hematoxylin and eosin stained breast tissue specimens. The system achieves an AUC of 0.962 for the binary classification of nonmalignant and malignant slides and obtains a three-class accuracy of 81.3% for classification of WSIs into normal/benign, DCIS, and IDC, demonstrating its potential for routine diagnostics.